Secure AI
Posts
Secure AI #2: Lots of AI security news!

Secure AI #2: Lots of AI security news!

Charles Gillman
August 10, 2023

Welcome to Secure AI, where we discuss the intersection of Cybersecurity, Privacy and IT Governance, Risk & Compliance.

This week we have lots of news in the research and vendor space including new attacks and defences!

In this newsletter…

This Weeks Article - AI + Analyst Pairing = Closing the Cyber Skills Gap
AI Security News - Latest news & news you may have missed
AI Governance News - Latest GRC related AI news
Vendor AI Security News - AI news from the vendor world
Recommended Reading - Deeper dives into AI & Security

This Weeks Article

AI + Analyst Pairing = Closing the Cyber Skills Gap

There is a lot to learn and understand across all facets of Cybersecurity. It takes years to gain a depth of skills across such a wide range of topics, from ransomware to Risk Management and everything in between.

Now with tools like ChatGPT and other Large Language Models (LLMs), we have access to a vast wealth of cybersecurity information that we can query and converse with in natural language. This functionality will help to skill up not just new starters in Cybersecurity but will help to upskill across the board.

We are already seeing industry-specific LLMs, including Google’s SecPaLM model, popping up. Google owns both Mandiant and VirusTotal and has used those sources, amongst others, to train SecPaLM. So when we start pairing cybersecurity-specific LLMs with cybersecurity resources, we have a force multiplier. In the simplest example, when paired up with a cybersecurity-specific LLM, it would allow a Level 1 SOC Analyst to begin operating as Level 2 Analyst, and a Level 2 Analyst as a Level 3 and so on.

To get started, we don’t need to rely on a cybersecurity-specific LLM; with the right prompts, we can create a cybersecurity-specific chatbot right now for use in our organisations with minimal effort. I created a proof of concept Cybersecurity Awareness Training bot* using ChatGPT-4 and an elaborate prompt written in Javascript Object Notation (JSON) to demonstrate this.

The JSON prompt turned ChatGPT-4 into an interactive Cybersecurity Awareness Coach that allowed the user to set their skill levels and topics of interest, and the bot would deliver content accordingly. This concept could readily be extended to start augmenting junior Cybersecurity resources right now.

So while ChatGPT, Bard, Claude and other LLMs are not going to solve the global cybersecurity skills shortage overnight, I believe that over the next year, we will see more AI tools that empower cybersecurity professionals to operate at a higher skill level and allow newcomers to the industry to skill up faster than those that came before them.

The write-up of the bot is on Medium along with the JSON prompt if you want to test it out https://medium.com/@fullstackciso/prompts-as-code-is-this-the-future-of-prompting-aac7fadf69cc*)

AI Security News

Story of the Week

A New Attack Impacts ChatGPT—and No One Knows How to Stop It

Researchers found a simple way to make ChatGPT, Bard, and other chatbots misbehave, proving that AI is hard to tame.

www.wired.com/story/ai-adversarial-attacks

Other News

Securing LLM Systems Against Prompt Injection | NVIDIA Technical Blog

This post explains prompt injection and shows how the NVIDIA AI Red Team identified vulnerabilities where prompt injection can be used to exploit three plug-ins included in the LangChain library.

developer.nvidia.com/blog/securing-llm-systems-against-prompt-injection

Meet the Brains Behind the Malware-Friendly AI Chat Service ‘WormGPT’ – Krebs on Security

Brian Krebs interviews the creator of WormGPT. It’s not about the money.

krebsonsecurity.com/2023/08/meet-the-brains-behind-the-malware-friendly-ai-chat-service-wormgpt

EyeSpy Proof-of-Concept

HYAS Labs releases EyeSpy, a proof-of-concept that introduces an entirely new type of fully autonomous, AI-synthesized, polymorphic malware.

www.hyas.com/blog/eyespy-proof-of-concept

Unmasking Hypnotized AI: The Hidden Risks of Large

To what extent can Large Language Models (LLMs) be hypnotized into giving responses that pose a clear security risk?

securityintelligence.com/posts/unmasking-hypnotized-ai-hidden-risks-large-language-models

MIT Develops 'Masks' to Protect Images From Manipulation by AI

PhotoGuard creates invisible masks that distort AI-generated images.

aibusiness.com/responsible-ai/mit-develops-masks-to-protect-images-from-manipulation-by-ai

In case you missed it…

OWASP Top 10 for LLMs

Welcome to the first iteration of the OWASP Top 10 for Large Language Models (LLMs) Applications. This document marks an exciting new chapter in the ongoing efforts to enhance security in the rapidly evolving field of artificial intelligence

owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v05.pdf

How Cybercriminals Can Perform Virtual Kidnapping Scams Using AI Voice Cloning Tools and ChatGPT - Security News

This article gives an overview of the elements of virtual kidnapping and how malicious actors use social engineering tactics and abuse AI voice cloning tools and ChatGPT to launch these attacks.

www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/how-cybercriminals-can-perform-virtual-kidnapping-scams-using-ai-voice-cloning-tools-and-chatgpt

New AI Tool 'FraudGPT' Emerges, Tailored for Sophisticated Attacks

FraudGPT, the latest cybercrime AI tool, is being sold on dark web marketplaces and Telegram channels.

thehackernews.com/2023/07/new-ai-tool-fraudgpt-emerges-tailored.html

AI Governance News

Regulating Generative AI

How Well Do LLMs Comply with the EU AI Act?

towardsdatascience.com/regulating-generative-ai-e8b22525d71a

GitHub - wealthsimple/llm-gateway

llm-gateway is a gateway for third party LLM providers such as OpenAI, Cohere, etc. It tracks data sent and received from these providers in a postgres database and runs PII scrubbing heuristics prior to sending.

github.com/wealthsimple/llm-gateway

Vendor AI News

Protecting data in the era of generative AI: Nightfall AI launches innovative security platform

Nightfall AI's platform is the first data loss prevention (DLP) platform that scales across the three top threat vectors CISOs need the most help securing when generative AI and ChatGPT are in use across their organizations.

venturebeat.com/security/protecting-data-in-the-era-of-generative-ai-nightfall-ai-launches-innovative-security-platform

Dig Security launches capabilities to protect data fed into LLMs

Dig has made enhancements to its data security offering to enable the detection, classification, and mapping of sensitive data used to train LLM models.

www.csoonline.com/article/647380/dig-security-launches-capabilities-to-protect-data-fed-into-llms.html

NetSPI Introduces ML/AI Penetration Testing: Elevating Security Measures for Machine Learning…

NetSPI introduces pioneering ML/AI Penetration Testing solution for fortified security in machine learning models.

medium.com/@multiplatform.ai/netspi-introduces-ml-ai-penetration-testing-elevating-security-measures-for-machine-learning-57233235884b

The Time is Now to Protect AI

Protect AI Raises $35M in Series A Funding.

protectai.com/blog/time-is-now

Vectra AI Delivers Attack Signal Intelligence Across Hybrid Cloud Domains - MSSP Alert

The Vectra AI Platform with Attack Signal Intelligence provides insights into active attacks across organizations' hybrid cloud domains.

www.msspalert.com/cybersecurity-services-and-products/ai/vectra-ai-delivers-attack-signal-intelligence-across-hybrid-cloud-domains