Mar 17, 2026 – Jeremy Snyder – The State of AI Security: Moving Beyond TheoryThe biggest shift evident at the [un]prompted AI Security Practitioners Conference was the move from purely theoretical discussions about “what could go wrong” to concrete, battle-tested methodologies for “what is going wrong and how we fix it.” It’s clear that AI security is rapidly evolving, from initial employee DLP use cases, to organization-wide focus around securing all THINGS A.I..This Week in AI SecurityWe published an episode of our This Week in AI Security Podcast right after the event, which you can watch here below.In the episode, I shared some of my key thoughts around several major themes:LLMs for Vulnerability Discovery and the Zero-Day Clock: Many researchers shared information on using LLMs to identify zero days, malware, and source code vulnerabilities. The most striking observation was the dramatic acceleration of the “meantime to availability of an exploit,” which has reduced from months to hours. This is a “call to arms” for the cybersecurity industry and raised the question of whether automatic patching is now required.Defensive Automation and Agentic Infrastructure: I heard presentations from companies like Google, OpenAI, and Meta about their security strategies, tooling, and efforts to leverage AI agents for security automation.New Attack Surfaces: This area included discussions on indirect prompt injection, new attack vectors in AI-automated systems like KYC pipelines and image recognition (OCR embedded in LLMs), and the vulnerabilities and legal implications of ubiquitous AI notetakers (meeting assistants).Prompt as Code (Conceptual Highlight): To me, the concept of “thinking about the prompt as code” from the Google Gmail team was one of the most interesting conceptual points, emphasizing the need to apply secure coding and hygiene practices to the prompt itself, as it serves as an instruction set.Real-World Case Studies: I noted good real-world case studies from various firms (Trail of Bits, Wiz, others), including the use of multi-agent triage to uncover breaches.Overall, huge kudos to the team over at Knostic!But that’s not all…There were a number of other topics that I didn’t have enough time to cover in the 15-minute episode. Here are some of my thoughts below. Operationalizing Threat Modeling for LLMsOne theme was the urgent need for threat modeling tailored specifically to Large Language Models (LLMs) and generative AI systems. Traditional application security models often fall short, failing to account for the unique attack surface introduced by model weights, training data pipelines, and prompts themselves.Key speaker sessions highlighted a new approach focusing on three main challenges:Model Theft & Extraction: Protecting intellectual property embedded in the model itself.Inference-Time Attacks (Prompt Injection, Evasion): Mitigating threats during real-time use.System-Level Integration Risks: Addressing vulnerabilities introduced when LLMs connect to external tools (RAG, code execution).A Shift in Attack Vectors: Focus on Evasion and MisuseWhile Prompt Injection remains a foundational concern, the conversation has matured to address more subtle and potentially damaging attack vectors.Adversarial Evasion TechniquesSeveral talks detailed advanced adversarial examples designed not just to trick the model into an undesirable output, but to subtly shift its behavior over time or bypass safety filters without obvious jailbreaking language. This requires a defensive posture that looks beyond simple keyword blocking and into semantic understanding and anomaly detection on input and output data.Misuse and Abuse by DesignThe focus is increasingly on how malicious actors can misuse the powerful capabilities of an AI system, even when it’s technically operating “as intended.” For example, using a coding assistant LLM to generate highly optimized malware code or leveraging an RAG system to exfiltrate proprietary data through cleverly crafted queries. This necessitates integrating “red teaming” early in the development lifecycle, simulating real-world abuse scenarios before deployment.The Tooling Landscape: What Practitioners Are UsingThe conference provided a fantastic overview of the tools that are actually making a difference in AI security labs today. The consensus is that no single tool provides a complete solution, so a layered defense strategy is essential.The Rise of Defense-in-Depth for AIThe core message is the need for an approach that includes:Application Layer: Prompt engineering guidelines and specific guardrails.Middleware/Proxy Layer: Dedicated AI security tools intercepting API calls for validation, sanitization, and logging.Model Layer: In-model defenses (e.g., constitutional AI, fine-tuning for robustness) and continuous monitoring of model performance and drift.Looking Ahead: The Human Element and Future ChallengesBeyond the technical deep-dives, the most engaging discussions
[…]
Content was cut in order to protect the source.Please visit the source for the rest of the article.
Read the original article: