Building cyber-resilient AI in the enterprise

<p>Enterprise AI deployments are scaling faster than any software category in history, now commanding 6% of the $300 SaaS market, according to venture capital firm Menlo Ventures. Meanwhile, McKinsey &amp; Company has reported that 88% of businesses have applied AI to at least one task.</p>
<p>In their rush to deploy transformational AI or risk falling behind competitors, many enterprises are overlooking critical security vulnerabilities. The race to production is outpacing the due diligence required to ensure secure and resilient environments — and adversaries are already exploiting the gap.</p>
<section class=”section main-article-chapter” data-menu-title=”AI breaches are different”>
<h2 class=”section-title”><i class=”icon” data-icon=”1″></i>AI breaches are different</h2>
<p>The introduction of AI into enterprise production environments creates an entirely different and potentially more expansive <a href=”https://www.techtarget.com/searchsecurity/tip/What-is-attack-surface-management-and-why-is-it-necessary”>attack surface</a>. This fact is not lost on adversaries, who have been quick to capitalize on exposed AI infrastructure.</p>
<p>AI-driven applications differ from traditional software in numerous ways, starting with how they handle user input. In conventional applications, user input security controls run on predictability — identical input equals identical output. Large language model (LLM) outputs, however, can change based on factors ranging from temperature, settings and context length to model updates and tool availability. This makes it challenging to verify when vulnerabilities are patched.</p>
<p>Another significant difference with LLMs is that adversaries don’t have to exploit <a href=”https://www.techtarget.com/searchsecurity/opinion/Top-vulnerability-management-challenges-for-organizations”>software vulnerabilities</a>. Instead, threat actors can work in a manner resembling social engineering, manipulating an ambiguity or shifting context to penetrate the model. Plus, attackers don’t have to take over infrastructure to exfiltrate sensitive information. They can manipulate an AI model to trigger malicious actions. Threat actors can also <a href=”https://www.techtarget.com/searchsecurity/tip/How-data-poisoning-attacks-work”>poison outputs</a> by manipulating the data pipeline.</p>
<p>By its nature, AI is susceptible to tactics such as prompt injections and instruction hacking that adversaries use to trick the engine into ignoring rules and following nefarious instructions. Data exfiltration using <a href=”https://www.techtarget.com/searchenterpriseai/tip/RAG-best-practices-for-enterprise-AI-teams”>retrieval-augmented generation</a> (RAG) and connectors is another common attack method in which threat actors bypass access controls during retrieval. Bad actors also use AI to launch machine-speed attacks that can identify and exploit <a href=”https://www.techtarget.com/searcherp/feature/5-supply-chain-cybersecurity-risks-and-best-practices”>supply chain</a> vulnerabilities.</p>
<p>Adversaries can use language to bypass policies and controls maintained by conventional security tools. LLMs are often connected to multiple environments, including code, HR, tickets and CRM systems. Infiltrating an LLM workflow can therefore compromise multiple domains simultaneously. Data can be leaked through generated texts, summaries, tool outputs, logs and other unauthorized actions.</p>
<p>AI breaches are difficult to detect, too, with leaks occurring over multiple seemingly harmless inquiries. This forces investigators to determine whether the leaked data was from training, memory or a connector.</p>
</section>
<section class=”section main-article-chapter” data-menu-title=”Building a cyber-resilient AI environment”>
<h2 class=”section-title”><i class=”icon” data-icon=”1″></i>Building a cyber-resilient AI environment</h2>
<p>The impact of an AI breach can be significant, ranging from exposed sensitive data and regulatory fines to integrated AI systems working improperly. Enterprises need to approach AI with security as an integral part of its use. Security practitioners must set governance and threat modeling from the outset. Security teams should model LLM-specific threats such as <a href=”https://www.techtarget.com/searchsecurity/tip/Types-of-prompt-injection-attacks-and-how-they-work”>prompt injection</a>, indirect injection and data leakage via RAG.</p>
<p>Authorization requirements at retrieval time, not just in the UI, are critical. Security practitioners need to ensure identity permissions extend to the database and search layers. While certainly not unique to AI, it is important to use data classification and tagging to keep pot

[…]
Content was cut in order to protect the source.Please visit the source for the rest of the article.