<p>As enterprises race to deploy AI across their operations, a perfect storm is brewing: New AI-generated attack vectors are colliding with employees’ growing emotional trust in chatbots and AI assistants, creating security blind spots that traditional defenses weren’t designed to handle.</p>
<p>Security teams are knee-deep in mitigating the threats that accompany AI adoption, from <a href=”https://www.techtarget.com/searchsecurity/tip/Types-of-prompt-injection-attacks-and-how-they-work”>prompt injections</a> and <a href=”https://www.techtarget.com/searchsecurity/tip/How-data-poisoning-attacks-work”>data poisoning</a> to bias exploitation, deepfakes, and models acting in unexpected ways. Though a constant struggle, this more technical concern accompanies the psychological issue of employees oversharing sensitive information with conversational AI systems they’ve come to trust as helpful and even friendly digital assistants — a challenge that is harder to address.</p>
<section class=”section main-article-chapter” data-menu-title=”The problem of oversharing”>
<h2 class=”section-title”><i class=”icon” data-icon=”1″></i>The problem of oversharing</h2>
<p>The personal use of generative AI and chatbots has broad social implications that bleed into the workplace. Psychologists understand that human beings tend to connect with anything that talks to them, <a href=”https://www.psychologytoday.com/us/blog/virtue-in-the-media-world/202405/chatbots-could-start-shaping-how-we-trust-and-who-we-trust” target=”_blank” rel=”noopener”>even if it’s a machine</a>. And although most users know that AI isn’t sentient, it can still elicit emotions — specifically, misplaced trust.<br><br>The line between workplace and personal AI is blurry, and some employees are bringing their bad habits to work. Many organizations have yet to establish firm policies for the use of AI assistants, and many employees use AI without awareness of their organization’s AI strategy, suggesting widespread use of personal tools outside official channels. According to a <a href=”https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part” target=”_blank” rel=”noopener”>Microsoft study</a>, 78% of users bring their own AI tools to work, with the practice being more common at small and midsize companies. Further, a National Cybersecurity Alliance and CybSafe <a href=”https://www.staysafeonline.org/articles/oh-behave-the-annual-cybersecurity-attitudes-and-behaviors-report-2025″ target=”_blank” rel=”noopener”>survey</a> found that 43% of employees who use AI for work tasks send sensitive data to AI applications without their employer’s knowledge.</p>
<p>This reality is creating a new problem for security teams. Employees, already conditioned to trust their personal AI assistants — everything from ChatGPT to AI friend apps — are more likely to let their guard down and share personally identifiable information or sensitive company data with systems that lack inherent privacy safeguards. In fact, many publicly available GenAI platforms clearly state in their T&Cs that they use inputs as training data.<br><br>There are real-world implications. For example, Samsung suffered <a href=”https://www.ciodive.com/news/Samsung-Electronics-ChatGPT-leak-data-privacy/647137/” target=”_blank” rel=”noopener”>several security incidents</a> related to AI assistants. In 2023, an engineer pasted proprietary source code for semiconductor equipment into ChatGPT to help correct errors, exposing confidential code used in the company’s chip manufacturing process. Another employee exposed sensitive business intelligence and internal discussions after feeding the content of a high-level meeting into ChatGPT.</p>
<p>According to Naynesh Patel, managing director of cybersecurity at Accenture, the ease of information sharing with AI assistants is problematic, and traditional enterprise security was not designed for it. “The concept of a text box — where you are able to put information in with little to no friction — and the fact that it’s helpful creates risk,” he said.</p>
</section>
<section class=”section main-article-chapter” data-menu-title=”Governance for AI trust”>
<h2 class=”section-title”><i class=”icon” data-icon=”1″></i>Governance for AI trust</h2>
<p>The convergence of technical vulnerabilities and human psychology requires CISOs and their teams to adopt controls to defend against data loss via AI.</p>
<p>According to Patel, the solution isn’t fixing AI; it’s rethinking how the organization itself governs AI use among its employees. He said that most data security failures aren’t model failures, but identity and <a href=”https://www.techtarget.com/searchsecurity/tip/What-C
[…]
Content was cut in order to protect the source.Please visit the source for the rest of the article.
Read the original article: