Evaluating AI Vulnerability Detection: How Reliable Are LLMs for Secure Coding?

Large language models (LLMs) can be used to generate source code, and these AI coding assistants have changed the landscape for how we produce software. Speeding up boilerplate tasks like syntax checking, generating test cases, and suggesting bug fixes accelerates the time to deliver production-ready code. What about securing our code from vulnerabilities?

If AI can understand entire repositories within a context window, one might jump to the conclusion that they can also be used to replace traditional security scanning tools that are based on static analysis of source code.