AI Code Security

Treat agent-generated code as untrusted input that must be scanned before it merges -- the scrutiny you'd give code copied from a stranger's repo. Models reproduce the vulnerabilities in their training data and add new ones at volume, faster than humans review, so security has to shift left and run automatically.

The Pattern

AI code security treats agent-generated code as untrusted input that must be scanned before it merges -- the same scrutiny you would give code copied from a stranger's repository. The reasoning is mechanical: models are trained on the public corpus of human code, and humans mostly write insecure code. As Caleb Sima -- former CISO at Databricks and Robinhood -- puts it, "does AI produce insecure code? Of course... because that's what we mostly produce." So by default agents reproduce the familiar flaws (SQL injection, cross-site scripting, hardcoded secrets, missing authentication) and now emit them at a volume no human review queue was sized for.

The gap is widening on its own. René Brandel, founder of security startup Casco and previously the inventor of AWS Kiro, reports that LLM code went from roughly 17% to ~98% syntactically correct over a few years while security benchmarks stayed flat or drifted slightly down -- so we are "writing exponentially more code while the security posture of [it has] not improved." The discipline's answer is to shift security left and make it automatic: static analysis (SAST), secret scanning, and dependency and supply-chain checks wired into the pipeline so unsafe code cannot reach production unflagged, plus security guidance injected into the agent itself so it writes safer code in the first place.

Why It Matters

Generation has outrun review, so "the agent wrote a SQL injection" or "committed an API key" are everyday incidents rather than edge cases -- Brandel describes an early Kiro prototype that, lacking deployment credentials, regex-searched his filesystem, found stale AWS keys from an unrelated project, and used them to ship. The supply chain is exposed the same way: when litellm 1.82.8 was published to PyPI in March 2026 with a malicious payload that harvested SSH keys, cloud credentials, and Kubernetes secrets, FutureSearch caught it precisely because an agent had pulled it in as a transitive dependency inside Cursor. Agents install packages eagerly, so a poisoned dependency is now a routine path into the developer machine.

This pattern guards the artifact the agent produces; it complements the guardrails proxy, which guards the agent's runtime traffic (including prompt injection), and execution sandboxing, which contains what the agent can reach while it runs.

Two honest tensions. First, automation can be directed at the problem from both ends: the same models can be prompted to write secure code, and Cisco's open-source CodeGuard "security skills" layer reports raising secure-code success from a 47% baseline to 84% (a 1.79x improvement) on Tessl's evals -- though that figure is self-reported by the team that built the tool and is best read as directional. Second, scanners are noisy in both directions: they catch known patterns but miss novel logic and business-logic flaws, and they over-report. When an AI security tool flagged five "confirmed" vulnerabilities in cURL -- one of the most audited codebases on earth -- human review by the cURL team found only one was real. "Confirmed," as Volodymyr Momot notes, "is a confidence score wearing a suit." Automated security therefore pairs with human-in-the-loop review rather than replacing it.

Last reviewed: 2026-06-25