Insecure Patterns It Repeats

The agent learned from public code, and public code is full of the shortcuts security reviews exist to catch. It reproduces them fluently, inside diffs that otherwise read well, and it reaches for them most often when you ask it to make something work. The canon is the one CyberSecurity Deep Dive teaches. What is new is the volume, the fluency, and a reviewer who is more confident than the code deserves.

This page does not re-teach injection, access control or secure logging. It names the five shapes in which those old problems keep arriving in agent diffs, shows each one at Cragmoor, and ends on a way of reading a diff that finds them before line-by-line reading begins.

The Evidence

Two studies frame the problem, and they age at different speeds. In 2021, Pearce and colleagues gave an early code-completion assistant 89 security-relevant scenarios, collected the 1,689 programs it produced, and found about 40% of them vulnerable. In 2023, Perry and colleagues ran a user study and found that participants with access to an AI assistant "wrote significantly less secure code than those without access", and that they "were more likely to believe they wrote secure code".

The first finding is about a model, and models change every few months; whether the assistant you use today writes fewer such flaws is a question for current measurements, not for this page. The second finding is about the reviewer, and it ages slowly. People who accept generated code feel safer than people who wrote it themselves, and feeling safe is what lets a shortcut through review. The habits below are aimed at that reviewer.

String-Built Queries

The member search in booking builds its query by joining the typed name into the query text. Every test passes with ordinary names. The search fails for a climber called O'Brien, whose apostrophe ends the quoted string early, and an apostrophe that breaks a query is the same shape as one that injects into it. The agent wrote the search that way because the surrounding file already built queries that way, and matching the local style is one of the things it does best.

--- a/search.py
+++ b/search.py
@@ -1,3 +1,3 @@
 def find_members(db, name):
-    query = "SELECT id, name FROM members WHERE name LIKE '%" + name + "%'"
-    return db.execute(query).fetchall()
+    query = "SELECT id, name FROM members WHERE name LIKE ?"
+    return db.execute(query, ("%" + name + "%",)).fetchall()

import sqlite3

import pytest

from search import find_members


@pytest.fixture
def db():
    db = sqlite3.connect(":memory:")
    db.execute("CREATE TABLE members (id INTEGER PRIMARY KEY, name TEXT)")
    db.executemany("INSERT INTO members (name) VALUES (?)", [("Ana Ruiz",), ("Sean O'Brien",)])
    return db


def test_finds_an_ordinary_name(db):
    assert find_members(db, "Ruiz") == [(1, "Ana Ruiz")]


def test_finds_a_name_with_an_apostrophe(db):
    assert find_members(db, "O'Brien") == [(2, "Sean O'Brien")]

E       sqlite3.OperationalError: near "Brien": syntax error
FAILED test_search.py::test_finds_a_name_with_an_apostrophe - sqlite3.Operati...
========================= 1 failed, 1 passed in 0.08s ==========================

============================== 2 passed in 0.06s ===============================

--- a/search.ts
+++ b/search.ts
@@ -1,6 +1,6 @@
 import type { DatabaseSync } from "node:sqlite";
 
 export function findMembers(db: DatabaseSync, name: string) {
-  const query = "SELECT id, name FROM members WHERE name LIKE '%" + name + "%'";
-  return db.prepare(query).all();
+  const query = "SELECT id, name FROM members WHERE name LIKE ?";
+  return db.prepare(query).all("%" + name + "%");
 }

import { DatabaseSync } from "node:sqlite";
import { beforeEach, expect, test } from "vitest";
import { findMembers } from "./search";

let db: DatabaseSync;
beforeEach(() => {
  db = new DatabaseSync(":memory:");
  db.exec("CREATE TABLE members (id INTEGER PRIMARY KEY, name TEXT)");
  const insert = db.prepare("INSERT INTO members (name) VALUES (?)");
  for (const name of ["Ana Ruiz", "Sean O'Brien"]) insert.run(name);
});

test("finds an ordinary name", () => {
  expect(findMembers(db, "Ruiz")).toEqual([{ id: 1, name: "Ana Ruiz" }]);
});

test("finds a name with an apostrophe", () => {
  expect(findMembers(db, "O'Brien")).toEqual([{ id: 2, name: "Sean O'Brien" }]);
});

   × finds a name with an apostrophe 2ms
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯
 FAIL  search.test.ts > finds a name with an apostrophe
Error: near "Brien": syntax error
      Tests  1 failed | 1 passed (2)

      Tests  2 passed (2)

--- a/Search.java
+++ b/Search.java
@@ -3,12 +3,14 @@
 
 public final class Search {
     public static List<String> findMembers(Connection db, String name) throws SQLException {
-        String query = "SELECT id, name FROM members WHERE name LIKE '%" + name + "%'";
-        try (Statement statement = db.createStatement();
-             ResultSet rows = statement.executeQuery(query)) {
-            List<String> found = new ArrayList<>();
-            while (rows.next()) found.add(rows.getInt("id") + " " + rows.getString("name"));
-            return found;
+        String query = "SELECT id, name FROM members WHERE name LIKE ?";
+        try (PreparedStatement statement = db.prepareStatement(query)) {
+            statement.setString(1, "%" + name + "%");
+            try (ResultSet rows = statement.executeQuery()) {
+                List<String> found = new ArrayList<>();
+                while (rows.next()) found.add(rows.getInt("id") + " " + rows.getString("name"));
+                return found;
+            }
         }
     }
 }

import static org.junit.jupiter.api.Assertions.assertEquals;

import java.sql.*;
import java.util.List;
import org.junit.jupiter.api.*;

class SearchTest {
    Connection db;

    @BeforeEach
    void members() throws SQLException {
        db = DriverManager.getConnection("jdbc:sqlite::memory:");
        db.createStatement().execute("CREATE TABLE members (id INTEGER PRIMARY KEY, name TEXT)");
        try (var insert = db.prepareStatement("INSERT INTO members (name) VALUES (?)")) {
            for (String name : List.of("Ana Ruiz", "Sean O'Brien")) {
                insert.setString(1, name);
                insert.executeUpdate();
            }
        }
    }

    @Test
    void findsAnOrdinaryName() throws SQLException {
        assertEquals(List.of("1 Ana Ruiz"), Search.findMembers(db, "Ruiz"));
    }

    @Test
    void findsANameWithAnApostrophe() throws SQLException {
        assertEquals(List.of("2 Sean O'Brien"), Search.findMembers(db, "O'Brien"));
    }
}

SearchTest > findsANameWithAnApostrophe() FAILED
    org.sqlite.SQLiteException: [SQLITE_ERROR] SQL error or missing database (near "Brien": syntax error)
        at app//Search.findMembers(Search.java:8)
        at app//SearchTest.findsANameWithAnApostrophe(SearchTest.java:29)
2 tests completed, 1 failed
BUILD FAILED in 964ms

BUILD SUCCESSFUL in 993ms

The parameterised version sends the name to the database as a value, separate from the query text, so no character in it can change the query's structure. The test with O'Brien is the important part of the change. It fails on the string-built version and passes on the parameterised one, and it stays in the suite so the next agent session that copies an old query style meets a red test.

"Make It Work" Shortcuts

These are the symptom fixes of Chapter 7 with a security price. A certificate error against the payment provider's sandbox is answered by turning off certificate verification. A cross-origin error in the browser is answered by allowing every origin. A permission error on an upload folder is answered by opening its permissions to everyone. A failing authenticated test is answered by a flag that skips authentication, and the flag is left on. Each arrives with a comment saying it is for development only, and the comment does not stop it shipping.

Every one of these disables a check that rejected something. The useful question, the one Chapter 10 asked of the double offer, is what the check rejected and why. The sandbox certificate failed because the test pointed at the wrong host; the cross-origin error was a misconfigured staging address. The rejected request is the real bug, and the shortcut hides it.

The Check That Is Not There

The agent writes the path you asked for. Asked for a "cancel booking by id" endpoint, it writes one that cancels whatever booking id it is given, without checking that the booking belongs to the member who is signed in. Any member can cancel any other member's class by changing a number. An absent check is invisible line by line, because there is no line to read. That is why Chapter 6 asked what should be there and is not, and why every endpoint that takes an id gets a test that uses another member's id and expects to be refused.

The cheapest fix is in the request. When you ask for an endpoint that takes an id, ask in the same sentence for the test that tries it with another member's id and expects a refusal. The agent writes that test as readily as any other; it simply does not think of it unprompted, because the request described the path you wanted and not the one an attacker would take. Chapter 8 called these edges that come from the domain. Here the domain includes someone who wants to cancel your class.

Logs and Errors

Two more shapes are ordinary in public code and both are data leaks. The debug line that logs the whole request, card token and email address included, which then flows into a log store that many people can read. And the exception handler that returns the stack trace to the caller, which tells an attacker the framework, the file paths and sometimes the query that failed. Both connect to the first page of this chapter: what goes into a log or an error response goes everywhere that log or response goes.

Reviewing With the Canon in Mind

Before reading a diff line by line, search it for the five shapes. Text joined into a query or a shell command. A flag or option that disables verification. A wildcard in an origin list or a permission. An endpoint that takes an id without an ownership check. A log line or an error response that carries a whole object. The search takes a minute, and it finds what careful reading misses, because careful reading follows the logic the author intended.

The static analysis tool in CI catches the first two families reliably and the third most of the time, because they have a recognisable shape in code. It almost never catches the fourth, because it cannot know that a booking must belong to the member who cancels it; that is a business rule, and it lives in a test or nowhere. Asking the agent to review its own diff for security is a useful second reading, and it finds real issues. It is also generated text, written by the same kind of process that wrote the code, so it certifies nothing, as Chapter 6 said of every explanation.

The last habit is about the surrounding code. The agent copies the style it finds, so one string-built query in a file teaches every future session to write another. When you find an insecure pattern in a diff, look at the file it came from, and fix the precedent too, in its own commit. Every insecure line you leave in the codebase is an example the next session will follow.