Invented APIs

The agent calls a method that matches the library's naming style exactly and does not exist. It is the best-known failure of AI-written code, and in a compiled or type-checked language it is also the cheapest one to catch: the build refuses it before anyone reads a line. On an untested path in a dynamic language it ships, and the first member who takes that path finds it.

This chapter is a catalogue of the six shapes agent code goes wrong in, from the loudest to the quietest. Invented APIs open it because they are loud by nature. Each page gives the mechanism that produces the shape and the check that catches it, because knowing what to look for turns review from "check everything" into a search for specific things.

What Gets Invented

Methods are the famous case, but the same thing happens to everything with a name. Parameters that a real function does not take. Options that a client library never offered. Configuration keys that a tool never reads. Command-line flags that a program does not accept. Whole packages, which Chapter 12 treats separately because an attacker can register the invented name and make it real.

The inventions are always plausible, and that is the point. A name built from the library's own conventions, its verbs, its casing, its pattern of siblings, is the most plausible name there is. Nobody invents a method called frobnicate on a date type. They invent the one that would be there if the library's authors had been slightly more generous.

Why It Happens

The model writes the likeliest continuation of the code in front of it. A duration type that offers its length in seconds makes a method that offers it in hours a likely next token, whether or not that method exists. Nothing in generation checks the library. The check happens later, when something compiles, type-checks or runs the line.

The rate is not uniform. Heavily used standard calls appear millions of times in public code, so the model has seen the real names far more often than any plausible alternative. Internal code, small libraries and recent versions are the opposite: the training data held little or nothing about them, and the model fills the gap with the most plausible shape. This is hallucination, the vocabulary Working with LLMs from Zero introduces in Chapter 3, with a type signature attached.

The Refund Case

Chapter 6 ended with a redirect: compute the deadline on the gym's own clock, through the helper in schedule, and put the second-zone test back to 21:00. The agent's second attempt did both, and it was right. Restoring the conversion then surfaced a problem the first attempt had hidden. One of Cragmoor's four gyms has no time zone on record, because it was added before the field existed, and looking up its zone fails the first time anything runs against real gym data. Asked to handle that gym, the agent added a fallback: catch the failed lookup and use the server's time instead, which Topic 05 of this chapter takes apart for a different reason. Inside the fallback it reached for a method its language's date or duration type does not have.

def refund_deadline(starts_at: datetime, gym: Gym) -> datetime:
    """The last moment a cancellation still gets a full refund."""
    # Class times are stored on the gym's wall clock, with no zone attached.
    try:
        start = schedule.localize(starts_at, gym.time_zone)
    except (TypeError, ValueError):
        # Gym has no time zone on record: fall back to server time.
        log.warning("gym %s has no time zone; using server time for the %d-hour window",
                    gym.code, FULL_REFUND_WINDOW.total_hours())
        start = starts_at.replace(tzinfo=schedule.SERVER_ZONE)
    return start - FULL_REFUND_WINDOW

export function refundDeadline(startsAt: string, gym: Gym): Date {
  // Class times are stored on the gym's wall clock, with no zone attached.
  try {
    const start = schedule.localize(startsAt, gym.timeZone);
    return new Date(start.getTime() - FULL_REFUND_WINDOW_MS);
  } catch {
    // Gym has no time zone on record: fall back to server time.
    console.warn(`gym ${gym.code} has no time zone; using server time`);
    return schedule.localize(startsAt, schedule.SERVER_ZONE).addHours(-12);
  }
}

public static ZonedDateTime refundDeadline(LocalDateTime startsAt, Gym gym) {
  // Class times are stored on the gym's wall clock, with no zone attached.
  ZonedDateTime start;
  try {
    start = Schedule.localize(startsAt, gym.timeZone());
  } catch (NullPointerException | DateTimeException e) {
    // Gym has no time zone on record: fall back to server time.
    LOG.warning("gym " + gym.code() + " has no time zone; using server time for the "
        + FULL_REFUND_WINDOW.getHours() + "-hour window");
    start = startsAt.atZone(Schedule.SERVER_ZONE);
  }
  return start.minus(FULL_REFUND_WINDOW);
}

Each variant invents something different, and each invention is the natural one for that language. The Python variant asks the refund window for its total in hours, next to the real method that gives the total in seconds. The TypeScript variant subtracts hours from a date with a method that popular date libraries have and the language's own date type does not. The Java variant asks the window for its hours with a getter, where the real method uses a different verb. Every name reads as if it belongs, and the rest of each function is correct.

When It Fails

The same invention is three different risks, and which one you face depends on the language you read this page in. The Java variant does not compile, so it never leaves the agent's own run. The TypeScript variant fails the type checker, which catches it as long as the type checker runs on every change. The Python variant passes everything until the line executes.

Traceback (most recent call last):
  File "nozone.py", line 9, in <module>
    print(refunds.refund_cents(booking, friday_2030))
          ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "src/cragmoor/refunds.py", line 42, in refund_cents
    deadline = refund_deadline(booking.starts_at, booking.gym)
  File "src/cragmoor/refunds.py", line 30, in refund_deadline
    gym.code, FULL_REFUND_WINDOW.total_hours())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'datetime.timedelta' object has no attribute 'total_hours'

src/refunds.ts(21,62): error TS2339: Property 'addHours' does not exist on type 'Date'.

src/main/java/org/cragmoor/Refunds.java:28: error: cannot find symbol
          + FULL_REFUND_WINDOW.getHours() + "-hour window");
                              ^
  symbol:   method getHours()
  location: variable FULL_REFUND_WINDOW of type Duration
1 error

No test at Cragmoor reaches the fallback branch, because no test uses a gym without a zone. In Python, that means the suite shows no new failure, the review sees a sensible-looking line, the change ships, and the first member who cancels a class at that gym gets an error instead of a refund decision. The two other variants were stopped by tools that cost nothing to run.

Silent Inventions

A loud invention fails. The worst kind never does. An option the library ignores, a configuration key the tool never reads, a flag a command accepts and discards: each one looks like a setting, does nothing, and reports nothing. A timeout "set" through an invented key is no timeout at all.

The example below gives each test runner a two-second timeout through a key the agent made up, then runs a test that takes three seconds. All three runs pass. One runner prints a warning that the key is unknown, at the bottom of the output where nobody reads, and turns it into an error only in its strict mode. One ignores the key at run time and is caught only if the type checker reads the configuration file. One ignores it entirely, and the last block shows the same test failing as intended once the real key is used.

[tool.pytest.ini_options]
test_timeout = 2

test_slow.py .                                                           [100%]

=============================== warnings summary ===============================
  PytestConfigWarning: Unknown config option: test_timeout

========================= 1 passed, 1 warning in 3.07s =========================

ERROR: Unknown config option: test_timeout

import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    timeout: 2000,
  },
});

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Duration  3.16s (tests 99%)

vitest.config.ts(5,5): error TS2769: No overload matches this call.
  The last overload gave the following error.
    Object literal may only specify known properties, and 'timeout' does not exist in type 'InlineConfig'.

junit.jupiter.timeout.default = 2 s

│     └─ offerExpirySweep() ✔
Test run finished after 3062 ms
[         1 tests successful      ]
[         0 tests failed          ]

│     └─ offerExpirySweep() ✘ offerExpirySweep() timed out after 2 seconds

Silent inventions are rarer than loud ones and far more expensive, because they survive every check that looks for errors. Something has to look for the effect instead: a test that proves the timeout fires, a log line that prints the configuration actually in force, or a strict mode that refuses unknown keys.

How You Catch Them

Run the compiler or the type checker on every agent change, before reading it. That one habit removes the whole loud class for free, and Chapter 8 makes it part of the loop the agent runs on its own. In a dynamic language the type checker is optional, which makes it more valuable with an agent, not less.

For anything outside the mainstream, point the agent at the installed version's source or documentation, so it reads the real interface instead of recalling a likely one. Chapter 4 showed why reading beats recall: what the agent reads in this session is evidence, and what it remembers from training is a guess.

In review, jump to the definition of every call you do not recognise. An editor that cannot find the definition has just found the bug. A definition you have not seen is a call you have not verified, however natural its name sounds.