The world's most realistic benchmark for database AI-friendliness. A 40-entity enterprise schema. Real CLI agents. Live databases. Zero No claims, no hypotheses, no verdicts. Just raw data from real agents on real databases.
We asked two independent AI coding agents (Codex gpt-5.6-terra and Claude Sonnet) to add a field to an existing codebase with a 40-entity schema (600+ lines of inline schema code) and 5 working model files. Each agent was compared only to itself — never cross-CLI. MongoDB.
No hypotheses. No verdicts. Just the raw numbers — you decide what they mean.
Each CLI counts tokens differently. Claude Code includes the system prompt, tool definitions, and cache read/creation in every token count — so its numbers are inherently higher. Codex counts usage events differently. These numbers are never compared cross-agent. Each agent is compared only to itself across the three database lanes. The percentage differences (e.g. +256%) are within-agent — they measure how much MORE the same agent spends on Postgres vs MongoDB, not how one agent compares to the other.
Every benchmark claims to be fair. This one proves it with 10 independent defenses — any one of which would catch bias if it existed.
No mocks. No simulated agents. No estimated tokens. Every number comes from a real AI agent writing real code against a real database. vanilla mode, anti-cheat, within-agent comparison — zero bias.
If your team runs 1,000 AI database-tasks per month, here's the monthly token bill per database design — based on the measured medians. MongoDB saves significant money on every agent.
Assumptions: 1,000 tasks/month. Token cost estimated at $3/M input tokens (Claude) and $2/M input tokens (Codex). within-agent comparison only — absolute costs are not cross-comparable between agents because CLIs count tokens differently. vanilla mode: no skills, no code reviews, no subagents. anti-cheat: source-pattern scan, protected files, cheat-signal detection.
Based on research into real enterprise database complexity (SAP, Salesforce, Workday). 40 entities across 6 business domains, written as inline schema code the agent must read. MongoDB stores them as 21 collections with embedded arrays. Postgres stores them as 40 normalized tables with foreign keys and CHECK constraints.
The agent receives a change ticket — exactly what a developer gets in an enterprise. It must understand the existing schema, find where the field goes, write the migration, update the query, and not break anything.
"Add a preferredPaymentMethod field (string, optional,
enum: credit_card / wire_transfer / ach / paypal) to the
accounts entity. Update the
getOrderSummary query to include the account's
preferredPaymentMethod in the result."
src/schema.mjs (600+ lines of inline schema) to
understand the 40-entity schema
src/accounts.mjs, src/orders.mjs,
src/invoices.mjs, src/products.mjs,
src/support.mjs — 5 existing model files with working
CRUD code
src/queries.mjs for the order summary query
npm test until it passes — tests check regression
AND new behavior