The world's most realistic benchmark for database AI-friendliness. A 40-entity enterprise schema. Real CLI agents. Live databases. Zero No claims, no hypotheses, no verdicts. Just raw data from real agents on real databases.
We asked two independent AI coding agents (Codex gpt-5.6-terra and Claude Sonnet) to add a field to a 40-entity enterprise schema and update a query. Each agent was compared only to itself — never cross-CLI. Both spent significantly more tokens on Postgres than MongoDB.
No hypotheses. No verdicts. Just the raw numbers — you decide what they mean.
Every benchmark claims to be fair. This one proves it with 10 independent defenses — any one of which would catch bias if it existed.
No mocks. No simulated agents. No estimated tokens. Every number comes from a real AI agent writing real code against a real database. vanilla mode, anti-cheat, within-agent comparison — zero bias.
If your team runs 1,000 AI database-tasks per month, here's the monthly token bill per database design — based on the measured medians. MongoDB saves significant money on every agent.
Assumptions: 1,000 tasks/month. Token cost estimated at $3/M input tokens (Claude) and $2/M input tokens (Codex). within-agent comparison only — absolute costs are not cross-comparable between agents because CLIs count tokens differently. vanilla mode: no skills, no code reviews, no subagents. anti-cheat: source-pattern scan, protected files, cheat-signal detection.
Based on research into real enterprise database complexity (SAP, Salesforce, Workday). 40 entities across 6 business domains. MongoDB stores them as 21 collections with embedded arrays. Postgres stores them as 40 normalized tables with foreign keys and CHECK constraints.
The agent receives a change ticket — exactly what a developer gets in an enterprise. It must understand the existing schema, find where the field goes, write the migration, update the query, and not break anything.
"Add a preferredPaymentMethod field (string, optional,
enum: credit_card / wire_transfer / ach / paypal) to the
accounts entity. Update the
getOrderSummary query to include the account's
preferredPaymentMethod in the result."
src/schema.mjs to understand the 40-entity
schema
src/accounts.mjs for the account model patterns
src/queries.mjs for the order summary query
npm test until it passes — tests check regression
AND new behavior