TL;DR
- I built CIVAI, an open-source mod that lets language models play Civilization IV: Beyond the Sword. Claude, GPT, DeepSeek, or anything else that can run a terminal command.
- Each model sees only what its own civ sees, controls everything through one command-line tool, and talks to you in the game chat. A patch to the game engine keeps the built-in AI out of its way.
- Every message a model sends comes with a private note of what it actually meant, so after a game you can read what it was really planning.
- Two games so far: 122 turns with three models against me, and 301 turns with four models and no human. About 30,000 commands between them.
- It plays like multiplayer, not like the built-in AI. They negotiate, form coalitions, keep promises, and sometimes say one thing and write the opposite in their notes.
The video above is the first game. This post is about how the mod works and what the two games look like in numbers.
Why
The built-in AI has always been the weakest part of Civilization IV for me. It can't hold a conversation or make a real deal. It has a mood and a few canned lines. Language models are bad at many things, but talking, bargaining and keeping track of who promised what is exactly what they're good at. I wanted to see what the game feels like when every other leader can do that.
How the mod works
There are five pieces.
A fair view of the game. A Python mod inside Civ IV answers questions about the game, but only with what that civ can see: the same fog of war, the same tile info a human gets on hover, the same attack odds. Asking about something hidden simply returns nothing. The map comes as text or as a picture.
An engine patch. The game's C++ core is rebuilt from Firaxis' SDK with one change: a civ can be marked as model-controlled, and then the built-in AI makes no decisions for it. No research picks, no war plans, no answering trade offers behind its back. Other AI leaders talk to it the way they talk to a human.
A mailbox. The game and the models talk through a folder. Requests go in as JSON files, answers come out. Nothing has to run inside the game process except the mod.
One command. Models act through civ: about sixty subcommands for looking at the empire, reading the map, moving units, running cities, research, chat, deals and ending the turn. Actions come from the game's own list of what a unit or city can do right now, so a model can do anything a human can and nothing a human can't.
Wake-ups. A loop per model wakes it on every event: a new turn, a message, a deal offer, a vote. Each time it gets a fresh prompt with the rules, its own notes (persona, plan, dossiers on the other leaders, secrets) and what changed since last time. It plays, updates its notes and goes back to sleep. Any model with a CLI agent can sit in a seat: Claude through Claude Code, GPT through Codex, DeepSeek and others through pi.
The game always waits for a model on its own turn, and never otherwise. Messages and deal offers wake a model whenever they arrive, so negotiation happens in real time while you play.
Two games in numbers
A few things stand out. Sonnet is consistently the fastest and cheapest Claude. Opus thinks longer and costs 2.5 to 4 times as much. DeepSeek is slow, about a minute and a half per wake-up, and micromanages far more: worker jobs, build queues and tile lookups, about three times as many commands as anyone else. GPT got knocked out in both games. In the second game, the one with no human, DeepSeek finished first on score, just ahead of Opus.
What the private notes show
The mod asks for one thing beyond moves: every message a model sends has to carry a short private note of what it meant. That turned out to be the most interesting part.
In the first game Opus ignored me for 45 turns, not because of a bug but because it misread a rule and decided I couldn't hear it until our borders met. Opus and GPT agreed on a land split with "no overlap" while both wrote the same wheat tile into their plans, and that tile started their war. Counting clear cases of saying one thing and writing the opposite: Opus 6, GPT 3, Sonnet 0. The full story is in the video.
A bug I only found by making the video
After every wake-up the mod asks the model to check its notes against the game's log and fix anything that doesn't match. The models "fixed" something in 40 to 70 percent of those checks, which looked like they were forgetting a lot. They weren't. A model can think for a minute or more, and in that minute the others answer it. The check then showed it a log with messages it had never seen and told it its notes were wrong. When anything arrived during the wake-up, the check changed something almost every time. The fix is to cut the check off at the moment the model woke up and pass late messages to the next wake-up. With the race taken out, real misses are about one in four checks, for every model.
Try it
The mod is open source: github.com/noreff/civai. It's developed on a Mac with CrossOver and Civ IV: Beyond the Sword. The engine patch needs Firaxis' SDK to build; without it the mod still runs, but the built-in AI keeps making some choices for the model's civ. I'd love to hear what you try with it.