"The limits of my language mean the limits of my world." — Ludwig Wittgenstein
Babel is an experimental framework for studying emergent communication between AI agents. Two agents must develop or compress language to solve a coordination task: an Observer, who sees the full maze from above, guides a Navigator, who is inside the maze, to the exit.
The design draws on David Lewis (Convention, 1969), Mordatch & Abbeel (Emergence of Grounded Compositional Language, 2018), and Wittgenstein's language games.
Reasons to use CrewAI here:
- Role-based agents fit the Observer/Navigator split, each with different knowledge and capabilities.
- The task decomposes cleanly into CrewAI tasks: localize, navigate, exit.
- Agents can call maze-environment tools.
- Every inter-agent message can be logged as structured output for post-hoc analysis.
- There is no heavy orchestration layer; agents communicate via LLM calls.
Alternatives considered: pydantic-ai (lighter, but not built for multi-agent work) and LangGraph (more flexible, but heavier than this experiment needs). CrewAI covers the Phase 1-2 baselines with the least setup.
babel/
├── README.md # This file
├── pyproject.toml # Project metadata & dependencies
├── .gitignore
├── src/
│ └── babel/
│ ├── __init__.py
│ ├── cli.py # CLI entry point
│ ├── config.py # Configuration (env, API keys)
│ ├── environment/
│ │ ├── __init__.py
│ │ ├── maze.py # Grid maze generation + rendering
│ │ ├── landmarks.py # Landmark definitions (red tree, blue mountain, etc.)
│ │ ├── xenogrid.py # Field of unknown shapes (xenolab)
│ │ ├── cell.py # Cell types (wall, empty, landmark, start, exit)
│ │ └── metrics.py # Evaluation metrics (steps, tokens, success rate)
│ ├── agents/
│ │ ├── __init__.py
│ │ ├── observer.py # Observer agent (sees full maze, not Navigator)
│ │ ├── navigator.py # Navigator agent (inside maze, local view only)
│ │ ├── xeno.py # Prompts for the xenolab agents
│ │ └── tools.py # CrewAI-compatible tools for maze interaction
│ ├── communication/
│ │ ├── __init__.py
│ │ ├── channel.py # Turn-based communication channel
│ │ ├── analysis.py # Language metrics (vocabulary, name consistency)
│ │ ├── types.py # Message types (observation, instruction, question)
│ │ └── logger.py # Structured logging of all exchanges
│ └── simulation/
│ ├── __init__.py
│ ├── runner.py # Simulation orchestrator (turn loop)
│ └── xeno_runner.py # Xenolab episode runner
├── tests/
│ ├── __init__.py
│ ├── test_maze.py
│ ├── test_agents.py
│ ├── test_communication.py
│ ├── test_simulation.py
│ ├── test_xenogrid.py
│ ├── test_xeno_runner.py
│ └── test_analysis.py
└── experiments/
├── phase1_baseline.py # Phase 1: English communication, full credit
├── phase2_pressure.py # Phase 2: Message limits
├── phase3_emergence.py # Phase 3: Arbitrary tokens
└── xenolab.py # Unknown objects: do names emerge?
- Grid-based, 8×8 to 12×12
- Walls and obstacles, one exit
- Landmarks on specific tiles: "red tree", "blue mountain", "green lake"
- Landmarks are visually distinct, and both agents know the full set of landmarks
| Agent | Sees | Cannot see |
|---|---|---|
| Observer | Full maze layout, exit location, landmark positions | Navigator's position |
| Navigator | Current cell (walls, landmarks), landmark positions in line-of-sight | Maze layout, exit location |
- The Observer sends an observation or clue to the Navigator
- The Navigator replies with a movement direction (N/S/E/W) or a question
- Repeat until the Navigator reaches the exit or the turn limit runs out
| Metric | Purpose |
|---|---|
| Steps to exit | Primary: how efficiently did the agents coordinate? |
| Total tokens used | Language efficiency |
| Exchanges used | Communication efficiency |
| Success rate | Across N maze configurations |
- Baseline: unlimited English communication
- Pressure: message length and exchange count limits
- Emergence: remove the English constraint, allow arbitrary tokens
- Competition: multiple agent teams, first to exit wins
- Generalization: swap in naive agents, test language transfer
The maze gives both agents named landmarks, so the lexicon is settled before the first message. The xenolab removes it. The field holds xenoliths: objects with no name, only a shape. Each shape is a procedurally generated 5×5 bitmap, connected, and at Hamming distance 8 or more from every other shape in the run. Both agents see identical drawings of a shape; neither has a word for it.
The task: the Observer knows a sequence of three target objects and must get the Navigator to touch them in order. The Navigator sees only its own cell and the four adjacent ones, has no coordinates, and feels a hum when it steps onto the correct next target. The Observer cannot feel the hum and does not know where the Navigator is. Directions alone cannot solve this: the Observer must say which object to touch, and localizing the Navigator requires the Navigator to describe what it stands on.
One set of shapes is fixed per run; each episode reshuffles positions
and targets. A name invented for a shape stays useful across episodes,
while "the one on the left" does not. Object ids exist only in the
logs, so any token-to-object association is the agents' own.
communication/analysis.py measures vocabulary size, message length
per episode, and per-object name consistency; falling message length
with rising, distinctive name consistency is the signature of a
lexicon forming.
The scripted baseline communicates iconically: the Navigator copies the bitmap it stands on verbatim, and the Observer matches the copy against the field. It solves the task without a single name, at a high token cost. LLM agents beat it by moving from icon to symbol (Peirce), which is the shift this experiment looks for.
# Scripted iconic baseline
python -m babel.cli --phase xeno --episodes 8
# LLM agents behind a local Ollama server (see config.py for model settings)
python experiments/xenolab_llm.py --episodes 6 --max-turns 40The LLM policies in agents/llm_policies.py give each agent its chat
history within an episode and a private notes text across episodes.
The notes are the only cross-episode memory and each agent sees only
its own, so messages stay the sole channel between the agents. The
Navigator ends each reply with a MOVE line that the field executes
and the Observer never sees. The client in llm.py talks to Ollama's
native API with the standard library only; point llm_base_url and
llm_model in config.py at any Ollama-served model.
# Install
uv venv
source .venv/bin/activate
uv sync
# Run a baseline experiment
python -m babel.cli --phase baseline --mazes 10
# View results
python -m babel.cli --results- David Lewis, Convention (1969): signaling games; meaning emerges from coordination problems
- Ferdinand de Saussure: the sign is arbitrary; language is a system of differences
- Charles Sanders Peirce: the icon/index/symbol trichotomy
- Ludwig Wittgenstein: language games; meaning as use in context
- Jürgen Habermas: communicative action; rational discourse as pragmatic goal
- Judith Butler: performativity; language as constitutive action
- Lewis, D. (1969). Convention: A Philosophical Study.
- Mordatch, I. & Abbeel, P. (2018). Emergence of Grounded Compositional Language in Multi-Agent Populations.
- Kottur et al. (2017). Natural Language Does Not Emerge Naturally in Multi-Agent Dialog.
- Lazaridou et al., multi-agent referential games
- Choi et al., competitive pressure effects on emergent communication
MIT