Neural Computers (The Data Pipeline)

📘 Blog   ·   🎬 Demo Gallery   ·   📄 Paper

News

• 2026 April: First open-source release of Neural Computers, starting with the data pipeline for CLI and GUI trajectory generation.

Abstract

Neural Computers (NCs) are neural networks that unify computation, memory, and I/O in a single latent runtime state. The long-term goal is the Completely Neural Computer (CNC): a general-purpose neural computer that challenges the layered architecture of conventional computers. As an initial step, this project studies whether executable dynamics can be learned solely from collected I/O traces, without access to instrumented program state, using CLI and GUI trajectories as training and evaluation data.

Quick Start

Environment Python 3.9+ · Docker · asciinema · agg · ffmpeg

1. CLI / Asciinema

python3 main.py cligen asciinema record --command "python3 --version"
python3 main.py cligen asciinema cast-to-mp4 workspace/cligen_general/casts

2. CLI / VHS

python3 main.py cligen vhs build-image
python3 main.py cligen vhs generate-basic --count 10 --prefix demo
python3 main.py cligen vhs make-manifest
python3 main.py cligen vhs run-manifest

3. GUI / Synthetic

pip install -r engine/gui/synthetic_data_collection/requirements.txt
python3 main.py guiworld build-image
python3 main.py guiworld synthetic --count 1 --max-workers 1

Runtime Generations from Neural Computers

_{Using training data generated by the code released in this repository.}

CLIGen (general / asciinema)

CLIGen (clean / vhs)

GUIWorld (GUI)

_{Alternating: Conventional Computer / Neural Computer}

Acknowledge

Our models (not released here) are built from Wan2.1 and Matrix-Game-2. The data engine for CLIGen (General) is built from Asciinema, the data engine for CLIGen (Clean) is from VHS, the data engine for GUIWorld (Random) is modified directly from Neural-OS, and the data engine for GUIWorld (CUA) is built according to Claude CUA.

Data Engine	Source
CLIGen (General)	Asciinema
CLIGen (Clean)	VHS
GUIWorld (Random)	Neural-OS
GUIWorld (CUA)	Claude CUA

Reference

@article{zhuge2026neural,
  title={Neural computers},
  author={Zhuge, Mingchen and Zhao, Changsheng and Liu, Haozhe and Zhou, Zijian and Liu, Shuming and Wang, Wenyi and Chang, Ernie and Lan, Gael Le and Fei, Junjie and Zhang, Wenxuan and others},
  journal={arXiv preprint arXiv:2604.06425},
  year={2026}
}