Supermix Expanse

A 132.6M-parameter language model, built and trained on a laptop CPU, carrying the whole male-cns v1.0 type graph — 11,751 cell types and 3,830,931 type-to-type edges — as a per-token recurrent core grafted into its residual stream beside branches distilled from other models. Held-out loss on connectome facts drops from 3.522 to 0.571 nats per token. The results do not oversell: only 1 of 32 grafted expert groups reproduces its teacher on held-out tokens, most of what transferred came from distillation, and writing new code or defining biomedical terms is called unusable after one CPU run.

Official site ↗

#language-model#graft#distillation#cpu-training#honest-results#malecns