Text generation now has a third option.
Beyond autoregressive and diffusion models.

@Stanford and @togethercompute proposed Next-Concept Modeling (NCM) that generates text through a hierarchy of compressed codes, building from a coarse representation to finer detail.

These codes are "concepts" in the name - learned representations, not explicit ideas or human-readable outlines.

▪️ NCM works in 2 parts:

1. It starts with a compressed representation.
A multiscale residual VQ-VAE (Vector Quantized Variational Autoencoder) learns to turn text into several levels of discrete codes and reconstruct text from them.
Level 1 is brief and coarse, and higher levels add what the reconstruction still lacks.

2. Then it adds finer detail.
A Transformer generates those codes from coarse to fine, one level at a time, predicting all codes in each level in parallel.

An important note: NCM adds detail at successive scales, doing less repeated work than diffusion models and skipping teacher–student distillation.

In tests, NCM showed:

- 10–31× faster generation than the strongest diffusion baselines, with comparable or better quality.
- Higher Sudoku accuracy than the tested autoregressive and diffusion, using fewer model evaluations.
~4–5× fewer training tokens than strong diffusion baselines.

That's for small models and specific benchmarks. Compares at large language-model scale are not established yet.

Text gen needs this breath of fresh air, and all new ideas are valuable. The NCM approach makes it cheaper, with fewer steps and less work are needed.