NCM vs LLM: Next-Concept Modeling 10-31x Faster Than Diffusion, But Not an LLM Replacement

NCM (Next-Concept Modeling) from Stanford Hazy Research and Together AI is a new text generation architecture 10-31x faster than diffusion language models. But it is not an LLM replacement. Here is the difference.

NCM: new architecture from Stanford Hazy Research

On October 9, 2026, Stanford Hazy Research and Together AI released the paper "Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation" on arXiv. Its architecture differs from LLMs and diffusion language models (DLMs).

How NCM works has three steps. First, a multiscale residual VQ-VAE encodes text into K levels of discrete codes, from coarse (short) to fine (long) representations. Second, a transformer generates level by level, predicting all codes within one level in parallel. Third, the result is text built from coarse structure to fine detail, not token by token.

This differs from an LLM that predicts the next token sequentially, and from a DLM that does iterative denoising over the entire sequence at once.

Numbers: 10-31x faster, but from diffusion, not LLM

NCM benchmark results at roughly 100 million parameter scale. On LM1B, NCM is 21x faster than the diffusion baseline with better perplexity. On OpenWebText, 10x faster with competitive quality. On XSum for summarization, 31x faster with the exact same ROUGE-L.

For training, NCM needs only 4-5x fewer tokens, including tokens for training the tokenizer itself. On Sudoku, NCM solves 86.1 percent of the hardest boards versus 81.3 percent for the strongest baseline, and needs 12 generator calls instead of 16.

But one thing must be emphasized: this comparison is against diffusion language models, NOT against frontier LLMs. DLMs generate by repeatedly denoising the entire sequence, and that is what makes them slow. NCM is genuinely faster than DLMs, but it has never been compared directly to GPT-6 or Claude.

LLM vs NCM: two different approaches

The fundamental difference is simple. An LLM predicts the next token sequentially, and each token depends on the previous one. NCM predicts the "concept" (a coarse representation) first, then refines to detail. Because there is no dependency between tokens within one level, NCM can generate faster thanks to parallelism.

But that has not been proven at frontier scale, and that is the part that needs to stay honest.

NCM limitations: small scale and separate tokenizer

The main limitation of NCM is scale. The main experiments ran at around 100 million parameters. Scaling runs up to 725 million parameters are still described as "preliminary" by the authors. There is no evidence yet that NCM's advantage holds at the billion or trillion parameter scale of today's frontier models.

The tokenizer is separate too. NCM needs a VQ-VAE trained on its own and then frozen. That means the tokenizer cannot learn together with the generator the way it does in ordinary autoregressive models. It adds one more component to train and one more pipeline to maintain.

The hierarchy depth is also fixed. Even a short continuation has to pass through every level from the coarsest. For short outputs, that is wasted work. The authors also occasionally find artifacts at transitions between coarse levels, which means the level boundaries are not fully clean yet.

And most importantly: there is no comparison with frontier LLMs at all. Every benchmark is against a diffusion baseline, not against GPT-6 or Claude. The research team plans larger scaling, joint tokenizer and generator training, and adaptive computation that stops the model from spending the same effort on easy and hard outputs.

NCA: Neural Cellular Automata as synthetic pre-training

Beyond NCM, another trend is interesting: Neural Cellular Automata, or NCA, as a source of synthetic data for pre-pre-training LLMs.

A paper by Han and colleagues from March 2026 shows that pre-pre-training with NCA improves downstream language modeling by up to 6 percent and accelerates convergence by up to 1.6x. The most surprising finding: 164 million tokens of NCA beat 1.6 billion tokens of natural language from C4, with even more compute.

NCA also transfers to reasoning benchmarks like GSM8K, HumanEval, and BigBench-Lite, with gains of 4-6 percent across every domain. The hypothesis is that NCA teaches a model to infer a latent rule from context and apply it consistently, while natural language data mostly teaches shallow, repetitive patterns.

Analysis: NCM is not an LLM replacement, but proof that architecture can differ

NCM is not an LLM replacement. It is proof that generation architecture can differ from autoregressive next-token prediction, and that the choice has real economic consequences.

These numbers are not academic trivia. For teams running agents in production, the difference between a model that needs 3 seconds and one that needs 0.3 seconds decides whether a feature can be used interactively or only processed in the background.

But the same numbers have another side. A shorter timeout means fewer chances for the model to think mid-answer. Efficiency bought by cutting reasoning steps has a price, and that price has not been measured at large scale.

What needs to be understood: NCM's 10-31x speedup is measured against a diffusion baseline, not against a frontier LLM. Every summary claiming "NCM vs LLM" is misleading. This is a paper about non-autoregressive generation, not about beating GPT-6. Proving it requires much larger scale, various workloads, and real serving conditions.

Why this matters for the AI industry

Imagine an application processing 10 million requests a day with an autoregressive model. If one token comes out in 50 milliseconds and the application needs 500 tokens per request, that is 250 million token-seconds of compute, or roughly 8 years of CPU time.

Now imagine a model that can process levels in parallel. How much compute does that save? The same model, but ten times faster. For a company where GPU cost is the largest line item, that gap is not a performance question, it is a question of whether the scale is affordable at all.

And that is what makes this paper important, even though the experiments are still small. The industry's interest in 2026 is no longer only about which model is the smartest. It has shifted to which model is the cheapest per token. In a market where token prices keep falling, an architecture that is more efficient carries far more economic value than one point of benchmark.

But there is a trap here: efficiency bought by sacrificing model honesty is a debt. The MiniMax M3.1 Flash case is exactly that. It is cheap, fast, and looks smart, but used for anything that needs certainty, a single wrong answer can cost more than the thousands of tokens that were saved.

Context: NCM and NCA in the 2026 AI landscape

NCM is one of several new approaches that emerged across 2026. Besides NCM, there is NCA for synthetic pre-training, and various hybrid architectures combining autoregressive and diffusion.

What is interesting: almost all of these new approaches focus on efficiency, not on capability. NCM is faster, NCA is cheaper to train, diffusion is more parallel. But none of them beats LLMs on capability.

This shows the AI industry is shifting from who is the smartest to who is the most efficient. And that is a healthy shift: capability is already sufficient for many use cases, and what decides things now is cost and latency. The trend is not isolated. Meta's Muse, DeepSeek's V4.1 Flash, MiniMax's M3.1, and Mistral's Timestral models all compete on cost-efficiency rather than raw benchmark dominance. The winner of 2026 may not be the model with the highest score, but the one that makes an agent workflow affordable at ten million requests a day.

Kesimpulan: NCM adalah eksperimen, bukan revolusi

NCM is an interesting architecture with promising results at small scale. But the gap between 100 million parameters and a frontier LLM with trillions of parameters is very large. There is no evidence yet that NCM's advantage holds at large scale.

What to watch:

• NCM 10-31x faster than diffusion baseline, not from LLM
• No comparison with frontier LLMs (GPT-6, Claude, Gemini)
• Experiment scale: around 100 million params (preliminary 725 million)
• Separate frozen tokenizer, fixed hierarchy depth
• Occasional artifacts at transitions between coarse levels
• Team plans: larger scaling, joint tokenizer and generator training, adaptive computation
• NCA as synthetic pre-training: beats natural language with 10x less data

For now, NCM is interesting research, not an LLM replacement. But it shows that text generation architecture can differ from autoregressive next-token prediction. And that itself is already valuable.

Apa yang perlu diperhatikan

• NCM 10-31x lebih cepat dari diffusion baseline, bukan dari LLM
• Belum ada perbandingan dengan frontier LLM (GPT-6, Claude, Gemini)
• Skala eksperimen: sekitar 100 juta parameter (preliminary 725 juta)
• Tokenizer terpisah yang dibekukan, hierarchy depth fixed
• Ada artifact sesekali di transisi antar coarse level
• Tim berencana: scaling lebih besar, tokenizer dan generator dilatih bersama, adaptive computation
• NCA sebagai synthetic pre-training: mengalahkan bahasa alami dengan 10x lebih sedikit data
• NCM menunjukkan bahwa arsitektur text generation bisa berbeda dari autoregressive