Unmasking Order in Diffusion Language Models

Pengcheng Huang · November 2026

A masked diffusion language model predicts tokens at several masked positions in one forward pass. Its sampler still has to decide which positions to reveal, how many to commit, and whether earlier decisions can be revised. Papers on unmasking order often change different parts of that decision. This guide separates them so readers can identify the closest prior work and trace each claim to its source.

Research question Representative papers What the method changes
Which position should be revealed first? LLaDA, UNCODE, BayesER Start from confidence-based remasking, reweight confidence using positional and token-informativeness priors, or estimate uncertainty from a posterior over model parameters.
How many positions can be committed together? Fast-dLLM, EB-Sampler Accept positions above a confidence threshold, or adapt how many positions to reveal under an entropy budget. Fast-dLLM also studies approximate KV caching.
How should the sampler account for the remaining masked intervals? Balance-Aware Pivot Decoding After confidence-based prefilling, favor central pivots to split masked intervals into shorter intervals that can be processed in parallel.
Can extra inference compute help select a better trajectory? LookUM Propose alternative unmasking paths and select among them using uncertainty over the resulting trajectories.
Can a separate policy learn which positions to reveal or revisit? Where-to-Unmask, TIPS Train a planner from ground-truth-guided rankings, or train a small controller that can both reveal and remask tokens while keeping the underlying model frozen.

Which tokens does confidence-first decoding reveal early?

Confidence is an estimate of how likely the current token prediction is at a position. It can favor an easy prediction even when revealing it contributes little to the rest of the sequence. UNCODE reports two recurring patterns under the uncertainty-based samplers it studies: a rigid tendency to reveal boundary positions early, and early preference for high-frequency, low-information tokens. Its sampler combines a positional prior and a corpus-based informativeness prior with the model’s confidence. In the reported confidence-based decoding trajectories, tokens such as spaces, punctuation, newlines, and the article “the” accounted for about 40% of the selections, compared with about 20% in the Qwen-2.5-7B-Instruct autoregressive baseline. On GSM8K, accuracy rose from 59.1% to 63.2% as the experiment increased the probability of suppressing these selections. Figure 3 shows the token groups and accuracy comparison. Section 4.2 describes how to change that ranking without training a planner. Paper sections · ACL BibTeX

BayesER uses a different notion of calibration. It averages predictions from an approximate parameter posterior and ranks positions by predictive entropy. It is relevant when discussing whether the model’s uncertainty estimates track error. The two approaches therefore motivate different comparisons: selecting informative positions versus improving the reliability of uncertainty estimates.

Version note. The first two arXiv versions of UNCODE used the name PC-Sampler. The ACL 2026 version is titled Empirical Analysis of Decoding Biases in Masked Diffusion Models. I am the first author of that paper.

LogicDiff separately studies when high-entropy logical connectives are revealed on reasoning tasks. Its intervention prioritizes those tokens. For an analysis of boundary positions and high-frequency tokens across uncertainty-based samplers, see UNCODE §3. The two analyses use different token groupings.

Position order and parallel commitment

Ranking positions and choosing a commit count can be varied separately. Fast-dLLM uses a confidence threshold for parallel commitment, while EB-Sampler adjusts the count with an approximate entropy bound. Balance-Aware Pivot Decoding explicitly considers how the positions selected now determine the remaining masked intervals. Its depth analysis concerns the balanced splitting rule. End-to-end speed also depends on the model, threshold, and implementation used in the experiment.

Local token dependence is a related question. CoDiLA adds a small autoregressive model to account for local dependencies within blocks. This changes how simultaneously proposed tokens are made coherent. A scheduler that changes only the order of positions addresses a different part of the sampler.

Search and learned policies

The methods differ in where their extra work happens. LookUM uses additional forward passes at inference to evaluate candidate unmasking paths. Where-to-Unmask uses oracle scores during training to supervise a planner and then ranks positions with that planner during inference. TIPS learns a visibility controller that can reveal or remask tokens. These distinctions matter when a manuscript says a method is training-free, keeps the backbone frozen, or allows previously revealed positions to change.

Citing and comparing results

  • For the rationale for a position policy, cite the work that identifies the relevant behavior. For example, use UNCODE §3 for observed decoding bias and Where-to-Unmask §3 for evidence from controlled oracle orderings.
  • For an inference comparison, match the model checkpoint, prompt, output budget, number of function evaluations, and what counts as a commitment or remask. If reporting latency, also state the cache configuration, hardware, and batch size. A throughput result and an accuracy result at matched function evaluations answer different questions. The IJCAI survey on efficient dLLM inference discusses practical latency comparisons in more detail.

Scope. This is a focused reading guide to position selection and commitment at inference in masked text diffusion. It draws on the papers linked above and the references of the IJCAI survey and UNCODE, checked November 19, 2026. It includes selected preprints and provides no common performance ranking.