Apple study probes confidence limits of discrete diffusion sampling

Apple researchers examine confidence in discrete diffusion sampling. Samplers such as remasking and uniform-state write several token positions per step, drawing values from marginal distributions. The work proves such a step matches the training distribution only when the written positions are conditionally independent given existing tokens, yet tokens in pixels, phonemes and words are inherently dependent.

Latest turnApple researchers show that a discrete diffusion step matches the training distribution only when the positions it writes are conditionally independent given the tokens already generated. That assumption breaks down in domains such as pixels, phonemes and words, where tokens are inherently dependent, limiting what samplers like remasking and uniform-state can claim.