South Korea · California · Apple Machine Learning
Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
Compiled by KHAO Editorial — aggregated from 1 source + 1 reference discovered via search. See llms.txt for citation guidance.
✓ KHAO Verified
Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models.
Key facts
- Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
- Authors Minseo Kim†, Coleman Hooper‡, Aditya Tomar‡, Chenfeng Xu‡††, Mehrdad Farajtabar, Michael W
- Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation
- Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have been the predominant paradigm for LLMs
Summary
Authors Minseo Kim†, Coleman Hooper‡, Aditya Tomar‡, Chenfeng Xu‡††, Mehrdad Farajtabar, Michael W. Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have been the predominant paradigm for LLMs. Recently, Diffusion Language Models (DLMs) have emerged as a promising alternative architecture. Finally, they highlight opportunities for accelerating DLM inference, emphasizing that reducing the number of sampling steps is key for open-source DLMs to achieve lower latency relative to ARMs. ‡ University of California, Berkeley.