Apple · Apple Machine Learning
Recently, several works have demonstrated the possibility of generating discrete data continuously by a simple flow matching
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
★ Tier-1 Source
They have further shown the feasibility of accelerated sampling via Categorical Flow Maps (CFMs), resulting in competitive sample quality in the few-step regime.
Key facts
- In this article, they train a 1.7B-parameter base flow model on 2.1T tokens and self-distill it into a CFM that generates diverse, high-quality text in as few as 4 inference steps while maintaining
- Authors Oscar Davis†**, Anastasiia Filippova, Victor Turrisi, Amitis Shidani, Pierre Ablin, Marco Cuturi, Louis Béthune — They have further shown the feasibility of accelerated sampling via Categorical Flow Maps (CFMs), resulting in competitive sample quality in the few-step regime
- Continuous diffusion and flow matching models could represent a powerful alternative to autoregressive approaches for language modelling (LM), as they unlock a host of advantages currently reserved
Summary
Authors Oscar Davis†**, Anastasiia Filippova, Victor Turrisi, Amitis Shidani, Pierre Ablin, Marco Cuturi, Louis Béthune. Continuous diffusion and flow matching models could represent a powerful alternative to autoregressive approaches for language modelling (LM), as they unlock a host of advantages currently reserved for continuous modalities, including accelerated sampling and tilting. In this article, they train a 1.7B-parameter base flow model on 2.1T tokens and self-distill it into a CFM that generates diverse, high-quality text in as few as 4 inference steps while maintaining near-data-level token entropy.