Scaling Categorical Flow Maps

Researchers trained a 1.7 billion parameter categorical flow map (CFM) model on 2.1 trillion tokens, achieving high-quality text generation in as few as 4 steps with near-data-level entropy. They also introduced a likelihood bound for CFMs, enabling scoring on standard language modeling benchmarks with competitive results, and discussed challenges in scaling these models.