FS-DFM (Few-Step Discrete Flow-Matching) is a few-step discrete flow matching language model jointly released by Apple and Ohio State University. 8-step sampling can be comparable to the 1024-step baseline quality, achieving up to 128 times acceleration.

FS-DFM: Few-step discrete flow matching text generation model

Core parameters and statistics

Project	Details
Product name	FS-DFM (Few-Step Discrete Flow-Matching)
Product Type	Open Source Research Model / Diffusion Language Model
Delivery Form	PyTorch Open Source Code
Supported languages	English
Development Organization	Apple ML Research / Ohio State University
Paper included	ICLR 2026
Open Source License	Customize Apple Open Source License
Model parameters	1.3B (pre-trained weights provided)

The above information is based on arXiv:2509.20624 and GitHub repository.

FS-DFM (Few-Step Discrete Flow-Matching) is a discrete flow matching language model jointly proposed by Apple and Ohio State University. The core breakthrough is that: It only takes 8 steps of sampling to reach the 1024-step perplexity level of the traditional discrete diffusion model, achieving a generation acceleration of up to 128 times. This achievement breaks the inherent bottleneck of "exchanging iteration steps for quality" in the diffusion language model, and compresses long text generation from minutes to seconds.

A brief comment in one sentence: This is not a chat product for end users, but an important research breakthrough for Apple to push the discrete diffusion model into practical use - it proves that "fewer-step sampling + flow matching" can compete with autoregressive models for throughput on language modeling tasks.

User and market recognition

• Academic Recognition: The paper has been included in the top international conference ICLR 2026, and the reviewers evaluated its substantial advancement in the direction of few-step sampling of discrete diffusion language models.
• Open Source Ecosystem: GitHub repository apple/ml-fs-dfm provides complete PyTorch implementation, pre-training weights (1.3B), training and evaluation scripts, and the community can directly reproduce and develop secondary.
• Media Attention: Multiple AI media (such as related technology interpretation videos have been viewed in the tens of thousands).
• Technical influence: The combination of "sampling steps as explicit parameter training + teacher distillation + custom discrete solver" proposed by FS-DFM provides a new baseline for subsequent discrete diffusion model research.

Cost advantage

Cost Dimension	Description
Model weights	Free and open source, providing 1.3B pre-training checkpoint download
Code	Completely open source, MIT-like license
Training cost	GPU resources required (CUDA 11.0+), training cost depends on hardware configuration
Inference cost	8 steps of sampling can achieve high quality, reducing inference computing power consumption by 128 times compared to the thousand-step baseline

The core cost value of FS-DFM is to reduce the inference overhead from O(thousand steps) to O(8 steps), which has direct significance for latency and throughput optimization in deployment scenarios. For researchers, the completely open source code and weights eliminate the cost of repeated implementation; for engineering teams, the characteristics of few-step sampling make it easier to integrate into real-time text generation pipelines.

It should be noted that training a set of FS-DFM models still requires the support of a large-scale GPU cluster. The 1.3B weight provided by the open source can meet most experimental and fine-tuning needs.

Main functions

• Few-step discrete flow matching: Use the number of sampling steps as an explicit parameter to train the model, so that it can be generated stably under different step budgets. Eight sampling steps can reach the perplexity level of traditional 1024-step discrete flow matching.
• Custom discrete solver: Provides mixture_euler, mixture_euler_with_cumulative_scalar and other discrete solvers, supporting two source distributions of uniform and mask.
• Teacher Distillation Framework: By distilling knowledge from a long-trajectory teacher model to a few-step student model, it improves the quality of few-step sampling and avoids probability overshoot or gradient instability.
• Full-process open source tool chain: including three major modules: pre-training, training, and evaluation. Provides 1.3B parameter pre-training checkpoint and supports perplexity and ELBO evaluation.
• Transformer architecture support: Based on the standard Transformer architecture, it supports configurable vocabulary size, Dropout regularization and distributed training.

[Expert View]: The real highlight of FS-DFM is not the simple speed improvement, but its "step generalization" design - after the model is trained with step consistency, the same weight can be flexibly switched under different delay/quality requirements. This means that engineering teams can dynamically adjust the number of sampling steps at deployment time based on hardware capabilities and real-time load, rather than maintaining independent models for each scenario. This elasticity is extremely valuable in LLM servitization deployments.

Model and version evolution

Version	Date	Event
arXiv first version	2025-09-24	The paper was first published on arXiv:2509.20624
GitHub Open Source	2025-09-24	apple/ml-fs-dfm warehouse is made public simultaneously, including complete code and weight
Accepted by ICLR 2026	2026-01	The paper was accepted by ICLR 2026 (the specific date is subject to the conference schedule)
Weight release	2026-02	Provide 1.3B FS-DFM and DFM pre-training checkpoint download

The project has not undergone commercial version iterations, and all releases are research-oriented open source versions.

Technical advantages

• Step-Consistent Training: Using the number of sampling steps as an explicit input parameter, the model shares the same set of weights under different step budgets, eliminating the need to configure separate training for each step. In engineering terms, this means that one training session can cover the full spectrum of inference scenarios from extremely fast (2-4 steps) to high quality (64-128 steps).

• Reliable Update Rule: The discrete solver of FS-DFM controls the direction of probability transfer through the energy barrier (Energy Barrier), and cooperates with DT piecewise normalization to ensure that each step moves the probability mass in the "correct direction" without overshooting. Compared with the common probability drift problem in traditional discrete diffusion, this mechanism significantly improves the generation stability in few-step scenarios.

• Teacher-Student Distillation: Distill knowledge from the RK4 teacher trajectory to the student model, so that the student model can still maintain high-quality output in a very small number of steps. The distilled 8-step model is on par with the 1024-step teacher model in terms of perplexity, reducing inference latency by two orders of magnitude.

• Flexible solver selection: Supports two source distribution strategies, uniform and mask, and multiple ODE solvers such as mixture_euler and mixture_euler_with_cumulative_scalar. Researchers can freely combine them according to different data characteristics.

How to use

FS-DFM is delivered as an open source PyTorch project and is not a SaaS service. Before use, you need to prepare Python 3.8+ and CUDA 11.0+ environment.

Installation

# Clone repository
git clone https://github.com/apple/ml-fs-dfm.git
cd ml-fs-dfm

#Create conda context
conda env create -f fsdfm_environment.yml
conda activate FSDFM

# Install development mode
pip install -e .

Model evaluation

# Download pre-trained weights and run evaluation
python fs_dfm/run_eval.py \
    --work_dir "/path/to/output" \
    --ngpus 1 \
    --perplexity_n_samples 320 \
    --eval_elbo \
    --eval_perplexity \
    --pre_trained_model_path "/path/to/checkpoint.pth"

Model training

python fs_dfm/run_train.py \
    data.cache_dir=${CACHE_DIR:-./cache_dir}

Key configuration parameters are located in fs_dfm/configs/config.yaml, including sampling_steps (number of sampling steps), source_distribution (source distribution type), temperature (sampling temperature), etc.

Typical usage process

Clone the repository → Configure the environment → Download the pre-trained weights → Run the evaluation/training script → Analyze the output results. For complete instructions, please refer to the GitHub repository README.

Product Pricing

FS-DFM is a purely open source research project and does not involve commercial pricing:

Project	Cost
Source code	Free (GitHub public repository)
Pre-trained weights (1.3B)	Free download
Paper Access	Free (arXiv Open Access)
Commercial License	Subject to the terms of the warehouse LICENSE file
Cloud reasoning	No official hosting service, need to deploy by yourself

Academic and industrial researchers can obtain all codes and weights at zero cost, but the GPU hardware resources required for training or inference are borne by the user.

Application scenarios

• Diffusion Language Model Research: As a few-step sampling baseline for discrete flow matching, used for comparative experiments, ablation studies and new solver development.
• Efficient generation of long text: Suitable for scenarios that require parallel generation of long sequences (such as 1024 tokens) but are sensitive to latency, such as summary generation, structured output, and code completion.
• Low-latency text inference service: 8-step sampling compresses the inference delay from tens of seconds at the thousand-step level to the second level, which is suitable for online service backends with high real-time requirements.
• Small sample fine-tuning and domain adaptation: Based on 1.3B pre-training weights, it can quickly adapt to specific fields (such as legal documents, medical texts) through fine-tuning on downstream tasks.

Applicable people

• AI researcher (NLP/generative model direction): FS-DFM provides a complete baseline for discrete flow matching few-step sampling and is suitable as a reference implementation and comparison method for diffusion language model research.
• Deep Learning Engineer: Interested in accelerated text generation with few-step sampling, and needs a reproducible open source solution for evaluation and integration.
• Large Model Inference Optimization Team: An engineering team that faces the throughput bottleneck of autoregressive model inference and is exploring non-autoregressive alternatives.
• College Students and Academic Institutions: Course projects, graduation projects or academic exploration can be carried out based on open source code and pre-trained weights.

[Not suitable for boundaries]:

• Not for end consumers (no Web UI, no App, no SaaS service).
• Not suitable for scenarios that require ChatGPT-like interactive dialogue (FS-DFM is a generative model rather than a dialogue model).
• Not suitable for zero-code/low-code users, requires PyTorch and command line capabilities.
• Direct generation of Chinese is not supported (the pre-training data is mainly in English, and Chinese generation requires additional fine-tuning).

Summary and Outlook

FS-DFM is a substantial advancement by Apple in the field of discrete diffusion language models. Through the triple combination of step consistency training + reliable update rules + teacher distillation, the sampling steps of the diffusion language model are compressed from thousands of steps to single digits, while maintaining quality without degradation. Generation of 1024 tokens in 8 steps, and 128 times acceleration in perplexity that is the same as the thousand-step baseline - these numbers make non-autoregressive text generation possible for the first time in the efficiency dimension to compete with autoregressive models.

From an industry perspective, the significance of FS-DFM is more than just a top conference paper: it shows that the technical route of "discrete flow matching" far exceeds traditional discrete diffusion in terms of short-step sampling potential, providing a new paradigm direction for efficient text generation in the future. If this route can be continuously verified on larger model scales (>10B) and more complex tasks, it is expected to change the architecture selection pattern in the field of long text generation.

[Procurement/Adoption Risk Assessment]:

• FS-DFM is a pure research project with no commercial SLA or technical support. Production deployment requires the team to bear the engineering costs.
• The scale of the 1.3B model is relatively small in the current large model ecosystem, and it cannot compete with commercial models such as GPT and Claude in terms of complex reasoning and instruction following capabilities.
• Open source license terms need to be carefully reviewed to confirm compliance in commercial scenarios.
• The project maintenance rhythm is subject to the research team, and long-term updates or problem responses are not guaranteed.
• It is recommended to position it as an "efficiency-enhancing component for specific scenarios" rather than a general text generation base, and prioritize evaluation in tasks that are delay-sensitive, have long sequences, and have controllable quality requirements.

Related tools: hugging-face, replicate

Version Info

• ICLR 2026 version ：The release version of the paper arXiv:2509.20624, which contains the complete model architecture and training code of FS-DFM, has been included in ICLR 2026. （2025-09-24）
• initial commit ：The paper was first published on arXiv, and the open source code was made public simultaneously. （2025-09-24）