GPDi Free

GPDiT (Generative Pre-trained Autoregressive Diffusion Transformer) is an autoregressive diffusion video generation model jointly proposed by the University of Science and Technology of China and Step Star. It performs frame-level autoregressive prediction in a continuous latent space, and integrates the high-quality generation capabilities of the diffusion model with the timing modeling advantages of the autoregressive model. It has reached the optimal level at the time on the MSRVTT and UCF-101 benchmarks.

Full review of #GPDiT

Core parameters and statistics

GPDiT (Generative Pre-trained Autoregressive Diffusion Transformer) was jointly proposed by the University of Science and Technology of China and Step Star. It aims to unify the high-quality generation capabilities of diffusion models and the temporal dependence modeling capabilities of autoregressive models within the same framework. Unlike discrete token prediction routes, GPDiT performs autoregressive predictions frame by frame in a continuous latent space, predicting one frame at a time and optimizing through diffusion loss, allowing the model to naturally capture inter-frame motion dynamics and semantic consistency.

Project	Specifications
Full name	Generative Pre-trained Autoregressive Diffusion Transformer
Publishing Agency	University of Science and Technology of China (USTC), StepFun
arXiv ID	2505.07344 (v5, 2025-10-08)
Subject Area	cs.CV (Computer Vision and Pattern Recognition), cs.AI (Artificial Intelligence)
Model size	GPDiT-B (85M parameters), GPDiT-H (2B parameters)
Technical Roadmap	Frame-wise Autoregressive Diffusion Transformer (Frame-wise Autoregressive Diffusion)
Core Innovation	Continuous latent space frame prediction + lightweight causal attention + parameter-free rotational temporal encoding
Training data	UCF-101, MSR-VTT, LAION-Aesthetic, internal video data set
Public form	arXiv paper (code and model are not open source)
Open Source License	arXiv Non-Exclusive Distribution License

Comparison of model specifications: GPDiT-B follows the DiT-B configuration, with 12 layers of Transformer, 768 hidden dimensions and 12 heads of attention, and a parameter amount of only 85M; GPDiT-H is extended to 24 layers, 2816 hidden dimensions, 22 heads of attention, and a parameter amount of 2B. This dual-track design allows researchers to use small models to verify ideas in resource-constrained scenarios, and when computing power is sufficient, switch to a large-scale version to pursue optimal generation quality.

Key performance figures: GPDiT-H-LONG achieves FVD 64 and FID 7.4 on MSRVTT zero-sample generation, surpassing SnapVideo (FVD 110) by about 42% in the same period; on UCF-101, it achieves FVD 218 and IS 66.6, and the IS indicator is about 58% higher than PixelDance (42.1). GPDiT-B (80M parameters) achieved FVD 214 on UCF-101 training, and the gap with FAR (FVD 194) of 130M parameters is within 10%, which illustrates the efficiency advantage of lightweight architecture.

Publicity Verification: The quantitative results of the paper on MSRVTT and UCF-101 can be verified, and the advantages compared with the same period methods are clear. However, the code and model weights are not open source, and third-party independent reproduction has not yet appeared. All values ​​are self-reported by the paper.

User and market recognition

GPDiT is currently in the academic paper stage, and market recognition is reflected in academic citations and community attention. There is no commercialization or enterprise adoption data yet.

Academic community attention: The paper was submitted in May 2025 to the final version in October, and has gone through 5 version iterations on arXiv, indicating that the author team is continuing to improve experiments and demonstrations. The paper appears in the Hugging Face Daily Papers trend, gaining some community exposure.

Institutional endorsement: The University of Science and Technology of China (USTC) has profound accumulation in the fields of CV and AI, and StepFun is an important player in the domestic video generation track (it has launched the Step-Video series). The joint publication of the two institutions gives the paper credibility at the level of industry-university-research integration.

Citation and Reproduction: As of the current point in time, there is no public third-party reproduction report or code repository for the paper. This is a normal cadence for an academic paper - most top CV conference papers don't see community replication until 3-6 months after publication. It is recommended to pay attention to whether official or third-party code will be open sourced in the future.

Industry Benchmark Position: GPDiT’s 2B parameter scale is in the same order of magnitude as SnapVideo (3.9B) and PixelDance (1.5B), but its performance of FVD 64 on MSRVTT was in the first echelon at the time. However, due to the lack of Demo and API, its C-side perception is far lower than commercial products such as Runway and Pika.

Cost advantage

GPDiT currently exists entirely in the form of papers, with a very simple cost structure—papers are available for free, but significant computing power investment is required to reproduce experiments.

Dimensions	Public information
Paper reading	arXiv is free and open to the public, supports PDF / HTML / TeX source code
Code and model	Not open source and cannot be downloaded and used
API service	Not provided
GPDiT-B recurrence cost (deduction)	32 H100 GPUs × 400k iterations × about 2-3 days
GPDiT-H recurrence cost (deduction)	Multi-stage training: 200k + 200k + 150k iterations, about 1-2 weeks in total × 64+ H100
Few-sample fine-tuning cost (deduction)	500 iterations × batch 4 × can be completed with a single card

The Free Truth: It is true that papers are free to read, but the hidden costs of "use" are extremely high. The training of GPDiT-B uses 32 H100 GPUs for 400k iterations. Calculated according to the cloud vendor H100, which is about $3-4/card hour, the computing power cost of a single complete reproduction is on the order of $10,000-20,000. The multi-stage training of GPDiT-H requires higher computing power, which is not affordable for general academic laboratories.

Implicit cost comparison with commercial APIs: If the goal is just to "generate videos", directly using Runway Gen-3 (~$0.05-0.10/second) or Pika (subscription $10-50/month) is much lower than the marginal cost of self-training GPDiT at the current stage. The value of GPDiT lies in technology exploration and architectural innovation, rather than out-of-the-box cost advantages.

Compliance Risk: This paper uses the arXiv non-exclusive distribution license, which allows scholarly citations and derivative research. However, if someone trains a commercial model based on the thesis solution in the future, they need to pay attention to the patents or intellectual property rights that StepFun and USTC may hold, and an FTO (freedom to implement) analysis should be conducted before commercial use.

Main functions

GPDiT’s capabilities focus on three levels: video generation, video representation learning and few-sample multi-task transfer:

• Frame-level autoregressive video generation: Predict future frames frame by frame in a continuous latent space, and each frame is generated based on all previously generated frames. This design breaks through the limitations of fixed-length video generation and can theoretically generate video sequences of any length. Compared with the discrete token autoregressive model, continuous spatial prediction avoids the information loss problem of the VQ tokenizer.

• Two causal attention variants (OF / OF2): OF (Only Former) variant eliminates attention calculations between clean frames during training, using spatial redundancy between video frames to reduce the amount of calculation by about 50%; OF2 variant retains the interaction between clean frames in exchange for stronger representation capabilities. Both are naturally compatible with KV Cache acceleration during inference, and the long sequence generation efficiency is significantly higher than the bidirectional attention scheme.

• Parameterless rotation time encoding: Reinterpret the diffusion forward process as a rotation operation on the complex plane, replacing the traditional adaLN-Zero module with the rotation angle $\theta_t$. This design removes the time condition submodule, which accounts for about 28% of the parameters of the DiT model, and has obvious advantages in parameter efficiency.

• Few-sample multi-task transfer: The pre-trained GPDiT model only needs 20 video samples (20 shots) and 500 fine-tuning iterations to transfer to a variety of downstream tasks such as style transfer, edge detection, depth estimation, human detection, image colorization, etc. This capability benefits from the natural condition splicing mechanism of the continuous autoregressive framework without the need for additional adaptation modules.

• Video Representation Learning: GPDiT’s middle-layer features can be directly used for video understanding tasks. Linear detection experiments show that GPDiT-H features have considerable representation quality on UCF-101 classification, and achieve 28.2% accuracy on ActivityNet-QA after replacing the CLIP visual encoder in the Video-ChatGPT framework.

Expert perspective: GPDiT's functional design has a "hidden synergy" - its autoregressive architecture naturally supports the unification of "generation + understanding". In traditional solutions, video generation models (such as Stable Video Diffusion) and video understanding models (such as VideoMAE) are two independent architectures. GPDiT's pre-trained weights on the generation task can be directly used as feature extractors for understanding tasks, and the improvement of generation quality is positively correlated with the improvement of representation capabilities (Figure 5c of the paper verifies the simultaneous improvement of FVD and classification accuracy), which means that the same model may serve both video content creation and video analysis scenarios.

Model and version evolution

Version context

GPDiT has gone through 5 version iterations on arXiv, and it took about 5 months from the first version to the final version:

Version	Date	Core Changes
v1	2025-05-12	First submission, proposing GPDiT-B (85M) and GPDiT-H (2B) architecture, reporting basic results on UCF-101 and MSRVTT
v2	2025-05-15	Revised experimental settings and supplemented ablation research
v3	2025-05-19	Improve related work and experimental descriptions
v4	2025-05-22	Supplementary linear detection experiment of video representation learning and more sample results
v5	2025-10-08	Final version, adding GPDiT-H-LONG variant, supporting 17-45 frame variable length generation, FVD reduced to 64 on MSRVTT

Model Variations

The GPDiT system consists of three main variants:

• GPDiT-B (85M parameters): Basic benchmark model, trained from scratch on UCF-101 for 400k iterations, suitable for resource-constrained verification and ablation experiments.
• GPDiT-H (2B parameters): Large-scale version, first warm-up 200k iterations on LAION-Aesthetic image data, then train 200k iterations on mixed image-video data, and finally fine-tune 150k iterations on pure video data.
• GPDiT-H-LONG: Based on GPDiT-H, use variable length (17-45 frames) video data to continue training for 150k iterations to enhance long sequence generation capabilities.

Technical advantages

GPDiT’s technical advantages can be broken down into three levels, each of which directly corresponds to specific efficiency or quality gains:

Frame-level causal attention is compatible with KV Cache: Traditional bidirectional attention faces two problems in long video generation-future frame information leakage destroys temporal consistency, and the inability to use KV Cache during inference causes the amount of calculation to increase with the square number of frames. GPDiT adopts frame-level causal attention, and each noisy frame only pays attention to the clean frames that have been generated in the past and itself. This design naturally supports KV Cache during inference, reducing the computational complexity of long sequence generation from $\mathcal{O}(F^2)$ to $\mathcal{O}(F)$. Experiments show that GPDiT maintains stable quality when generating videos longer than the training length, while bidirectional attention schemes suffer severe degradation.

Lightweight Causal Attention: Standard causal attention needs to maintain two sets of representations of clean frames and noisy frames at the same time during training, resulting in doubling of video memory and calculation. GPDiT observes a large amount of spatial redundancy between adjacent frames of the video and therefore directly skips attention calculations between clean frames during training. This optimization reduces attention computation by approximately 50% without sacrificing generation quality (the FVD difference between GPDiT-B-OF and GPDiT-B-OF2 ​​is only 2 points: 216 vs 214).

Parameterless rotation time coding (Rotation-based Time Conditioning): Mainstream diffusion Transformers such as DiT use adaLN-Zero to inject time step information. This module accounts for about 28% of the total parameters of the model. GPDiT reinterprets the diffusion forward process $x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1-\bar{\alpha}_t} \epsilon$ as a 2D complex plane rotation:

$$R(\theta_t) = \begin{pmatrix} \cos\theta_t & \sin\theta_t \ -\sin\theta_t & \cos\theta_t \end{pmatrix}$$

Where $\cos\theta_t = \sqrt{\bar{\alpha}_t}$, $\sin\theta_t = \sqrt{1-\bar{\alpha}_t}$. The model implicitly injects temporal information through reverse rotation, completely removing the MLP parameters of adaLN-Zero, significantly reducing the number of parameters while maintaining comparable generation quality.

Continuous spatial autoregression vs discrete token autoregression: Existing autoregressive video methods (such as VideoPoet, OmniTokenizer) rely on VQ-VAE to quantize video frames into discrete tokens. This mapping involves irreversible information loss. GPDiT operates in a continuous latent space and uses diffusion loss instead of cross-entropy loss for frame prediction, avoiding the accuracy bottleneck of discretization. This advantage is particularly obvious on few-shot tasks - GPDiT can complete condition generation through sequence splicing without additional adapters, while discrete token methods usually require training of specialized mapping modules.

How to use

GPDiT is currently only publicly available in the form of academic papers, with no directly usable code, model weights or online demos. The usage path is divided into the following levels:

How to use	Suitable for the crowd	Current feasibility	Resource requirements
Read papers	Researchers, algorithm engineers	✅ arXiv full text public	None
Reference architecture design	Video generation researcher	✅ The paper provides a complete technical description	None
Reproduce GPDiT-B	Academic laboratory with computing power	⚠️ Need to be implemented by yourself, no official code	32× H100 GPU
Reproduce GPDiT-H	Large research team	⚠️ Need to be implemented by yourself, no official code	64+ H100 GPU, multi-stage training
Few-sample fine-tuning experiment	Team with pre-trained weights	❌ Model weights are not disclosed	Single-card GPU
Commercial integration	Enterprise users	❌ Code/model not open source	N/A

Usage Suggestions: For teams that want to follow this direction, it is recommended to carefully read the architectural design (§4) and ablation experiments (§5) of the paper first, focusing on the implementation details of lightweight causal attention and the mathematical derivation of rotation time encoding. If the team has the intention to reproduce, it can start with GPDiT-B, verify the basic FVD indicators on UCF-101 (paper report 214-216), and then decide whether to extend to GPDiT-H.

Product Pricing

GPDiT is an academic research project and does not have a commercial product pricing system. The following is an overview of the costs of various forms:

Project	Description
Paper access	arXiv is permanently free and supports PDF / HTML / TeX download
Code and models	Not open source, no pricing information
API service	Not provided
Commercial License	Unpublished (need to contact author team or institutional technology transfer office)
Self-training computing power cost (deduction)	GPDiT-B is about $10K-20K (32× H100, 2-3 days); GPDiT-H is about $50K-100K+ (64+ H100, 1-2 weeks)

Cost comparison with commercial video generation solutions:

Plan	Typical monthly cost	Applicable scenarios	Production quality
GPDiT (self-training, deduction)	$10K-100K (one-time)	Academic research, architecture exploration	Self-reported paper MSRVTT FVD 64
Runway Gen-3	$0.05-0.10/second	Content creation, advertising production	Commercial-grade image quality and consistency
Pika subscription	$10-50/month	Personal creator, social media video	Quick production, moderate control
Stable Video Diffusion (open source)	Self-hosted GPU cost	Self-deployment scenario	Continuous optimization by the open source community

The cost performance of GPDiT at the current stage is far lower than that of commercial solutions. Its value proposition is not to "replace existing video generation tools", but to provide technical verification and academic reference for the next generation architecture.

Application scenarios

The technical characteristics of GPDiT determine that it is more suitable for research and engineering exploration scenarios rather than ready-to-use production environments:

• Research on Autoregressive Diffusion Fusion Architecture: GPDiT provides a complete frame-level autoregressive diffusion framework, on which researchers can explore new attention variants, temporal conditioning mechanisms or multi-modal extensions. Its lightweight architecture is particularly suitable for ablation experiments – the 85M parameter GPDiT-B allows for rapid iterative verification of new ideas.

• Research on Long Sequence Video Generation: GPDiT’s causal attention and KV Cache support make the computational cost of long sequence generation controllable. The paper verifies the stability of GPDiT-H-LONG in the 17-45 frame range. For research tasks that require tens of seconds of continuous video (such as autonomous driving scene prediction, motion simulation), GPDiT's architecture provides a more natural solution than fixed-length diffusion models.

• Unified generation-understanding model exploration: GPDiT's experiments show that there is a positive correlation between generation quality and representation ability, which means that the route of "one model doing both generation and understanding" is feasible. For teams that want to build a visual foundation model, GPDiT provides a unified training paradigm reference from the generation side.

• Less-sample video task customization: For specific fields (such as medical image conversion, industrial inspection visualization), GPDiT's few-sample migration capability means that it can be quickly adapted with a very small amount of annotated data. The threshold of 20 shots + 500 iterations is much lower than training a dedicated model from scratch.

Not suitable for scenarios:

• Ready-to-use video content production: No code, no model, no demo, GPDiT is currently completely unavailable to content creators.
• Real-time interactive generation: The iterative denoising process of the diffusion model is naturally not suitable for real-time scenarios, and the inference efficiency of GPDiT is not evaluated in real-time in the paper.
• Industrial-grade video generation that requires high-precision timing control: As an academic paper-level model, GPDiT does not have engineering-level guarantees in terms of controllability and consistency.

Applicable people

• Researchers in the field of video generation: Researchers who are concerned about the cross-fusion of diffusion models and autoregressive models can use GPDiT as a baseline framework for expansion or comparison. It is recommended to start with the ablation experiment (§5.2-5.4) of the paper to understand the performance differences between OF and OF2 attention variants on different tasks.

• Computer Vision Algorithm Engineer: Engineers who want to track the cutting-edge technology of video generation, suitable for intensive reading of the technical solution of the paper (§4), focusing on the implementation path of lightweight causal attention - the gains in reasoning efficiency of this design have engineering implementation value.

• Multimodal Basic Model Team: A team that is exploring the unification of generation and understanding. GPDiT's finding that "generation ability and representation ability are positively related" (Figure 5c) deserves further study. You can verify in your own framework whether this phenomenon is reproduced in other architectures.

• Academic laboratories (including master's and doctoral students): The 85M parameter scale of GPDiT-B is relatively friendly to student laboratories. If the team has 4-8 GPUs, it can try to reproduce the core experiment on UCF-101 as a learning project in the direction of diffusion Transformer.

Dissuade the crowd:

• Content Creator/Self-Media Operator: GPDiT currently has no available entrance. It is recommended to turn to commercial tools such as Runway, Pika or Kling.
• Product teams who need to deliver video generation functionality within 1-2 weeks: GPDiT's return period is measured in months.
• Technical decision makers without deep learning background: Evaluating GPDiT requires understanding the underlying principles of diffusion models and autoregressive models, and is not suitable for procurement evaluation.

Summary and Outlook

GPDiT has made a meaningful contribution on the technical roadmap - unifying frame-level autoregressive prediction and diffusion generation in continuous latent space, and proposing two practical optimizations of lightweight causal attention and rotational time encoding. On MSRVTT and UCF-101, GPDiT-H reached the optimal or near-optimal level at the time, and GPDiT-B achieved competitive results with larger models with 85M parameters, proving the efficiency advantage of the architectural design.

It can be seen from the discussion section of the paper that the author team has realized the limitations of the current work: the experimental scale is limited by computing power, the model is limited to video modality, and multi-modal extensions such as language have yet to be explored. The paper's VQA accuracy of 28.2% on ActivityNet-QA (vs. 35.2% on Video-ChatGPT) also shows that there is still room for improvement in GPDiT's representation learning capabilities.

Follow-up observation points:

1. Code open source: The complete reproduction of GPDiT relies on official code release. If the paper is accepted by top conferences such as CVPR/ICCV, the probability of the code being open source will increase significantly. It is recommended to follow the GitHub accounts of USTC or StepFun.
2. Larger Scaling: The experimental scale of Paper 2B parameters is medium in the context of large models in 2025. If subsequently expanded to 7B-13B parameters, the architectural advantages may become more obvious.
3. Multimodal Extensions: The authors of GPDiT explicitly state plans to explore unified modeling of multimodal inputs such as language. This will determine whether it can evolve from a "video generation model" to a "visual basic model".
4. Collaboration with the Step-Video series: Step Star is the publisher of both Step-Video-T2V and Step-Video-TI2V, and GPDiT’s technology accumulation may be integrated into Step-Video’s product line.

Procurement/Adoption Risk Assessment: GPDiT is not suitable as a procurement target or product integration dependency at this stage. Academic teams can incorporate it into the research baseline as an important reference; industrial teams recommend waiting for the code to be open sourced and community verified before evaluating technology adoption. Any commercialization plan based on GPDiT must first complete an FTO analysis to confirm the intellectual property boundaries of USTC and StepFun. If the team's goal is only to "use high-quality video generation in 2026", it is recommended to prioritize evaluating commercial APIs (Runway, Pika, Kling, Sora) or mature diffusion open source solutions (Stable Video Diffusion, VideoCrafter), and use GPDiT as a technical reserve for the next generation architecture.

Related tools: runway, pika

Version Info

• GPDiT-H-LONG ：arXiv v5 update, proposes GPDiT-H-LONG variant, supports 17-45 frame variable length video generation, and FVD is reduced to 64 on MSRVTT. （2025-10-08）
• GPDiT First Edition ：The first submission to arXiv proposes two specifications: the autoregressive diffusion Transformer infrastructure and GPDiT-B/GPDiT-H. （2025-05-12）
• GPDiT-H Update ：Supplement experimental data and ablation studies to improve the evaluation of GPDiT-H in video generation and representation learning. （2025-05-22）
• GPDiT-H-LONG ：Added GPDiT-H-LONG variant and long video generation evaluation, released final version. There is no official precise date yet. （2025-10-08）