FlowDirector is a training-free video editing framework launched by Westlake University AGI Lab and Central South University. It is based on ordinary differential equation (ODE)-driven editing flow generation technology, combined with spatial attention flow correction (SAFC) and differential average guidance (DAG) mechanisms to achieve high-precision semantic-level video content editing. Supports tasks such as object replacement, texture conversion, attribute modification, etc., and can be edited using the pre-trained T2V model without additional training.

FlowDirector

Core parameters and statistics

Projects	Information
Tool name	FlowDirector
Development Team	Westlake University AGI Lab / Central South University
Core entrance	GitHub: github.com/Westlake-AGI-Lab/FlowDirector
Delivery form	Open source research framework (Python)
Core technology	ODE-driven editing flow (Editing Flow), SAFC, DAG
License	Open source (see GitHub repository for specific license)
Required pre-training model	Compatible with mainstream Text-to-Video (T2V) model
Editing type	Object replacement, texture conversion, attribute modification, object addition/deletion

The core contribution of FlowDirector is to solve two long-standing pain points in video editing: timing inconsistency and structural distortion. Traditional methods rely on latent space inversion, which is prone to timing flickers and content distortion during the process of encoding video to latent space and then decoding. FlowDirector models the editing process directly in data space and uses ODE-driven smooth transition paths to avoid information loss caused by inverse mapping.

User and market recognition

• Academic Impact: The paper was published on arXiv (2506.05046), jointly produced by Westlake University AGI Lab and Central South University. In multiple video editing benchmark tests, FlowDirector performed outstandingly in three key indicators: instruction following, temporal consistency, and background preservation.
• Open Source Ecosystem: The code has been open sourced on GitHub, and community developers can reproduce experiments, secondary development, and verify the effects on their own data sets. Open source lowers the threshold for technical verification, and other research teams can directly use the results of FlowDirector for baseline comparison.
• Comparison with similar frameworks: Compared with existing video editing methods such as InstructVideo, Video-P2P, and TokenFlow, the core advantages of FlowDirector are no training (no need to fine-tune the model for each video) and structure preservation (ODE flow is naturally more stable than inverse mapping). The disadvantage is that it is currently more focused on research prototypes and is still far away from out-of-the-box product-level tools.

Cost advantage

• Completely Open Source and Free: Code and model weights are available from GitHub, with no API call fees. The computing resources required depend on the GPU specifications of the running device - the larger the memory, the higher the resolution and frame rate it can handle.
• Comparison reference: Commercial video editing tools (such as Runway Gen-2, Pika Labs, Topaz Video AI) are billed by the second or by the month, and the cost of a 30-second editing task ranges from a few yuan to dozens of yuan. FlowDirector has no direct usage cost, but you need to bear the GPU computing power overhead. If you already have a GPU device (RTX 4090 or above), the electricity cost of running a single edit is negligible; if you rely on cloud GPU rental (AutoDL and other platforms), the cost of a single edit is about a few yuan per hour.
• Suitable for teams that are cost-sensitive but have GPU resources, or users who need to control marginal costs in batch editing scenarios.

Main functions

• Precise Semantic Editing: Make semantic-level modifications to video content based on natural language instructions. For example, replace "bear" in the video with "dinosaur", or change "day" into "night". The editing process does not require manual keyframe annotation or frame-by-frame processing.
• Spatial and temporal consistency protection: Maintain the temporal coherence and spatial structure integrity of the video during the editing process. This is the core competitiveness of FlowDirector - other methods are prone to screen flickering or content misalignment in long video editing. FlowDirector alleviates this problem to a certain extent through ODE streaming.
• Local Editing and Global Protection: Based on the Spatial Attention Flow Correction (SAFC) mechanism, only the target area related to the editing instruction is operated, and the original content and motion status of non-target areas are frozen. Users can precisely control "where to change" and "where not to change", rather than letting the AI ​​make guesses.
• Trainless Editing: Use pre-trained text-to-video (T2V) models directly, without the need for additional model training or fine-tuning for each video or each editing task. Processing a new video only requires one forward pass, and the time and computing power costs are significantly lower than methods that require video-by-video fine-tuning.
• Multiple types of editing support: Covers various editing tasks such as object replacement (car→dragon), texture conversion (leather→metallic), local attribute modification (hair color change), object addition/deletion (add a hat, remove a person), etc.

Model and version evolution

• arXiv paper release (June 2025): The FlowDirector technology solution is disclosed for the first time, including three core innovations: Editing Flow Generation, SAFC and DAG. Published evaluation data on multiple benchmarks.
• GitHub Open Source: The open source code repository is synchronized with the paper, including inference code, sample scripts and pre-training weight calling interfaces.
• The current version is in the research prototype stage and there is no commercialization roadmap. Subsequent iteration directions may include: supporting higher resolutions (currently limited by GPU memory), integrating more T2V model bases, reducing computing resource requirements, etc.

Technical advantages

• Data Space Editing Flow: Does not rely on latent space inverse mapping, directly calculates the speed field difference between the source video and the target video in the original pixel space, and generates a smooth editing path. This fundamentally avoids the common structural distortion in the inverse mapping method - for example, the former is modified directly on the original image, while the latter is compressed into a thumbnail and then decompressed and modified back. The difference in information loss is decisive.
• Spatial Attention Flow Correction (SAFC): Extract attention maps relevant to the editing task and generate accurate spatial masks. During the ODE-driven editing process, a mask is superimposed on the velocity field, freezing the velocity field components in non-target areas and protecting the original state of the background and irrelevant areas. Intuitive understanding of SAFC: the model knows where to focus its attention and only changes the "focused" area.
• Differential Average Guidance (DAG): Inspired by Classifier-Free Guidance, it generates multiple candidate edit streams and calculates the difference signal between them, using the difference signal to enhance the accuracy of semantic alignment. Explanation: Rather than trusting the results of a single edit, it is better to find the "consensus direction" among multiple candidate solutions to reduce random bias.

How to use

The usage of FlowDirector is biased toward research and technical verification, and is not packaged into a Web application.

GET CODE:

git clone https://github.com/Westlake-AGI-Lab/FlowDirector.git

Depend on the environment: -Python 3.8+

• PyTorch (2.0 or above recommended)
• CUDA (11.8 or above recommended)
• Pre-trained T2V model weights

Typical steps:

1. Configure the Python environment and install dependencies according to the warehouse README.
2. Download the compatible pre-trained T2V model checkpoint.
3. Prepare to input video and edit instruction text.
4. Run the editing script: specify the input video path, editing prompt, output path and parameter configuration.
5. Wait for the inference to complete and view the editing results in the output directory.

Technical threshold description: FlowDirector is currently aimed at researchers and engineers with deep learning background. If you need to make it practical, you can consider encapsulating it into a Streamlit or Gradio web interface, or integrating it into the ComfyUI node ecosystem.

Product Pricing

Project	Description
Software License	Completely Open Source and Free
Computing resources	Need to bring your own GPU (RTX 4090 or above recommended)
Cloud GPU solution	The rental cost is about 2-8 yuan/hour (depending on the instance specifications), and a single edit takes several minutes to dozens of minutes
Commercial License	Please check the specific open source license terms of the GitHub repository

Compared with commercial video editing APIs (such as Runway's per-second billing), FlowDirector has a marginal cost advantage in batch and high-frequency editing scenarios - after a fixed investment in computing power, the incremental cost of additional editing approaches zero.

Application scenarios

• Video Special Effects Production: Use text commands to replace ordinary objects in the video with special effects elements. In the early proof-of-concept stage of the film and television industry, FlowDirector can be used to quickly generate samples to evaluate editing effects, without spending hours of labor by special effects artists on each version of the samples.
• Quick iteration of advertising videos: Quickly adjust video content according to changes in copywriting - replace product appearance, change background context. Advertising agencies can run comparison samples of multiple sets of editing plans at the same time, select the best plan, and then enter the fine production stage.
• Animation and Game Materials: Quickly modify character appearance or scene style in animated short films, reducing the workload of manual adjustment frame by frame. Suitable for independent animators and small game studios to quickly verify visual solutions in the early prototyping stage.
• Short video content production: Content creators make secondary creative edits to existing video materials - transforming ordinary scenery into a cyberpunk style and adding fantasy elements to real-life videos. An edit takes several minutes to ten minutes, which is one to two orders of magnitude faster than traditional frame-by-frame processing.
• Personalized Video Editing: Individual users can modify elements in pet videos and travel short films according to their own creative ideas to make them more interesting. GPU device support is required, but no professional skills are required for the editing process.

Applicable people

• AI researchers and computer vision engineers: FlowDirector’s target user group. You can reproduce the results of the paper, improve the algorithm based on the source code, and verify the editing effect on your own data set.
• Film and television post-production special effects artist: Technical special effects personnel who are interested in AI video editing technology and want to incorporate it into their workflow. A certain Python foundation is required to configure and use it, but the editing effects can be incorporated into early creative proposals.
• AI content creators and independent developers: People who have GPU resources and are willing to try cutting-edge video editing technology. The editing quality of FlowerDirector is at the leading level among equivalent training-free methods, and it is suitable as a basic component of batch video processing pipelines.

Boundaries not applicable:

• Scenarios that require real-time or near-real-time processing speed - FlowDirector inference takes minutes and is not suitable for live broadcast or real-time editing.
• There are extreme requirements for picture quality of 4K and above - currently limited by GPU memory, the output resolution is limited.
• Non-technical users - Currently it is not packaged as a web product, and the threshold for command line operations and Python bounded configuration is high.
• Projects that have strict requirements on the commercial copyright of output results - the copyright ownership of AI-generated/edited content is still a legal gray area.

Summary and Outlook

FlowDirector proposes a simple and effective technical solution in the field of video editing - using data space ODE flow to replace latent space inverse mapping, which solves the problems of timing inconsistency and structural distortion from the architectural level. Both SAFC and DAG improve reliability from the two dimensions of "precise positioning of the editing area" and "enhanced semantic alignment" respectively. For researchers, this is a baseline framework with clear ideas; for teams that want to implement video editing capabilities in actual projects, it proves that the quality of video editing without training can reach practical levels.

Current Limitations:

• It is still a research prototype and has not been packaged into a product form that can be commercially deployed.
• The requirements for GPU memory are high, which limits the video resolution and number of frames that can be processed.
• Users are required to bring their own technical capabilities to configure the environment, adjust parameters and troubleshoot problems.

Follow-up observation points:

• Whether a team will encapsulate it as a ComfyUI node or web service - this will directly determine the degree of practicality.
• Whether to support higher resolution and longer videos - this determines whether it can enter the film and television production process.
• The number of community contributions and forks - reflects the health of the open source ecosystem.

Related tools: runway, pika

Version Info

• initial release ：The initial public version supports precise semantic video editing SAFC local protection mechanism and DAG semantic enhancement based on ODE editing flow. （~2025-06）
• Paper release ：The arXiv paper is released, disclosing technical details and benchmark results. （~2025-06）