AudioCraft is a Meta open source AI audio framework that includes MusicGen music generation, AudioGen sound effect generation and EnCodec audio compression.

AudioCraft

Core parameters and statistics of AudioCraft

AudioCraft is the open source PyTorch audio generation framework of the Meta AI Research Institute. It is officially positioned as a "one-stop audio generation code library", covering the three major tasks of music generation, sound effect synthesis and neural audio compression. It is currently one of the most popular text-to-audio (Text-to-Audio) frameworks in the global open source community.

Projects	Public Information
Core components	MusicGen (music generation), AudioGen (sound effect generation), EnCodec (audio codec), MAGNeT (non-autoregressive text to audio), Multi Band Diffusion (diffusion decoder), JASCO (chord/melody/drum conditional generation)
Model scale	MusicGen provides three parameter versions of 300M / 1.5B / 3.3B
Input method	Text description + optional reference audio (melody continuation/style guidance)
Output format	Mono/stereo WAV, supports continuous generation for up to several minutes
Code License	MIT License (fully open source)
Model Weight License	CC-BY-NC 4.0 (Non-Commercial Use Only)
Hardware requirements	NVIDIA GPU inference recommended; 300M version can run on consumer-grade graphics cards with 6GB video memory
Python version	3.9+, depends on PyTorch 2.1.0+
GitHub Stars	~23,500 stars / ~2,700 forks (as of 2026-07)
Developer Contributions	34 contributors, mainly from the Meta AI research team

Parameter scale and hardware mapping: The 300M version can perform real-time inference on RTX 3060 (6GB); the 1.5B version recommends 8GB+ video memory; the 3.3B version requires 16GB+ video memory or model parallelism. Developers should "select the model size according to the available video memory" instead of blindly pursuing the maximum parameters. 300M can already produce usable music clips in most short video soundtrack scenarios.

Community activity: The GitHub repository has approximately 23.5k stars, 2.7k forks, and 34 contributors. Although the last active submission to the main repository was more than a year ago, the number of downloads of model weights on Hugging Face continues to grow, indicating that the community’s popularity on the consumer side has not subsided.

AudioCraft’s users and market recognition

AudioCraft's market recognition is concentrated at the level of the open source research community and application developers, and the number of commercial customers and revenue data have not been officially disclosed.

GitHub Ecological Signal: 23.5k stars and 2.7k forks indicate that AudioCraft has surpassed the early experimental stage and is in the top position among AI audio open source projects. A large number of third-party derivative projects (such as Gradio WebUI, Colab notebook Hugging Face Spaces online demo) form an ecosystem around it.

Academic citation: The MusicGen paper was accepted by NeurIPS 2023, and derivative research and replication projects around its architecture continue to appear. The combined citations of the two papers MusicGen and AudioGen have entered the forefront of the audio generation field.

Industry Application Coverage: Many game studios and independent content creators have disclosed sound effects pipelines based on AudioCraft in technology blogs; however, Meta itself has not announced any enterprise-level commercial cases, and commercialization completely relies on the community and third parties.

B-side adoption bottleneck: Model weights use the CC-BY-NC 4.0 license, which means that commercial use requires a separate authorization from Meta. This is the main uncertainty in enterprise-level adoption, and it is also the structural reason why the current market recognition is "applauded but not popular".

AudioCraft’s Cost Advantage

AudioCraft's "cost advantage" lies in zero license fee model access and self-control of computing power for localized deployment, rather than the obvious low price of SaaS-style pay-as-you-go billing.

C client/individual users: completely zero cost. You can pull the code directly from GitHub, download the model weights from Hugging Face and run it locally, or experience it online through the official Demo page. The only costs are local GPU hardware and power consumption; the 300M model runs on a consumer-grade graphics card, and the electricity bill generated per time is negligible.

Developers/API users: The model itself is free, but deploying inference requires your own GPU infrastructure. Estimated using cloud GPU instances (such as RTX 4090 on-demand instances), the computational cost of a single 30-second music generation is approximately US$0.01-0.03. In comparison, the cost per call to commercial APIs (such as OpenAI Jukebox-like services) is typically 10-50 times higher. If the team already has a GPU cluster, AudioCraft's marginal inference cost can approach zero.

Enterprise/Private: No license fees, but you need to bear the full model deployment and operation costs: GPU server purchase or cloud instance rental Python contextual maintenance, model version management, and inference API packaging. When making budget comparisons, corporate executives should not simply equate "open source and free" with "total cost of ownership is zero." Operation and maintenance manpower, GPU update cycles and technical debt are often higher than the subscription fees for SaaS services.

Hidden costs: The model weights of the CC-BY-NC 4.0 license cannot be directly used for commercial use. If an enterprise requires commercial authorization, it must confirm the terms separately with the Meta legal team. This process may cause unforeseen legal costs and time delays. In addition, the maintenance rhythm of open source projects is uncontrollable - the latest submission of the main warehouse is more than a year ago, and long-term dependence on an inactive upstream project involves technology stack risks.

Main functions of AudioCraft

AudioCraft's capabilities revolve around the two main lines of "audio generation + audio encoding". The core model group covers the entire link from music to ambient sound to compression.

• MusicGen (text/melody to music): receives text descriptions (such as "soothing piano jazz") to generate corresponding style music clips; supports melody conditioning (Melody Conditioning) - upload a humming or ready-made melody, and the model will continue to write a complete arrangement based on this. Acceptance Concerns: Structural coherence in long audio (over 30 seconds) is still limited, and the second half of the paragraph is prone to topic drift.
• AudioGen (Text to Sound Effects): Specializes in ambient sound effects and onomatopoeia generation. Enter descriptions such as "rain hitting windows", "dog barking", "traffic noise", etc. to output the corresponding sound effect clips. Acceptance Concerns: The ability to control precise timing (such as producing an explosion sound at the exact 3rd second) is weak, and is more suitable for "ambient sound effects" rather than "precise onomatopoeia".
• EnCodec (Neural Audio Codec): Meta's self-developed end-to-end neural audio compression model, which maps original audio into discrete token streams, is the underlying coding foundation of MusicGen/AudioGen. Support for variable rates from 1.5 kbps to 24 kbps provides discrete audio representation for subsequent models.
• MAGNeT (non-autoregressive extension): Introducing a non-autoregressive parallel decoding strategy based on the MusicGen architecture. The inference speed is several times higher than that of the autoregressive baseline, making it suitable for delay-sensitive real-time generation scenarios.
• JASCO (Multi-Conditional Music Generation): The latest extended model (v1.4.0a2) supports using chord progressions, melody lines, and drum tracks as explicit condition inputs into the music generation process, which is suitable for advanced scenarios that require precise arrangement control.
• Multi Band Diffusion (diffusion decoding enhancement): As an alternative decoder to EnCodec, it uses a multi-band diffusion model to improve the fidelity of the generated audio while maintaining compression efficiency.
• AudioSeal (Audio Watermark): The built-in AIGC audio watermark tool supports embedding tags imperceptible to the human ear in the generated audio for source tracing and copyright protection of AI content.

Synergies between functions: EnCodec is more than just a standalone compression tool - it provides a unified discrete token representation layer for MusicGen, AudioGen and MAGNeT, allowing the three generative models to share the same set of encoding semantic spaces, making decoders interchangeable in the inference pipeline to achieve "quality vs speed" trade-offs. This means that developers can improve sound quality with one click by replacing the EnCodec decoder with Multi Band Diffusion without changing the generation model itself.

AudioCraft model and version evolution

AudioCraft's version history is based on GitHub Tags. Since its initial release in June 2023, it has experienced an evolution from a research prototype to a multi-model framework.

Initial release (v0.0.1 - v0.0.2)

• v0.0.1 (2023-06-09): Initial open source release, only contains model evaluation code and is not trainable. Released as supporting open source material for the paper "Simple and Controllable Music Generation".
• v0.0.2 (2023-08-01): Added Gradio local Demo to support Extended Generation (infinite length calculation) and PyTorch 2.0 memory efficient attention. Fixed the Top-p sampling problem and added tanh to the compressor output to prevent clipping.

Training capability open (v1.0.0 - v1.1.0)

• v1.0.0 (2023-09-07): Milestone version. Added complete training code for EnCodec, AudioGen, MusicGen and Multi Band Diffusion, and released AudioGen pre-trained weights. Marks the upgrade of AudioCraft from "model demonstration" to "trainable framework".
• v1.1.0 (2023-11-06): Remove direct dependency on torchaudio and use ffmpeg CLI to handle audio I/O instead. Fixed CFG (Classifier-Free Guidance) coverage issue and CLAP sampling rate error. Three backwards-incompatible changes were introduced (see CHANGELOG for details).

Stereo and Architecture Extensions (v1.2.0 - v1.3.0)

• v1.2.0 (2024-01-11): Critical Release. Added support for stereo models; fixed issue where commitment loss was only applied to the first layer of RVQ; removed compressed model state from LM checkpoints to maintain loading consistency.
• v1.3.0 (2024-05-02): Integrate the MAGNeT non-autoregressive model and its Hugging Face checkpoint with the Gradio Demo. Fix typo and setup.py packaging scope. Added FSDP (Fully Sharded Data Parallel) support for PyTorch 2.1.0.

Experimental branch (v1.4.0a)

• v1.4.0a1 (2024-06-03): Alpha version. Added AudioSeal watermark training code and PESQ audio quality evaluation index; added audio enhancement tools such as pink noise generation, resampling, and filtering.
• v1.4.0a2 (2025-01-14): Alpha version. Released the JASCO model (conditional music generation, supporting chord/melody/drum conditions), and related checkpoints were uploaded to Hugging Face simultaneously.

Version Feature Summary: AudioCraft experienced intensive feature iterations in the second half of 2023 (an official version every 2-3 months). The pace slowed down after entering 2024, and v1.4.0 stayed in the Alpha stage. As of July 2026, the latest stable version is v1.3.0 (2024-05-02). Teams should use v1.3.0 as a baseline evaluation when planning production deployment, and keep an eye on Alpha features such as JASCO and AudioSeal, but they are not suitable for production yet.

AudioCraft’s technical advantages

AudioCraft's technical advantage does not lie in the performance breakthrough of a single model, but in the architectural design of "shared token representation layer between models" and the flexibility of the triple decoding path of "autoregressive + non-autoregressive + diffusion".

Unified Coding Layer (EnCodec): All generative models share EnCodec as the front end, which maps the original audio signal into a discrete token sequence. The direct effect of this shared layer is that new generative models (MAGNeT, JASCO) can reuse the trained EnCodec encoder/decoder and only need to train the intermediate LM or diffusion component, greatly reducing the repeated investment in training data and computing resources.

Token interleaving strategy: AudioCraft adopts a simple interleaving mode for multi-stream parallel tokens. Different from previous work that requires modeling multiple streams separately, single autoregressive LM simultaneously captures the long-term dependencies and local details of audio through a specific token order. This mechanism directly explains "why AudioCraft can use a single model to process music and sound effects at the same time": it does not design dedicated modules for different audio types at the model architecture layer, but allows LM to learn the token pattern that distinguishes music and sound effects from the data.

Triple decoding ecology: The token output by the same LM can be rolled back to the waveform through three decoders - original EnCodec (fastest, basic quality), Multi Band Diffusion (slower, higher fidelity), and custom decoders from the future community. This flexibility of "once generation, multi-level decoding" allows developers to make quality/speed trade-offs during the inference phase without retraining the model.

Hugging Face Ecological Integration: Model weights are hosted in Hugging Face Hub and can be loaded directly through the transformers and audiocraft libraries. The community has built web UIs, API wrappers, and MCP connectors around this, lowering the engineering barrier from research code to working services.

Boundary Note: AudioCraft is not specifically optimized in the direction of speech synthesis (TTS) and is not suitable to replace specialized speech synthesis models (such as Bark, VALL-E). Its pronunciation of Chinese lyrics is also less accurate than business plans optimized for Chinese.

How to use AudioCraft

AudioCraft's usage portals are divided into three categories: online experience, local Python inference and self-owned model training.

Online Experience: Through Meta’s official Demo page (audiocraft.metademolab.com), you can directly try out the core generation capabilities of MusicGen and AudioGen without the need for a local GPU. Suitable for non-technical users to quickly evaluate the generation effect.

Native Python inference (recommended path):

# Install
pip install -U audiocraft

#MusicGen inference example
from audiocraft.models import MusicGen
from audiocraft.data.audio import audio_write

model = MusicGen.get_pretrained('facebook/musicgen-small') # 300M parameters
model.set_generation_params(duration=8) # Generate 8 seconds of audio

wav = model.generate([ # Batch text input
    "Soothing Piano Jazz",
    "Exciting electronic rock"
])

for idx, one_wav in enumerate(wav):
    audio_write(f'output_{idx}', one_wav.cpu(), model.sample_rate)

Parameter Description: MusicGen.get_pretrained() accepts three preset sizes: 'small' (300M), 'medium' (1.5B), 'large' (3.3B); set_generation_params(duration=8, top_k=250, top_p=0.0, temperature=1.0, cfg_coef=3.0) cfg_coef controls the strength of text conditional following - higher values better match the music to the text description, but at the expense of variety. The first run will automatically download the model weights (small is about 1.2GB, large is about 12GB) and is stored in ~/.cache/audiocraft/.

Training Portal: The complete training code is available starting from v1.0.0. Define the data set, model architecture and training parameters through the configuration system (YAML + Hydra), and execute dora run to start training. For detailed procedures, see the GitHub repository docs/TRAINING.md.

Community Extensions: There are dozens of AudioCraft-based Gradio Demo on Hugging Face Spaces, which can be experienced in the browser without installation; third-party developers have also contributed REST API packaging Discord Bot and MCP Server integration.

AudioCraft Product Pricing

AudioCraft follows a two-tier pricing structure of "free code + non-commercial restrictions on model weights", which is not entirely equivalent to "free open source" in the traditional sense.

• C client/individual users: Code and model weights are available for free, and there is no need to pay any fees for personal creation, learning and research. The official Demo page can be experienced online, and local running requires your own GPU.
• Developer/Independent Commercial: The code is released under the MIT license. Developers can freely modify, distribute, and integrate into commercial products, as long as the copyright statement is retained. However, the model weights use the CC-BY-NC 4.0 license - any for-profit use (including but not limited to commercial SaaS services, app embedding for advertising monetization, and internal efficiency tools within the enterprise) requires a separate authorization from Meta. This separation arrangement of "open source code and limited weight" is the cost trap that developers should be most concerned about.
• Corporate/Private: No standardized pricing. If an enterprise requires commercial model weight authorization, it must contact the Meta legal team to confirm the terms and fees. In addition, enterprises need to bear engineering investment in building GPU infrastructure Python operation and maintenance pipelines and inference API packaging during privatized deployment.

Price comparison with competing products: Based on a 30-second music generation, the GPU cost of AudioCraft (self-deployed) is about $0.01-0.03, while similar commercial APIs such as Soundraw subscription are about $16.99/month (unlimited generation but not commercially available), and AIVA subscription is €15/month (commercially available but with quantity restrictions). AudioCraft has significant cost advantages in enterprise-level batch generation scenarios, but the premise is that the team has GPU operation and maintenance capabilities and can solve the commercial authorization of model weights.

AudioCraft application scenarios

AudioCraft's core scenarios focus on the field of "budget-sensitive, high-frequency trial and error, and copyright-required audio content production".

• Short video and social media soundtrack: Content creators can quickly generate background music for TikTok, Reels, and YouTube Shorts, and generate 15-30 second clips at a time. The cost of text-driven iterative trial and error is almost zero. Revenue Quantification Deduction: The soundtrack selection time for a regular short video is shortened from "10-20 minutes to browse the library and audition" to "30 seconds to enter the description + 10 seconds to generate the preview", and the efficiency is increased by more than 10 times.
• Game sound effects mass production: Independent games or small studios use AudioGen to batch generate ambient sound effects (wind sounds, footsteps, door sounds) based on text descriptions, replacing traditional sound effects library procurement or outsourcing customization. Quantification of benefits Deduction: For a project containing 50 sound effects, the outsourcing cost is about $500-2000. The production cost using AudioCraft is close to zero, but 10-20% of manual screening and post-production fine-tuning time needs to be reserved.
• Music Creation Inspiration Assistance: The composer uploads a humming or piano clip through Melody Conditioning, allowing the model to generate multiple arrangement variations as a starting point for creation. This does not directly produce a finished product, but it can significantly shorten the time from idea to demo during the composition bottleneck period.
• Podcast and audio content post-production: Generate transition sound effects, background pad sounds, and paragraph transition music to enrich the auditory level. Especially suitable for solo podcasters - in traditional post-production, you need to purchase a sound effects library or outsource mixing. The clips generated by AudioCraft are copyright-free and can be mixed directly into the track.
• Education and Research: The academic community uses AudioCraft as the baseline framework for audio generation research. Because of its modular design and open training code, the cost of experimental verification of new models is much lower than building it from scratch.

Not suitable for scenes: Scenes that require extremely high generation accuracy (such as commercial soundtracks that require rhythm alignment accurate to the second, movie-level sound effects design that requires multi-track combination control) are not suitable for pure text-driven generation methods; scenes that require accurate pronunciation of Chinese lyrics, or strict requirements for speech synthesis quality should also be avoided.

Who is AudioCraft suitable for?

AudioCraft uses an open source framework to cover a variety of roles, from researchers to creative workers, but the depth of use and thresholds for each group vary significantly.

• AI Audio Researcher: Can quickly reproduce baselines, modify architecture, and design new experiments based on training code and modular components. Prerequisites: Basic GPU training for PyTorch deep learning. AudioCraft is one of the most open-source research frameworks in the field of audio generation.
• Independent developers and full-stack engineers: Build custom APIs or web applications by encapsulating Python inference code. Prerequisite: Python engineering capabilities GPU server or cloud instance. The 300M model is enough to support MVP verification, and there is no need to use a large model in the early stage.
• Content creators and self-media people: Use MusicGen/AudioGen to generate soundtracks and sound effects through the official Demo or community Gradio page. Prerequisites: No technical requirements, but a browser and network are required. In commercial scenarios, attention should be paid to the model weight permission boundaries.
• Small games and independent studios: Use the AudioGen pipeline to replace some of the sound outsourcing work. Prerequisite: It is best to have a technical member with Python basics in the team who is responsible for model deployment and batch script maintenance.

Not suitable for the crowd: Business users who have a strong need for "zero deployment, out-of-the-box use" (no technical background and unwilling to touch the command line) should give priority to commercial AI music SaaS; media companies that require large-scale production-level stability (24/7 reasoning SLA, enterprise-level technical support) currently have no official channel guarantee; projects that require Chinese speech synthesis or accurate lyrics generation have no mature solutions in the AudioCraft ecosystem.

Summary and Outlook

AudioCraft is currently the most complete AI audio generation framework in the open source community. Its core value lies in "allowing music, sound effects, and compression tasks to share the same model architecture and training pipeline through a unified EnCodec token representation layer." This is essentially different from the previous route of independent modeling of each task.

There are four current limitations: First, the model-weighted CC-BY-NC 4.0 license is a structural obstacle to commercialization. The community has been looking forward to Meta's open commercial license for two years but no substantial progress has been seen; second, the maintenance activity of the main repository has dropped to near stagnation - v1.4.0 has stayed in the Alpha stage for more than 18 months with no follow-up official version; third, the generation quality is excellent in short clips (within 15 seconds), but exceeds 30 The topic coherence and sense of structure are significantly attenuated in seconds; fourth, the semantic following ability of non-English text conditions (especially Chinese) is significantly weaker than that of English, and Chinese users require additional prompt engineering tuning.

Follow-up observation points: Whether Meta will reactivate the update rhythm of AudioCraft in future audio extensions of the Llama series; whether community-derived models (such as the Chinese music model based on MusicGen fine-tuning) can fill long-tail needs that are not officially covered; and whether the Spaces application of AudioCraft in the Hugging Face ecosystem continues to grow.

Procurement/Adoption Risk Assessment: Independent content creators and small game studios with limited budgets can give priority to using the 300M model for internal trial and error and prototype verification, starting at zero cost and without legal risks (non-commercial use). Three verification actions must be completed before enterprise-level procurement: (1) Contact Meta Legal to confirm the model weight commercial authorization conditions and fees; (2) Assess whether the GPU infrastructure and operation and maintenance team capabilities can support long-term inference pipelines; (3) Establish upstream warehouse activity monitoring - if v1.4.0 has not entered the stable version within 12 months, you should consider preparing alternatives (such as community fine-tuning models or commercial APIs on Hugging Face). The most reliable strategy at present is a dual-track parallelism of "using open source for prototype verification and commercial API for production expansion".

Related tools: elevenlabs, udio

Version Info

• AudioCraft 1.2 ：Optimize the quality of MusicGen long audio generation, and add a new model distillation version (smaller size, faster inference). （~2026-02）
• AudioCraft 1.0 ：Initial open source release, including MusicGen, AudioGen and EnCodec core models. （~2025-05）