CodeQwen is a code-specific large language model launched by the Alibaba Tongyi Qianwen (Qwen) team. It is based on the Qwen1.5 architecture and is pre-trained on approximately 3 trillion tokens of code data. It supports 92 programming languages ​​and has a context length of 64K. CodeQwen1.5-7B reaches 83.5% pass@1 (Chat version) on HumanEval, and performs well in code generation, long context modeling, code editing Text-to-SQL and other tasks, making it the first echelon in open source code models.

CodeQwen’s Tool Analysis

Core parameters and statistics

Parameter	Value
Full model name	CodeQwen1.5-7B / CodeQwen1.5-7B-Chat
Development Team	Alibaba Cloud Tongyi Qianwen Team (Qwen Team)
Infrastructure	Qwen1.5 (Transformer Decoder-only)
Parameter scale	7B
Amount of pre-training data	About 3 trillion tokens (code-related data)
Supported programming languages	92 types
Context length	64K tokens
Attention mechanism	Group Query Attention (GQA)
License Agreement	Tongyi Qianwen RESEARCH LICENSE AGREEMENT (Available for Research, application required for commercial use)
Open source repository	GitHub: QwenLM/Qwen1.5
Model Weight Platform	Hugging Face / ModelScope

CodeQwen is a large language model in the Alibaba Tongyi Qianwen family specially designed for coding scenarios. Different from the general Qwen model, CodeQwen uses about 3 trillion tokens of code data for special training in the pre-training stage, covering 92 programming languages. Its core gain comes from the specialization of data distribution - not the larger parameter scale, but the higher density and diversity of codes in the training data, making the model significantly better than the same-sized general model in code generation, understanding, repair and other tasks.

User and market recognition

• GitHub Ecosystem: CodeQwen1.5 series models are hosted in the QwenLM/Qwen1.5 warehouse, and the overall Qwen organization has received 21.4k+ Stars on GitHub. CodeQwen1.5-7B has about 3,154 monthly downloads on Hugging Face (as of the query date).
• Benchmark results: CodeQwen1.5-7B-Chat reached 83.5% pass@1 on HumanEval 0-shot, surpassing DeepSeek-Coder-Instruct 6.7B (78.6%) and CodeLlama-7B (33.5%) of the same size, ranking first in the open source 7B level code model. MBPP 0-shot reaches 77.7%, which is also ahead of competing products of the same size.
• Long context evaluation: In the "Needle in the Code" synthetic evaluation, CodeQwen1.5 can accurately reproduce the inserted custom function within the 64K length range, indicating that its long context code understanding ability meets the design goals.
• SWE-Bench performance: CodeQwen1.5-Chat scored 0.89 on SWE-Bench, surpassing ChatGPT-3.5, indicating that its ability to solve real warehouse-level software development problems is initially competitive.

Cost advantage

CodeQwen is positioned as an open source code model, and its cost structure is different at three levels: individual developers, academic researchers, and enterprises.

• C-side/individual developers: Model weights are completely free and open source (Research License), and can be deployed and run on local GPUs through tools such as Hugging Face Transformers or Ollama. In terms of video memory requirements, BF16 accuracy requires approximately 16.99GB of GPU video memory (7B model), which is reduced to approximately 8.21GB after Int4 quantification, and consumer-grade graphics cards (such as RTX 3090/4090) can run it. In terms of inference speed, it is about 40.93 tokens/s under BF16 and about 50.09 tokens/s under Int4.
• Developer/API call: Tongyi Qianwen series models (qwen-turbo/qwen-plus) can be called through Alibaba Cloud DashScope API, and billing is based on Token. The specific pricing is subject to the official DashScope page. In addition, vLLM/SGLang services can be deployed through the Hugging Face Inference API or by yourself. The deployment cost depends on the GPU specifications.
• Enterprise/Private Deployment: Enterprises can conduct privatized deployment and fine-tuning based on open source weights. For commercial use, you need to apply for commercial authorization from Alibaba Cloud (fill in the application form). The specific authorization terms shall be subject to the official reply. For compliance scenarios such as finance and government affairs that require privatized deployment of Code Assistant, CodeQwen's open source features reduce the risk of data outbound and third-party dependence.

Main functions

• Code Generation (Text-to-Code): Translate natural language descriptions into executable code. Supports 92 languages ​​including Python, C++, Java, JavaScript, TypeScript, Go, and Rust. On the HumanEval and MBPP benchmarks, Chat version pass@1 reaches 83.5% and 77.7% respectively, which is close to the GPT-3.5 level.
• Long context code understanding: A context window of 64K tokens enables it to handle repository-level code files. Extended based on technologies such as NTK-aware interpolation and LogN attention scaling, it can accurately retrieve and reproduce code fragments within a 64K length range in actual tests.
• Code Editing and Debugging: On CodeEditorBench, CodeQwen1.5 achieved a SOTA effect of 7B parameters in the four dimensions of Debugging, Translation, Language Switching, and Code Polishing.
• Text-to-SQL: Supports converting natural language queries into SQL statements. In the two Text-to-SQL benchmarks Spider and Bird, CodeQwen1.5-Chat ranks second behind GPT-4 (DIN-SQL prompt method). This capability enables it to bridge the interaction gap between non-programmers and databases.
• Tool Calling and Agent Capabilities: Qwen-Chat series supports ReAct-style tool calling and Function Calling. In the Chinese tool usage evaluation, CodeQwen1.5-7B-Chat's tool selection accuracy reached 95.5%, tool input Rouge-L reached 0.900, and the false positive rate was only 11.6%.

Model and version evolution

CodeQwen currently has only one major release series - CodeQwen1.5, released on April 16, 2024. The series consists of two models:

CodeQwen1.5-7B (Base)

Code-based model, pre-trained on approximately 3 trillion tokens of code data. Suitable for downstream tasks such as code completion and fine-tuning. There is no built-in dialogue capability, users need to build the Prompt format or perform SFT by themselves.

CodeQwen1.5-7B-Chat (Chat)

The dialogue version aligned with SFT and RLHF based on the Base model natively supports interactive scenarios such as multiple rounds of code dialogue, code generation, debugging, and SQL queries. The review mainly revolves around the Chat version.

It is worth noting that CodeQwen was born in the Qwen1.5 era (early 2024). With the release of Qwen2, Qwen2.5 and the latest Qwen3 series, the Tongyi Qianwen team continues to iterate on code capabilities - the Qwen3 series has integrated code generation capabilities, and subsequent dedicated models such as Qwen3-Coder may become the spiritual successor of CodeQwen. But currently CodeQwen1.5 is still the only code-specific open source model named "Code" in the Qwen series.

Technical advantages

• Specialized pre-training data: CodeQwen1.5 is pre-trained on about 3 trillion tokens of code data instead of general text data. These code data cover real codes in multiple languages, multiple frameworks, and multiple fields, making the model's understanding of code syntax, semantics, and programming paradigms much deeper than a general model of the same scale.
• Group Query Attention (GQA): Uses the group query attention mechanism to reduce KV Cache usage during inference and improve the throughput efficiency of multi-round dialogue and long sequence generation. GQA is particularly important in continuous editing scenarios of code generation - when developers frequently view and modify code snippets, GQA can effectively reduce inference latency.
• Long context expansion technology: Extend the context length from pre-trained 8K to 64K through a combination of NTK-aware interpolation Window Attention and LogN attention scaling technologies. This allows the model to read the complete warehouse-level code file at once, thereby understanding the contextual dependencies of the entire file when modifying a function.
• Multi-language coverage: Supports code generation in 92 programming languages, covering everything from mainstream languages ​​(Python, Java, C++, JavaScript) to niche languages ​​(such as Verilog, Racket, COBOL, etc.). In the MultiPL-E evaluation, the performance was balanced on the eight mainstream languages ​​​​of Python, C++, Java, PHP, TypeScript, C#, Bash, and JavaScript, and it was not biased towards a single language.
• Quantification Friendly: The official provides Int4 and Int8 quantized versions. After quantization, the inference speed is increased by about 20%-40%, and the video memory usage is reduced by about 50% (7B BF16: 16.99GB → Int4: 8.21GB), making it possible for consumer-grade graphics cards to run. Quantified performance loss is no more than 2-3 percentage points in benchmarks.

How to use

CodeQwen has flexible deployment and invocation methods, covering all scenarios from local experiments to production deployment.

Local Reasoning (Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/CodeQwen1.5-7B-Chat"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    trust_remote_code=True
).eval()

response, history = model.chat(tokenizer, "Write a quick sort function in Python", history=None)
print(response)

OpenAI Compatible API Deployment (vLLM)

pip install vllm
python -m vllm.entrypoints.openai.api_server \
    --model Qwen/CodeQwen1.5-7B-Chat \
    --trust-remote-code \
    --dtype bfloat16

Then call via OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    api_key="<YOUR_API_KEY>",
    base_url="http://localhost:8000/v1"
)

chat_response = client.chat.completions.create(
    model="Qwen/CodeQwen1.5-7B-Chat",
    messages=[{"role": "user", "content": "Implementing an LRU Cache in C++"}],
    max_tokens=2048,
    temperature=0.2
)
print(chat_response.choices[0].message.content)

Local lightweight deployment (Ollama/llama.cpp)

CodeQwen1.5 supports one-click deployment through Ollama, and can also perform CPU inference through llama.cpp. The model weights have been uploaded to Hugging Face, and the community has provided a quantized version in GGUF format.

Alibaba Cloud DashScope API

Tongyi Qianwen series models (including coding capabilities) can be called online through Alibaba Cloud DashScope:

import dashscope
from dashscope import Generation

dashscope.api_key = "<YOUR_DASHSCOPE_API_KEY>"
response = Generation.call(
    model="qwen-turbo",
    messages=[{"role": "user", "content": "Write a Python decorator to measure function execution time"}]
)
print(response.output.text)

Product Pricing

• Open source weights: The model weights of CodeQwen1.5-7B and CodeQwen1.5-7B-Chat are free to download on Hugging Face and ModelScope, using Tongyi Qianwen RESEARCH LICENSE AGREEMENT (can be used for research, commercial use requires authorization from Alibaba Cloud).
• DashScope API: Call Tongyi Qianwen series models through Alibaba Cloud DashScope, billed by Token. The specific prices of qwen-turbo and qwen-plus are subject to the official DashScope pricing page, which is usually lower than the API price of the GPT series models.
• Private Deployment Cost: Depends on GPU hardware selection. The Int4 quantized version can run on a single RTX 3090/4090 (24GB of video memory), and the BF16 version recommends using a GPU with more than 24GB of video memory. Enterprise deployment also needs to consider inference framework (vLLM/SGLang), load balancing and operation and maintenance costs.
• Free quota: DashScope API provides free calling quota for new users (subject to the latest official Alibaba Cloud policy). The open source deployment itself has no usage limit.

Application scenarios

• AI Programming Assistant Private Deployment: Enterprises can deploy CodeQwen on internal servers as an open source alternative to GitHub Copilot. It is especially suitable for scenarios such as finance, government affairs, and military industry that have strict requirements for code data privacy. Connect to the model API through the VSCode extension or JetBrains plug-in to achieve code completion, interpretation and review. Deployment requires the enterprise to have GPU infrastructure or be able to accept cloud API calls.
• Text-to-SQL data query: Non-technical business personnel describe query requirements through natural language, and CodeQwen converts them into executable SQL statements. Self-service query scenarios suitable for data analysts, operators and business managers to reduce dependence on the data team. It should be noted that the accuracy of complex multi-table JOIN queries may be lower than GPT-4. It is recommended to add manual review to key queries.
• Code Teaching and Learning Assistance: CodeQwen-Chat can serve as a programming learning partner, explaining code snippets, providing examples, and pointing out errors. For beginners in programming, it provides a conversational step-by-step guide to the coding process. However, the model may generate unsafe code or writing methods with security risks, so the awareness of safe coding needs to be emphasized in teaching.
• Code Migration and Refactoring: Migrate code from old languages ​​(such as COBOL, Fortran) to modern languages ​​(such as Python, Java), or refactor code between different frameworks. CodeQwen’s cross-language translation capabilities come from its pre-training coverage on 92 languages. It is recommended to perform adequate unit testing verification on the migrated code.

Applicable people

• Software Developers: Use CodeQwen to assist with code generation, debugging, and interpretation during the coding process. Supports mainstream IDE integration (via Ollama or vLLM's API interface), suitable for individual developers and small teams looking for a free open source code assistant. Basic model local deployment capabilities are required for the best experience.
• Data Engineers & Analysts: Leverage Text-to-SQL capabilities to quickly query databases, or use Python scripts to automate data processing tasks. CodeQwen's performance in SQL generation is close to GPT-4 and is reliable enough for regular queries. Not suitable for ultra-complex queries and high-concurrency scenarios (it is recommended to use commercial database AI assistants).
• AI application developers: Embed CodeQwen as a back-end code generation module into a self-developed AI application, and deploy it by yourself through open source weight, without relying on the pricing and frequency control restrictions of third-party APIs. It is necessary to evaluate the commercial use restrictions of the Research License, and you must apply for authorization from Alibaba Cloud before commercial use.
• Academic Researchers: Research the technical route, fine-tuning methods and evaluation benchmarks of large code models. CodeQwen1.5's open source weights and full technical report provide a reproducible baseline model for code LLM research. Suitable for research in code intelligence, software engineering automation and other directions.

Not suitable for boundaries: CodeQwen is not suitable for general conversation scenarios (compared to the Qwen2 general model of the same period, its chatting and open domain creation capabilities are insufficient); it is not suitable for scenarios that require 80K+ ultra-long contexts (the upper limit of the context window is 64K); it is not suitable for online services that require real-time low latency (inference speed is limited when not optimized by vLLM); it is not suitable for direct use of model output in production contexts with extremely high code security requirements (manual Code Review is required).

Summary and Outlook

CodeQwen1.5-7B is an important attempt by the Alibaba Tongyi Qianwen team in the direction of code-specific models. It was also one of the strongest open source code models with a 7B parameter level at the time. Its core advantages lie in: deep code understanding capabilities brought by pre-training on 3 trillion token code data, 64K context window processing capabilities for warehouse-level code, and extensive coverage of 92 languages. Its performance on multiple code benchmarks such as HumanEval, MBPP, and SWE-Bench has confirmed its competitiveness.

Current limitations: CodeQwen is currently only 7B in size and lacks a version with larger parameter scale to hit higher benchmarks; the Research License is not commercial-friendly, and companies need to apply for additional authorization; the model was released relatively early (April 2024), and the subsequent Qwen3 series has built-in code capabilities into the general model.

Procurement and Adoption Risk Assessment: For teams that want to privatize the deployment of code assistants, CodeQwen1.5 is a low-risk starting solution - the open source weight can be obtained for free at 7B, the scale can be run on consumer-grade graphics cards, and the community ecosystem is complete. It is recommended to use Ollama or Transformers to perform PoC verification locally to confirm that the code generation quality meets business requirements, and then evaluate commercial authorization requirements. For teams pursuing the latest coding capabilities, the Qwen3-Coder series can be considered as a successor solution. Before all production environments are used, a code security audit process should be established to conduct necessary security review and test coverage on the model-generated code.

Related tools: github-copilot, cursor

Version Info

• CodeQwen1.5-7B-Chat ：The first public version is based on Qwen1.5 architecture, 7B parameters, supports 92 programming languages ​​and 64K contexts, and reaches 83.5% pass@1 on HumanEval. （2024-04-16）
• CodeQwen1.5-7B ：The code base model (Base) is pre-trained on approximately 3 trillion token code data and supports code completion and fine-tuning. There is no official precise date yet, please refer to the blog release date of 2024-04-16. （2024-04-16）