DeepInfra is an platform that provides OpenAI-compatible API interfaces, hosts 100+ open source models (LLM, vision, speech, image generation, embedding, etc.), and supports private model deployment and GPU cluster leasing. The platform has a self-built data center, SOC 2 / ISO 27001 certification, and zero data retention strategy. Complete $107 million Series B financing in 2026.

DeepInfra

Core parameters and statistics

DeepInfra is positioned as an AI inference cloud infrastructure, providing OpenAI-compatible inference APIs to developers and enterprises. It is not a single model, but a model hosting and inference service platform, covering multiple capabilities such as text generation, visual understanding, image generation, speech recognition, embedding and reordering.

Projects	Public Information
Official positioning	AI inference cloud — OpenAI-compatible API, 100s of open-source models
Platform form	Inference API, private model deployment, GPU cluster rental
Model scale	100+ open source models (LLM, multi-modal, image, speech, embedding)
API compatible	OpenAI SDK compatible (base_url can be replaced and used)
Service Tiers	Standard (1x), Priority (1.5x), Flex (0.8x)
Concurrency upper limit	Default 200 concurrent requests/account (can apply for improvement)
First Token Latency (TTFT)	Undisclosed general data; Priority level is significantly lower than Standard, the details vary depending on the model and load, and are subject to actual testing
Token throughput (TPM/RPM)	The general frequency control value is not disclosed; the platform is billed by token and there is no hard rate limit, but the 200 concurrency limit indirectly restricts the throughput
Compliance Certification	SOC 2, ISO 27001
Data Center	8 self-built data centers in the United States
Data strategy	Zero retention
Company establishment	September 2022
Cumulative financing	US$107 million Series B (2026-05)
Support Platform	Web Dashboard, REST API

Service level mechanism: DeepInfra's three service levels are distinguished by request scheduling priority. Standard is the default scheduling, Priority queues requests before standard traffic to obtain faster first token latency (price 1.5 times), and Flex offers a lower price in exchange for slow response and occasional unavailability (price 0.8 times), which is suitable for non-production or asynchronous scenarios.

Covered model categories: The platform simultaneously hosts multiple types of models such as text generation, vision/OCR, embedding/reordering, image generation (FLUX series), speech recognition (Whisper/Voxtral), video generation, etc. All of them can be called with one API Key.

User and market recognition

DeepInfra's market recognition is mainly reflected in three levels: developer ecosystem, corporate customers and capital market:

Developer Adoption: DeepInfra is the inference provider with the largest number of hosted models on OpenRouter. The platform provides OpenAI compatible API. Developers only need to point base_url to https://api.deepinfra.com/v1/openai to migrate with zero code changes. The GitHub organization has 47 repositories, including the official TypeScript/Node SDK, Python SDK (community maintained), LangChain integrated CLI tool deepctl, and a rich library of cookbook examples.

Enterprise customers: The official website publicly displays well-known users including OpenRouter, Quora, Vercel, Adobe, Ubisoft, Meta, ByteDance, LinkedIn, Microsoft, Deloitte, Salesforce, Hugging Face, Rakuten, etc., covering technology, consulting, finance, games and other fields. These customers use DeepInfra to power production inference workloads for their AI products.

Capital Markets: Completed $107 million Series B financing in May 2026, co-led by 500 Global and Georges Harik (co-founder of imo.im), with participation from NVIDIA, Samsung Next, Supermicro, Felicis, Crescent Cove, A.Capital Ventures, Peak6, Upper90, etc. Token processing volume has increased 25x since Series A.

Industry Recognition: Establishing early infrastructure collaboration with NVIDIA to support the Nemotron model NemoClaw Agent framework and NVIDIA Dynamo inference software, and being the first to deploy Blackwell GPUs.

Cost advantage

DeepInfra's cost structure is designed around "pay only for the computing you actually use" with no minimums or long-term contracts.

C-side/individual: DeepInfra does not provide chat products directly for C-side (its independent product DeepGPT may meet this demand). Individual developers use it through API, billed by Token or execution time, and there is no monthly fee threshold. Embed models start as low as $0.005/million Tokens.

API/Developer: LLM is billed separately by input/output token, and supports cache hit discounts (for example, DeepSeek V4-Pro cache input is only $0.10/million tokens, which is much lower than standard input $1.30). Image models are charged based on the generated resolution and number of iterations (e.g. FLUX-2-dev $0.01 × (w/1024) × (h/1024) × (iters/28)). Speech models are billed based on execution time. Taking DeepSeek V4-Flash as an example, the input is $0.09/million Tokens and the output is $0.18/million Tokens, which is a very competitive price in the open source model inference market.

Model Example	Context	Input Price ($/M Token)	Output Price ($/M Token)
DeepSeek-V4-Flash	1024k	$0.09 ($0.018 cached)	$0.18
DeepSeek-V4-Pro	1024k	$1.30 ($0.10 cached)	$2.60
Qwen3.6-35B-A3B	256k	$0.15	$0.95
Llama-4-Maverick-17B-128E	1024k	$0.20	$0.80
gemma-4-26B-A4B-it	256k	$0.07	$0.34
Mistral-Small-3.2-24B	125k	$0.075	$0.20

Enterprise/Privatization: Supports two privatization modes. The first is custom LLM deployment (Custom LLMs), which runs its own models on dedicated GPUs and is billed by GPU hours: A100 $0.89/h, H100 $2.20/h, H200 $2.69/h, B200 $3.69/h, B300 $4.89/h, billed at the minute level, and billed on a weekly basis. The second is DeepCluster exclusive cluster. The 5-year B300 cluster is only $1.98/GPU-h (compared to public cloud $6.50/GPU-h), a discount of about 70%. 256–5000 GPUs are available, including Tier 3 data center and 99.982% availability SLA.

Hidden costs: The hidden costs of the API model mainly include the concurrency limit during high-frequency calls (default 200 concurrency), the communication cost of contacting the business to increase the quota after exceeding it, and the 1.5 times increase in price at the Priority level. Enterprise privatization deployment also needs to evaluate supporting investments in computer room space, power and network bandwidth.

Main functions

DeepInfra's capabilities are designed around "one API to call all models". Core functions include:

• OpenAI Compatible Reasoning API: Supports text generation, multi-modal understanding, embedding, and reordering of 100+ open source models. Using OpenAI SDK or any compatible client, just modify base_url and api_key to call. Supports streaming output JSON schema, tool calling/Function Calling, context caching and other features.

• Multi-modal and visual understanding: Supports image understanding, OCR text extraction, and document analysis of multi-modal models such as Gemma 4, Qwen-VL, and Kimi K2.6. Just pass in the image URL or Base64 via the standard Chat Completions API.

• Image and video generation: Hosts the full range of FLUX (FLUX-2-dev, FLUX-2-pro, FLUX-2-max, etc.), image generation models such as Stable Diffusion, and text-to-video models. Billing is based on resolution and number of iterations, with high flexibility.

• Embedding and reranking: Provides mainstream embedding models such as bge, e5, gte, and multilingual-e5, as well as reranking capabilities, suitable for RAG, semantic search, and information retrieval scenarios. As low as $0.005/million Tokens.

• Speech recognition and synthesis: Supports OpenAI Whisper series models and Voxtral series speech models, covering speech-to-text and text-to-speech scenarios.

• Private model deployment (Custom LLMs): Deploy your own fine-tuned model on DeepInfra's A100/H100/H200/B200/B300 GPU, with automatic expansion and contraction, and private endpoint isolation. Suitable for teams with high data sovereignty requirements or customized reasoning logic.

• GPU Cluster Rental (DeepCluster): Provides B300 exclusive cluster in the form of a 5-year long-term contract with direct SSH connection, suitable for training, fine-tuning, or other scenarios that require continuous GPU computing power.

Functional synergy: The real product power of DeepInfra lies not in a single function, but in the combination of "single API entry + multiple model types + flexible switching of service levels". The development team does not need to connect with different suppliers for image generation, text reasoning, and embedded retrieval. One integration can cover multiple AI loads. The overlay of Private models and GPU clusters allows enterprises to complete the entire AI workload package from experimentation to production, and from inference to training on the same platform.

Model and version evolution

Continuous iterative updates, the latest version introduces performance optimization and new features. Historical version information can be viewed on the official release page. There is no complete public version evolution timeline yet. It is recommended to pay attention to the official announcement to understand the rhythm of feature updates.

Technical advantages

DeepInfra's technical competitiveness comes from the system-level engineering capabilities of "vertical integration + inference optimization". Each capability revolves around the causal chain of "why it is faster/more economical/more stable":

Own infrastructure -> Eliminate middle-tier price increases and resource competition: Unlike competing products that build inference layers on public clouds (AWS/GCP/Azure), DeepInfra self-purchases and operates GPU hardware in 8 data centers in the United States, with full-stack auto-control from chips to APIs. This architecture produces structural advantages at three levels - in terms of cost, it eliminates the markup layer of cloud vendors (take DeepCluster B300 as an example, $1.98/GPU-hr compared to public cloud $6.50/GPU-hr); in terms of latency, it avoids fluctuations caused by overbooking of physical machines; in terms of capacity, it can quickly complete large-scale deployment after the release of a new model without being restricted by cloud vendor quotas. Applicable scenarios: High-throughput, continuously running Agent inference and production-level LLM services.

Inference optimization software stack -> Higher GPU utilization and lower cost per token: The team is deeply involved in the engineering development of NVIDIA TensorRT-LLM and vLLM (both have official forks on GitHub), and also has joint optimization with the NVIDIA Dynamo distributed inference framework. This means that DeepInfra can achieve higher GPU utilization than general cloud platforms at the operator level (FlashAttention, PagedAttention, Continuous Batching) and scheduling level (KV cache management, dynamic batch processing). The higher the utilization rate, the lower the fixed cost amortization per million Tokens, which is ultimately reflected in the terminal price advantage. Applicable scenarios: Large-scale inference scenarios that are sensitive to Token unit price.

NVIDIA Ecosystem Deep Collaboration -> Be the First to Get Next-Generation Hardware Dividends: As an early infrastructure partner of NVIDIA’s open AI ecosystem, DeepInfra is the first to deploy Blackwell B200/B300 GPUs and plans to achieve a 20-fold improvement in inference cost efficiency through Dynamo on the Vera Rubin architecture. This partnership not only means priority in hardware selection, but more importantly, DeepInfra can perform adaptation and optimization of NVIDIA's next-generation inference software stack (such as Dynamo, NemoClaw) while it is still in the development stage, instead of passively following up after GA. Applicable scenarios: Enterprises that need to take advantage of the latest GPU architecture dividends for long-term cost optimization.

Automatic expansion and contraction -> Zero idle cost: The platform's built-in automatic expansion and contraction mechanism dynamically adjusts the number of model copies based on the request load. Automatically add replicas during traffic peaks to ensure response speed, and release replicas during low traffic periods to avoid idle GPU billing. Combined with Custom LLM’s minute-level billing granularity, users only pay for the actual GPU time used. Applicable scenarios: AI applications with large fluctuations in traffic patterns, such as e-commerce promotions, social fission, etc.

Service level scheduling -> Differentiated resource allocation does not affect the overall cost: Standard/Priority/Flex three-level scheduling is not a simple rate limit, but achieves differentiated resource allocation through request queue priority management. The price premium of Priority 1.5x comes from moving requests to the front of the queue, not from increasing the total amount of hardware. Flex 0.8x price discounts are achieved by scheduling requests during low load periods or using idle capacity. This design provides users with cost-latency independent choices without significantly increasing the total cost of the platform. Applicable scenarios: Multi-load mixed scenarios (Priority for production traffic, Flex for background batch processing).

Adaptation Boundary: The architectural strength of DeepInfra is in inference rather than training - it does not provide a model pre-training platform, nor does it directly provide one-click fine-tuning services (users need to prepare their own weights and then deploy them through Custom LLM). For teams that require one-stop "training + inference" services, an external training framework may be needed. In addition, the current data center is limited to 8 nodes in the United States. For Asia-Pacific and European customers, network latency may become a bottleneck. It is recommended to conduct cross-region latency testing before actual deployment.

How to use

The core usage path of DeepInfra is API calling, and it also provides a webpage Dashboard for model browsing, API Key management and usage monitoring.

Quick start

1. Visit deepinfra.com to register an account.
2. Generate API Key (https://deepinfra.com/dash/api_keys) in Dashboard.
3. Use OpenAI SDK or directly initiate an HTTP request to call the model.

API call example

Use Python OpenAI SDK to call DeepSeek-V4-Flash:

from openai import OpenAI

client = OpenAI(
    api_key="<YOUR_DEEPINFRA_API_KEY>",
    base_url="https://api.deepinfra.com/v1/openai",
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Flash",
    messages=[{"role": "user", "content": "Hello! what is deepinfra、"}],
    temperature=0.7,
    max_tokens=1024,
    stream=True,
)
for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

Curl call example:

curl -s https://api.deepinfra.com/v1/openai/chat/completions \
  -H "Authorization: Bearer <YOUR_DEEPINFRA_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V4-Flash",
    "messages": [{"role": "user", "content": "Hello!"}],
    "temperature": 0.7,
    "max_tokens": 1024,
    "stream": true
  }'

Service level settings

Specify scheduling priority via the service_tier parameter in the API request:

{
  "model": "deepseek-ai/DeepSeek-V4-Flash",
  "messages": [{"role": "user", "content": "Hello!"}],
  "service_tier": "priority"
}

Optional values: "standard" (default, 1x price), "priority" (1.5x price), "flex" (0.8x price, non-production scenario).

How to use	Suitable for people	Features
REST API (OpenAI compatible)	Developer/integrator	Replace base_url to migrate, SDK supports Python, Node.js, LangChain
Web Dashboard	Evaluation/Experimentation	Browse models, generate API Keys, view usage and bills
deepctl CLI	DevOps/SRE	Command line management of inference tasks and clusters
Custom LLMs	Enterprise/Model Owner	Deploy private fine-tuned models and automatically scale up and down
DeepCluster	High Performance Computing Team	B300 dedicated cluster, SSH direct connection, 5-year contract

Product Pricing

The pricing model is subject to the official real-time page. Usually a freemium or subscription system is used, and basic functions can be used for free. Advanced functions or high-frequency use require paid subscriptions, and users are advised to evaluate the optimal solution based on actual usage.

Application scenarios

DeepInfra's typical implementation scenarios cover the full spectrum from personal experiments to enterprise production:

• Production-level LLM inference: Enterprises integrate open source models such as DeepSeek, Qwen, and Llama into production processes such as customer service systems, content generation, and code assistance. OpenAI-compatible APIs make migration costs nearly zero, and the benefits are reflected in reduced costs and increased model autonomy after switching from closed-source APIs to open-source models.

• RAG and Knowledge Retrieval: Combining DeepInfra's embedding models (bge, e5 series) and reordering capabilities to build a retrieval enhancement generation system. The same API Key calls embedding and generating models, simplifying the technology stack. Pay attention to the maximum token limit (512 or 8K) and multi-language adaptability of different embedding models.

• Image and video generation: Use FLUX series models to batch generate e-commerce product images, advertising creatives, and concept designs. Billing is based on resolution and number of iterations, which is suitable for teams that need to produce images frequently. The FLUX-2-klein series is a lightweight model that can be generated faster and at lower cost.

• Multimodal document understanding: Use Gemma 4, Qwen-VL and other models for invoice recognition, contract information extraction, and form OCR. Document screenshots are passed in directly through the Vision API, eliminating the need for a pre-processing pipeline.

• Agent and tool calling: DeepSeek V4-Pro, Claude Fable 5 and other models support Function Calling and tool calling, which can be used to build AI Agent applications. Priority service tiers ensure low latency for Agent multi-step reasoning.

• Model fine-tuning and private deployment: Deploy your own fine-tuning model on DeepInfra GPU, combined with automatic expansion and contraction to serve production traffic. Suitable for enterprises that have data sovereignty requirements or need to customize model behavior.

Applicable people

DeepInfra's multi-layer service form covers the following four types of roles:

• AI application developers: Need to quickly integrate inference capabilities, focusing on API compatibility, model richness, and price. DeepInfra's OpenAI compatible API makes it possible to migrate from OpenAI with almost no code modifications, making it suitable for model selection verification and production deployment.

• DevOps and Infrastructure Team: Responsible for GPU cluster management and inference cost optimization. DeepInfra's deepctl CLI, private deployment, and DeepCluster options give it a complete gradient of control from managed APIs to bare metal GPUs.

• AI researchers and experimenters: need to frequently switch between different models for comparative evaluation. DeepInfra’s 100+ model library and unified API reduce the engineering overhead of multi-model evaluation.

• Enterprise AI Platform Leader: Evaluate inference vendors for compliance, security, and cost. DeepInfra's SOC 2 / ISO 27001 certification, zero data retention policy, and self-built data centers in the United States are key differentiators from public cloud APIs.

Not suitable for boundaries: DeepInfra is not suitable for scenarios that require deep binding to a single model vendor ecosystem (such as enterprise contracts that require Azure OpenAI), immediate needs for global multi-region deployment (currently mainly US data centers), or personal experiment scenarios that require very low inference and are not sensitive to latency (in this case, free or cheap alternatives are more suitable).

Summary and Outlook

DeepInfra's core competitiveness lies in the vertical integration of "open source model ecosystem + OpenAI compatible API + self-built inference infrastructure". It is not the cheapest inference API (some model Flex tiers can achieve very low cost), nor is it the most feature-rich (no model training services are provided), but for development teams and enterprises that need to use open source models at scale in production environments, it provides a rare balance in the current market between compatibility, model breadth, and infrastructure controllability.

Current limitations: Data centers are currently concentrated in the United States, with limited global coverage; the default 200 concurrency limit requires additional business communication for high-traffic scenarios; the platform itself does not provide model training or fine-tuning services, and only focuses on the inference layer.

Procurement/Adoption Risk Assessment: For entrepreneurial teams and small and medium-sized developers, it is recommended to first use the Standard level + pay-as-you-go prototype to run through the prototype, verify API compatibility and model effects, and then evaluate whether Priority level or private deployment is needed based on actual usage. Before purchasing, enterprises need to confirm: the concurrent quota increase process, the specific terms of the data retention strategy (especially whether input and output are used for service improvement), the early termination conditions of the DeepCluster contract, and future coverage plans for non-U.S. data centers. For teams already using the OpenAI API, DeepInfra is a low-migration cost alternative to open source models, but should not be considered a complete replacement for closed source models - some scenarios (e.g. very long context role-playing, domain-specific closed source models) will still require retaining multiple vendors.

Related tools: hugging-face, replicate

Platform version evolution of DeepInfra

As a SaaS platform, DeepInfra does not have a software version number in the traditional sense. Its evolution is reflected in model support expansion, hardware upgrades and financing milestones.

Initial stage of the platform (2022–2024)

• September 2022: Company founded, founded by ex-imo team.
• 2023–2024: Launch initial version of inference API, gradually add open source model support, and establish US data center infrastructure.

High-speed expansion stage (2025)

• Model library expanded to 100+: covering mainstream open source LLM, multi-modal, image generation and speech models.
• Hardware upgrade: Introducing H200 and B200 GPU support to improve inference throughput and energy efficiency.
• Compliance Certification: Obtained SOC 2 and ISO 27001 certification, and implemented a zero data retention strategy.

Series B Milestones (2026-05)

• May 4, 2026: Announced $107 million in Series B financing, with Token processing volume increasing 25 times compared to Series A.
• DeepCluster Release: Launched B300 exclusive GPU cluster rental service, 5-year contract $1.98/GPU-h.
• Continuous access to new models: The latest models such as Anthropic Claude Fable 5, Claude Sonnet 5, GLM-5.2, Kimi K2.7-Code, Qwen3.6, etc. are all available online quickly after release.

Version Info

• DeepInfra Series B version ：Series B financing raised US$107 million; Token processing volume increased 25 times compared to Series A; DeepCluster B300 cluster was launched; Claude Fable 5/Sonnet 5, GLM-5.2, Kimi K2.7-Code and other models were added; 8 US data centers were operated. （2026-05-04）
• DeepStart Entrepreneurship Plan ：Launched a 1 billion free tokens program for start-ups, requiring financing of US$250,000-10 million and established no more than 2 years ago. There is no official precise date yet. （~2026-04）
• Series A Milestones ：Completed Series A financing; since then, Token processing volume has increased 25 times; expanded model library to 100+; obtained SOC 2/ISO 27001 certification. There is no official precise date and amount yet. （~2025）
• Initial version of platform ：DeepInfra was established, launched the initial version of the AI ​​inference API, and built an inference optimization infrastructure from scratch. （2022-09）