🛒 AI Token transfer station construction and operation solutions for technical teams and enterprise IT managers, covering API aggregation gateway construction, multi-key load balancing, cost monitoring and budget control, access security management and open source solution deployment.

AI Token transfer station construction and operation plan

Solution overview

With the deep penetration of large model APIs in business scenarios such as R&D, operations, and customer service, the number of internal calls to many large model APIs such as OpenAI, Claude, DeepSeek, and Tongyi Qianwen has increased exponentially. Each model manufacturer has its own independent access method, pricing system, key management strategy and rate limit, causing the R&D team to be tired of maintaining multiple sets of API integrations, and managers facing cost control and security risks. AI Token transfer station (API Proxy/Relay) is the unified gateway infrastructure created to solve the above problems.

This solution is aimed at technology companies, startups and SaaS product teams with R&D teams of more than 5 people and average monthly API calls exceeding one million Tokens. It provides a complete implementation path to build a Token transfer station from scratch. The solution covers open source gateway selection and deployment, multi-model aggregation access, intelligent routing and load balancing, cost monitoring and budget control, access security and auditing, as well as daily operation and maintenance and continuous optimization. Expected benefits include: reducing API call costs by 20-40%, shortening the R&D integration cycle from days to minutes, and achieving unified visualization of usage across the entire team.

Target users: Technical team leaders, DevOps engineers, AI Infra engineers, enterprise IT managers.

Prerequisites:

• Have basic operating capabilities of Linux server or container orchestration (Docker/K8s)
• Have an API Key from at least one large model manufacturer (such as OpenAI API or DeepSeek)
• Understand basic network concepts (domain name, reverse proxy, HTTPS)
• Monthly API budget is no less than RMB 500 (with room for cost optimization)

Toolchain list

Preparation

Before starting implementation, please confirm the following preparations one by one:

• [ ] Apply for API Keys from at least 2 large model manufacturers (recommended ≥3 to experience routing capabilities)
• [ ] Prepare a Linux server (2 cores 4G or above, 4 cores 8G recommended) or Kubernetes cluster
• [ ] Install Docker and Docker Compose (version ≥20.10)
• [ ] Prepare a domain name (optional, for HTTPS access and reverse proxy)
• [ ] Budget cap and maximum concurrency per model determined
• [ ] Internally confirmed API call compliance policy and data security boundary

Step-by-step guide

Step 1: Requirements Assessment and Architecture Design

⏱ Estimated time: 0.5-1 day 🎯 Goal: Clarify the access model, estimate the call volume, and determine the deployment architecture ⚠️ Prerequisites: None

Operation instructions

The architectural design of the Token transfer station directly determines the subsequent deployment scale and operating costs. Don’t blindly choose a deployment option without doing a capacity assessment—the staging architecture required for a small toolchain team versus an internal AI platform facing hundreds of business users is vastly different.

Specific operations

1. Inventory of existing model calls: Count the types of models currently used by the team (such as GPT-4o, Claude Sonnet, DeepSeek-V4, etc.), and record the average daily requests, average number of input/output tokens, and number of users of each model.
2. Clear access goals: Determine the model vendors that need to be aggregated by the transfer station (including at least OpenAI API, Claude, DeepSeek, Tongyi Qianwen and other mainstream manufacturers), as well as models that may be connected in the future.
3. Determine the deployment scale:
   • Team level (≤50 users, daily average ≤1 million Tokens): stand-alone Docker deployment, no K8s required
   • Department level (50-500 users, average daily 1 million-10 million Tokens): multi-node deployment + Redis cluster
   • Enterprise level (500+ users, daily average ≥10 million Tokens): K8s cluster + independent monitoring and logging platform
4. Choose an open source gateway solution:
   • Priority recommendation One API (GitHub 25K+ Stars): mature community, complete documentation, suitable for most technical teams
   • Choose New API (One API community branch) when you need more model support and more granular billing
   • Choose LiteLLM when minimalist deployment (pip install) is required, suitable for Python technology stack teams
   • If you don’t want to operate and maintain yourself, you can choose OpenRouter SaaS service

Verification method

Output the "Token Transfer Station Architecture Design Document", including: access model list, estimated concurrency and storage requirements, deployment architecture diagram, and reasons for selection. The team technical review passed.

Step 2: Open source gateway deployment and initialization

⏱ Estimated time: 1-2 days 🎯 Goal: Complete the basic deployment and initial configuration of the gateway service ⚠️ Prerequisites: Server is ready, Docker installation is complete, domain name (optional) DNS points to the server

Operation instructions

Take One API (or New API) as an example to show the standard deployment process. One API is currently the most widely used open source project in the field of domestic AI Token transfer stations. Its Docker one-click deployment mode reduces the deployment threshold from hours to 10 minutes.

Specific operations

1. Get deployment files:

   # Pull One API Docker image
   docker pull justsong/one-api
   
   # Or use New API (community enhanced version)
   docker pull ghcr.io/songquanpeng/new-api

2. Start via Docker Compose (recommended):

   # docker-compose.yml
   version: '3.8'
   services:
    one-api:
      image: justsong/one-api
      container_name: one-api
      restart: always
      ports:
        - "3000:3000"
      volumes:
        - ./data:/data
      environment:
        - SESSION_SECRET=your-secret-key
        - SQL_DSN=one-api.db
        - REDIS_CONN_STRING=redis://redis:6379/0
    redis:
      image: redis:7-alpine
      container_name: one-api-redis
      restart: always
      ports:
        - "6379:6379"
      volumes:
        - ./redis-data:/data

3. Initial access:

   • Visit http://yourserverIP:3000 -Default administrator account: root, password: 123456
   • Change the default password immediately after logging in for the first time

4. Configure HTTPS (required for production environment): Using Nginx reverse proxy, it is recommended to use acme.sh or certbot to automatically apply for a Let's Encrypt certificate:

   # /etc/nginx/sites-available/relay.yourdomain.com
   server {
      listen 443 ssl;
      server_name relay.yourdomain.com;
      ssl_certificate /etc/letsencrypt/live/relay.yourdomain.com/fullchain.pem;
      ssl_certificate_key /etc/letsencrypt/live/relay.yourdomain.com/privkey.pem;
      location/{
          proxy_pass http://127.0.0.1:3000;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header X-Forwarded-Proto $scheme;
      }
   }

5. Configuration data persistence:

   • SQLite is suitable for small scale (single file /data/one-api.db)
   • MySQL/PostgreSQL is suitable for medium and large scale (database service needs to be started separately)
   • Redis must be configured for caching and rate limiting

Verification method

• Visit https://relay.yourdomain.com to log in to the management panel normally
• docker ps confirms that both one-api and redis containers are running normally
• You can log in again normally after changing the default password

Step 3: Upstream model access and routing configuration

⏱ Estimated time: 0.5-1 day 🎯 Goal: Complete API Key configuration and routing strategies for all upstream model vendors ⚠️ Prerequisites: The gateway deployment is completed and you have the API Key of each manufacturer.

Operation instructions

This step is the core value of the Token transfer station—unifying the scattered vendor API keys into a management platform, and then exposing them to the entire team through a transfer station address. Each team member only needs to remember one API address and no longer needs to apply for and manage vendor keys individually.

Specific operations

1. Add channel in the management panel: Enter the management panel → Channel → Add channel, configure in order by model manufacturer:

   Manufacturer	Type	Model	Recommended Key Quantity
   OpenAI API	OpenAI	gpt-4o / gpt-4.1 / o3-mini	3-5 (load balancing)
   Claude	Anthropic	claude-sonnet-4 / claude-opus-4	2-3
   DeepSeek	DeepSeek	deepseek-chat / deepseek-reasoner	3-5
   Tongyi Qianwen	Alibaba Cloud DashScope	qwen-max / qwen-plus	2-3

2. Configure Key load balancing policy:

   • Round-Robin: distribute requests evenly, suitable for multiple Keys of the same specification
   • Weighted polling: the primary key carries 70% of the traffic, and the backup key carries 30%
   • Failover: Automatically switch to the backup key when the primary key times out or returns an error
   • Minimum latency: Automatically select the Key with the fastest response (requires gateway version support)

3. Configure model routing mapping:

   • Unify the model names exposed to the outside world, for example, map gpt-4o, claude-sonnet-4-20250514, etc. to user-friendly short names.
   • Configure Alternate Model: Automatically downgrade to an alternative when the preferred model quota is exhausted (e.g. gpt-4o → gpt-4o-mini)
   • Configure Cost Priority Routing: Allow users to choose the "cheapest" model for non-critical tasks

4. Create user and token:

   • Create user groups according to team roles (development group, operations group, management group)
   • Generate an independent API Key for each user (different from the upstream manufacturer Key)
   • Configure available model ranges and quota caps for each user

Verification method

• Use curl to test relay API calls:

  curl https://relay.yourdomain.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your transfer station Key" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

• Call 10 times continuously to confirm that Key polling takes effect (can be viewed in the management panel log)
• Deliberately using a wrong Key to confirm that the failover mechanism is triggered normally

Step 4: Cost control and usage monitoring system

⏱ Estimated time: 1 day 🎯 Goal: Establish usage monitoring, budget alarm and cost analysis system ⚠️ Prerequisites: Model access and routing configuration completed

Operation instructions

Controllable costs are the key difference between Token transfer stations and bare APIs from manufacturers. A transfer station without cost control is just a new call entrance, but a transfer station equipped with a complete monitoring and budgeting system can truly help managers control AI expenditures.

Specific operations

1. Configure usage statistics:

   • One API management panel has built-in complete "Log" and "Statistics" modules
   • View Token consumption by time range (today/this week/this month), by user, and by model
   • Export CSV data for financial reconciliation

2. Set budget alarm:

   • Set daily quota and monthly quota for each user/group
   • Configure overage processing policy: reject when exceeded / downgrade to cheaper model / notify administrator for approval
   • Set Global Budget Upper Limit: Automatically notify when the total consumption of the month reaches the threshold

3. Configure cost routing policy:

   • Define model price list (manually configure the cost per million Tokens of each model)
   • For non-critical business scenarios, create "economy mode" routing: automatically selects the cheapest available model
   • Scheduled tasks: Summarize the previous day's costs every early morning and send reports

4. Integrate external monitoring (optional, recommended for medium and large-scale deployment):

   • Export gateway metrics to Prometheus
   • Create a visual panel in Grafana: real-time QPS, Token consumption trend, cost proportion of each model, and delay distribution
   • Configure alarm rules: Single user's daily consumption surges by 300%, and overall availability is lower than 99%

Cost Optimization Benefit Estimation

Verification method

• Create a test user and set the daily quota to 1000 Tokens. After confirming that the quota is exceeded, the user will be rejected and receive a clear error message.
• Sending the same request using two models with different pricing confirms that cost routing is performed as configured
• Check the statistics panel to confirm that yesterday's usage data is accurate

Step 5: Security hardening and access control

⏱ Estimated time: 0.5-1 day 🎯 Goal: Improve API Key management, IP whitelist, audit logs and data security ⚠️ Precondition: User and token system have been created

Operation instructions

The Token transfer station concentrates the API Key and call traffic of the entire team. Once compromised, it may lead to Key leakage, budget theft and even data leakage. Security hardening is not optional, but a prerequisite for production deployment.

Specific operations

1. API Key Security Policy:

   • Upstream manufacturer Key encrypted storage: ensures that even if the database is stolen, the Key cannot be restored directly
   • User Key can be rotated regularly and supports setting expiration time
   • Disable the complete Key from being displayed in clear text on the front-end page (supported by default)

2. IP and Network Access Control:

   • Configure Nginx or gateway level IP whitelist: only allow access from company egress IP or VPN IP
   • For mobile office scenarios, configure Cloudflare Access or similar zero-trust proxy
   • Disable public access to admin panel (restrict /admin path via Nginx)

3. Audit Log:

   • Enable complete request logging: record the user, model, number of tokens, time taken, and status code of each call
   • Log retention policy: 30 days online retention, 1 year archive retention
   • Configure abnormal behavior alarms: calling the same Key from multiple IPs in a short period of time, high-frequency calls in the early morning, etc.

4. Data Compliance:

   • Clearly inform users that the transfer station will record request metadata, but will not store the complete request/response body (unless content auditing is turned on)
   • Configure data transmission encryption: Make sure both the admin panel and API entrance use HTTPS
   • Confirm data export compliance with legal affairs: When using domestic models (Tongyi Qianwen, DeepSeek) to call domestic APIs, the data does not cross the border

Verification method

• Use an IP that is not in the whitelist to call the API and confirm that it is correctly rejected.
• View the audit log to find all call records in the past 24 hours
• Try to access the database through SQL injection and other means, and confirm that the Key field is stored encrypted

Step 6: Cache acceleration and performance optimization

⏱ Estimated time: 0.5-1 day 🎯 Goal: Configure response caching, streaming optimization and connection pool reuse ⚠️ Precondition: Redis service is running normally

Operation instructions

For a large number of repeated system prompts, fixed template questions or monitoring queries, caching can significantly reduce the cost and delay of repeated requests. The response time after a cache hit in non-streaming scenarios can be reduced from seconds to milliseconds.

Specific operations

1. Configure request cache:

   • Enable the "caching" function in the One API management panel
   • Configure cache TTL (recommended 300-600 seconds, adjusted according to business scenarios)
   • Note: Streaming requests (stream=true) are not cached by default

2. Streaming performance optimization:

   • Configure Nginx's proxy_buffering off; to ensure that the SSE stream is not interrupted
   • Adjust the gateway's connection pool size (default 100, can be increased according to the amount of concurrency)
   • Enable HTTP/2 to reduce connection establishment overhead

3. Database performance tuning:

   • SQLite is suitable for scenarios with an average daily consumption of less than 1 million Tokens.
   • If the scale exceeds this size, it is recommended to migrate to PostgreSQL and configure the connection pool (pgbouncer)
   • Regularly clean up expired logs: retain detailed logs for the past 30 days, and delete historical data after archiving

4. CDN acceleration (optional):

   • Deploy multiple transit station instances in multiple regions around the world
   • Use DNS intelligent resolution to route users to the nearest transit node
   • Or use Cloudflare Workers to do front-end gateway offloading and forwarding

Verification method

• Send the exact same non-streaming request twice, the first time should show "cache miss" and the second time should show "cache hit" with over 80% improvement in response time
• Send 50 concurrent requests at the same time to confirm that throughput and latency are within acceptable limits
• redis-cli info stats confirm cache hit rate

Step 7: Daily operation and maintenance and continuous optimization

⏱Estimated time: Ongoing (about 1 day for first time setup) 🎯 Goal: Establish operation and maintenance SOPs for daily inspections, version upgrades, and emergency response ⚠️ Prerequisites: All the above configurations are completed

Operation instructions

The launch of the Token transfer station is just the beginning. Upstream manufacturers' model updates, API version changes, price adjustments, and changes in user needs all require continuous investment in operation and maintenance to maintain the efficiency and stability of the transfer station.

Specific operations

1. Daily inspection list:

   • Daily: Check usage trends, see if there are any abnormal surges, and confirm that all model interfaces are available
   • Weekly: Review audit logs for bad requests, analyze cache hit ratio, check disk and memory usage
   • Monthly: Cost analysis report, user permissions review, Key rotation, gateway version check

2. Version upgrade process:

   # 1. View the current version and changelog
   docker exec one-api ./one-api -v
   # 2. Pull the latest image
   docker pull justsong/one-api:latest
   # 3. Back up data
   cp /data/one-api.db /data/one-api.db.bak.$(date +%Y%m%d)
   # 4. Restart the container
   docker compose down && docker compose up -d
   # 5. Verify upgrade
   curl https://relay.yourdomain.com/api/status

3. Emergency Response Plan:

   • Upstream vendor API failure: automatic switch to equivalent model of alternate vendor
   • Gateway itself fails: use health check script to automatically restart the service
   • Budget exhausted: After receiving the alarm, the administrator quickly adjusts the quota or increases the budget.
   • Security incident: Immediately revoke the suspected leaked Key and trace back the audit log to locate the cause

4. Continuous optimization direction:

   • Evaluate the price and performance ratio of each manufacturer's latest models every quarter and adjust cost routing strategies
   • Add new model access based on user feedback
   • Connect with the internal OA/monitoring system to realize automatic approval flow and work order processing

Verification method

• Simulate the scenario where all keys from upstream manufacturers become invalid and confirm that the downgrade strategy takes effect.
• Execute a complete version upgrade process to confirm that the data is intact
• Generate monthly cost analysis report to compare with the previous month

Expected results

Comparison of key indicators

Acceptance criteria

• [ ] At least 4 large model manufacturers have successfully accessed the API and passed the call test
• [ ] Each manufacturer must configure at least 2 Keys, and the load balancing strategy can be verified
• [ ] User management, quota control, and usage statistics functions are normal
• [ ] Budget alarm is correctly triggered before exceeding the limit
• [ ] IP whitelist access control takes effect
• [ ] Cache hit rate > 10% (depending on the business scenario)
• [ ] Audit log completely records data for more than 7 days
• [ ] The operation and maintenance SOP document has been written and the handover has been completed within the team

Frequently Asked Questions and Troubleshooting

Q: I am an individual developer and only have a few API call requirements. Is it necessary to build a transfer station? A: If you only use one vendor and the call volume does not exceed 100,000 Tokens per month, it is easier to use the vendor API directly. But if you use 2-3 models at the same time for comparative testing or different task offloading, a lightweight transfer station can help you manage Keys and record costs in a unified manner. It is recommended to use LiteLLM (pip install is enough) or directly use the OpenRouter SaaS service.

Q: What is the difference between One API and New API? How to choose? A: New API is a community-derived version of One API. Based on One API, it adds more model channel support (such as Azure, Vertex AI, Cloudflare Workers AI), a more complete billing panel, and a more friendly management interface. If you are deploying for the first time, it is recommended to choose New API directly; if you need maximum stability and a longer community verification cycle, choose One API.

Q: Does setting up a relay station mean that all API requests have to go through my server, increasing latency? A: Yes, the request will take one more hop. However, when deployed in the same area, the increased delay is usually within 3-10ms and is almost imperceptible. It is recommended to deploy the transfer station on a cloud service provider that is close to the main upstream API nodes (for example, domestic business is deployed on Alibaba Cloud, and Tongyi Qianwen and DeepSeek directed to Alibaba Cloud only increase the intranet delay).

Q: If the key of my upstream manufacturer is leaked elsewhere (not through my transit station), will my transit station be affected? A: As long as the Key is revoked in the transfer station management panel in time and replaced with a new one, it will not affect the normal use of the transfer station. The transfer station itself is not responsible for the security of the upstream Key, but the transfer station's audit logs can help you quickly locate leak time points and abnormal calls.

Q: How to ensure the availability of transfer stations? Do you need multi-node deployment? A: For team-level use (< 50 users), single-node deployment with Docker automatic restart strategy can achieve 99.5% availability. For enterprise-level use, it is recommended to adopt a multi-node + load balancer + database master-slave architecture, with availability up to 99.9%.

Q: Will the transfer station cache my conversation content? How to ensure data security? A: By default, One API / New API only records request metadata (user, model, number of tokens, time taken) and does not store the specific content of the request and response. If you turn on the content auditing function, the conversation content will be recorded in the log. At this time, you should ensure that the log storage is encrypted and complies with data compliance requirements. It is recommended to double check the log configuration after first deployment.

Q: Can I use a transfer station for API resale? A: Both One API and New API support user management and billing functions, and can support internal cost center settlement from a technical perspective. However, external resale involves terms of service compliance issues-the terms of service of OpenAI, Anthropic, etc. usually prohibit unauthorized resale of APIs. Recommended for compliance sharing only with internal teams or partners.

Cycle and cost estimation

Implementation cycle

Monthly operating costs

The above costs do not include upstream large model API call fees. Upstream API costs vary greatly depending on usage. Through the optimized routing and caching strategies of this solution, upstream costs can be reduced by 20-40%.

Analysis of advantages and disadvantages

Advantages

1. Unified access portal: Teams only need one API address to call multiple models, reducing integration complexity and reducing the coupling of business code to manufacturer APIs.
2. Visible and controllable costs: The complete usage statistics, budget alarm and cost routing system allows managers to transform from "black box expenditure" to "quantitative management".
3. High availability architecture: Multi-Key load balancing + failover + backup model degradation, the impact of a single point of failure on the business is reduced from hours to seconds.
4. Security centralized management and control: The trinity of user-level API Key, IP whitelist, and audit logs greatly reduces the scope of losses after the Key is leaked.
5. Open and extensible: The open source gateway supports custom channel plug-ins, which can quickly connect to new manufacturers or internal self-built model inference services.

Disadvantages and Risks

1. Operation and maintenance dependence: The transfer station itself requires continuous server maintenance and version upgrades, which increases the team's operation and maintenance burden. If the team does not have a DevOps role, it can lead to lagging gateway versions and delayed security vulnerability remediation.
2. Extra one-hop delay: The request passes through the transit station and increases the network path. Although the delay increase is usually negligible (3-10ms), it may be perceived in real-time conversation scenarios with extremely low latency requirements.
3. Single point of failure risk: If the transfer station itself goes down and is not configured for high availability, the entire team's AI API calls will be interrupted. Must be paired with health checks and automatic recovery mechanisms.
4. Cost accounting deviation: The pricing of upstream manufacturers changes frequently, and the price list of the transfer station needs to be updated in a timely manner, otherwise the cost report and the actual bill may be different.
5. Blurred Compliance Boundaries: The logs of transfer records may involve compliance issues such as data export and privacy protection, and require legal confirmation before being officially launched online.

Tool summary

Next action

If you have confirmed that this plan is suitable for your team, it is recommended to proceed at the following pace:

• Week 1: Complete steps one to three, and run through the entire process from "gateway deployment" to "multi-model invocation" in the test environment
• Week 2: Complete steps four through six, configure monitoring, security, and caching, and invite 2-3 early adopters for beta testing
• Week 3: Optimize configuration based on Beta feedback, write operation and maintenance documents, and promote to the whole team
• Week 4 and beyond: Enter daily operation and maintenance mode, continue to track cost optimization effects, and evaluate model and gateway version updates quarterly