History

Add OpenAI compatibility support and improve resource cleanup

- Introduced OpenAI compatibility configurations for external providers, enabling model alias routing via the OpenAI API format. - Enhanced provider logic in `GetProviderName` to handle OpenAI aliases and added new helper functions for compatibility checks. - Updated API handlers and client initialization to support OpenAI compatibility models. - Improved resource cleanup across clients by closing response bodies and streams using deferred functions.

Luis Pater committed 1 year agov4.1.0

Commit ed8873f1 parent 9102ff018 files · +671 −72Browse files

Unified · Split

18 files
  1. README.md +80 −29
  2. README_CN.md +70 −22
  3. config.example.yaml +26 −9
  4. internal/api/handlers/claude/code_handlers.go +1 −1
  5. internal/api/handlers/gemini/gemini-cli_handlers.go +1 −1
  6. internal/api/handlers/gemini/gemini_handlers.go +1 −1
  7. internal/api/handlers/handlers.go +3 −1
  8. internal/api/handlers/openai/openai_handlers.go +1 −1
  9. internal/client/claude_client.go +4 −0
  10. internal/client/codex_client.go +4 −0
  11. internal/client/gemini-cli_client.go +6 −0
  12. internal/client/gemini_client.go +4 −0
  13. internal/client/openai-compatibility_client.go +360 −0
  14. internal/client/qwen_client.go +4 −0
  15. internal/cmd/run.go +12 −0
  16. internal/config/config.go +29 −0
  17. internal/constant/constant.go +6 −5
  18. internal/util/provider.go +59 −2
README.md+80−29
@@ -6,9 +6,9 @@ A proxy server that provides OpenAI/Gemini/Claude compatible API interfaces for · 5 unmodified lines
6 6
7It now also supports OpenAI Codex (GPT models) and Claude Code via OAuth.7It now also supports OpenAI Codex (GPT models) and Claude Code via OAuth.
8 8
9so you can use local or multi‑account CLI access with OpenAI‑compatible clients and SDKs.9So you can use local or multi-account CLI access with OpenAI-compatible clients and SDKs.
10 10
11Now, We added the first Chinese provider: [Qwen Code](https://github.com/QwenLM/qwen-code).11The first Chinese provider has now been added: [Qwen Code](https://github.com/QwenLM/qwen-code).
12 12
13## Features13## Features
14 14
@@ -19,12 +19,13 @@ Now, We added the first Chinese provider: [Qwen Code](https://github.com/QwenLM/ · 4 unmodified lines
19- Streaming and non-streaming responses19- Streaming and non-streaming responses
20- Function calling/tools support20- Function calling/tools support
21- Multimodal input support (text and images)21- Multimodal input support (text and images)
22- Multiple accounts with round‑robin load balancing (Gemini, OpenAI, Claude and Qwen)22- Multiple accounts with round-robin load balancing (Gemini, OpenAI, Claude and Qwen)
23- Simple CLI authentication flows (Gemini, OpenAI, Claude and Qwen)23- Simple CLI authentication flows (Gemini, OpenAI, Claude and Qwen)
24- Generative Language API Key support24- Generative Language API Key support
25- Gemini CLI multi‑account load balancing25- Gemini CLI multi-account load balancing
26- Claude Code multi‑account load balancing26- Claude Code multi-account load balancing
27- Qwen Code multi‑account load balancing27- Qwen Code multi-account load balancing
28- OpenAI-compatible upstream providers via config (e.g., OpenRouter)
28 29
29## Installation30## Installation
30 31
@@ -59,13 +60,13 @@ You can authenticate for Gemini, OpenAI, and/or Claude. All can coexist in the s · 28 unmodified lines
59 ```bash60 ```bash
60 ./cli-proxy-api --login61 ./cli-proxy-api --login
61 ```62 ```
62 If you are an old gemini code user, you may need to specify a project ID:63 If you are an existing Gemini Code user, you may need to specify a project ID:
63 ```bash64 ```bash
64 ./cli-proxy-api --login --project_id <your_project_id>65 ./cli-proxy-api --login --project_id <your_project_id>
65 ```66 ```
66 The local OAuth callback uses port `8085`.67 The local OAuth callback uses port `8085`.
67 68
68 Options: add `--no-browser` to print the login URL instead of opening a browser. The local OAuth callback uses port `1455`.69 Options: add `--no-browser` to print the login URL instead of opening a browser. The local OAuth callback uses port `8085`.
69 70
70- OpenAI (Codex/GPT via OAuth):71- OpenAI (Codex/GPT via OAuth):
71 ```bash72 ```bash
@@ -126,7 +127,7 @@ Request body example: · 54 unmodified lines
126```127```
127 128
128Notes:129Notes:
129- Use a `gemini-*` model for Gemini (e.g., `gemini-2.5-pro`), a `gpt-*` model for OpenAI (e.g., `gpt-5`), a `claude-*` model for Claude (e.g., `claude-3-5-sonnet-20241022`), or a `qwen-*` model for Qwen (e.g., `qwen3-coder-plus`). The proxy will route to the correct provider automatically.130- Use a `gemini-*` model for Gemini (e.g., "gemini-2.5-pro"), a `gpt-*` model for OpenAI (e.g., "gpt-5"), a `claude-*` model for Claude (e.g., "claude-3-5-sonnet-20241022"), or a `qwen-*` model for Qwen (e.g., "qwen3-coder-plus"). The proxy will route to the correct provider automatically.
130 131
131#### Claude Messages (SSE-compatible)132#### Claude Messages (SSE-compatible)
132 133
@@ -226,7 +227,7 @@ console.log(await claudeResponse.json()); · 93 unmodified lines
226- claude-3-5-haiku-20241022227- claude-3-5-haiku-20241022
227- qwen3-coder-plus228- qwen3-coder-plus
228- qwen3-coder-flash229- qwen3-coder-flash
229- Gemini models auto‑switch to preview variants when needed230- Gemini models auto-switch to preview variants when needed
230 231
231## Configuration232## Configuration
232 233
@@ -238,21 +239,28 @@ The server uses a YAML configuration file (`config.yaml`) located in the project · 5 unmodified lines
238 239
239### Configuration Options240### Configuration Options
240 241
241| Parameter | Type | Default | Description |242| Parameter | Type | Default | Description |
242|---------------------------------------|----------|--------------------|-----------------------------------------------------------------------------------------------|243|-----------------------------------------|----------|--------------------|---------------------------------------------------------------------------------------------------------|
243| `port` | integer | 8317 | The port number on which the server will listen |244| `port` | integer | 8317 | The port number on which the server will listen. |
244| `auth-dir` | string | "~/.cli-proxy-api" | Directory where authentication tokens are stored. Supports using `~` for home directory |245| `auth-dir` | string | "~/.cli-proxy-api" | Directory where authentication tokens are stored. Supports using `~` for the home directory. |
245| `proxy-url` | string | "" | Proxy url, support socks5/http/https protocol, example: socks5://user:pass@192.168.1.1:1080/ |246| `proxy-url` | string | "" | Proxy URL. Supports socks5/http/https protocols. Example: socks5://user:pass@192.168.1.1:1080/ |
246| `request-retry` | integer | 0 | Request retry times, if the response http code is 403, 408, 500, 502, 503, 504, it will retry |247| `request-retry` | integer | 0 | Number of times to retry a request. Retries will occur if the HTTP response code is 403, 408, 500, 502, 503, or 504. |
247| `quota-exceeded` | object | {} | Configuration for handling quota exceeded |248| `quota-exceeded` | object | {} | Configuration for handling quota exceeded. |
248| `quota-exceeded.switch-project` | boolean | true | Whether to automatically switch to another project when a quota is exceeded |249| `quota-exceeded.switch-project` | boolean | true | Whether to automatically switch to another project when a quota is exceeded. |
249| `quota-exceeded.switch-preview-model` | boolean | true | Whether to automatically switch to a preview model when a quota is exceeded |250| `quota-exceeded.switch-preview-model` | boolean | true | Whether to automatically switch to a preview model when a quota is exceeded. |
250| `debug` | boolean | false | Enable debug mode for verbose logging |251| `debug` | boolean | false | Enable debug mode for verbose logging. |
251| `api-keys` | string[] | [] | List of API keys that can be used to authenticate requests |252| `api-keys` | string[] | [] | List of API keys that can be used to authenticate requests. |
252| `generative-language-api-key` | string[] | [] | List of Generative Language API keys |253| `generative-language-api-key` | string[] | [] | List of Generative Language API keys. |
253| `claude-api-key` | object | {} | List of Claude API keys |254| `claude-api-key` | object | {} | List of Claude API keys. |
254| `claude-api-key.api-key` | string | "" | Claude API key |255| `claude-api-key.api-key` | string | "" | Claude API key. |
255| `claude-api-key.base-url` | string | "" | Custom Claude API endpoint, if you use the third party API endpoint |256| `claude-api-key.base-url` | string | "" | Custom Claude API endpoint, if you use a third-party API endpoint. |
257| `openai-compatibility` | object[] | [] | Upstream OpenAI-compatible providers configuration (name, base-url, api-keys, models). |
258| `openai-compatibility.*.name` | string | "" | The name of the provider. It will be used in the user agent and other places. |
259| `openai-compatibility.*.base-url` | string | "" | The base URL of the provider. |
260| `openai-compatibility.*.api-keys` | string[] | [] | The API keys for the provider. Add multiple keys if needed. Omit if unauthenticated access is allowed. |
261| `openai-compatibility.*.models` | object[] | [] | The actual model name. |
262| `openai-compatibility.*.models.*.name` | string | "" | The models supported by the provider. |
263| `openai-compatibility.*.models.*.alias` | string | "" | The alias used in the API. |
256 264
257### Example Configuration File265### Example Configuration File
258 266
@@ -266,10 +274,10 @@ auth-dir: "~/.cli-proxy-api" · 7 unmodified lines
266# Enable debug logging274# Enable debug logging
267debug: false275debug: false
268 276
269# Proxy url, support socks5/http/https protocol, example: socks5://user:pass@192.168.1.1:1080/277# Proxy URL. Supports socks5/http/https protocols. Example: socks5://user:pass@192.168.1.1:1080/
270proxy-url: ""278proxy-url: ""
271 279
272# Request retry times, if the response http code is 403, 408, 500, 502, 503, 504, it will retry280# Number of times to retry a request. Retries will occur if the HTTP response code is 403, 408, 500, 502, 503, or 504.
273request-retry: 3281request-retry: 3
274 282
275# Quota exceeded behavior283# Quota exceeded behavior
@@ -294,8 +302,51 @@ claude-api-key: · 18 unmodified lines
294 - api-key: "sk-atSM..." # use the official claude API key, no need to set the base url302 - api-key: "sk-atSM..." # use the official claude API key, no need to set the base url
295 - api-key: "sk-atSM..."303 - api-key: "sk-atSM..."
296 base-url: "https://www.example.com" # use the custom claude API endpoint304 base-url: "https://www.example.com" # use the custom claude API endpoint
305
306# OpenAI compatibility providers
307openai-compatibility:
308 - name: "openrouter" # The name of the provider; it will be used in the user agent and other places.
309 base-url: "https://openrouter.ai/api/v1" # The base URL of the provider.
310 api-keys: # The API keys for the provider. Add multiple keys if needed. Omit if unauthenticated access is allowed.
311 - "sk-or-v1-...b780"
312 - "sk-or-v1-...b781"
313 models: # The models supported by the provider.
314 - name: "moonshotai/kimi-k2:free" # The actual model name.
315 alias: "kimi-k2" # The alias used in the API.
316```
317
318### OpenAI Compatibility Providers
319
320Configure upstream OpenAI-compatible providers (e.g., OpenRouter) via `openai-compatibility`.
321
322- name: provider identifier used internally
323- base-url: provider base URL
324- api-keys: optional list of API keys (omit if provider allows unauthenticated requests)
325- models: list of mappings from upstream model `name` to local `alias`
326
327Example:
328
329```yaml
330openai-compatibility:
331 - name: "openrouter"
332 base-url: "https://openrouter.ai/api/v1"
333 api-keys:
334 - "sk-or-v1-...b780"
335 - "sk-or-v1-...b781"
336 models:
337 - name: "moonshotai/kimi-k2:free"
338 alias: "kimi-k2"
297```339```
298 340
341Usage:
342
343Call OpenAI's endpoint `/v1/chat/completions` with `model` set to the alias (e.g., `kimi-k2`). The proxy routes to the configured provider/model automatically.
344
345Also, you may call Claude's endpoint `/v1/messages`, Gemini's `/v1beta/models/model-name:streamGenerateContent` or `/v1beta/models/model-name:generateContent`.
346
347And you can always use Gemini CLI with `CODE_ASSIST_ENDPOINT` set to `http://127.0.0.1:8317` for these OpenAI-compatible provider's models.
348
349
299### Authentication Directory350### Authentication Directory
300 351
301The `auth-dir` parameter specifies where authentication tokens are stored. When you run the login command, the application will create JSON files in this directory containing the authentication tokens for your Google accounts. Multiple accounts can be used for load balancing.352The `auth-dir` parameter specifies where authentication tokens are stored. When you run the login command, the application will create JSON files in this directory containing the authentication tokens for your Google accounts. Multiple accounts can be used for load balancing.
@@ -327,8 +378,8 @@ export CODE_ASSIST_ENDPOINT="http://127.0.0.1:8317" · 25 unmodified lines
327The server will relay the `loadCodeAssist`, `onboardUser`, and `countTokens` requests. And automatically load balance the text generation requests between the multiple accounts.378The server will relay the `loadCodeAssist`, `onboardUser`, and `countTokens` requests. And automatically load balance the text generation requests between the multiple accounts.
328 379
329> [!NOTE] 380> [!NOTE]
330> This feature only allows local access because I couldn't find a way to authenticate the requests. 381> This feature only allows local access because there is currently no way to authenticate the requests.
331> I hardcoded `127.0.0.1` into the load balancing.382> 127.0.0.1 is hardcoded for load balancing.
332 383
333## Claude Code with multiple account load balancing384## Claude Code with multiple account load balancing
334 385
README_CN.md+70−22
@@ -6,9 +6,9 @@ · 5 unmodified lines
6 6
7现已支持通过 OAuth 登录接入 OpenAI Codex（GPT 系列）和 Claude Code。7现已支持通过 OAuth 登录接入 OpenAI Codex（GPT 系列）和 Claude Code。
8 8
9可与本地或多账户方式配合，使用任何 OpenAI 兼容的客户端与 SDK。9您可以使用本地或多账户的CLI方式，通过任何与OpenAI兼容的客户端和SDK进行访问。
10 10
11现在，我们添加了第一个中国提供商：[Qwen Code](https://github.com/QwenLM/qwen-code)。11现已新增首个中国提供商：[Qwen Code](https://github.com/QwenLM/qwen-code)。
12 12
13## 功能特性13## 功能特性
14 14
@@ -25,6 +25,7 @@ · 10 unmodified lines
25- 支持 Gemini CLI 多账户轮询25- 支持 Gemini CLI 多账户轮询
26- 支持 Claude Code 多账户轮询26- 支持 Claude Code 多账户轮询
27- 支持 Qwen Code 多账户轮询27- 支持 Qwen Code 多账户轮询
28- 通过配置接入上游 OpenAI 兼容提供商（例如 OpenRouter）
28 29
29## 安装30## 安装
30 31
@@ -59,12 +60,14 @@ · 28 unmodified lines
59 ```bash60 ```bash
60 ./cli-proxy-api --login61 ./cli-proxy-api --login
61 ```62 ```
62 如果您是旧版 gemini code 用户，可能需要指定项目 ID：63 如果您是现有的 Gemini Code 用户，可能需要指定一个项目ID：
63 ```bash64 ```bash
64 ./cli-proxy-api --login --project_id <your_project_id>65 ./cli-proxy-api --login --project_id <your_project_id>
65 ```66 ```
66 本地 OAuth 回调端口为 `8085`。67 本地 OAuth 回调端口为 `8085`。
67 68
69 选项：加上 `--no-browser` 可打印登录地址而不自动打开浏览器。本地 OAuth 回调端口为 `8085`。
70
68- OpenAI（Codex/GPT，OAuth）：71- OpenAI（Codex/GPT，OAuth）：
69 ```bash72 ```bash
70 ./cli-proxy-api --codex-login73 ./cli-proxy-api --codex-login
@@ -123,7 +126,7 @@ POST http://localhost:8317/v1/chat/completions · 52 unmodified lines
123```126```
124 127
125说明：128说明：
126- 使用 `gemini-*` 模型（如 `gemini-2.5-pro`）走 Gemini，使用 `gpt-*` 模型（如 `gpt-5`）走 OpenAI，使用 `claude-*` 模型（如 `claude-3-5-sonnet-20241022`）走 Claude，使用 `qwen-*` 模型（如 `qwen3-coder-plus`）走 Qwen，服务会自动路由到对应提供商。129- 使用 "gemini-*" 模型（例如 "gemini-2.5-pro"）来调用 Gemini，使用 "gpt-*" 模型（例如 "gpt-5"）来调用 OpenAI，使用 "claude-*" 模型（例如 "claude-3-5-sonnet-20241022"）来调用 Claude，或者使用 "qwen-*" 模型（例如 "qwen3-coder-plus"）来调用 Qwen。代理服务会自动将请求路由到相应的提供商。
127 130
128#### Claude 消息（SSE 兼容）131#### Claude 消息（SSE 兼容）
129 132
@@ -235,21 +238,28 @@ console.log(await claudeResponse.json()); · 105 unmodified lines
235 238
236### 配置选项239### 配置选项
237 240
238| 参数 | 类型 | 默认值 | 描述 |241| 参数 | 类型 | 默认值 | 描述 |
239|---------------------------------------|----------|--------------------|------------------------------------------------------------------------|242|---------------------------------------|----------|--------------------|---------------------------------------------------------------------------------------------|
240| `port` | integer | 8317 | 服务器监听的端口号 |243| `port` | integer | 8317 | 服务器将监听的端口号。 |
241| `auth-dir` | string | "~/.cli-proxy-api" | 存储身份验证令牌的目录。支持使用 `~` 表示主目录 |244| `auth-dir` | string | "~/.cli-proxy-api" | 存储身份验证令牌的目录。支持使用 `~` 来表示主目录。 |
242| `proxy-url` | string | "" | 代理 URL，支持 socks5/http/https 协议，示例：socks5://user:pass@192.168.1.1:1080/ |245| `proxy-url` | string | "" | 代理URL。支持socks5/http/https协议。例如：socks5://user:pass@192.168.1.1:1080/ |
243| `request-retry` | integer | 0 | 请求失败重试次数，如果响应的http代码为403、408、500、502、503、504则会自动重试 |246| `request-retry` | integer | 0 | 请求重试次数。如果HTTP响应码为403、408、500、502、503或504，将会触发重试。 |
244| `quota-exceeded` | object | {} | 处理配额超限的配置 |247| `quota-exceeded` | object | {} | 用于处理配额超限的配置。 |
245| `quota-exceeded.switch-project` | boolean | true | 当配额超限时是否自动切换到另一个项目 |248| `quota-exceeded.switch-project` | boolean | true | 当配额超限时，是否自动切换到另一个项目。 |
246| `quota-exceeded.switch-preview-model` | boolean | true | 当配额超限时是否自动切换到预览模型 |249| `quota-exceeded.switch-preview-model` | boolean | true | 当配额超限时，是否自动切换到预览模型。 |
247| `debug` | boolean | false | 启用调试模式以进行详细日志记录 |250| `debug` | boolean | false | 启用调试模式以获取详细日志。 |
248| `api-keys` | string[] | [] | 可用于验证请求的 API 密钥列表 |251| `api-keys` | string[] | [] | 可用于验证请求的API密钥列表。 |
249| `generative-language-api-key` | string[] | [] | 生成式语言 API 密钥列表 |252| `generative-language-api-key` | string[] | [] | 生成式语言API密钥列表。 |
250| `claude-api-key` | object | {} | Claude API 密钥列表 |253| `claude-api-key` | object | {} | Claude API密钥列表。 |
251| `claude-api-key.api-key` | string | "" | Claude API 密钥 |254| `claude-api-key.api-key` | string | "" | Claude API密钥。 |
252| `claude-api-key.base-url` | string | "" | 自定义 Claude API 端点（如果你使用的是第三方 Claude API 端点） |255| `claude-api-key.base-url` | string | "" | 自定义的Claude API端点，如果您使用第三方的API端点。 |
256| `openai-compatibility` | object[] | [] | 上游OpenAI兼容提供商的配置（名称、基础URL、API密钥、模型）。 |
257| `openai-compatibility.*.name` | string | "" | 提供商的名称。它将被用于用户代理（User Agent）和其他地方。 |
258| `openai-compatibility.*.base-url` | string | "" | 提供商的基础URL。 |
259| `openai-compatibility.*.api-keys` | string[] | [] | 提供商的API密钥。如果需要，可以添加多个密钥。如果允许未经身份验证的访问，则可以省略。 |
260| `openai-compatibility.*.models` | object[] | [] | 实际的模型名称。 |
261| `openai-compatibility.*.models.*.name` | string | "" | 提供商支持的模型。 |
262| `openai-compatibility.*.models.*.alias` | string | "" | 在API中使用的别名。 |
253 263
254### 配置文件示例264### 配置文件示例
255 265
@@ -263,10 +273,10 @@ auth-dir: "~/.cli-proxy-api" · 7 unmodified lines
263# 启用调试日志273# 启用调试日志
264debug: false274debug: false
265 275
266# 代理 URL，支持 socks5/http/https 协议，示例：socks5://user:pass@192.168.1.1:1080/276# 代理URL。支持socks5/http/https协议。例如：socks5://user:pass@192.168.1.1:1080/
267proxy-url: ""277proxy-url: ""
268 278
269# 请求失败重试次数，如果响应的http代码为403、408、500、502、503、504则会自动重试279# 请求重试次数。如果HTTP响应码为403、408、500、502、503或504，将会触发重试。
270request-retry: 3280request-retry: 3
271 281
272 282
@@ -292,8 +302,46 @@ claude-api-key: · 19 unmodified lines
292 - api-key: "sk-atSM..." # use the official claude API key, no need to set the base url302 - api-key: "sk-atSM..." # use the official claude API key, no need to set the base url
293 - api-key: "sk-atSM..."303 - api-key: "sk-atSM..."
294 base-url: "https://www.example.com" # use the custom claude API endpoint304 base-url: "https://www.example.com" # use the custom claude API endpoint
305
306# OpenAI 兼容提供商
307openai-compatibility:
308 - name: "openrouter" # 提供商的名称；它将被用于用户代理和其它地方。
309 base-url: "https://openrouter.ai/api/v1" # 提供商的基础URL。
310 api-keys: # 提供商的API密钥。如果需要，可以添加多个密钥。如果允许未经身份验证的访问，则可以省略。
311 - "sk-or-v1-...b780"
312 - "sk-or-v1-...b781"
313 models: # 提供商支持的模型。
314 - name: "moonshotai/kimi-k2:free" # 实际的模型名称。
315 alias: "kimi-k2" # 在API中使用的别名。
295```316```
296 317
318### OpenAI 兼容上游提供商
319
320通过 `openai-compatibility` 配置上游 OpenAI 兼容提供商（例如 OpenRouter）。
321
322- name：内部识别名
323- base-url：提供商基础地址
324- api-keys：可选，多密钥轮询（若提供商支持无鉴权可省略）
325- models：将上游模型 `name` 映射为本地可用 `alias`
326
327示例：
328
329```yaml
330openai-compatibility:
331 - name: "openrouter"
332 base-url: "https://openrouter.ai/api/v1"
333 api-keys:
334 - "sk-or-v1-...b780"
335 - "sk-or-v1-...b781"
336 models:
337 - name: "moonshotai/kimi-k2:free"
338 alias: "kimi-k2"
339```
340
341使用方式：在 `/v1/chat/completions` 中将 `model` 设为别名（如 `kimi-k2`），代理将自动路由到对应提供商与模型。
342
343并且，对于这些与OpenAI兼容的提供商模型，您始终可以通过将CODE_ASSIST_ENDPOINT设置为 http://127.0.0.1:8317 来使用Gemini CLI。
344
297### 身份验证目录345### 身份验证目录
298 346
299`auth-dir` 参数指定身份验证令牌的存储位置。当您运行登录命令时，应用程序将在此目录中创建包含 Google 账户身份验证令牌的 JSON 文件。多个账户可用于轮询。347`auth-dir` 参数指定身份验证令牌的存储位置。当您运行登录命令时，应用程序将在此目录中创建包含 Google 账户身份验证令牌的 JSON 文件。多个账户可用于轮询。
@@ -325,7 +373,7 @@ export CODE_ASSIST_ENDPOINT="http://127.0.0.1:8317" · 25 unmodified lines
325服务器将中继 `loadCodeAssist`、`onboardUser` 和 `countTokens` 请求。并自动在多个账户之间轮询文本生成请求。373服务器将中继 `loadCodeAssist`、`onboardUser` 和 `countTokens` 请求。并自动在多个账户之间轮询文本生成请求。
326 374
327> [!NOTE] 375> [!NOTE]
328> 此功能仅允许本地访问，因为找不到一个可以验证请求的方法。 376> 此功能仅允许本地访问，因为找不到一个可以验证请求的方法。
329> 所以只能强制只有 `127.0.0.1` 可以访问。377> 所以只能强制只有 `127.0.0.1` 可以访问。
330 378
331## Claude Code 的使用方法379## Claude Code 的使用方法
config.example.yaml+26−9
@@ -1,23 +1,29 @@
1# Server configuration1# Server port
2port: 83172port: 8317
3
4# Authentication directory (supports ~ for home directory)
3auth-dir: "~/.cli-proxy-api"5auth-dir: "~/.cli-proxy-api"
4debug: true6
7# Enable debug logging
8debug: false
9
10# Proxy URL. Supports socks5/http/https protocols. Example: socks5://user:pass@192.168.1.1:1080/
5proxy-url: ""11proxy-url: ""
6 12
7# Request retry times, if the response http code is 403, 408, 500, 502, 503, 504, it will retry13# Number of times to retry a request. Retries will occur if the HTTP response code is 403, 408, 500, 502, 503, or 504.
8request-retry: 314request-retry: 3
9 15
10# Quota exceeded behavior16# Quota exceeded behavior
11quota-exceeded:17quota-exceeded:
12 switch-project: true18 switch-project: true # Whether to automatically switch to another project when a quota is exceeded
13 switch-preview-model: true19 switch-preview-model: true # Whether to automatically switch to a preview model when a quota is exceeded
14 20
15# API keys for client authentication21# API keys for authentication
16api-keys:22api-keys:
17 - "12345"23 - "your-api-key-1"
18 - "23456"24 - "your-api-key-2"
19 25
20# Generative language API keys26# API keys for official Generative Language API
21generative-language-api-key:27generative-language-api-key:
22 - "AIzaSy...01"28 - "AIzaSy...01"
23 - "AIzaSy...02"29 - "AIzaSy...02"
@@ -29,3 +35,14 @@ claude-api-key: · 5 unmodified lines
29 - api-key: "sk-atSM..." # use the official claude API key, no need to set the base url35 - api-key: "sk-atSM..." # use the official claude API key, no need to set the base url
30 - api-key: "sk-atSM..."36 - api-key: "sk-atSM..."
31 base-url: "https://www.example.com" # use the custom claude API endpoint37 base-url: "https://www.example.com" # use the custom claude API endpoint
38
39# OpenAI compatibility providers
40openai-compatibility:
41 - name: "openrouter" # The name of the provider; it will be used in the user agent and other places.
42 base-url: "https://openrouter.ai/api/v1" # The base URL of the provider.
43 api-keys: # The API keys for the provider. Add multiple keys if needed. Omit if unauthenticated access is allowed.
44 - "sk-or-v1-...b780"
45 - "sk-or-v1-...b781"
46 models: # The models supported by the provider.
47 - name: "moonshotai/kimi-k2:free" # The actual model name.
48 alias: "kimi-k2" # The alias used in the API.
internal/api/handlers/claude/code_handlers.go+1−1
@@ -148,7 +148,7 @@ outLoop: · 147 unmodified lines
148 // Detects when the HTTP client has disconnected and cleans up resources148 // Detects when the HTTP client has disconnected and cleans up resources
149 case <-c.Request.Context().Done():149 case <-c.Request.Context().Done():
150 if c.Request.Context().Err().Error() == "context canceled" {150 if c.Request.Context().Err().Error() == "context canceled" {
151 log.Debugf("GeminiClient disconnected: %v", c.Request.Context().Err())151 log.Debugf("claude client disconnected: %v", c.Request.Context().Err())
152 cliCancel() // Cancel the backend request to prevent resource leaks152 cliCancel() // Cancel the backend request to prevent resource leaks
153 return153 return
154 }154 }
internal/api/handlers/gemini/gemini-cli_handlers.go+1−1
@@ -188,7 +188,7 @@ outLoop: · 187 unmodified lines
188 // Handle client disconnection.188 // Handle client disconnection.
189 case <-c.Request.Context().Done():189 case <-c.Request.Context().Done():
190 if c.Request.Context().Err().Error() == "context canceled" {190 if c.Request.Context().Err().Error() == "context canceled" {
191 log.Debugf("Client disconnected: %v", c.Request.Context().Err())191 log.Debugf("gemini cli client disconnected: %v", c.Request.Context().Err())
192 cliCancel() // Cancel the backend request.192 cliCancel() // Cancel the backend request.
193 return193 return
194 }194 }
internal/api/handlers/gemini/gemini_handlers.go+1−1
@@ -290,7 +290,7 @@ outLoop: · 289 unmodified lines
290 // Handle client disconnection.290 // Handle client disconnection.
291 case <-c.Request.Context().Done():291 case <-c.Request.Context().Done():
292 if c.Request.Context().Err().Error() == "context canceled" {292 if c.Request.Context().Err().Error() == "context canceled" {
293 log.Debugf("GeminiClient disconnected: %v", c.Request.Context().Err())293 log.Debugf("gemini client disconnected: %v", c.Request.Context().Err())
294 cliCancel() // Cancel the backend request.294 cliCancel() // Cancel the backend request.
295 return295 return
296 }296 }
internal/api/handlers/handlers.go+3−1
@@ -136,6 +136,8 @@ func (h *BaseAPIHandler) GetClient(modelName string, isGenerateContent ...bool) · 135 unmodified lines
136 log.Debugf("Claude Model %s is quota exceeded for account %s", modelName, cliClient.GetEmail())136 log.Debugf("Claude Model %s is quota exceeded for account %s", modelName, cliClient.GetEmail())
137 } else if cliClient.Provider() == "qwen" {137 } else if cliClient.Provider() == "qwen" {
138 log.Debugf("Qwen Model %s is quota exceeded for account %s", modelName, cliClient.GetEmail())138 log.Debugf("Qwen Model %s is quota exceeded for account %s", modelName, cliClient.GetEmail())
139 } else if cliClient.Type() == "openai-compatibility" {
140 log.Debugf("OpenAI Compatibility Model %s is quota exceeded for provider %s", modelName, cliClient.Provider())
139 }141 }
140 cliClient = nil142 cliClient = nil
141 continue143 continue
@@ -145,7 +147,7 @@ func (h *BaseAPIHandler) GetClient(modelName string, isGenerateContent ...bool) · 3 unmodified lines
145 }147 }
146 148
147 if len(reorderedClients) == 0 {149 if len(reorderedClients) == 0 {
148 if util.GetProviderName(modelName) == "claude" {150 if util.GetProviderName(modelName, h.Cfg) == "claude" {
149 // log.Debugf("Claude Model %s is quota exceeded for all accounts", modelName)151 // log.Debugf("Claude Model %s is quota exceeded for all accounts", modelName)
150 return nil, &interfaces.ErrorMessage{StatusCode: 429, Error: fmt.Errorf(`{"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your account's rate limit. Please try again later."}}`)}152 return nil, &interfaces.ErrorMessage{StatusCode: 429, Error: fmt.Errorf(`{"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your account's rate limit. Please try again later."}}`)}
151 }153 }
internal/api/handlers/openai/openai_handlers.go+1−1
@@ -278,7 +278,7 @@ outLoop: · 277 unmodified lines
278 // Handle client disconnection.278 // Handle client disconnection.
279 case <-c.Request.Context().Done():279 case <-c.Request.Context().Done():
280 if c.Request.Context().Err().Error() == "context canceled" {280 if c.Request.Context().Err().Error() == "context canceled" {
281 log.Debugf("Client disconnected: %v", c.Request.Context().Err())281 log.Debugf("qwen client disconnected: %v", c.Request.Context().Err())
282 cliCancel() // Cancel the backend request.282 cliCancel() // Cancel the backend request.
283 return283 return
284 }284 }
internal/client/claude_client.go+4−0
@@ -183,6 +183,7 @@ func (c *ClaudeClient) SendRawMessage(ctx context.Context, modelName string, raw · 182 unmodified lines
183 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}183 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}
184 }184 }
185 185
186 _ = respBody.Close()
186 c.AddAPIResponseData(ctx, bodyBytes)187 c.AddAPIResponseData(ctx, bodyBytes)
187 188
188 var param any189 var param any
@@ -238,6 +239,9 @@ func (c *ClaudeClient) SendRawMessageStream(ctx context.Context, modelName strin · 49 unmodified lines
238 return239 return
239 }240 }
240 delete(c.modelQuotaExceeded, modelName)241 delete(c.modelQuotaExceeded, modelName)
242 defer func() {
243 _ = stream.Close()
244 }()
241 245
242 scanner := bufio.NewScanner(stream)246 scanner := bufio.NewScanner(stream)
243 buffer := make([]byte, 10240*1024)247 buffer := make([]byte, 10240*1024)
internal/client/codex_client.go+4−0
@@ -132,6 +132,7 @@ func (c *CodexClient) SendRawMessage(ctx context.Context, modelName string, rawJ · 131 unmodified lines
132 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}132 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}
133 }133 }
134 134
135 _ = respBody.Close()
135 c.AddAPIResponseData(ctx, bodyBytes)136 c.AddAPIResponseData(ctx, bodyBytes)
136 137
137 var param any138 var param any
@@ -188,6 +189,9 @@ func (c *CodexClient) SendRawMessageStream(ctx context.Context, modelName string · 50 unmodified lines
188 return189 return
189 }190 }
190 delete(c.modelQuotaExceeded, modelName)191 delete(c.modelQuotaExceeded, modelName)
192 defer func() {
193 _ = stream.Close()
194 }()
191 195
192 scanner := bufio.NewScanner(stream)196 scanner := bufio.NewScanner(stream)
193 buffer := make([]byte, 10240*1024)197 buffer := make([]byte, 10240*1024)
internal/client/gemini-cli_client.go+6−0
@@ -497,6 +497,7 @@ func (c *GeminiCLIClient) SendRawMessage(ctx context.Context, modelName string, · 496 unmodified lines
497 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}497 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}
498 }498 }
499 499
500 _ = respBody.Close()
500 c.AddAPIResponseData(ctx, bodyBytes)501 c.AddAPIResponseData(ctx, bodyBytes)
501 502
502 newCtx := context.WithValue(ctx, "alt", alt)503 newCtx := context.WithValue(ctx, "alt", alt)
@@ -571,6 +572,11 @@ func (c *GeminiCLIClient) SendRawMessageStream(ctx context.Context, modelName st · 68 unmodified lines
571 delete(c.modelQuotaExceeded, modelName)572 delete(c.modelQuotaExceeded, modelName)
572 break573 break
573 }574 }
575 defer func() {
576 if stream != nil {
577 _ = stream.Close()
578 }
579 }()
574 580
575 newCtx := context.WithValue(ctx, "alt", alt)581 newCtx := context.WithValue(ctx, "alt", alt)
576 var param any582 var param any
internal/client/gemini_client.go+4−0
@@ -249,6 +249,7 @@ func (c *GeminiClient) SendRawMessage(ctx context.Context, modelName string, raw · 248 unmodified lines
249 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}249 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}
250 }250 }
251 251
252 _ = respBody.Close()
252 c.AddAPIResponseData(ctx, bodyBytes)253 c.AddAPIResponseData(ctx, bodyBytes)
253 254
254 var param any255 var param any
@@ -301,6 +302,9 @@ func (c *GeminiClient) SendRawMessageStream(ctx context.Context, modelName strin · 46 unmodified lines
301 return302 return
302 }303 }
303 delete(c.modelQuotaExceeded, modelName)304 delete(c.modelQuotaExceeded, modelName)
305 defer func() {
306 _ = stream.Close()
307 }()
304 308
305 newCtx := context.WithValue(ctx, "alt", alt)309 newCtx := context.WithValue(ctx, "alt", alt)
306 var param any310 var param any
internal/client/openai-compatibility_client.go+360−0
@@ -0,0 +1,360 @@
1// Package client defines the interface and base structure for AI API clients.
2// It provides a common interface that all supported AI service clients must implement,
3// including methods for sending messages, handling streams, and managing authentication.
4package client
5
6import (
7 "bufio"
8 "bytes"
9 "context"
10 "fmt"
11 "io"
12 "net/http"
13 "strings"
14 "sync"
15 "time"
16
17 "github.com/gin-gonic/gin"
18 "github.com/luispater/CLIProxyAPI/internal/auth"
19 "github.com/luispater/CLIProxyAPI/internal/config"
20 . "github.com/luispater/CLIProxyAPI/internal/constant"
21 "github.com/luispater/CLIProxyAPI/internal/interfaces"
22 "github.com/luispater/CLIProxyAPI/internal/translator/translator"
23 "github.com/luispater/CLIProxyAPI/internal/util"
24 log "github.com/sirupsen/logrus"
25 "github.com/tidwall/sjson"
26)
27
28// OpenAICompatibilityClient implements the Client interface for external OpenAI-compatible API providers.
29// This client handles requests to external services that support OpenAI-compatible APIs,
30// such as OpenRouter, Together.ai, and other similar services.
31type OpenAICompatibilityClient struct {
32 ClientBase
33 compatConfig *config.OpenAICompatibility
34 currentAPIKeyIndex int
35}
36
37// NewOpenAICompatibilityClient creates a new OpenAI compatibility client instance.
38//
39// Parameters:
40// - cfg: The application configuration.
41// - compatConfig: The OpenAI compatibility configuration for the specific provider.
42//
43// Returns:
44// - *OpenAICompatibilityClient: A new OpenAI compatibility client instance.
45// - error: An error if the client creation fails.
46func NewOpenAICompatibilityClient(cfg *config.Config, compatConfig *config.OpenAICompatibility) (*OpenAICompatibilityClient, error) {
47 if compatConfig == nil {
48 return nil, fmt.Errorf("compatibility configuration is required")
49 }
50
51 if len(compatConfig.APIKeys) == 0 {
52 return nil, fmt.Errorf("at least one API key is required for OpenAI compatibility provider: %s", compatConfig.Name)
53 }
54
55 httpClient := util.SetProxy(cfg, &http.Client{})
56 client := &OpenAICompatibilityClient{
57 ClientBase: ClientBase{
58 RequestMutex: &sync.Mutex{},
59 httpClient: httpClient,
60 cfg: cfg,
61 modelQuotaExceeded: make(map[string]*time.Time),
62 },
63 compatConfig: compatConfig,
64 currentAPIKeyIndex: 0,
65 }
66
67 return client, nil
68}
69
70// Type returns the client type.
71func (c *OpenAICompatibilityClient) Type() string {
72 return OPENAI
73}
74
75// Provider returns the provider name for this client.
76func (c *OpenAICompatibilityClient) Provider() string {
77 return c.compatConfig.Name
78}
79
80// CanProvideModel checks if this client can provide the specified model alias.
81//
82// Parameters:
83// - modelName: The name/alias of the model to check.
84//
85// Returns:
86// - bool: True if the model alias is supported, false otherwise.
87func (c *OpenAICompatibilityClient) CanProvideModel(modelName string) bool {
88 for _, model := range c.compatConfig.Models {
89 if model.Alias == modelName {
90 return true
91 }
92 }
93 return false
94}
95
96// GetUserAgent returns the user agent string for OpenAI compatibility API requests.
97func (c *OpenAICompatibilityClient) GetUserAgent() string {
98 return fmt.Sprintf("cli-proxy-api-%s", c.compatConfig.Name)
99}
100
101// TokenStorage returns nil as this client doesn't use traditional token storage.
102func (c *OpenAICompatibilityClient) TokenStorage() auth.TokenStorage {
103 return nil
104}
105
106// GetCurrentAPIKey returns the current API key to use, with rotation support.
107func (c *OpenAICompatibilityClient) GetCurrentAPIKey() string {
108 if len(c.compatConfig.APIKeys) == 0 {
109 return ""
110 }
111
112 key := c.compatConfig.APIKeys[c.currentAPIKeyIndex]
113 // Rotate to next key for load balancing
114 c.currentAPIKeyIndex = (c.currentAPIKeyIndex + 1) % len(c.compatConfig.APIKeys)
115 return key
116}
117
118// GetActualModelName returns the actual model name to use with the external API
119// based on the provided alias.
120func (c *OpenAICompatibilityClient) GetActualModelName(alias string) string {
121 for _, model := range c.compatConfig.Models {
122 if model.Alias == alias {
123 return model.Name
124 }
125 }
126 return alias // fallback to alias if not found
127}
128
129// APIRequest makes an HTTP request to the OpenAI-compatible API.
130//
131// Parameters:
132// - ctx: The context for the request.
133// - modelName: The model name to use.
134// - endpoint: The API endpoint path.
135// - rawJSON: The raw JSON request data.
136// - alt: Alternative response format (not used for OpenAI compatibility).
137// - stream: Whether this is a streaming request.
138//
139// Returns:
140// - io.ReadCloser: The response body reader.
141// - *interfaces.ErrorMessage: An error message if the request fails.
142func (c *OpenAICompatibilityClient) APIRequest(ctx context.Context, modelName string, endpoint string, rawJSON []byte, alt string, stream bool) (io.ReadCloser, *interfaces.ErrorMessage) {
143 // Replace the model alias with the actual model name in the request
144 actualModelName := c.GetActualModelName(modelName)
145 modifiedJSON, errReplace := sjson.SetBytes(rawJSON, "model", actualModelName)
146 if errReplace != nil {
147 return nil, &interfaces.ErrorMessage{
148 StatusCode: http.StatusInternalServerError,
149 Error: fmt.Errorf("failed to replace model name: %w", errReplace),
150 }
151 }
152
153 // Create the HTTP request
154 url := strings.TrimSuffix(c.compatConfig.BaseURL, "/") + endpoint
155 req, errReq := http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(modifiedJSON))
156 if errReq != nil {
157 return nil, &interfaces.ErrorMessage{
158 StatusCode: http.StatusInternalServerError,
159 Error: fmt.Errorf("failed to create request: %w", errReq),
160 }
161 }
162
163 // Set headers
164 req.Header.Set("Content-Type", "application/json")
165 apiKey := c.GetCurrentAPIKey()
166 if apiKey != "" {
167 req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", apiKey))
168 }
169 req.Header.Set("User-Agent", c.GetUserAgent())
170
171 if stream {
172 req.Header.Set("Accept", "text/event-stream")
173 req.Header.Set("Cache-Control", "no-cache")
174 }
175
176 log.Debugf("OpenAI Compatibility [%s] API request: %s", c.compatConfig.Name, util.HideAPIKey(apiKey))
177
178 // Send the request
179 resp, err := c.httpClient.Do(req)
180 if err != nil {
181 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: fmt.Errorf("failed to execute request: %v", err)}
182 }
183
184 if resp.StatusCode < 200 || resp.StatusCode >= 300 {
185 defer func() {
186 if err = resp.Body.Close(); err != nil {
187 log.Printf("warn: failed to close response body: %v", err)
188 }
189 }()
190 bodyBytes, _ := io.ReadAll(resp.Body)
191 // log.Debug(string(jsonBody))
192 return nil, &interfaces.ErrorMessage{StatusCode: resp.StatusCode, Error: fmt.Errorf("%s", string(bodyBytes))}
193 }
194
195 return resp.Body, nil
196}
197
198// SendRawMessage sends a raw message to the OpenAI-compatible API.
199//
200// Parameters:
201// - ctx: The context for the request.
202// - modelName: The model alias name to use.
203// - rawJSON: The raw JSON request data.
204// - alt: Alternative response format parameter.
205//
206// Returns:
207// - []byte: The response data from the API.
208// - *interfaces.ErrorMessage: An error message if the request fails.
209func (c *OpenAICompatibilityClient) SendRawMessage(ctx context.Context, modelName string, rawJSON []byte, alt string) ([]byte, *interfaces.ErrorMessage) {
210 handler := ctx.Value("handler").(interfaces.APIHandler)
211 handlerType := handler.HandlerType()
212 rawJSON = translator.Request(handlerType, c.Type(), modelName, rawJSON, false)
213
214 respBody, err := c.APIRequest(ctx, modelName, "/chat/completions", rawJSON, alt, false)
215 if err != nil {
216 if err.StatusCode == 429 {
217 now := time.Now()
218 c.modelQuotaExceeded[modelName] = &now
219 }
220 return nil, err
221 }
222 delete(c.modelQuotaExceeded, modelName)
223 bodyBytes, errReadAll := io.ReadAll(respBody)
224 if errReadAll != nil {
225 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}
226 }
227
228 _ = respBody.Close()
229 c.AddAPIResponseData(ctx, bodyBytes)
230
231 var param any
232 bodyBytes = []byte(translator.ResponseNonStream(handlerType, c.Type(), ctx, modelName, bodyBytes, &param))
233
234 return bodyBytes, nil
235}
236
237// SendRawMessageStream sends a raw streaming message to the OpenAI-compatible API.
238//
239// Parameters:
240// - ctx: The context for the request.
241// - modelName: The model alias name to use.
242// - rawJSON: The raw JSON request data.
243// - alt: Alternative response format parameter.
244//
245// Returns:
246// - <-chan []byte: A channel that will receive response chunks.
247// - <-chan *interfaces.ErrorMessage: A channel that will receive error messages.
248func (c *OpenAICompatibilityClient) SendRawMessageStream(ctx context.Context, modelName string, rawJSON []byte, alt string) (<-chan []byte, <-chan *interfaces.ErrorMessage) {
249 handler := ctx.Value("handler").(interfaces.APIHandler)
250 handlerType := handler.HandlerType()
251 rawJSON = translator.Request(handlerType, c.Type(), modelName, rawJSON, true)
252
253 dataTag := []byte("data: ")
254 doneTag := []byte("data: [DONE]")
255 errChan := make(chan *interfaces.ErrorMessage)
256 dataChan := make(chan []byte)
257 // log.Debugf(string(rawJSON))
258 // return dataChan, errChan
259 go func() {
260 defer close(errChan)
261 defer close(dataChan)
262
263 // Set streaming flag in the request
264 rawJSON, _ = sjson.SetBytes(rawJSON, "stream", true)
265
266 newCtx := context.WithValue(ctx, "gin", ctx.Value("gin").(*gin.Context))
267
268 stream, err := c.APIRequest(newCtx, modelName, "/chat/completions", rawJSON, alt, true)
269 if err != nil {
270 if err.StatusCode == 429 {
271 now := time.Now()
272 c.modelQuotaExceeded[modelName] = &now
273 }
274 errChan <- err
275 return
276 }
277 delete(c.modelQuotaExceeded, modelName)
278 defer func() {
279 _ = stream.Close()
280 }()
281
282 scanner := bufio.NewScanner(stream)
283
284 if translator.NeedConvert(handlerType, c.Type()) {
285 var param any
286 for scanner.Scan() {
287 line := scanner.Bytes()
288 if bytes.HasPrefix(line, dataTag) {
289 if bytes.Equal(line, doneTag) {
290 break
291 }
292 lines := translator.Response(handlerType, c.Type(), newCtx, modelName, line[6:], &param)
293 for i := 0; i < len(lines); i++ {
294 dataChan <- []byte(lines[i])
295 }
296 }
297 }
298 } else {
299 // No translation needed, stream data directly
300 for scanner.Scan() {
301 line := scanner.Bytes()
302 if bytes.HasPrefix(line, dataTag) {
303 if bytes.Equal(line, doneTag) {
304 break
305 }
306 c.AddAPIResponseData(newCtx, line[6:])
307 dataChan <- line[6:]
308 }
309 }
310 }
311
312 if scanner.Err() != nil {
313 errChan <- &interfaces.ErrorMessage{StatusCode: 500, Error: scanner.Err()}
314 }
315 }()
316
317 return dataChan, errChan
318}
319
320// SendRawTokenCount sends a token count request (not implemented for OpenAI compatibility).
321// This method is required by the Client interface but not supported by OpenAI compatibility clients.
322func (c *OpenAICompatibilityClient) SendRawTokenCount(ctx context.Context, modelName string, rawJSON []byte, alt string) ([]byte, *interfaces.ErrorMessage) {
323 return nil, &interfaces.ErrorMessage{
324 StatusCode: http.StatusNotImplemented,
325 Error: fmt.Errorf("token counting not supported for OpenAI compatibility clients"),
326 }
327}
328
329// GetEmail returns a placeholder email for this OpenAI compatibility client.
330// Since these clients don't use traditional email-based authentication,
331// we return the provider name as an identifier.
332func (c *OpenAICompatibilityClient) GetEmail() string {
333 return fmt.Sprintf("openai-compatibility-%s", c.compatConfig.Name)
334}
335
336// IsModelQuotaExceeded checks if the specified model has exceeded its quota.
337// For OpenAI compatibility clients, this is based on tracked quota exceeded times.
338func (c *OpenAICompatibilityClient) IsModelQuotaExceeded(model string) bool {
339 if quota, exists := c.modelQuotaExceeded[model]; exists && quota != nil {
340 // Check if quota exceeded time is less than 5 minutes ago
341 if time.Since(*quota) < 5*time.Minute {
342 return true
343 }
344 // Clear expired quota tracking
345 delete(c.modelQuotaExceeded, model)
346 }
347 return false
348}
349
350// SaveTokenToFile returns nil as this client type doesn't use traditional token storage.
351func (c *OpenAICompatibilityClient) SaveTokenToFile() error {
352 // No token file to save for OpenAI compatibility clients
353 return nil
354}
355
356// RefreshTokens is not applicable for OpenAI compatibility clients as they use API keys.
357func (c *OpenAICompatibilityClient) RefreshTokens(ctx context.Context) error {
358 // API keys don't need refreshing
359 return nil
360}
internal/client/qwen_client.go+4−0
@@ -128,6 +128,7 @@ func (c *QwenClient) SendRawMessage(ctx context.Context, modelName string, rawJS · 127 unmodified lines
128 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}128 return nil, &interfaces.ErrorMessage{StatusCode: 500, Error: errReadAll}
129 }129 }
130 130
131 _ = respBody.Close()
131 c.AddAPIResponseData(ctx, bodyBytes)132 c.AddAPIResponseData(ctx, bodyBytes)
132 133
133 var param any134 var param any
@@ -186,6 +187,9 @@ func (c *QwenClient) SendRawMessageStream(ctx context.Context, modelName string, · 52 unmodified lines
186 return187 return
187 }188 }
188 delete(c.modelQuotaExceeded, modelName)189 delete(c.modelQuotaExceeded, modelName)
190 defer func() {
191 _ = stream.Close()
192 }()
189 193
190 scanner := bufio.NewScanner(stream)194 scanner := bufio.NewScanner(stream)
191 buffer := make([]byte, 10240*1024)195 buffer := make([]byte, 10240*1024)
internal/cmd/run.go+12−0
@@ -150,6 +150,18 @@ func StartService(cfg *config.Config, configPath string) { · 149 unmodified lines
150 }150 }
151 }151 }
152 152
153 if len(cfg.OpenAICompatibility) > 0 {
154 // Initialize clients for OpenAI compatibility configurations
155 for _, compatConfig := range cfg.OpenAICompatibility {
156 log.Debugf("Initializing OpenAI compatibility client for provider: %s", compatConfig.Name)
157 compatClient, errClient := client.NewOpenAICompatibilityClient(cfg, &compatConfig)
158 if errClient != nil {
159 log.Fatalf("failed to create OpenAI compatibility client for %s: %v", compatConfig.Name, errClient)
160 }
161 cliClients = append(cliClients, compatClient)
162 }
163 }
164
153 // Create and start the API server with the pool of clients in a separate goroutine.165 // Create and start the API server with the pool of clients in a separate goroutine.
154 apiServer := api.NewServer(cfg, cliClients)166 apiServer := api.NewServer(cfg, cliClients)
155 log.Infof("Starting API server on port %d", cfg.Port)167 log.Infof("Starting API server on port %d", cfg.Port)
internal/config/config.go+29−0
@@ -41,6 +41,9 @@ type Config struct { · 40 unmodified lines
41 RequestRetry int `yaml:"request-retry"`41 RequestRetry int `yaml:"request-retry"`
42 42
43 ClaudeKey []ClaudeKey `yaml:"claude-api-key"`43 ClaudeKey []ClaudeKey `yaml:"claude-api-key"`
44
45 // OpenAICompatibility defines OpenAI API compatibility configurations for external providers.
46 OpenAICompatibility []OpenAICompatibility `yaml:"openai-compatibility"`
44}47}
45 48
46// QuotaExceeded defines the behavior when API quota limits are exceeded.49// QuotaExceeded defines the behavior when API quota limits are exceeded.
@@ -64,6 +67,32 @@ type ClaudeKey struct { · 17 unmodified lines
64 BaseURL string `yaml:"base-url"`67 BaseURL string `yaml:"base-url"`
65}68}
66 69
70// OpenAICompatibility represents the configuration for OpenAI API compatibility
71// with external providers, allowing model aliases to be routed through OpenAI API format.
72type OpenAICompatibility struct {
73 // Name is the identifier for this OpenAI compatibility configuration.
74 Name string `yaml:"name"`
75
76 // BaseURL is the base URL for the external OpenAI-compatible API endpoint.
77 BaseURL string `yaml:"base-url"`
78
79 // APIKeys are the authentication keys for accessing the external API services.
80 APIKeys []string `yaml:"api-keys"`
81
82 // Models defines the model configurations including aliases for routing.
83 Models []OpenAICompatibilityModel `yaml:"models"`
84}
85
86// OpenAICompatibilityModel represents a model configuration for OpenAI compatibility,
87// including the actual model name and its alias for API routing.
88type OpenAICompatibilityModel struct {
89 // Name is the actual model name used by the external provider.
90 Name string `yaml:"name"`
91
92 // Alias is the model name alias that clients will use to reference this model.
93 Alias string `yaml:"alias"`
94}
95
67// LoadConfig reads a YAML configuration file from the given path,96// LoadConfig reads a YAML configuration file from the given path,
68// unmarshals it into a Config struct, applies environment variable overrides,97// unmarshals it into a Config struct, applies environment variable overrides,
69// and returns it.98// and returns it.
internal/constant/constant.go+6−5
@@ -1,9 +1,10 @@
1package constant1package constant
2 2
3const (3const (
4 GEMINI = "gemini"4 GEMINI = "gemini"
5 GEMINICLI = "gemini-cli"5 GEMINICLI = "gemini-cli"
6 CODEX = "codex"6 CODEX = "codex"
7 CLAUDE = "claude"7 CLAUDE = "claude"
8 OPENAI = "openai"8 OPENAI = "openai"
9 OPENAI_COMPATIBILITY = "openai-compatibility"
9)10)
internal/util/provider.go+59−2
@@ -5,25 +5,33 @@ package util · 4 unmodified lines
5 5
6import (6import (
7 "strings"7 "strings"
8
9 "github.com/luispater/CLIProxyAPI/internal/config"
8)10)
9 11
10// GetProviderName determines the AI service provider based on the model name.12// GetProviderName determines the AI service provider based on the model name.
11// It analyzes the model name string to identify which service provider it belongs to.13// It analyzes the model name string to identify which service provider it belongs to.
14// First checks for OpenAI compatibility aliases, then falls back to standard provider detection.
12//15//
13// Supported providers:16// Supported providers:
14// - "gemini" for Google's Gemini models17// - "gemini" for Google's Gemini models
15// - "gpt" for OpenAI's GPT models18// - "gpt" for OpenAI's GPT models
16// - "claude" for Anthropic's Claude models19// - "claude" for Anthropic's Claude models
17// - "qwen" for Alibaba's Qwen models20// - "qwen" for Alibaba's Qwen models
21// - "openai-compatibility" for external OpenAI-compatible providers
18// - "unknow" for unrecognized model names22// - "unknow" for unrecognized model names
19//23//
20// Parameters:24// Parameters:
21// - modelName: The name of the model to identify the provider for.25// - modelName: The name of the model to identify the provider for.
26// - cfg: The application configuration containing OpenAI compatibility settings.
22//27//
23// Returns:28// Returns:
24// - string: The name of the provider.29// - string: The name of the provider.
25func GetProviderName(modelName string) string {30func GetProviderName(modelName string, cfg *config.Config) string {
26 if strings.Contains(modelName, "gemini") {31 // First check if this model name is an OpenAI compatibility alias
32 if IsOpenAICompatibilityAlias(modelName, cfg) {
33 return "openai-compatibility"
34 } else if strings.Contains(modelName, "gemini") { // Fall back to standard provider detection
27 return "gemini"35 return "gemini"
28 } else if strings.Contains(modelName, "gpt") {36 } else if strings.Contains(modelName, "gpt") {
29 return "gpt"37 return "gpt"
@@ -37,6 +45,55 @@ func GetProviderName(modelName string) string { · 7 unmodified lines
37 return "unknow"45 return "unknow"
38}46}
39 47
48// IsOpenAICompatibilityAlias checks if the given model name is an alias
49// configured for OpenAI compatibility routing.
50//
51// Parameters:
52// - modelName: The model name to check
53// - cfg: The application configuration containing OpenAI compatibility settings
54//
55// Returns:
56// - bool: True if the model name is an OpenAI compatibility alias, false otherwise
57func IsOpenAICompatibilityAlias(modelName string, cfg *config.Config) bool {
58 if cfg == nil {
59 return false
60 }
61
62 for _, compat := range cfg.OpenAICompatibility {
63 for _, model := range compat.Models {
64 if model.Alias == modelName {
65 return true
66 }
67 }
68 }
69 return false
70}
71
72// GetOpenAICompatibilityConfig returns the OpenAI compatibility configuration
73// and model details for the given alias.
74//
75// Parameters:
76// - alias: The model alias to find configuration for
77// - cfg: The application configuration containing OpenAI compatibility settings
78//
79// Returns:
80// - *config.OpenAICompatibility: The matching compatibility configuration, or nil if not found
81// - *config.OpenAICompatibilityModel: The matching model configuration, or nil if not found
82func GetOpenAICompatibilityConfig(alias string, cfg *config.Config) (*config.OpenAICompatibility, *config.OpenAICompatibilityModel) {
83 if cfg == nil {
84 return nil, nil
85 }
86
87 for _, compat := range cfg.OpenAICompatibility {
88 for _, model := range compat.Models {
89 if model.Alias == alias {
90 return &compat, &model
91 }
92 }
93 }
94 return nil, nil
95}
96
40// InArray checks if a string exists in a slice of strings.97// InArray checks if a string exists in a slice of strings.
41// It iterates through the slice and returns true if the target string is found,98// It iterates through the slice and returns true if the target string is found,
42// otherwise it returns false.99// otherwise it returns false.