forked from xiaohei/taiji-AI-PAD
29 KiB
29 KiB
LiteLLM Callback 计费方案实施指南
版本: v1.0.0
创建时间: 2026-01-08
目标: 实现基于 LiteLLM Custom Callback 的实时模型 Token 计费
1. 方案概述
1.1 架构设计
sequenceDiagram
participant Agent as Agent/租户应用
participant LiteLLM as LiteLLM Gateway
participant MCP as mcp-server
participant DB as PostgreSQL
Agent->>LiteLLM: 1. 调用模型 API
LiteLLM->>LiteLLM: 2. 执行模型调用
LiteLLM-->>Agent: 3. 返回响应
LiteLLM->>MCP: 4. POST /api/v1/billing/litellm-callback
Note over LiteLLM,MCP: 异步回调，包含 Token 用量
MCP->>MCP: 5. 解析回调数据
MCP->>DB: 6. 写入 model_billing_records
MCP->>DB: 7. 更新用户 EU 余额
MCP-->>LiteLLM: 8. 返回 200 OK
1.2 核心优势
| 特性 | 说明 |
|---|---|
| 实时性 | 每次模型调用后立即回调，无延迟 |
| 准确性 | 直接从 LiteLLM 获取精确的 Token 用量 |
| 可靠性 | LiteLLM 内置重试机制 |
| 解耦性 | mcp-server 无需轮询，被动接收数据 |
2. 实施步骤清单
2.1 总体任务
- Step 1: 创建数据库表
model_billing_records - Step 2: 创建 Pydantic Schema 定义
- Step 3: 创建 Webhook 路由
/api/v1/billing/litellm-callback - Step 4: 实现计费逻辑（Token → EU 转换）
- Step 5: 配置 LiteLLM Custom Callback
- Step 6: 修改 Dashboard 查询逻辑
- Step 7: 测试和验证
3. 详细实施方案
3.1 Step 1: 数据库迁移
文件: services/mcp-server/migrations/012_add_model_billing_records.sql
-- 模型调用计费记录表
-- 存储 LiteLLM Callback 推送的 Token 用量数据
CREATE TABLE IF NOT EXISTS model_billing_records (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
-- 关联信息（从 LiteLLM metadata 解析）
tenant_id UUID REFERENCES users(id),
channel_id UUID REFERENCES channels(id),
-- LiteLLM 回调数据
litellm_call_id VARCHAR(100) UNIQUE NOT NULL, -- LiteLLM 调用 ID
api_key VARCHAR(255), -- 使用的 API Key（脱敏）
team_id VARCHAR(100), -- LiteLLM Team ID
-- 模型信息
model_name VARCHAR(100) NOT NULL, -- 如 azure/gpt-4
model_id VARCHAR(100), -- LiteLLM 内部模型 ID
-- Token 用量
input_tokens INTEGER NOT NULL DEFAULT 0,
output_tokens INTEGER NOT NULL DEFAULT 0,
total_tokens INTEGER NOT NULL DEFAULT 0,
-- 费用计算
input_cost NUMERIC(12, 6) DEFAULT 0, -- 输入成本（美元）
output_cost NUMERIC(12, 6) DEFAULT 0, -- 输出成本（美元）
total_cost NUMERIC(12, 6) DEFAULT 0, -- 总成本（美元）
eu_consumed NUMERIC(12, 4) DEFAULT 0, -- EU 消耗
-- 调用信息
request_id VARCHAR(100), -- 请求 ID
call_type VARCHAR(50) DEFAULT 'completion', -- completion, embedding, etc.
status VARCHAR(20) DEFAULT 'success', -- success, error
-- 时间信息
start_time TIMESTAMP WITH TIME ZONE,
end_time TIMESTAMP WITH TIME ZONE,
response_time_ms INTEGER, -- 响应时间（毫秒）
-- 原始数据（用于调试）
raw_callback_data JSONB,
-- 索引
CONSTRAINT idx_model_billing_tenant UNIQUE (tenant_id, litellm_call_id)
);
-- 创建索引
CREATE INDEX idx_model_billing_created ON model_billing_records(created_at);
CREATE INDEX idx_model_billing_tenant_id ON model_billing_records(tenant_id);
CREATE INDEX idx_model_billing_channel_id ON model_billing_records(channel_id);
CREATE INDEX idx_model_billing_model ON model_billing_records(model_name);
CREATE INDEX idx_model_billing_team ON model_billing_records(team_id);
-- 添加注释
COMMENT ON TABLE model_billing_records IS '模型调用计费记录，由 LiteLLM Callback 推送';
COMMENT ON COLUMN model_billing_records.litellm_call_id IS 'LiteLLM 调用唯一标识';
COMMENT ON COLUMN model_billing_records.eu_consumed IS '消耗的执行单元，根据 Token 和定价计算';
3.2 Step 2: 数据模型定义
文件: services/mcp-server/models.py（添加新模型）
class ModelBillingRecord(BaseModel, Base):
"""模型调用计费记录
存储 LiteLLM Callback 推送的 Token 用量数据。
每次模型调用后，LiteLLM 会异步回调 mcp-server，
mcp-server 解析数据并写入此表。
"""
__tablename__ = "model_billing_records"
# 关联信息
tenant_id = Column(GUID(), ForeignKey("users.id"))
channel_id = Column(GUID(), ForeignKey("channels.id"))
# LiteLLM 回调数据
litellm_call_id = Column(String(100), unique=True, nullable=False)
api_key = Column(String(255)) # 脱敏存储
team_id = Column(String(100))
# 模型信息
model_name = Column(String(100), nullable=False)
model_id = Column(String(100))
# Token 用量
input_tokens = Column(Integer, default=0)
output_tokens = Column(Integer, default=0)
total_tokens = Column(Integer, default=0)
# 费用计算
input_cost = Column(sa.Numeric(12, 6), default=0)
output_cost = Column(sa.Numeric(12, 6), default=0)
total_cost = Column(sa.Numeric(12, 6), default=0)
eu_consumed = Column(sa.Numeric(12, 4), default=0)
# 调用信息
request_id = Column(String(100))
call_type = Column(String(50), default="completion")
status = Column(String(20), default="success")
# 时间信息
start_time = Column(DateTime)
end_time = Column(DateTime)
response_time_ms = Column(Integer)
# 原始数据
raw_callback_data = Column(JSON)
# 关联关系
tenant = relationship("User")
channel = relationship("Channel")
__table_args__ = (
Index("idx_model_billing_created", "created_at"),
Index("idx_model_billing_tenant_id", tenant_id),
Index("idx_model_billing_channel_id", channel_id),
Index("idx_model_billing_model", model_name),
Index("idx_model_billing_team", team_id),
)
3.3 Step 3: Webhook 路由实现
文件: services/mcp-server/app/routes/billing_webhook.py（新建）
"""
LiteLLM Callback Webhook 路由
接收 LiteLLM 的实时回调数据，记录模型调用的 Token 用量和费用。
LiteLLM Callback 数据格式参考：
https://docs.litellm.ai/docs/proxy/logging#custom-callback
"""
import logging
from datetime import datetime
from typing import Optional, Dict, Any, List
from decimal import Decimal
from fastapi import APIRouter, Depends, HTTPException, Request, status, Header
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import select, update
from pydantic import BaseModel, Field
from database import get_db
from models import ModelBillingRecord, User, Channel, TenantModelKey, ModelPricing
from config import settings
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/billing", tags=["计费 Webhook"])
# ==================== Pydantic Schemas ====================
class LiteLLMUsage(BaseModel):
"""LiteLLM Token 用量"""
prompt_tokens: int = Field(0, alias="prompt_tokens")
completion_tokens: int = Field(0, alias="completion_tokens")
total_tokens: int = Field(0, alias="total_tokens")
class Config:
populate_by_name = True
class LiteLLMCallbackData(BaseModel):
"""LiteLLM Callback 数据结构
参考: https://docs.litellm.ai/docs/proxy/logging
"""
# 调用标识
call_id: str = Field(..., alias="id")
call_type: str = Field("completion", alias="call_type") # completion, embedding
# 模型信息
model: str
model_id: Optional[str] = None
# API Key 信息
api_key: Optional[str] = None
team_id: Optional[str] = None
# Token 用量
usage: Optional[LiteLLMUsage] = None
# 费用（LiteLLM 计算）
response_cost: Optional[float] = None
# 时间信息
startTime: Optional[str] = None # ISO 格式
endTime: Optional[str] = None
response_time: Optional[float] = None # 秒
# 状态
status: str = "success" # success, failure
# 元数据（包含 tenant_id, channel_id 等）
metadata: Optional[Dict[str, Any]] = None
# 原始请求/响应（可选）
messages: Optional[List[Dict]] = None
response: Optional[Dict] = None
class Config:
populate_by_name = True
class CallbackResponse(BaseModel):
"""Callback 响应"""
success: bool
message: str
record_id: Optional[str] = None
# ==================== 辅助函数 ====================
def mask_api_key(key: Optional[str]) -> Optional[str]:
"""脱敏 API Key"""
if not key:
return None
if len(key) <= 10:
return "***"
return f"{key[:8]}...{key[-4:]}"
async def get_tenant_from_metadata(
metadata: Optional[Dict],
api_key: Optional[str],
team_id: Optional[str],
db: AsyncSession
) -> tuple[Optional[str], Optional[str]]:
"""从 metadata 或 API Key 解析租户和渠道 ID
解析优先级：
1. metadata 中的 tenant_id 和 channel_id
2. 通过 api_key 查询 TenantModelKey 表
3. 通过 team_id 查询 Channel 表
"""
tenant_id = None
channel_id = None
# 1. 从 metadata 解析
if metadata:
tenant_id = metadata.get("tenant_id")
channel_id = metadata.get("channel_id")
if tenant_id and channel_id:
return tenant_id, channel_id
# 2. 通过 api_key 查询
if api_key and not tenant_id:
# 注意：api_key 可能是加密的，需要匹配
result = await db.execute(
select(TenantModelKey).where(
TenantModelKey.litellm_key_id == api_key
)
)
tenant_key = result.scalar_one_or_none()
if tenant_key:
tenant_id = str(tenant_key.tenant_id)
channel_id = str(tenant_key.channel_id) if tenant_key.channel_id else None
# 3. 通过 team_id 查询渠道
if team_id and not channel_id:
result = await db.execute(
select(Channel).where(Channel.litellm_team_id == team_id)
)
channel = result.scalar_one_or_none()
if channel:
channel_id = str(channel.id)
return tenant_id, channel_id
async def calculate_eu_from_tokens(
model_name: str,
input_tokens: int,
output_tokens: int,
response_cost: Optional[float],
db: AsyncSession
) -> Decimal:
"""根据 Token 用量计算 EU 消耗
计算逻辑：
1. 如果 LiteLLM 返回了 response_cost，直接使用
2. 否则查询 ModelPricing 表计算
3. 最后按 EU 转换率转换
EU 转换率：1 EU = $0.01（可配置）
"""
EU_PER_DOLLAR = Decimal("100") # 1 美元 = 100 EU
if response_cost is not None and response_cost > 0:
# 直接使用 LiteLLM 计算的成本
return Decimal(str(response_cost)) * EU_PER_DOLLAR
# 查询模型定价
result = await db.execute(
select(ModelPricing).where(
ModelPricing.model_name == model_name,
ModelPricing.is_active == True
)
)
pricing = result.scalar_one_or_none()
if pricing:
# 计算成本（每 1K tokens）
input_cost = (Decimal(input_tokens) / 1000) * pricing.input_price_per_1k
output_cost = (Decimal(output_tokens) / 1000) * pricing.output_price_per_1k
total_cost = input_cost + output_cost
# 如果有 EU 转换率配置，使用配置值
if pricing.eu_per_1k_tokens:
total_tokens = input_tokens + output_tokens
return (Decimal(total_tokens) / 1000) * pricing.eu_per_1k_tokens
return total_cost * EU_PER_DOLLAR
# 默认：每 1K tokens = 0.1 EU
total_tokens = input_tokens + output_tokens
return Decimal(total_tokens) / 1000 * Decimal("0.1")
async def update_tenant_eu_balance(
tenant_id: str,
eu_consumed: Decimal,
db: AsyncSession
):
"""更新租户 EU 余额
扣减 eu_balance，增加 total_eu_consumed
"""
await db.execute(
update(User).where(User.id == tenant_id).values(
eu_balance=User.eu_balance - eu_consumed,
total_eu_consumed=User.total_eu_consumed + eu_consumed
)
)
# ==================== Webhook 端点 ====================
@router.post("/litellm-callback", response_model=CallbackResponse)
async def litellm_callback(
request: Request,
db: AsyncSession = Depends(get_db),
x_litellm_signature: Optional[str] = Header(None, alias="X-LiteLLM-Signature")
):
"""LiteLLM Callback Webhook 端点
接收 LiteLLM 的实时回调数据，记录模型调用的 Token 用量。
**安全说明**：
- 建议配置 X-LiteLLM-Signature 进行签名验证
- 生产环境应限制来源 IP
**回调数据示例**：
```json
{
"id": "chatcmpl-xxx",
"call_type": "completion",
"model": "azure/gpt-4",
"api_key": "sk-xxx",
"team_id": "team-xxx",
"usage": {
"prompt_tokens": 100,
"completion_tokens": 50,
"total_tokens": 150
},
"response_cost": 0.0045,
"startTime": "2026-01-08T10:00:00Z",
"endTime": "2026-01-08T10:00:02Z",
"status": "success",
"metadata": {
"tenant_id": "uuid",
"channel_id": "uuid"
}
}
```
"""
try:
# 解析请求体
body = await request.json()
logger.info(f"收到 LiteLLM Callback: call_id={body.get('id')}")
# 验证签名（可选，生产环境建议启用）
# if settings.litellm_callback_secret:
# if not verify_signature(x_litellm_signature, body):
# raise HTTPException(status_code=401, detail="签名验证失败")
# 解析回调数据
callback_data = LiteLLMCallbackData(**body)
# 检查是否已处理（幂等性）
existing = await db.execute(
select(ModelBillingRecord).where(
ModelBillingRecord.litellm_call_id == callback_data.call_id
)
)
if existing.scalar_one_or_none():
logger.info(f"Callback 已处理，跳过: {callback_data.call_id}")
return CallbackResponse(
success=True,
message="Already processed"
)
# 解析租户和渠道
tenant_id, channel_id = await get_tenant_from_metadata(
callback_data.metadata,
callback_data.api_key,
callback_data.team_id,
db
)
# 提取 Token 用量
input_tokens = callback_data.usage.prompt_tokens if callback_data.usage else 0
output_tokens = callback_data.usage.completion_tokens if callback_data.usage else 0
total_tokens = callback_data.usage.total_tokens if callback_data.usage else 0
# 计算 EU 消耗
eu_consumed = await calculate_eu_from_tokens(
callback_data.model,
input_tokens,
output_tokens,
callback_data.response_cost,
db
)
# 解析时间
start_time = None
end_time = None
response_time_ms = None
if callback_data.startTime:
try:
start_time = datetime.fromisoformat(callback_data.startTime.replace("Z", "+00:00"))
except Exception:
pass
if callback_data.endTime:
try:
end_time = datetime.fromisoformat(callback_data.endTime.replace("Z", "+00:00"))
except Exception:
pass
if callback_data.response_time:
response_time_ms = int(callback_data.response_time * 1000)
# 计算成本
input_cost = Decimal("0")
output_cost = Decimal("0")
total_cost = Decimal(str(callback_data.response_cost or 0))
# 创建计费记录
record = ModelBillingRecord(
tenant_id=tenant_id,
channel_id=channel_id,
litellm_call_id=callback_data.call_id,
api_key=mask_api_key(callback_data.api_key),
team_id=callback_data.team_id,
model_name=callback_data.model,
model_id=callback_data.model_id,
input_tokens=input_tokens,
output_tokens=output_tokens,
total_tokens=total_tokens,
input_cost=input_cost,
output_cost=output_cost,
total_cost=total_cost,
eu_consumed=eu_consumed,
request_id=callback_data.call_id,
call_type=callback_data.call_type,
status=callback_data.status,
start_time=start_time,
end_time=end_time,
response_time_ms=response_time_ms,
raw_callback_data=body
)
db.add(record)
# 更新租户 EU 余额
if tenant_id:
await update_tenant_eu_balance(tenant_id, eu_consumed, db)
await db.commit()
logger.info(
f"LiteLLM Callback 处理成功: call_id={callback_data.call_id}, "
f"model={callback_data.model}, tokens={total_tokens}, eu={eu_consumed}"
)
return CallbackResponse(
success=True,
message="Callback processed",
record_id=str(record.id)
)
except HTTPException:
raise
except Exception as e:
logger.error(f"LiteLLM Callback 处理失败: {e}", exc_info=True)
# 返回 200 避免 LiteLLM 重试（可根据需求调整）
return CallbackResponse(
success=False,
message=f"Processing error: {str(e)}"
)
@router.get("/litellm-callback/health")
async def callback_health():
"""Callback 端点健康检查"""
return {"status": "ok", "endpoint": "/api/v1/billing/litellm-callback"}
3.4 Step 4: 注册路由
文件: services/mcp-server/app/routes/__init__.py（修改）
# 在现有导入后添加
from app.routes.billing_webhook import router as billing_webhook_router
# 在 include_router 部分添加
app.include_router(billing_webhook_router)
3.5 Step 5: LiteLLM 配置
文件: services/model-gateway/config/litellm.yaml（修改）
# 基础设置
general_settings:
master_key: "os.environ/LITELLM_MASTER_KEY"
database_url: "os.environ/DATABASE_URL"
# ... 其他配置保持不变 ...
# 回调配置 - 添加 mcp-server webhook
success_callback: ["langfuse", "webhook"]
failure_callback: ["langfuse", "webhook"]
# 回调配置
callbacks:
# 成功回调
success_callback:
- callback_name: "langfuse"
callback_type: "success"
callback_vars:
langfuse_public_key: "os.environ/LANGFUSE_PUBLIC_KEY"
langfuse_secret_key: "os.environ/LANGFUSE_SECRET_KEY"
langfuse_host: "os.environ/LANGFUSE_HOST"
# 新增：mcp-server webhook 回调
- callback_name: "webhook"
callback_type: "success"
callback_vars:
webhook_url: "os.environ/MCP_SERVER_CALLBACK_URL"
# 可选：签名密钥
# webhook_secret: "os.environ/LITELLM_CALLBACK_SECRET"
# 失败回调
failure_callback:
- callback_name: "langfuse"
callback_type: "failure"
callback_vars:
langfuse_public_key: "os.environ/LANGFUSE_PUBLIC_KEY"
langfuse_secret_key: "os.environ/LANGFUSE_SECRET_KEY"
langfuse_host: "os.environ/LANGFUSE_HOST"
# 新增：mcp-server webhook 回调
- callback_name: "webhook"
callback_type: "failure"
callback_vars:
webhook_url: "os.environ/MCP_SERVER_CALLBACK_URL"
3.6 Step 6: 环境变量配置
文件: .env 或 docker-compose.yml
# LiteLLM Gateway 配置
MCP_SERVER_CALLBACK_URL=http://mcp-server:8000/api/v1/billing/litellm-callback
# 可选：Callback 签名密钥（用于验证请求来源）
LITELLM_CALLBACK_SECRET=your-secret-key
3.7 Step 7: 修改 Dashboard 查询
文件: services/mcp-server/monitoring.py（修改 EU 消耗查询）
async def get_eu_consumption_stats(
self,
tenant_id: Optional[str] = None,
channel_id: Optional[str] = None,
start_date: Optional[datetime] = None,
end_date: Optional[datetime] = None,
db: AsyncSession = None
) -> Dict[str, Any]:
"""获取 EU 消耗统计
合并查询：
1. agent_billing_records - Agent 运行时计费
2. model_billing_records - 模型 Token 计费
"""
from models import AgentBillingRecord, ModelBillingRecord
# Agent 运行时 EU 消耗
agent_query = select(
func.coalesce(func.sum(AgentBillingRecord.cost), 0).label("agent_eu")
)
if tenant_id:
agent_query = agent_query.where(AgentBillingRecord.user_id == tenant_id)
if channel_id:
agent_query = agent_query.where(AgentBillingRecord.channel_id == channel_id)
if start_date:
agent_query = agent_query.where(AgentBillingRecord.created_at >= start_date)
if end_date:
agent_query = agent_query.where(AgentBillingRecord.created_at <= end_date)
agent_result = await db.execute(agent_query)
agent_eu = agent_result.scalar() or Decimal("0")
# 模型 Token EU 消耗
model_query = select(
func.coalesce(func.sum(ModelBillingRecord.eu_consumed), 0).label("model_eu")
)
if tenant_id:
model_query = model_query.where(ModelBillingRecord.tenant_id == tenant_id)
if channel_id:
model_query = model_query.where(ModelBillingRecord.channel_id == channel_id)
if start_date:
model_query = model_query.where(ModelBillingRecord.created_at >= start_date)
if end_date:
model_query = model_query.where(ModelBillingRecord.created_at <= end_date)
model_result = await db.execute(model_query)
model_eu = model_result.scalar() or Decimal("0")
total_eu = agent_eu + model_eu
return {
"totalEuConsumed": float(total_eu),
"agentEuConsumed": float(agent_eu),
"modelEuConsumed": float(model_eu),
"breakdown": {
"agentRuntime": float(agent_eu),
"modelTokens": float(model_eu)
}
}
4. 测试验证
4.1 单元测试
文件: services/mcp-server/tests/test_billing_webhook.py
import pytest
from httpx import AsyncClient
from datetime import datetime
@pytest.mark.asyncio
async def test_litellm_callback_success(client: AsyncClient, db):
"""测试正常的 LiteLLM Callback"""
callback_data = {
"id": "chatcmpl-test-001",
"call_type": "completion",
"model": "azure/gpt-4",
"api_key": "sk-test-key",
"team_id": "team-test",
"usage": {
"prompt_tokens": 100,
"completion_tokens": 50,
"total_tokens": 150
},
"response_cost": 0.0045,
"startTime": "2026-01-08T10:00:00Z",
"endTime": "2026-01-08T10:00:02Z",
"status": "success",
"metadata": {
"tenant_id": "test-tenant-uuid",
"channel_id": "test-channel-uuid"
}
}
response = await client.post(
"/api/v1/billing/litellm-callback",
json=callback_data
)
assert response.status_code == 200
data = response.json()
assert data["success"] == True
assert data["record_id"] is not None
@pytest.mark.asyncio
async def test_litellm_callback_idempotent(client: AsyncClient, db):
"""测试 Callback 幂等性"""
callback_data = {
"id": "chatcmpl-test-002",
"model": "azure/gpt-4",
"usage": {"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150}
}
# 第一次调用
response1 = await client.post("/api/v1/billing/litellm-callback", json=callback_data)
assert response1.status_code == 200
# 第二次调用（相同 call_id）
response2 = await client.post("/api/v1/billing/litellm-callback", json=callback_data)
assert response2.status_code == 200
assert response2.json()["message"] == "Already processed"
4.2 集成测试
# 1. 启动服务
docker-compose up -d mcp-server litellm-gateway
# 2. 模拟 LiteLLM Callback
curl -X POST http://localhost:8000/api/v1/billing/litellm-callback \
-H "Content-Type: application/json" \
-d '{
"id": "test-call-001",
"model": "azure/gpt-4",
"usage": {
"prompt_tokens": 100,
"completion_tokens": 50,
"total_tokens": 150
},
"response_cost": 0.0045,
"status": "success"
}'
# 3. 验证记录
psql -d taiji -c "SELECT * FROM model_billing_records ORDER BY created_at DESC LIMIT 5;"
# 4. 验证 Dashboard EU 消耗
curl http://localhost:8000/api/v1/monitoring/dashboard \
-H "Authorization: Bearer $TOKEN"
4.3 端到端测试
# 1. 通过 LiteLLM 发起真实模型调用
curl -X POST http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-tenant-key" \
-H "Content-Type: application/json" \
-d '{
"model": "azure/gpt-4",
"messages": [{"role": "user", "content": "Hello"}]
}'
# 2. 等待 Callback（通常 1-2 秒）
sleep 2
# 3. 检查计费记录
psql -d taiji -c "SELECT model_name, input_tokens, output_tokens, eu_consumed FROM model_billing_records ORDER BY created_at DESC LIMIT 1;"
5. 部署检查清单
5.1 部署前检查
- 数据库迁移已执行
model_billing_records表已创建billing_webhook.py路由已注册- 环境变量
MCP_SERVER_CALLBACK_URL已配置 - LiteLLM 配置已更新（添加 webhook callback）
- 网络连通性：LiteLLM → mcp-server
5.2 部署后验证
- Callback 端点健康检查通过
- 模拟 Callback 请求成功
- 真实模型调用触发 Callback
- 计费记录正确写入数据库
- Dashboard EU 消耗显示正确
6. 监控和告警
6.1 关键指标
| 指标 | 说明 | 告警阈值 |
|---|---|---|
callback_requests_total |
Callback 请求总数 | - |
callback_errors_total |
Callback 错误数 | > 10/min |
callback_latency_ms |
Callback 处理延迟 | > 500ms |
model_billing_records_count |
计费记录数 | - |
6.2 日志监控
# 监控 Callback 日志
tail -f /var/log/mcp-server/app.log | grep "LiteLLM Callback"
# 监控错误
tail -f /var/log/mcp-server/app.log | grep -E "Callback.*失败|error"
7. 故障排查
7.1 常见问题
| 问题 | 可能原因 | 解决方案 |
|---|---|---|
| Callback 未收到 | LiteLLM 配置错误 | 检查 success_callback 配置 |
| Callback 返回 500 | 数据库连接失败 | 检查 DB 连接 |
| EU 消耗为 0 | 租户 ID 解析失败 | 检查 metadata 配置 |
| 重复记录 | 幂等性检查失败 | 检查 litellm_call_id 唯一约束 |
7.2 调试命令
# 检查 LiteLLM Callback 配置
curl http://localhost:4000/config | jq '.callbacks'
# 检查 mcp-server 路由
curl http://localhost:8000/openapi.json | jq '.paths | keys | map(select(contains("billing")))'
# 检查数据库记录
psql -d taiji -c "SELECT COUNT(*) FROM model_billing_records WHERE created_at > NOW() - INTERVAL '1 hour';"
8. 相关文件索引
| 文件 | 说明 |
|---|---|
services/mcp-server/migrations/012_add_model_billing_records.sql |
数据库迁移（待创建） |
services/mcp-server/app/routes/billing_webhook.py |
Webhook 路由（待创建） |
services/mcp-server/models.py |
数据模型（需添加 ModelBillingRecord） |
services/model-gateway/config/litellm.yaml |
LiteLLM 配置（需修改） |
services/mcp-server/monitoring.py |
Dashboard 查询（需修改） |
9. 后续优化
9.1 短期优化
- 签名验证：启用
X-LiteLLM-Signature验证请求来源 - 批量处理：支持批量 Callback 减少请求数
- 异步队列：使用 NATS 异步处理 Callback，提高吞吐量
9.2 长期优化
- 实时仪表板：WebSocket 推送 EU 消耗变化
- 预算告警：EU 余额不足时实时告警
- 成本分析：按模型、时间段分析成本趋势
- 数据归档：定期归档历史计费记录
10. 总结
10.1 实施优先级
高优先级（必须完成）：
├── Step 1: 数据库迁移
├── Step 2: 数据模型
├── Step 3: Webhook 路由
└── Step 5: LiteLLM 配置
中优先级（建议完成）：
├── Step 6: Dashboard 查询修改
└── Step 7: 测试验证
低优先级（可选）：
├── 签名验证
├── 批量处理
└── 异步队列
10.2 预计工作量
| 步骤 | 工作内容 | 复杂度 |
|---|---|---|
| Step 1 | 数据库迁移 | 低 |
| Step 2 | 数据模型 | 低 |
| Step 3 | Webhook 路由 | 中 |
| Step 4 | 注册路由 | 低 |
| Step 5 | LiteLLM 配置 | 低 |
| Step 6 | Dashboard 修改 | 中 |
| Step 7 | 测试验证 | 中 |
10.3 风险评估
| 风险 | 影响 | 缓解措施 |
|---|---|---|
| LiteLLM 版本兼容性 | 中 | 测试环境验证 |
| 网络延迟 | 低 | 异步处理 |
| 数据丢失 | 高 | 幂等性设计 + 日志 |
| 性能瓶颈 | 中 | 批量处理 + 队列 |