taijibaga/taiji-AI-PAD
9
0
Fork 0
forked from xiaohei/taiji-AI-PAD
Code Pull Requests Activity

更新计费与资源管理API

Browse Source
This commit is contained in:
Ubuntu
2025-12-26 07:49:38 +00:00
parent ca824333c1
commit 0d4c57c6b0
16 changed files with 4392 additions and 2040 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Docs/前后端调试说明/API接口文档.md
+798 -12
View File
@@ -1,6 +1,6 @@
# taiji-AI-PAD API 接口文档
**版本**: v2.5.0
**版本**: v2.6.0
**更新时间**: 2025年12月26日
**基础URL**:
- Data Ingestion 服务: `http://localhost:8001` (容器内8000→主机8001)
@@ -20,16 +20,17 @@
4. [用户侧平台 API](#用户侧平台-api)
5. [渠道合作伙伴 API](#渠道合作伙伴-api)
6. [超级管理员 API](#超级管理员-api)
7. [供应商管理 API](#供应商管理-api)
8. [MCP 监控 API](#mcp-监控-api)
9. [WebSocket API](#websocket-api)
10. [通用响应格式](#通用响应格式)
11. [错误码说明](#错误码说明)
12. [认证说明](#认证说明)
13. [业务规则](#业务规则)
14. [交互式 API 文档](#交互式-api-文档)
15. [前端集成示例](#前端集成示例)
16. [完整 API 测试流程](#完整-api-测试流程)
7. [计费与资源管理 API](#计费与资源管理-api) ✨ **新增**
8. [供应商管理 API](#供应商管理-api)
9. [MCP 监控 API](#mcp-监控-api)
10. [WebSocket API](#websocket-api)
11. [通用响应格式](#通用响应格式)
12. [错误码说明](#错误码说明)
13. [认证说明](#认证说明)
14. [业务规则](#业务规则)
15. [交互式 API 文档](#交互式-api-文档)
16. [前端集成示例](#前端集成示例)
17. [完整 API 测试流程](#完整-api-测试流程)
---
@@ -2634,6 +2635,781 @@ curl -X PUT "http://localhost:8002/api/admin/resources/agents/agent-uuid-1/confi
---
## 计费与资源管理 API
**基础URL**: `http://localhost:8002/api/billing-admin`
> **权限说明**: 以下接口需要管理员权限（super_admin、billing_admin、operations_admin）
### 配额管理
#### 1. 获取用户配额信息
**GET** `/api/billing-admin/quota/user/{user_id}`
获取指定用户的配额汇总信息，包括余额、速率限制、活跃预警等。
**响应示例**:
```json
{
"success": true,
"data": {
"hasQuota": true,
"alertType": null,
"balance": {
"balance": 2450.50,
"creditLimit": 5000.00,
"available": 7450.50,
"dailyAvgCost": 85.30,
"estimatedDays": 87.3
},
"rateLimit": {
"currentRpm": 12,
"rpmLimit": 60,
"allowed": true
},
"activeAlerts": 0,
"alerts": []
}
}
```
---
#### 2. 获取渠道配额信息
**GET** `/api/billing-admin/quota/channel/{channel_id}`
**响应示例**:
```json
{
"success": true,
"data": {
"hasQuota": true,
"alertType": null,
"channelCredit": 100000.00,
"monthlyUsage": 15680.50,
"usagePercent": 15.68
}
}
```
---
#### 3. 获取配额预警列表
**GET** `/api/billing-admin/quota/alerts`
**查询参数**:
- `user_id` (string, 可选): 用户ID筛选
- `channel_id` (string, 可选): 渠道ID筛选
**响应示例**:
```json
{
"success": true,
"data": {
"alerts": [
{
"id": "alert-uuid-1",
"alertType": "balance_warning",
"thresholdPercent": 20,
"currentValue": 150.00,
"thresholdValue": 200.00,
"status": "active",
"createdAt": "2025-12-26T10:00:00Z"
}
],
"count": 1
}
}
```
---
#### 4. 确认配额预警
**PUT** `/api/billing-admin/quota/alerts/{alert_id}/acknowledge`
**响应示例**:
```json
{
"success": true,
"message": "预警已确认"
}
```
---
#### 5. 解决配额预警
**PUT** `/api/billing-admin/quota/alerts/{alert_id}/resolve`
**响应示例**:
```json
{
"success": true,
"message": "预警已解决"
}
```
---
### 资源监控
#### 6. 获取平台资源概览
**GET** `/api/billing-admin/resources/overview`
获取平台整体资源使用概览（管理员视图）。
**响应示例**:
```json
{
"success": true,
"data": {
"todayCalls": 1580,
"monthCalls": 45680,
"activeUsersToday": 85,
"activeAgents": 125,
"monthTotalEu": 4568,
"timestamp": "2025-12-26T12:00:00Z"
}
}
```
---
#### 7. 获取用户资源使用汇总
**GET** `/api/billing-admin/resources/user/{user_id}`
**查询参数**:
- `start_date` (string, 必需): 开始日期 (ISO 8601)
- `end_date` (string, 必需): 结束日期 (ISO 8601)
**响应示例**:
```json
{
"success": true,
"data": {
"totalCpuSeconds": 12580.5,
"totalMemoryMbSeconds": 458720.0,
"totalNetworkBytes": 156800000,
"totalStorageBytes": 52428800,
"totalApiCalls": 1580,
"startDate": "2025-12-01T00:00:00Z",
"endDate": "2025-12-26T23:59:59Z"
}
}
```
---
#### 8. 获取资源使用趋势
**GET** `/api/billing-admin/resources/trends`
**查询参数**:
- `user_id` (string, 必需): 用户ID
- `period` (string, 可选): 时间范围，可选值: `7d`, `30d`, `90d` (默认: `7d`)
- `granularity` (string, 可选): 粒度，可选值: `hourly`, `daily` (默认: `daily`)
**响应示例**:
```json
{
"success": true,
"data": {
"trends": [
{
"periodStart": "2025-12-25T00:00:00Z",
"periodEnd": "2025-12-25T23:59:59Z",
"cpuSeconds": 458.5,
"memoryMbSeconds": 16720.0,
"networkBytes": 5680000,
"apiCalls": 58
}
]
}
}
```
---
#### 9. 获取Agent资源统计
**GET** `/api/billing-admin/resources/agent/{agent_id}`
**查询参数**:
- `start_date` (string, 必需): 开始日期
- `end_date` (string, 必需): 结束日期
**响应示例**:
```json
{
"success": true,
"data": {
"agentId": "agent-uuid-1",
"totalExecutions": 1580,
"avgExecutionTime": 145.2,
"totalEuConsumed": 158.0,
"successRate": 98.5,
"startDate": "2025-12-01T00:00:00Z",
"endDate": "2025-12-26T23:59:59Z"
}
}
```
---
### 事件管理
#### 10. 获取待处理事件
**GET** `/api/billing-admin/events/pending`
获取待处理的计费事件列表。
**查询参数**:
- `limit` (int, 可选): 返回数量限制 (默认: 100, 最大: 1000)
**响应示例**:
```json
{
"success": true,
"data": {
"events": [
{
"id": "event-uuid-1",
"eventId": "evt_abc123",
"eventType": "execution.end",
"userId": "user-uuid-1",
"agentId": "agent-uuid-1",
"payload": {
"success": true,
"duration_ms": 1250,
"eu_consumed": 0.13
},
"status": "pending",
"createdAt": "2025-12-26T12:00:00Z"
}
],
"count": 1
}
}
```
---
#### 11. 重试失败事件
**POST** `/api/billing-admin/events/retry-failed`
**查询参数**:
- `max_retries` (int, 可选): 最大重试次数 (默认: 3, 最大: 10)
**响应示例**:
```json
{
"success": true,
"data": {
"retriedCount": 5
},
"message": "已重试 5 个事件"
}
```
---
#### 12. 获取事件统计
**GET** `/api/billing-admin/events/stats`
**查询参数**:
- `start_date` (string, 必需): 开始日期
- `end_date` (string, 必需): 结束日期
**响应示例**:
```json
{
"success": true,
"data": {
"startDate": "2025-12-01T00:00:00Z",
"endDate": "2025-12-26T23:59:59Z",
"byStatus": {
"completed": 4520,
"pending": 15,
"failed": 3
},
"byType": {
"execution.end": 4200,
"execution.start": 4200,
"balance.deduct": 138
},
"total": 4538
}
}
```
---
### 追踪管理
#### 13. 获取执行追踪详情
**GET** `/api/billing-admin/traces/execution/{execution_id}`
获取单个执行的完整追踪信息。
**响应示例**:
```json
{
"success": true,
"data": {
"executionId": "exec-uuid-1",
"traceId": "trace-abc123",
"spans": [
{
"spanId": "span-1",
"parentSpanId": null,
"operationName": "agent_execute",
"operationType": "agent_call",
"startedAt": "2025-12-26T12:00:00Z",
"endedAt": "2025-12-26T12:00:01.250Z",
"durationMs": 1250,
"status": "success",
"inputData": {"prompt": "***REDACTED***"},
"outputData": {"result": "..."},
"tokensUsed": 450,
"euConsumed": 0.13
}
],
"totalDurationMs": 1250,
"totalEuConsumed": 0.13,
"spanCount": 1
}
}
```
---
#### 14. 查询追踪记录
**GET** `/api/billing-admin/traces`
**查询参数**:
- `user_id` (string, 可选): 用户ID筛选
- `agent_id` (string, 可选): Agent ID筛选
- `status` (string, 可选): 状态筛选 (`running`, `success`, `error`)
- `start_date` (string, 可选): 开始日期
- `end_date` (string, 可选): 结束日期
- `page` (int, 可选): 页码 (默认: 1)
- `page_size` (int, 可选): 每页数量 (默认: 20, 最大: 100)
**响应示例**:
```json
{
"success": true,
"data": {
"total": 150,
"page": 1,
"pageSize": 20,
"totalPages": 8,
"traces": [
{
"traceId": "trace-abc123",
"executionId": "exec-uuid-1",
"agentId": "agent-uuid-1",
"agentName": "weather-agent",
"userId": "user-uuid-1",
"startedAt": "2025-12-26T12:00:00Z",
"endedAt": "2025-12-26T12:00:01.250Z",
"durationMs": 1250,
"spanCount": 3,
"totalTokens": 450,
"totalEu": 0.13
}
]
}
}
```
---
#### 15. 获取追踪统计
**GET** `/api/billing-admin/traces/stats`
**查询参数**:
- `user_id` (string, 可选): 用户ID筛选
- `start_date` (string, 必需): 开始日期
- `end_date` (string, 必需): 结束日期
**响应示例**:
```json
{
"success": true,
"data": {
"startDate": "2025-12-01T00:00:00Z",
"endDate": "2025-12-26T23:59:59Z",
"totalTraces": 4200,
"totalSpans": 12600,
"totalDurationMs": 5250000,
"avgDurationMs": 1250.0,
"totalTokens": 1890000,
"totalEu": 546.0,
"byStatus": {
"success": 4150,
"error": 50
},
"byOperationType": [
{"operationType": "agent_call", "count": 4200, "avgDurationMs": 850.0},
{"operationType": "tool_call", "count": 6300, "avgDurationMs": 120.5},
{"operationType": "llm_call", "count": 2100, "avgDurationMs": 980.0}
]
}
}
```
---
### 审计日志
#### 16. 查询审计日志
**GET** `/api/billing-admin/audit/logs`
**查询参数**:
- `user_id` (string, 可选): 用户ID筛选
- `action` (string, 可选): 操作类型筛选
- `resource_type` (string, 可选): 资源类型筛选
- `success` (bool, 可选): 成功/失败筛选
- `start_date` (string, 可选): 开始日期
- `end_date` (string, 可选): 结束日期
- `page` (int, 可选): 页码 (默认: 1)
- `page_size` (int, 可选): 每页数量 (默认: 20, 最大: 100)
**可用的action类型**:
| 操作类型 | 说明 |
|---------|------|
| `auth.login` | 用户登录 |
| `auth.logout` | 用户登出 |
| `auth.password_change` | 密码修改 |
| `user.create` | 创建用户 |
| `user.update` | 更新用户 |
| `user.delete` | 删除用户 |
| `channel.create` | 创建渠道 |
| `channel.update` | 更新渠道 |
| `channel.delete` | 删除渠道 |
| `agent.create` | 创建Agent |
| `agent.delete` | 删除Agent |
| `application.approve` | 审批通过 |
| `application.reject` | 审批拒绝 |
| `billing.charge` | 计费扣款 |
| `provider.pricing_update` | 更新模型定价 |
**响应示例**:
```json
{
"success": true,
"data": {
"total": 500,
"page": 1,
"pageSize": 20,
"totalPages": 25,
"logs": [
{
"id": "log-uuid-1",
"action": "channel.create",
"actionName": "创建渠道",
"resourceType": "channel",
"resourceId": "channel-uuid-1",
"userId": "admin-uuid-1",
"userName": "超级管理员",
"success": true,
"details": {"channelName": "合作渠道A"},
"errorMessage": null,
"ipAddress": "192.168.1.100",
"createdAt": "2025-12-26T10:00:00Z"
}
]
}
}
```
---
#### 17. 获取审计日志汇总
**GET** `/api/billing-admin/audit/summary`
**查询参数**:
- `start_date` (string, 必需): 开始日期
- `end_date` (string, 必需): 结束日期
**响应示例**:
```json
{
"success": true,
"data": {
"startDate": "2025-12-01T00:00:00Z",
"endDate": "2025-12-26T23:59:59Z",
"total": 1580,
"successTotal": 1550,
"failTotal": 30,
"byAction": [
{"action": "auth.login", "actionName": "用户登录", "count": 850, "successCount": 820, "failCount": 30},
{"action": "agent.create", "actionName": "创建Agent", "count": 45, "successCount": 45, "failCount": 0}
],
"byResourceType": [
{"resourceType": "user", "count": 520},
{"resourceType": "agent", "count": 380},
{"resourceType": "channel", "count": 150}
]
}
}
```
---
#### 18. 获取用户活动历史
**GET** `/api/billing-admin/audit/user/{user_id}/activity`
**查询参数**:
- `days` (int, 可选): 天数 (默认: 30, 最大: 90)
**响应示例**:
```json
{
"success": true,
"data": {
"activity": [
{
"action": "auth.login",
"actionName": "用户登录",
"resourceType": "user",
"resourceId": "user-uuid-1",
"success": true,
"ipAddress": "192.168.1.100",
"createdAt": "2025-12-26T08:00:00Z"
}
]
}
}
```
---
### 供应商健康检查
#### 19. 获取所有供应商健康状态
**GET** `/api/billing-admin/providers/health`
**响应示例**:
```json
{
"success": true,
"data": {
"providers": [
{
"providerId": "provider-uuid-1",
"providerName": "OpenAI",
"provider": "openai",
"status": "active",
"isHealthy": true,
"lastResponseTimeMs": 45,
"lastCheckAt": "2025-12-26T12:00:00Z",
"uptime24h": 99.9
}
]
}
}
```
---
#### 20. 获取供应商健康详情
**GET** `/api/billing-admin/providers/{provider_id}/health`
**查询参数**:
- `hours` (int, 可选): 统计时间范围 (默认: 24, 最大: 168)
**响应示例**:
```json
{
"success": true,
"data": {
"providerId": "provider-uuid-1",
"providerName": "OpenAI",
"currentStatus": "active",
"period": "24h",
"totalChecks": 1440,
"healthyCount": 1438,
"unhealthyCount": 2,
"uptimePercent": 99.86,
"avgResponseTimeMs": 42.5,
"maxResponseTimeMs": 180,
"minResponseTimeMs": 25,
"recentChecks": [
{
"isHealthy": true,
"responseTimeMs": 45,
"statusCode": 200,
"errorMessage": null,
"createdAt": "2025-12-26T12:00:00Z"
}
]
}
}
```
---
#### 21. 执行供应商健康检查
**POST** `/api/billing-admin/providers/health-check`
立即执行所有供应商的健康检查。
**响应示例**:
```json
{
"success": true,
"data": {
"timestamp": "2025-12-26T12:00:00Z",
"totalProviders": 3,
"healthyCount": 3,
"unhealthyCount": 0,
"checks": [
{
"providerId": "provider-uuid-1",
"providerName": "OpenAI",
"isHealthy": true,
"responseTimeMs": 45,
"statusCode": 200,
"errorMessage": null
}
]
}
}
```
---
### 模型定价管理
#### 22. 获取模型定价列表
**GET** `/api/billing-admin/pricing/models`
**查询参数**:
- `provider_id` (string, 可选): 供应商ID筛选
- `model_name` (string, 可选): 模型名称模糊搜索
**响应示例**:
```json
{
"success": true,
"data": {
"pricing": [
{
"id": "pricing-uuid-1",
"providerId": "provider-uuid-1",
"providerName": "OpenAI",
"modelName": "gpt-4o",
"inputPricePer1k": 0.005,
"outputPricePer1k": 0.015,
"euPer1kTokens": 0.1,
"maxContextLength": 128000,
"maxOutputTokens": 4096,
"isActive": true,
"effectiveFrom": "2025-12-01T00:00:00Z"
}
]
}
}
```
---
#### 23. 创建/更新模型定价
**POST** `/api/billing-admin/pricing/models`
**请求体**:
```json
{
"providerId": "provider-uuid-1",
"modelName": "gpt-4o-mini",
"inputPricePer1k": 0.00015,
"outputPricePer1k": 0.0006,
"euPer1kTokens": 0.05,
"maxContextLength": 128000,
"maxOutputTokens": 16384
}
```
**请求参数说明**:
- `providerId` (string, 必需): 供应商ID
- `modelName` (string, 必需): 模型名称
- `inputPricePer1k` (float, 必需): 输入价格（每1K tokens），单位USD
- `outputPricePer1k` (float, 必需): 输出价格（每1K tokens），单位USD
- `euPer1kTokens` (float, 可选): EU转换率（默认: 0.1）
- `maxContextLength` (int, 可选): 最大上下文长度（默认: 4096）
- `maxOutputTokens` (int, 可选): 最大输出tokens（默认: 2048）
**响应示例**:
```json
{
"success": true,
"data": {
"id": "pricing-uuid-2",
"modelName": "gpt-4o-mini"
},
"message": "模型定价已更新"
}
```
---
#### 24. 计算模型调用成本
**POST** `/api/billing-admin/pricing/calculate`
计算指定模型调用的成本预估。
**查询参数**:
- `model_name` (string, 必需): 模型名称
- `input_tokens` (int, 必需): 输入tokens数量
- `output_tokens` (int, 必需): 输出tokens数量
**请求示例**:
```bash
curl -X POST "http://localhost:8002/api/billing-admin/pricing/calculate?model_name=gpt-4o&input_tokens=1000&output_tokens=500" \
-H "Authorization: Bearer $TOKEN"
```
**响应示例**:
```json
{
"success": true,
"data": {
"modelName": "gpt-4o",
"inputTokens": 1000,
"outputTokens": 500,
"inputCost": 0.005,
"outputCost": 0.0075,
"totalCost": 0.0125,
"euConsumed": 0.15
}
}
```
---
## 供应商管理 API
**基础URL**: `http://localhost:8002/api/providers`
@@ -2926,7 +3702,7 @@ def get_dashboard(token: str):
---
**文档版本**: v2.5.0
**文档版本**: v2.6.0
**最后更新**: 2025年12月26日
**维护者**: taiji-AI-PAD 项目组
@@ -3057,6 +3833,16 @@ echo "=== API测试完成 ==="
## 更新日志
- **v2.6.0** (2025-12-26): **新增计费与资源管理API**
- ✅ 新增配额管理API（用户配额、渠道配额、配额预警）
- ✅ 新增资源监控API（平台概览、用户资源、使用趋势、Agent统计）
- ✅ 新增事件管理API（待处理事件、重试失败、事件统计）
- ✅ 新增追踪管理API（执行追踪详情、追踪查询、追踪统计）
- ✅ 新增审计日志API（日志查询、汇总统计、用户活动历史）
- ✅ 新增供应商健康检查API（健康状态、健康详情、手动检查）
- ✅ 新增模型定价管理API（定价列表、创建/更新定价、成本计算）
- ✅ 新增数据模型：TokenBlacklist、ResourceUsage、QuotaAlert、ModelPricing、ProviderHealthCheck、AgentTrace、BillingEvent
- ✅ 增强JWT认证：登出时将Token加入黑名单
- **v2.5.0** (2025-12-26): **权限系统重构**
- ✅ 重新设计权限系统，区分计费管理员和运维管理员
- ✅ billing_admin（计费管理员）：完整写入权限（创建渠道、管理租户、计费操作、审批申请）
Docs/项目文档/API接口文档_监控功能.md
-723
View File
@@ -1,723 +0,0 @@
# taiji-AI-PAD API 接口文档 - 监控功能
**版本**: v2.0
**创建时间**: 2025年12月23日
**最后更新**: 2025年12月23日
---
## 📋 目录
1. [概述](#概述)
2. [基础信息](#基础信息)
3. [监控API端点](#监控api端点)
4. [请求/响应示例](#请求响应示例)
5. [错误处理](#错误处理)
6. [集成示例](#集成示例)
---
## 1. 概述
本文档描述了 taiji-AI-PAD 平台监控功能的 API 接口。监控功能提供系统健康检查、性能指标、资源使用、服务统计、性能趋势和系统告警等功能。
### 1.1 功能特性
- ✅ 系统健康检查
- ✅ 实时性能指标（CPU、内存、磁盘）
- ✅ 服务统计信息（Agent、执行、工具、用户）
- ✅ 性能趋势分析
- ✅ 系统告警
- ✅ 监控仪表板（聚合数据）
### 1.2 监控指标
- **系统资源**: CPU使用率、内存使用、磁盘使用
- **服务指标**: 活跃Agent数、执行次数、成功率、平均响应时间
- **业务指标**: 日活用户、EU消耗、成本统计
- **告警信息**: 资源告警、服务告警
---
## 2. 基础信息
### 2.1 基础URL
```
http://localhost:8002
```
### 2.2 认证方式
当前版本无需认证，未来版本将支持 JWT Token 认证。
### 2.3 响应格式
所有API响应均为 JSON 格式，使用 UTF-8 编码。
### 2.4 HTTP状态码
| 状态码 | 说明 |
|--------|------|
| 200 | 请求成功 |
| 400 | 请求参数错误 |
| 500 | 服务器内部错误 |
---
## 3. 监控API端点
### 3.1 系统健康检查
#### GET /health
获取系统健康状态。
**请求参数**: 无
**响应示例**:
```json
{
"status": "healthy",
"timestamp": "2025-12-23T07:30:00.000000",
"services": {
"database": "healthy",
"redis": "healthy",
"nats": "healthy"
}
}
```
**响应字段说明**:
- `status`: 系统整体状态 (`healthy`, `degraded`, `unhealthy`)
- `timestamp`: 检查时间戳
- `services`: 各服务健康状态
---
### 3.2 系统性能指标
#### GET /api/v1/monitoring/metrics
获取系统实时性能指标。
**请求参数**: 无
**响应示例**:
```json
{
"timestamp": "2025-12-23T07:30:00.000000",
"system": {
"cpu_usage_percent": 15.5,
"memory_usage_percent": 45.2,
"memory_used_mb": 2048.5,
"memory_total_mb": 4096.0,
"disk_usage_percent": 32.1,
"disk_used_gb": 128.5,
"disk_total_gb": 400.0
},
"services": {
"active_agents": 10,
"total_executions_24h": 1250,
"success_rate_percent": 98.5,
"avg_execution_time_ms": 125.5,
"daily_active_users": 25
},
"billing": {
"total_eu_consumed_24h": 1250.5,
"total_cost_24h": 12.50
}
}
```
**响应字段说明**:
- `system`: 系统资源使用情况
- `cpu_usage_percent`: CPU使用率（%）
- `memory_usage_percent`: 内存使用率（%）
- `memory_used_mb`: 已使用内存（MB）
- `memory_total_mb`: 总内存（MB）
- `disk_usage_percent`: 磁盘使用率（%）
- `disk_used_gb`: 已使用磁盘（GB）
- `disk_total_gb`: 总磁盘空间（GB）
- `services`: 服务指标（过去24小时）
- `active_agents`: 活跃Agent数量
- `total_executions_24h`: 总执行次数
- `success_rate_percent`: 成功率（%）
- `avg_execution_time_ms`: 平均执行时间（毫秒）
- `daily_active_users`: 日活用户数
- `billing`: 计费统计（过去24小时）
- `total_eu_consumed_24h`: 总EU消耗
- `total_cost_24h`: 总成本
---
### 3.3 服务统计信息
#### GET /api/v1/monitoring/stats
获取服务统计信息。
**请求参数**:
| 参数名 | 类型 | 必填 | 说明 |
|--------|------|------|------|
| service | string | 否 | 服务类型，可选值: `all`, `agents`, `executions`, `tools`, `users`，默认: `all` |
**请求示例**:
```
GET /api/v1/monitoring/stats?service=agents
```
**响应示例**:
```json
{
"timestamp": "2025-12-23T07:30:00.000000",
"stats": {
"agents": {
"total": 50,
"active": 45,
"inactive": 5,
"avg_executions": 125.5,
"avg_success_rate": 98.2
},
"executions": {
"total_7d": 8750,
"completed": 8600,
"failed": 100,
"running": 50,
"avg_time_ms": 125.5,
"total_eu": 8750.5
},
"tools": {
"total": 20,
"active": 18,
"total_calls": 12500,
"avg_success_rate": 99.5,
"avg_response_time_ms": 50.2
},
"users": {
"total": 100,
"active": 95,
"admins": 5
}
}
}
```
**响应字段说明**:
- `agents`: Agent统计
- `total`: 总Agent数
- `active`: 活跃Agent数
- `inactive`: 非活跃Agent数
- `avg_executions`: 平均执行次数
- `avg_success_rate`: 平均成功率
- `executions`: 执行统计（过去7天）
- `total_7d`: 总执行次数
- `completed`: 成功完成数
- `failed`: 失败数
- `running`: 运行中数
- `avg_time_ms`: 平均执行时间（毫秒）
- `total_eu`: 总EU消耗
- `tools`: 工具统计
- `total`: 总工具数
- `active`: 活跃工具数
- `total_calls`: 总调用次数
- `avg_success_rate`: 平均成功率
- `avg_response_time_ms`: 平均响应时间（毫秒）
- `users`: 用户统计
- `total`: 总用户数
- `active`: 活跃用户数
- `admins`: 管理员数
---
### 3.4 性能趋势数据
#### GET /api/v1/monitoring/trends
获取性能趋势数据。
**请求参数**:
| 参数名 | 类型 | 必填 | 说明 |
|--------|------|------|------|
| metric | string | 否 | 指标类型，可选值: `executions`, `eu_consumption`，默认: `executions` |
| period | string | 否 | 时间范围，可选值: `24h`, `7d`, `30d`，默认: `24h` |
| interval | string | 否 | 时间间隔，可选值: `1h`, `6h`, `1d`，默认: `1h` |
**请求示例**:
```
GET /api/v1/monitoring/trends?metric=executions&period=7d&interval=6h
```
**响应示例** (metric=executions):
```json
{
"metric": "executions",
"period": "7d",
"interval": "6h",
"data": [
{
"timestamp": "2025-12-23T00:00:00",
"count": 125,
"avg_time_ms": 120.5,
"success_rate": 98.5
},
{
"timestamp": "2025-12-23T06:00:00",
"count": 150,
"avg_time_ms": 125.2,
"success_rate": 99.0
}
]
}
```
**响应示例** (metric=eu_consumption):
```json
{
"metric": "eu_consumption",
"period": "24h",
"interval": "1h",
"data": [
{
"timestamp": "2025-12-23T00:00:00",
"eu_consumed": 50.5,
"cost": 0.50
},
{
"timestamp": "2025-12-23T01:00:00",
"eu_consumed": 52.3,
"cost": 0.52
}
]
}
```
**响应字段说明**:
- `metric`: 指标类型
- `period`: 时间范围
- `interval`: 时间间隔
- `data`: 趋势数据数组
- `timestamp`: 时间点
- `count`: 执行次数（executions指标）
- `avg_time_ms`: 平均执行时间（executions指标）
- `success_rate`: 成功率（executions指标）
- `eu_consumed`: EU消耗（eu_consumption指标）
- `cost`: 成本（eu_consumption指标）
---
### 3.5 系统告警
#### GET /api/v1/monitoring/alerts
获取系统告警信息。
**请求参数**:
| 参数名 | 类型 | 必填 | 说明 |
|--------|------|------|------|
| severity | string | 否 | 严重程度过滤，可选值: `warning`, `critical`, `info` |
**请求示例**:
```
GET /api/v1/monitoring/alerts?severity=critical
```
**响应示例**:
```json
{
"timestamp": "2025-12-23T07:30:00.000000",
"alerts": [
{
"severity": "warning",
"type": "high_cpu",
"message": "CPU使用率过高: 85.5%",
"timestamp": "2025-12-23T07:29:00.000000"
},
{
"severity": "critical",
"type": "low_disk",
"message": "磁盘空间不足: 92.1%",
"timestamp": "2025-12-23T07:25:00.000000"
}
],
"count": 2
}
```
**响应字段说明**:
- `timestamp`: 查询时间
- `alerts`: 告警列表
- `severity`: 严重程度 (`warning`, `critical`, `info`)
- `type`: 告警类型 (`high_cpu`, `high_memory`, `low_disk`, `high_failure_rate`)
- `message`: 告警消息
- `timestamp`: 告警时间
- `count`: 告警总数
**告警类型说明**:
- `high_cpu`: CPU使用率 > 80%
- `high_memory`: 内存使用率 > 85%
- `low_disk`: 磁盘使用率 > 90%
- `high_failure_rate`: 过去1小时内失败执行 > 10次
---
### 3.6 监控仪表板
#### GET /api/v1/monitoring/dashboard
获取监控仪表板数据（聚合所有监控信息）。
**请求参数**: 无
**响应示例**:
```json
{
"timestamp": "2025-12-23T07:30:00.000000",
"health": {
"status": "healthy",
"timestamp": "2025-12-23T07:30:00.000000",
"uptime_seconds": 86400,
"services": {
"database": "healthy",
"redis": "healthy",
"nats": "healthy"
}
},
"metrics": {
"timestamp": "2025-12-23T07:30:00.000000",
"system": {
"cpu_usage_percent": 15.5,
"memory_usage_percent": 45.2,
"disk_usage_percent": 32.1
},
"services": {
"active_agents": 10,
"total_executions_24h": 1250,
"success_rate_percent": 98.5
},
"billing": {
"total_eu_consumed_24h": 1250.5,
"total_cost_24h": 12.50
}
},
"stats": {
"agents": {
"total": 50,
"active": 45
},
"executions": {
"total_7d": 8750,
"completed": 8600
},
"tools": {
"total": 20,
"active": 18
},
"users": {
"total": 100,
"active": 95
}
},
"alerts": {
"items": [
{
"severity": "warning",
"type": "high_cpu",
"message": "CPU使用率过高: 85.5%",
"timestamp": "2025-12-23T07:29:00.000000"
}
],
"count": 1,
"critical_count": 0,
"warning_count": 1
}
}
```
**响应字段说明**:
- `health`: 系统健康状态
- `metrics`: 系统性能指标
- `stats`: 服务统计信息
- `alerts`: 系统告警
- `items`: 告警列表
- `count`: 告警总数
- `critical_count`: 严重告警数
- `warning_count`: 警告告警数
---
## 4. 请求/响应示例
### 4.1 cURL 示例
#### 获取系统性能指标
```bash
curl -X GET "http://localhost:8002/api/v1/monitoring/metrics"
```
#### 获取Agent统计
```bash
curl -X GET "http://localhost:8002/api/v1/monitoring/stats?service=agents"
```
#### 获取执行趋势（7天，6小时间隔）
```bash
curl -X GET "http://localhost:8002/api/v1/monitoring/trends?metric=executions&period=7d&interval=6h"
```
#### 获取严重告警
```bash
curl -X GET "http://localhost:8002/api/v1/monitoring/alerts?severity=critical"
```
#### 获取监控仪表板
```bash
curl -X GET "http://localhost:8002/api/v1/monitoring/dashboard"
```
### 4.2 Python 示例
```python
import httpx
import asyncio
async def get_monitoring_data():
base_url = "http://localhost:8002"
async with httpx.AsyncClient() as client:
# 获取系统指标
metrics = await client.get(f"{base_url}/api/v1/monitoring/metrics")
print("系统指标:", metrics.json())
# 获取服务统计
stats = await client.get(f"{base_url}/api/v1/monitoring/stats?service=all")
print("服务统计:", stats.json())
# 获取性能趋势
trends = await client.get(
f"{base_url}/api/v1/monitoring/trends",
params={"metric": "executions", "period": "24h", "interval": "1h"}
)
print("性能趋势:", trends.json())
# 获取告警
alerts = await client.get(f"{base_url}/api/v1/monitoring/alerts")
print("系统告警:", alerts.json())
# 获取监控仪表板
dashboard = await client.get(f"{base_url}/api/v1/monitoring/dashboard")
print("监控仪表板:", dashboard.json())
asyncio.run(get_monitoring_data())
```
### 4.3 JavaScript 示例
```javascript
const baseUrl = 'http://localhost:8002';
// 获取系统指标
async function getMetrics() {
const response = await fetch(`${baseUrl}/api/v1/monitoring/metrics`);
const data = await response.json();
console.log('系统指标:', data);
}
// 获取服务统计
async function getStats(service = 'all') {
const response = await fetch(`${baseUrl}/api/v1/monitoring/stats?service=${service}`);
const data = await response.json();
console.log('服务统计:', data);
}
// 获取性能趋势
async function getTrends(metric = 'executions', period = '24h', interval = '1h') {
const url = new URL(`${baseUrl}/api/v1/monitoring/trends`);
url.searchParams.append('metric', metric);
url.searchParams.append('period', period);
url.searchParams.append('interval', interval);
const response = await fetch(url);
const data = await response.json();
console.log('性能趋势:', data);
}
// 获取告警
async function getAlerts(severity = null) {
let url = `${baseUrl}/api/v1/monitoring/alerts`;
if (severity) {
url += `?severity=${severity}`;
}
const response = await fetch(url);
const data = await response.json();
console.log('系统告警:', data);
}
// 获取监控仪表板
async function getDashboard() {
const response = await fetch(`${baseUrl}/api/v1/monitoring/dashboard`);
const data = await response.json();
console.log('监控仪表板:', data);
}
// 使用示例
getMetrics();
getStats('agents');
getTrends('executions', '7d', '6h');
getAlerts('critical');
getDashboard();
```
---
## 5. 错误处理
### 5.1 错误响应格式
```json
{
"detail": "错误描述信息"
}
```
### 5.2 常见错误
| HTTP状态码 | 错误类型 | 说明 |
|-----------|---------|------|
| 400 | Bad Request | 请求参数错误 |
| 500 | Internal Server Error | 服务器内部错误 |
### 5.3 错误处理示例
```python
import httpx
async def get_metrics_safe():
try:
async with httpx.AsyncClient() as client:
response = await client.get("http://localhost:8002/api/v1/monitoring/metrics")
response.raise_for_status()
return response.json()
except httpx.HTTPStatusError as e:
print(f"HTTP错误: {e.response.status_code}")
print(f"错误信息: {e.response.text}")
except Exception as e:
print(f"其他错误: {e}")
```
---
## 6. 集成示例
### 6.1 实时监控仪表板
```python
import asyncio
import httpx
from datetime import datetime
async def update_dashboard():
"""每30秒更新一次监控仪表板"""
base_url = "http://localhost:8002"
while True:
try:
async with httpx.AsyncClient() as client:
response = await client.get(f"{base_url}/api/v1/monitoring/dashboard")
data = response.json()
# 显示关键指标
print(f"\n[{datetime.now()}] 监控仪表板")
print(f"系统状态: {data['health']['status']}")
print(f"CPU使用率: {data['metrics']['system']['cpu_usage_percent']:.1f}%")
print(f"内存使用率: {data['metrics']['system']['memory_usage_percent']:.1f}%")
print(f"活跃Agent: {data['metrics']['services']['active_agents']}")
print(f"24小时执行次数: {data['metrics']['services']['total_executions_24h']}")
print(f"成功率: {data['metrics']['services']['success_rate_percent']:.2f}%")
print(f"告警数量: {data['alerts']['count']} (严重: {data['alerts']['critical_count']})")
except Exception as e:
print(f"获取监控数据失败: {e}")
await asyncio.sleep(30)
# 运行监控
asyncio.run(update_dashboard())
```
### 6.2 告警通知
```python
import httpx
import asyncio
async def check_alerts():
"""检查系统告警并发送通知"""
base_url = "http://localhost:8002"
async with httpx.AsyncClient() as client:
# 获取严重告警
response = await client.get(f"{base_url}/api/v1/monitoring/alerts?severity=critical")
alerts = response.json()
if alerts['count'] > 0:
print(f"⚠️ 发现 {alerts['count']} 个严重告警:")
for alert in alerts['alerts']:
print(f" - {alert['message']} ({alert['type']})")
# 这里可以添加通知逻辑（邮件、短信、Slack等）
# 获取警告告警
response = await client.get(f"{base_url}/api/v1/monitoring/alerts?severity=warning")
alerts = response.json()
if alerts['count'] > 0:
print(f"⚠️ 发现 {alerts['count']} 个警告:")
for alert in alerts['alerts']:
print(f" - {alert['message']} ({alert['type']})")
asyncio.run(check_alerts())
```
---
## 7. 最佳实践
### 7.1 监控频率建议
- **系统指标**: 每30秒-1分钟查询一次
- **服务统计**: 每5-10分钟查询一次
- **性能趋势**: 根据需求，建议每1小时查询一次
- **系统告警**: 每1-5分钟检查一次
### 7.2 性能优化
- 使用 `/api/v1/monitoring/dashboard` 端点获取聚合数据，减少请求次数
- 对于趋势数据，合理选择时间范围和间隔，避免查询过大数据集
- 使用缓存机制，避免频繁查询数据库
### 7.3 告警阈值建议
- **CPU使用率**: > 80% 警告，> 90% 严重
- **内存使用率**: > 85% 警告，> 95% 严重
- **磁盘使用率**: > 85% 警告，> 90% 严重
- **失败率**: > 5% 警告，> 10% 严重
---
## 8. 更新日志
### v2.0 (2025-12-23)
- ✅ 新增系统性能指标API
- ✅ 新增服务统计信息API
- ✅ 新增性能趋势数据API
- ✅ 新增系统告警API
- ✅ 新增监控仪表板API
- ✅ 优化健康检查API
---
**文档版本**: v2.0
**最后更新**: 2025年12月23日
**维护者**: taiji-AI-PAD 开发团队
Docs/项目文档/MCP-Server功能任务分工.md
+23 -15
View File
@@ -95,7 +95,7 @@ services/mcp-server/
```
services/mcp-server/
├── models.py # 数据库模型
├── models.py # 数据库模型（含新增模型）
├── database.py # 数据库连接
├── monitoring.py # 监控配置
├── app/
@@ -103,12 +103,20 @@ services/mcp-server/
│ ├── permissions.py # 权限管理
│ ├── billing.py # 计费逻辑
│ ├── schemas.py # 管理相关数据模型
│ ├── token_blacklist.py # Token黑名单管理 ✅ 新增
│ ├── resource_monitor.py # 资源使用监控 ✅ 新增
│ ├── quota_manager.py # 配额预警管理 ✅ 新增
│ ├── audit.py # 审计日志模块 ✅ 新增
│ ├── events.py # NATS事件处理 ✅ 新增
│ ├── tracing.py # Agent轨迹追踪 ✅ 新增
│ ├── provider_health.py # 供应商健康检查 ✅ 新增
│ └── routes/
│ ├── admin.py # 超级管理员API
│ ├── channel.py # 渠道合作伙伴API
│ ├── user.py # 用户管理API
│ ├── auth.py # 认证端点
│ ├── providers.py # 模型供应商管理
│ ├── billing_admin.py # 计费与资源管理API ✅ 新增
│ ├── monitoring.py # 监控端点
│ ├── metrics.py # Prometheus指标
│ └── health.py # 健康检查
@@ -129,7 +137,7 @@ services/mcp-server/
| 任务ID | 任务名称 | 描述 | 预计工时 | 状态 |
|--------|----------|------|----------|------|
| B-2.1 | JWT认证增强 | Token刷新、黑名单机制 | 12h | ⚠️ 待开始 |
| B-2.1 | JWT认证增强 | Token刷新、黑名单机制 | 12h | ✅ 已完成 |
| B-2.2 | RBAC权限系统 | 基于角色的访问控制 (T5.3.3) | 16h | ✅ 已完成 |
| B-2.3 | 多租户隔离 | 多租户数据隔离(RLS) (T5.2.3) | 20h | ⚠️ 待开始 |
| B-2.4 | Pomerium集成 | Pomerium网关部署 (T5.3.1) | 16h | ⚠️ 待开始 |
@@ -138,10 +146,10 @@ services/mcp-server/
| 任务ID | 任务名称 | 描述 | 预计工时 | 状态 |
|--------|----------|------|----------|------|
| B-3.1 | EU计算公式 | EU计费公式实现 (T5.1.1) | 16h | ⚠️ 部分完成 |
| B-3.2 | 资源使用监控 | 资源使用采集与统计 (T5.1.2) | 20h | ⚠️ 待开始 |
| B-3.3 | NATS事件采集 | 计费事件消息队列 (T5.1.3) | 16h | ⚠️ 待开始 |
| B-3.4 | 预付费配额管理 | 配额预警与限制 (T5.1.4) | 12h | ⚠️ 待开始 |
| B-3.1 | EU计算公式 | EU计费公式实现 (T5.1.1) | 16h | ✅ 已完成 |
| B-3.2 | 资源使用监控 | 资源使用采集与统计 (T5.1.2) | 20h | ✅ 已完成 |
| B-3.3 | NATS事件采集 | 计费事件消息队列 (T5.1.3) | 16h | ✅ 已完成 |
| B-3.4 | 预付费配额管理 | 配额预警与限制 (T5.1.4) | 12h | ✅ 已完成 |
| B-3.5 | 充值与扣费 | 余额管理与账单生成 | 12h | ✅ 已完成 |
#### 4. 供应商管理 (P1 - 中优先级)
@@ -149,17 +157,17 @@ services/mcp-server/
| 任务ID | 任务名称 | 描述 | 预计工时 | 状态 |
|--------|----------|------|----------|------|
| B-4.1 | 供应商CRUD | 模型供应商增删改查 | 8h | ✅ 已完成 |
| B-4.2 | 模型定价管理 | 供应商模型定价配置 | 12h | ⚠️ 待开始 |
| B-4.3 | 供应商健康检查 | 供应商可用性监控 | 8h | ⚠️ 待开始 |
| B-4.2 | 模型定价管理 | 供应商模型定价配置 | 12h | ✅ 已完成 |
| B-4.3 | 供应商健康检查 | 供应商可用性监控 | 8h | ✅ 已完成 |
#### 5. 监控与审计 (P1 - 中优先级)
| 任务ID | 任务名称 | 描述 | 预计工时 | 状态 |
|--------|----------|------|----------|------|
| B-5.1 | Prometheus集成 | Prometheus指标导出 (T5.4.1) | 12h | ✅ 已完成 |
| B-5.2 | Agent轨迹追踪 | 执行轨迹记录与查询 (T5.4.2) | 20h | ⚠️ 待开始 |
| B-5.3 | 合规审计日志 | 操作日志与审计 (T5.4.3) | 16h | ⚠️ 待开始 |
| B-5.4 | 实时监控仪表盘 | 监控数据API | 12h | ⚠️ 部分完成 |
| B-5.2 | Agent轨迹追踪 | 执行轨迹记录与查询 (T5.4.2) | 20h | ✅ 已完成 |
| B-5.3 | 合规审计日志 | 操作日志与审计 (T5.4.3) | 16h | ✅ 已完成 |
| B-5.4 | 实时监控仪表盘 | 监控数据API | 12h | ✅ 已完成 |
#### 6. 用户管理 (P1 - 中优先级)
@@ -167,15 +175,15 @@ services/mcp-server/
|--------|----------|------|----------|------|
| B-6.1 | 用户资料管理 | 个人信息与偏好设置 | 8h | ✅ 已完成 |
| B-6.2 | 用户计费查询 | 用户账单与使用记录 | 8h | ✅ 已完成 |
| B-6.3 | 密码与安全 | 密码修改、安全设置 | 8h | ⚠️ 待开始 |
| B-6.3 | 密码与安全 | 密码修改、安全设置 | 8h | ✅ 已完成 |
### 📊 开发者B工时统计
| 优先级 | 任务数 | 已完成 | 进行中 | 待开始 | 总工时 |
|--------|--------|--------|--------|--------|--------|
| P0 | 12 | 5 | 0 | 7 | 160h |
| P1 | 10 | 4 | 0 | 6 | 112h |
| **合计** | **22** | **9** | **0** | **13** | **272h** |
| P0 | 12 | 10 | 0 | 2 | 160h |
| P1 | 10 | 10 | 0 | 0 | 112h |
| **合计** | **22** | **20** | **0** | **2** | **272h** |
---
Docs/项目文档/工程排期计划.md
-384
View File
@@ -1,384 +0,0 @@
# taiji-AI-PAD 工程排期计划
## 📋 项目总览
**项目名称**: Agent 赋能平台 (taiji-AI-PAD)
**项目类型**: 全栈工程化平台
**技术架构**: 五层技术平面
**预计总工期**: 18-24个月
**团队规模建议**: 12-15人
## 🎯 项目目标
构建一个将AI Agents从实验性脚本演进为工业级生产力单元的全栈工程化平台，通过标准化的智力资源分发与治理体系，整合异构数据，支持多模型动态切换，并具备透明的计费与安全隔离机制。
## 📅 分阶段排期
### Phase 1: 基础设施与数据接入层 (3-4个月)
**时间**: 2025年1月 - 2025年4月
**关键里程碑**:
- 完成全域数据接入系统
- 实现APILLAMA技术栈
- 建立RapidAPI生态集成
**详细排期**:
- **Week 1-2**: 项目初始化与开发环境搭建
- **Week 3-6**: RapidAPI集成与统一API Key管理
- **Week 7-10**: APILLAMA模型部署与API文档转换
- **Week 11-14**: OpenAPI/Swagger动态加载机制
- **Week 15-16**: 第一阶段测试与优化
### Phase 2: 模型抽象与治理层 (2-3个月)
**时间**: 2025年4月 - 2025年7月
**关键里程碑**:
- LiteLLM网关部署
- 多模型路由与负载均衡
- 上下文管理与成本控制
**详细排期**:
- **Week 1-3**: LiteLLM集成与100+模型API支持
- **Week 4-6**: 高可用路由与故障转移机制
- **Week 7-9**: 上下文窗口管理与会话截断
- **Week 10-12**: 性能监控与链路追踪集成
### Phase 3: Agent协议化封装 (3-4个月)
**时间**: 2025年7月 - 2025年11月
**关键里程碑**:
- MCP协议实现
- A2A通信协议支持
- 单体Agent标准化
**详细排期**:
- **Week 1-4**: MCP Server/Client实现
- **Week 5-8**: A2A协议与Agent Card系统
- **Week 9-12**: 单体Agent封装与标准化
- **Week 13-16**: Agent注册与发现机制
### Phase 4: 本地编排与集成平台 (2-3个月)
**时间**: 2025年11月 - 2026年2月
**关键里程碑**:
- 主流框架适配器
- MCP-First集成策略
- IDE与客户端支持
**详细排期**:
- **Week 1-3**: LangChain/CrewAI/AutoGen适配器
- **Week 4-6**: Cursor/Claude Desktop集成
- **Week 7-9**: 动态发现与热加载机制
- **Week 10-12**: 本地编排工具开发
### Phase 5: 计费治理与安全平台 (4-5个月)
**时间**: 2026年2月 - 2026年7月
**关键里程碑**:
- EU计费系统
- 多租户安全隔离
- 生产环境部署
**详细排期**:
- **Week 1-4**: 执行单元(EU)计费引擎
- **Week 5-8**: Firecracker/gVisor安全隔离
- **Week 9-12**: 多租户数据与网络隔离
- **Week 13-16**: Pomerium身份认证集成
- **Week 17-20**: 监控、审计与合规系统
### Phase 6: 优化与上线 (2-3个月)
**时间**: 2026年7月 - 2026年10月
**关键里程碑**:
- 性能优化与压力测试
- 文档完善与培训
- 正式上线与运营支持
## 🔄 并行开发策略
### 可并行模块
1. **数据接入层 + 模型治理层**: 两个团队可并行开发
2. **前端界面 + 后端API**: UI/UX团队可提前开始
3. **安全隔离 + 计费系统**: 基础设施团队独立进行
4. **文档编写 + 测试用例**: 贯穿整个开发过程
### 关键依赖关系
- Phase 2 依赖 Phase 1 的API标准化
- Phase 3 依赖 Phase 2 的模型抽象层
- Phase 4 依赖 Phase 3 的Agent标准
- Phase 5 需要前四个阶段的基础支撑
## ⚠️ 风险评估与应对
### 高风险项目
1. **APILLAMA模型性能**: 可能需要额外的模型微调时间
2. **多模型兼容性**: 不同厂商API的差异化处理
3. **安全隔离复杂度**: Firecracker/gVisor的生产环境稳定性
### 应对策略
1. 提前准备备选技术方案
2. 建立每周技术评审机制
3. 关键模块预留20%缓冲时间
## 📊 资源分配建议
### 人员配置 (12-15人)
- **架构师**: 1人 (全程)
- **后端开发**: 4-5人
- **前端开发**: 2人
- **DevOps工程师**: 2人
- **测试工程师**: 2人
- **产品经理**: 1人
- **项目经理**: 1人
### 技术栈培训计划
- **Month 1**: Golang, NATS, LiteLLM基础培训
- **Month 2**: MCP协议, A2A通信深度培训
- **Month 3**: Firecracker, 容器安全培训
- **Month 4**: 监控系统, 计费引擎培训
## 🎯 成功标准
### 技术指标
- API响应时间 < 100ms (P95)
- 系统可用性 > 99.9%
- 支持1000+并发Agent
- 覆盖100+模型API
### 业务指标
- 支持主流开发框架集成
- 透明的EU计费体系
- 完整的安全隔离机制
- 企业级合规认证
---
## 📊 当前项目状态 (2025年12月21日)
### ✅ 已完成工作 (总体完成度: 约 92%)
#### 1. 基础设施层 - 100% ✅
- ✅ PostgreSQL 数据库部署和配置
- ✅ Redis 缓存服务部署
- ✅ NATS 消息队列部署
- ✅ Prometheus 监控服务部署
- ✅ Grafana 可视化服务部署
- ✅ 阿里云镜像源配置（显著提升构建速度）
#### 2. API Gateway - 100% ✅
- ✅ Nginx 反向代理配置
- ✅ 路由规则配置（MCP Server、Data Ingestion、LiteLLM Gateway）
- ✅ 服务发现和负载均衡
- ✅ 开发环境 HTTPS 配置
#### 3. LiteLLM Gateway - 100% ✅
- ✅ LiteLLM 网关部署和配置
- ✅ Prisma 兼容性修复（降级到 5.8.0）
- ✅ OpenRouter 集成（Claude 3.5 Sonnet、GPT-4o-mini）
- ✅ API Key 管理（环境变量统一管理）
- ✅ 模型调用功能测试通过
#### 4. MCP Server - 90% ✅
- ✅ Agent CRUD 操作（创建、读取、更新、删除）
- ✅ Agent 执行框架
- ✅ WebSocket 实时通信
- ✅ 工具列表管理
- ✅ 健康检查
- ✅ 数据库模型和 Schema
- ✅ HTTP 工具调用（通过 LiteLLM Gateway）
- ✅ LLM 工具调用框架
#### 5. Data Ingestion 基础功能 - 80% ⚠️
- ✅ 健康检查
- ✅ OpenAPI 规范解析（基本实现）
- ✅ 工具生成框架（基本实现）
- ✅ 统计信息收集
- ✅ 缓存管理
- ✅ 环境变量统一管理（.env 文件）
#### 6. 代码和部署管理 - 100% ✅
- ✅ Git 代码管理（已推送到 main 分支）
- ✅ 容器镜像构建和推送（私有注册表）
- ✅ 密钥安全管理（.env 文件已排除）
- ✅ 文档完善（环境变量配置说明）
### ⚠️ 待完成工作 (业务逻辑完成度: 约 65%)
#### 高优先级 - 核心业务功能
**1. RapidAPI 集成 - 完成度: 100%** ✅
- ✅ `sync_endpoints()` 方法 - 同步 RapidAPI 端点列表
- ✅ `test_endpoint()` 方法 - 测试 API 端点调用
- ✅ `get_api_data()` 方法 - 实际 API 数据获取
- ✅ `search_apis()` 方法 - API 搜索功能
- ✅ Redis 缓存集成
- **状态**: 已完成，功能正常
**2. APILLAMA 算法 - 完成度: 100%** ✅
- ✅ 集成 OpenRouter API，使用 Llama 3.1 8B Instruct 模型
- ✅ `initialize()` 方法 - OpenRouter API 连接和初始化
- ✅ `process_api_doc()` 方法 - 核心 LLM 增强处理逻辑
- ✅ `_process_document_fallback()` 方法 - Fallback 处理机制
- ✅ 支持多种输出格式（Pydantic、JSON Schema、OpenAPI）
- ✅ 方法名已统一为 `process_api_doc`
- **状态**: 已完成，功能正常
**3. OpenAPI 解析器 - 完成度: 100%** ✅
- ✅ 方法名已统一为 `parse_spec`
- ✅ URL 下载和解析功能
- ✅ 文件缓存和 Redis 缓存
- ✅ 支持 YAML 和 JSON 格式
- **状态**: 已完成，功能正常
#### 中优先级 - 增强功能
**4. MCP Server 函数工具调用 - 完成度: 100%** ✅
- ✅ `_execute_function_tool()` 方法 - 本地 Python 函数调用
- ✅ 沙箱安全机制实现
- ✅ 函数注册表 (16个内置函数)
- ✅ 参数验证和错误处理
- ✅ 超时控制和资源限制
- **状态**: 已完成，功能正常，测试通过
**5. Prometheus Metrics 收集 - 完成度: 100%** ✅
- ✅ Data Ingestion 服务指标收集逻辑
- ✅ HTTP 请求指标（总数、耗时、状态码）
- ✅ API 处理指标（RapidAPI、APILLAMA、OpenAPI）
- ✅ 系统健康指标（Redis、NATS 连接状态）
- ✅ 缓存指标（命中率、未命中率）
- ✅ Prometheus 格式输出实现
- ✅ `/metrics` 端点正常工作
- **状态**: 已完成，Prometheus 可正常抓取数据
#### 低优先级 - 优化功能
**6. 工具生成器增强**
- ⚠️ 改进参数提取逻辑
- ⚠️ 添加类型推断
- ⚠️ 支持复杂 Schema
- **预计工作量**: 2-3 小时
**7. 缓存策略优化**
- ⚠️ 实现智能缓存策略
- ⚠️ 添加缓存失效机制
- ⚠️ 优化缓存命中率
- **预计工作量**: 1-2 小时
### 📋 详细完成度统计
| 模块 | 完成度 | 状态 | 优先级 |
|------|--------|------|--------|
| 基础设施服务 | 100% | ✅ 完成 | - |
| API Gateway | 100% | ✅ 完成 | - |
| LiteLLM Gateway | 100% | ✅ 完成 | - |
| MCP Server 核心功能 | 90% | ✅ 基本完成 | - |
| MCP Server 工具调用 | 70% | ⚠️ 部分完成 | 中 |
| Data Ingestion 基础 | 100% | ✅ 完成 | - |
| RapidAPI 集成 | 100% | ✅ 完成 | - |
| APILLAMA 算法 | 100% | ✅ 完成 | - |
| OpenAPI 解析器 | 100% | ✅ 完成 | - |
| Prometheus Metrics | 100% | ✅ 完成 | - |
| 工具生成器 | 100% | ✅ 完成 | - |
| **总体业务逻辑** | **98%** | **✅ 基本完成** | - |
### 🎯 下一步行动计划
#### Phase 2 准备工作
1. **性能优化和压力测试** (1-2 周)
- 进行负载测试
- 优化 API 响应时间
- 优化缓存策略
- 数据库查询优化
2. **完善监控和告警** (3-5 天)
- 配置 Grafana 仪表板
- 设置告警规则
- 完善日志聚合
3. **文档和示例完善** (2-3 天)
- API 使用示例
- 最佳实践文档
- 故障排查指南
#### Phase 2 开始（模型抽象与治理层）
4. **LiteLLM 网关增强** (2-3 周)
- 多模型路由优化
- 负载均衡策略
- 成本控制机制
5. **上下文管理优化** (1-2 周)
- 上下文窗口管理
- 会话截断策略
- 上下文压缩
### 📝 当前版本信息
- **代码版本**: v1.2.1
- **最新提交**: `feat: 实现MCP Server函数工具调用和沙箱安全机制`
- **Git 仓库**: http://gitee.ath.cx:3000/xiaohei/taiji-AI-PAD.git
- **容器注册表**: reg.ath.cx:3000/xiaohei/
- **已发布镜像**:
- `taiji-ai-pad_litellm-gateway:latest` (1.36GB)
- `taiji-ai-pad_data-ingestion:latest` (677MB)
- `taiji-ai-pad_mcp-server:latest` (735MB)
### ✅ 最新完成工作 (2025-12-22)
1. **MCP Server 函数工具调用** ✅ (最新完成)
- 实现函数注册表 (16个内置安全函数)
- 实现沙箱执行器 (超时控制、参数验证、资源限制)
- 实现 `_execute_function_tool()` 方法
- 实现完整的错误处理机制
- 测试通过率: 100%
2. **APILLAMA OpenRouter 集成** ✅
- 集成 OpenRouter API，使用 `meta-llama/llama-3.1-8b-instruct` 模型
- 实现 LLM 增强处理逻辑
- 实现 Fallback 机制（无 API Key 时使用规则处理）
- 支持多种输出格式（Pydantic、JSON Schema、OpenAPI）
3. **RapidAPI 客户端完整实现** ✅
- 实现完整的 RapidAPI 客户端功能
- 支持搜索、同步、测试端点
- 集成 Redis 缓存机制
4. **Prometheus Metrics 完整实现** ✅
- 实现 HTTP 请求指标收集
- 实现 API 处理指标（RapidAPI、APILLAMA、OpenAPI）
- 实现系统健康指标
- 实现缓存命中率指标
5. **OpenAPI 解析器增强** ✅
- 支持从 URL 下载和解析
- 实现文件缓存和 Redis 缓存
6. **工具生成器完善** ✅
7. **API 接口文档** ✅
- 生成完整的 API 接口文档
- 包含所有端点的详细说明和示例
- 提供前端集成示例
- 完善工具生成逻辑
- 集成 Redis 和 NATS
- 支持 APILLAMA 增强
### ⚠️ 已解决问题
1. ✅ **方法名不匹配** - 已修复所有方法调用问题
2. ✅ **核心业务逻辑缺失** - RapidAPI 和 APILLAMA 已完整实现
3. ✅ **监控功能缺失** - Prometheus Metrics 已完整实现
4. ✅ **MCP Server 函数工具调用** - 已完整实现，包括沙箱安全机制
### 🔄 与原始排期的对应关系
**当前进度对应 Phase 1 (基础设施与数据接入层)**
- ✅ Week 1-2: 项目初始化与开发环境搭建 - **已完成**
- ✅ Week 3-6: RapidAPI集成与统一API Key管理 - **已完成** (完整实现)
- ✅ Week 7-10: APILLAMA模型部署与API文档转换 - **已完成** (集成OpenRouter API)
- ✅ Week 11-14: OpenAPI/Swagger动态加载机制 - **已完成** (完整实现)
- ✅ Week 15-16: 第一阶段测试与优化 - **已完成** (核心功能测试通过)
**Phase 1 完成度**: 100% ✅
**预计 Phase 2 开始时间**: 2025年1月（比原计划提前约 3 个月）
---
**更新时间**: 2025年12月22日
**版本**: v1.2.1
**负责人**: 项目组
**状态**: Phase 1 核心功能 100% 完成，MCP Server 函数工具调用已实现，平台基础设施完全就绪
Docs/项目文档/认证及后台管理系统设计.md
-864
View File
@@ -1,864 +0,0 @@
# taiji-AI-PAD 认证及后台管理系统设计文档
**版本**: v1.0
**创建时间**: 2025年12月22日
**设计者**: 项目组
---
## 📋 目录
1. [系统概述](#系统概述)
2. [架构设计](#架构设计)
3. [认证系统设计](#认证系统设计)
4. [权限管理系统](#权限管理系统)
5. [后台管理功能](#后台管理功能)
6. [API 设计](#api-设计)
7. [数据库设计](#数据库设计)
8. [安全设计](#安全设计)
9. [实施计划](#实施计划)
---
## 1. 系统概述
### 1.1 目标
构建一个完整的认证和后台管理系统，包括：
- 用户认证（登录、注册、密码管理）
- 基于角色的权限控制（RBAC）
- 后台管理界面和 API
- 审计日志和操作追踪
- API Key 管理
- 多租户支持
### 1.2 核心功能
#### 认证功能
- ✅ 用户注册/登录
- ✅ JWT Token 认证
- ✅ 密码加密存储（bcrypt）
- ✅ Token 刷新机制
- ✅ 密码重置
- ✅ 账户激活/禁用
#### 权限管理
- ✅ 基于角色的访问控制（RBAC）
- ✅ 权限粒度控制
- ✅ API Key 权限管理
- ✅ 资源级别的权限控制
#### 后台管理
- ✅ 用户管理（CRUD）
- ✅ Agent 管理
- ✅ 工具管理
- ✅ 系统监控
- ✅ 审计日志查看
- ✅ 计费管理
- ✅ 系统配置
---
## 2. 架构设计
### 2.1 系统架构图
```
┌─────────────────────────────────────────────────────────┐
│ 前端层 (Admin UI) │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ 登录页面 │ │ 用户管理 │ │ Agent管理 │ │ 系统监控 │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
└──────────────────────┬──────────────────────────────────┘
│ HTTPS
▼
┌─────────────────────────────────────────────────────────┐
│ API Gateway (Nginx) │
│ ┌──────────────────────────┐ │
│ │ 认证中间件 (JWT验证) │ │
│ │ 权限检查中间件 (RBAC) │ │
│ └──────────────────────────┘ │
└──────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ MCP Server (FastAPI) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ 认证模块 │ │ 权限管理模块 │ │ 后台管理模块 │ │
│ │ - 登录/注册 │ │ - RBAC │ │ - 用户管理 │ │
│ │ - JWT Token │ │ - 权限检查 │ │ - Agent管理 │ │
│ │ - 密码管理 │ │ - API Key │ │ - 系统监控 │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└──────────────────────┬──────────────────────────────────┘
│
┌───────────────┼───────────────┐
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ PostgreSQL │ │ Redis │ │ NATS │
│ - 用户数据 │ │ - Token缓存 │ │ - 事件发布 │
│ - 权限数据 │ │ - 会话管理 │ │ - 审计日志 │
│ - 审计日志 │ │ - 限流数据 │ │ │
└──────────────┘ └──────────────┘ └──────────────┘
```
### 2.2 模块划分
#### 2.2.1 认证模块 (`auth.py`)
- 用户注册/登录
- JWT Token 生成和验证
- 密码加密和验证
- Token 刷新
- 密码重置
#### 2.2.2 权限模块 (`permissions.py`)
- 角色定义和管理
- 权限检查装饰器
- API Key 权限验证
- 资源权限验证
#### 2.2.3 后台管理模块 (`admin.py`)
- 用户管理 API
- Agent 管理 API
- 工具管理 API
- 系统监控 API
- 审计日志 API
#### 2.2.4 审计模块 (`audit.py`)
- 操作日志记录
- 登录日志
- API 调用日志
- 异常日志
---
## 3. 认证系统设计
### 3.1 认证流程
#### 3.1.1 用户登录流程
```
用户 → 提交用户名/密码
↓
验证用户名和密码 (bcrypt)
↓
生成 JWT Token (包含用户ID、角色、权限)
↓
返回 Token 和用户信息
↓
客户端存储 Token (localStorage/cookie)
↓
后续请求携带 Token (Authorization Header)
↓
服务器验证 Token
↓
允许/拒绝访问
```
#### 3.1.2 Token 结构
```json
{
"sub": "user_id",
"username": "admin",
"email": "admin@example.com",
"roles": ["admin", "user"],
"permissions": ["user:read", "user:write", "agent:manage"],
"iat": 1234567890,
"exp": 1234571490,
"type": "access" // access 或 refresh
}
```
### 3.2 密码安全
- **加密算法**: bcrypt (cost factor: 12)
- **密码要求**:
- 最小长度: 8 字符
- 必须包含: 大小写字母、数字
- 可选: 特殊字符
- **密码重置**:
- 通过邮箱发送重置链接
- 重置链接有效期: 1 小时
- 使用临时 Token
### 3.3 Token 管理
- **Access Token**:
- 有效期: 60 分钟
- 用途: API 访问认证
- **Refresh Token**:
- 有效期: 7 天
- 用途: 刷新 Access Token
- 存储: Redis (可撤销)
- **Token 撤销**:
- 登出时撤销 Refresh Token
- 支持强制撤销所有 Token
---
## 4. 权限管理系统
### 4.1 角色定义
#### 4.1.1 系统角色
| 角色 | 描述 | 权限范围 |
|------|------|----------|
| **super_admin** | 超级管理员 | 所有权限 |
| **admin** | 管理员 | 用户管理、Agent管理、系统配置 |
| **developer** | 开发者 | Agent创建、工具使用、API调用 |
| **user** | 普通用户 | 基础功能、自己的Agent |
| **viewer** | 只读用户 | 查看权限，无修改权限 |
#### 4.1.2 权限定义
```
资源:操作 格式
用户权限:
- user:read - 查看用户
- user:write - 创建/修改用户
- user:delete - 删除用户
- user:manage - 完整用户管理
Agent权限:
- agent:read - 查看Agent
- agent:write - 创建/修改Agent
- agent:delete - 删除Agent
- agent:execute - 执行Agent
- agent:manage - 完整Agent管理
工具权限:
- tool:read - 查看工具
- tool:write - 创建/修改工具
- tool:delete - 删除工具
- tool:use - 使用工具
系统权限:
- system:read - 查看系统信息
- system:config - 系统配置
- system:monitor - 系统监控
- audit:read - 查看审计日志
```
### 4.2 权限检查机制
#### 4.2.1 装饰器方式
```python
@require_permission("agent:manage")
async def create_agent(...):
pass
@require_role("admin")
async def admin_function(...):
pass
```
#### 4.2.2 依赖注入方式
```python
from auth import get_current_user, require_permission
async def endpoint(
current_user: User = Depends(get_current_user),
_: None = Depends(require_permission("agent:read"))
):
pass
```
### 4.3 API Key 权限
- **API Key 类型**:
- `readonly`: 只读权限
- `write`: 读写权限
- `admin`: 管理员权限
- **API Key 限制**:
- 速率限制
- IP 白名单
- 过期时间
---
## 5. 后台管理功能
### 5.1 用户管理
#### 功能列表
- ✅ 用户列表（分页、搜索、筛选）
- ✅ 创建用户
- ✅ 编辑用户信息
- ✅ 禁用/启用用户
- ✅ 重置用户密码
- ✅ 查看用户详情
- ✅ 用户权限管理
- ✅ 用户 Agent 列表
- ✅ 用户使用统计
#### 数据展示
- 用户基本信息
- 注册时间、最后登录时间
- 状态（活跃/禁用）
- 角色和权限
- Agent 数量
- API 调用统计
### 5.2 Agent 管理
#### 功能列表
- ✅ Agent 列表（分页、搜索、筛选）
- ✅ 查看 Agent 详情
- ✅ 编辑 Agent 配置
- ✅ 启用/禁用 Agent
- ✅ 删除 Agent
- ✅ Agent 执行历史
- ✅ Agent 性能统计
- ✅ Agent 权限管理
#### 数据展示
- Agent 基本信息
- 所属用户
- 状态和版本
- 执行统计（总数、成功率、平均耗时）
- 工具列表
- 配置信息
### 5.3 工具管理
#### 功能列表
- ✅ 工具列表（分页、搜索、筛选）
- ✅ 工具详情查看
- ✅ 工具分类管理
- ✅ 工具权限配置
- ✅ 工具使用统计
- ✅ 工具健康检查
### 5.4 系统监控
#### 功能列表
- ✅ 系统健康状态
- ✅ 服务运行状态（Redis、NATS、数据库）
- ✅ 实时指标（请求数、响应时间、错误率）
- ✅ 资源使用情况（CPU、内存、磁盘）
- ✅ 活跃用户数
- ✅ API 调用统计
- ✅ 错误日志查看
### 5.5 审计日志
#### 功能列表
- ✅ 操作日志列表（分页、搜索、筛选）
- ✅ 登录日志
- ✅ API 调用日志
- ✅ 异常日志
- ✅ 日志导出
- ✅ 日志统计分析
#### 日志内容
- 操作时间
- 操作用户
- 操作类型
- 操作对象
- 操作结果
- IP 地址
- User Agent
### 5.6 计费管理
#### 功能列表
- ✅ 用户计费记录
- ✅ 计费统计
- ✅ 账单生成
- ✅ 配额管理
- ✅ 使用量统计
---
## 6. API 设计
### 6.1 认证 API
#### 6.1.1 用户注册
```
POST /api/v1/auth/register
Request:
{
"username": "string",
"email": "string",
"password": "string",
"full_name": "string"
}
Response:
{
"user_id": "uuid",
"username": "string",
"email": "string",
"message": "注册成功"
}
```
#### 6.1.2 用户登录
```
POST /api/v1/auth/login
Request:
{
"username": "string",
"password": "string"
}
Response:
{
"access_token": "string",
"refresh_token": "string",
"token_type": "bearer",
"expires_in": 3600,
"user": {
"id": "uuid",
"username": "string",
"email": "string",
"roles": ["string"],
"permissions": ["string"]
}
}
```
#### 6.1.3 Token 刷新
```
POST /api/v1/auth/refresh
Headers:
Authorization: Bearer <refresh_token>
Response:
{
"access_token": "string",
"token_type": "bearer",
"expires_in": 3600
}
```
#### 6.1.4 用户登出
```
POST /api/v1/auth/logout
Headers:
Authorization: Bearer <access_token>
Response:
{
"message": "登出成功"
}
```
#### 6.1.5 密码重置
```
POST /api/v1/auth/password/reset
Request:
{
"email": "string"
}
Response:
{
"message": "重置链接已发送到邮箱"
}
POST /api/v1/auth/password/reset/confirm
Request:
{
"token": "string",
"new_password": "string"
}
Response:
{
"message": "密码重置成功"
}
```
### 6.2 后台管理 API
#### 6.2.1 用户管理
```
GET /api/v1/admin/users # 用户列表
GET /api/v1/admin/users/{user_id} # 用户详情
POST /api/v1/admin/users # 创建用户
PUT /api/v1/admin/users/{user_id} # 更新用户
DELETE /api/v1/admin/users/{user_id} # 删除用户
POST /api/v1/admin/users/{user_id}/disable # 禁用用户
POST /api/v1/admin/users/{user_id}/enable # 启用用户
POST /api/v1/admin/users/{user_id}/reset-password # 重置密码
GET /api/v1/admin/users/{user_id}/agents # 用户Agent列表
GET /api/v1/admin/users/{user_id}/stats # 用户统计
```
#### 6.2.2 Agent 管理
```
GET /api/v1/admin/agents # Agent列表
GET /api/v1/admin/agents/{agent_id} # Agent详情
PUT /api/v1/admin/agents/{agent_id} # 更新Agent
DELETE /api/v1/admin/agents/{agent_id} # 删除Agent
POST /api/v1/admin/agents/{agent_id}/disable # 禁用Agent
POST /api/v1/admin/agents/{agent_id}/enable # 启用Agent
GET /api/v1/admin/agents/{agent_id}/executions # 执行历史
GET /api/v1/admin/agents/{agent_id}/stats # 性能统计
```
#### 6.2.3 系统监控
```
GET /api/v1/admin/system/health # 系统健康
GET /api/v1/admin/system/metrics # 系统指标
GET /api/v1/admin/system/stats # 系统统计
GET /api/v1/admin/system/logs # 系统日志
```
#### 6.2.4 审计日志
```
GET /api/v1/admin/audit/logs # 审计日志列表
GET /api/v1/admin/audit/logs/{log_id} # 日志详情
GET /api/v1/admin/audit/login-logs # 登录日志
GET /api/v1/admin/audit/api-logs # API调用日志
GET /api/v1/admin/audit/error-logs # 错误日志
POST /api/v1/admin/audit/logs/export # 导出日志
```
---
## 7. 数据库设计
### 7.1 用户表 (users) - 已有
```sql
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
username VARCHAR(50) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
hashed_password VARCHAR(255) NOT NULL,
full_name VARCHAR(100),
is_active BOOLEAN DEFAULT TRUE,
is_admin BOOLEAN DEFAULT FALSE,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
### 7.2 角色表 (roles) - 新增
```sql
CREATE TABLE roles (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(50) UNIQUE NOT NULL,
description TEXT,
is_system BOOLEAN DEFAULT FALSE, -- 系统角色不可删除
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
### 7.3 权限表 (permissions) - 新增
```sql
CREATE TABLE permissions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
resource VARCHAR(50) NOT NULL, -- user, agent, tool, system
action VARCHAR(50) NOT NULL, -- read, write, delete, manage
description TEXT,
created_at TIMESTAMP DEFAULT NOW(),
UNIQUE(resource, action)
);
```
### 7.4 用户角色关联表 (user_roles) - 新增
```sql
CREATE TABLE user_roles (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role_id UUID NOT NULL REFERENCES roles(id) ON DELETE CASCADE,
assigned_at TIMESTAMP DEFAULT NOW(),
assigned_by UUID REFERENCES users(id),
UNIQUE(user_id, role_id)
);
```
### 7.5 角色权限关联表 (role_permissions) - 新增
```sql
CREATE TABLE role_permissions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
role_id UUID NOT NULL REFERENCES roles(id) ON DELETE CASCADE,
permission_id UUID NOT NULL REFERENCES permissions(id) ON DELETE CASCADE,
granted_at TIMESTAMP DEFAULT NOW(),
UNIQUE(role_id, permission_id)
);
```
### 7.6 API Key 表 (api_keys) - 已有，需增强
```sql
CREATE TABLE api_keys (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
key_hash VARCHAR(255) NOT NULL, -- 存储哈希值
name VARCHAR(100), -- Key名称
permissions JSONB, -- 权限列表
rate_limit INTEGER DEFAULT 100, -- 速率限制
ip_whitelist TEXT[], -- IP白名单
expires_at TIMESTAMP, -- 过期时间
last_used_at TIMESTAMP,
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
### 7.7 审计日志表 (audit_logs) - 已有，需增强
```sql
CREATE TABLE audit_logs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id),
action VARCHAR(50) NOT NULL, -- login, logout, create, update, delete
resource_type VARCHAR(50), -- user, agent, tool, system
resource_id UUID,
details JSONB, -- 详细信息
ip_address INET,
user_agent TEXT,
status VARCHAR(20), -- success, failure, error
error_message TEXT,
created_at TIMESTAMP DEFAULT NOW()
);
```
---
## 8. 安全设计
### 8.1 认证安全
- **密码安全**:
- bcrypt 加密（cost factor: 12）
- 密码复杂度要求
- 密码历史记录（防止重复使用）
- **Token 安全**:
- JWT 签名验证
- Token 过期时间
- Refresh Token 轮换
- Token 黑名单（Redis）
### 8.2 权限安全
- **最小权限原则**: 默认无权限，需要显式授权
- **权限继承**: 角色权限可继承
- **资源级权限**: 支持资源级别的权限控制
- **API Key 安全**:
- Key 哈希存储
- 速率限制
- IP 白名单
- 过期时间
### 8.3 数据安全
- **SQL 注入防护**: 使用 ORM 参数化查询
- **XSS 防护**: 输入验证和输出转义
- **CSRF 防护**: Token 验证
- **敏感数据加密**:
- 密码: bcrypt
- API Key: 哈希存储
- 配置信息: 环境变量
### 8.4 审计安全
- **操作日志**: 所有关键操作记录
- **登录日志**: 记录所有登录尝试
- **异常监控**: 记录异常和错误
- **日志保留**: 至少保留 90 天
---
## 9. 实施计划
### 9.1 第一阶段：基础认证（1-2天）
**任务清单**:
- [ ] 创建认证模块 (`auth.py`)
- [ ] 密码加密和验证函数
- [ ] JWT Token 生成和验证
- [ ] 登录/注册 API
- [ ] Token 刷新 API
- [ ] 创建认证中间件
- [ ] JWT 验证中间件
- [ ] 用户信息注入
- [ ] 更新数据库模型
- [ ] 确认 User 模型
- [ ] 创建 Role、Permission 模型
- [ ] 创建关联表
- [ ] 创建认证 API 端点
- [ ] POST /api/v1/auth/register
- [ ] POST /api/v1/auth/login
- [ ] POST /api/v1/auth/refresh
- [ ] POST /api/v1/auth/logout
**预计工作量**: 1-2 天
### 9.2 第二阶段：权限管理（2-3天）
**任务清单**:
- [ ] 创建权限模块 (`permissions.py`)
- [ ] 角色定义和管理
- [ ] 权限检查装饰器
- [ ] 权限验证函数
- [ ] 初始化系统角色和权限
- [ ] 创建系统角色（super_admin, admin, developer, user, viewer）
- [ ] 创建系统权限
- [ ] 分配角色权限
- [ ] 创建权限管理 API
- [ ] 角色管理 API
- [ ] 权限管理 API
- [ ] 用户角色分配 API
- [ ] 实现权限检查中间件
- [ ] 装饰器方式
- [ ] 依赖注入方式
**预计工作量**: 2-3 天
### 9.3 第三阶段：后台管理 API（3-4天）
**任务清单**:
- [ ] 创建后台管理模块 (`admin.py`)
- [ ] 用户管理 API
- [ ] Agent 管理 API
- [ ] 工具管理 API
- [ ] 系统监控 API
- [ ] 实现审计日志模块 (`audit.py`)
- [ ] 日志记录函数
- [ ] 日志查询 API
- [ ] 日志导出功能
- [ ] 创建后台管理 API 端点
- [ ] 用户管理 API（CRUD）
- [ ] Agent 管理 API
- [ ] 系统监控 API
- [ ] 审计日志 API
**预计工作量**: 3-4 天
### 9.4 第四阶段：API Key 管理（1-2天）
**任务清单**:
- [ ] 增强 API Key 功能
- [ ] API Key 生成和验证
- [ ] API Key 权限管理
- [ ] API Key 速率限制
- [ ] API Key IP 白名单
- [ ] 创建 API Key 管理 API
- [ ] 创建 API Key
- [ ] 查看 API Key 列表
- [ ] 更新 API Key
- [ ] 删除 API Key
- [ ] 撤销 API Key
**预计工作量**: 1-2 天
### 9.5 第五阶段：测试和优化（1-2天）
**任务清单**:
- [ ] 单元测试
- [ ] 认证功能测试
- [ ] 权限功能测试
- [ ] 后台管理 API 测试
- [ ] 集成测试
- [ ] 端到端测试
- [ ] 安全测试
- [ ] 性能优化
- [ ] 查询优化
- [ ] 缓存优化
- [ ] 文档完善
- [ ] API 文档
- [ ] 使用示例
**预计工作量**: 1-2 天
### 9.6 总工作量估算
| 阶段 | 工作量 | 优先级 |
|------|--------|--------|
| 第一阶段：基础认证 | 1-2 天 | P0 |
| 第二阶段：权限管理 | 2-3 天 | P0 |
| 第三阶段：后台管理 API | 3-4 天 | P0 |
| 第四阶段：API Key 管理 | 1-2 天 | P1 |
| 第五阶段：测试和优化 | 1-2 天 | P0 |
| **总计** | **8-13 天** | - |
---
## 10. 技术选型
### 10.1 认证技术
- **JWT**: python-jose[cryptography]
- **密码加密**: passlib[bcrypt]
- **Token 存储**: Redis（用于刷新 Token 和黑名单）
### 10.2 权限管理
- **RBAC**: 自定义实现
- **权限检查**: FastAPI 依赖注入
### 10.3 数据库
- **ORM**: SQLAlchemy 2.0
- **数据库**: PostgreSQL
- **迁移工具**: Alembic
### 10.4 缓存
- **Redis**: Token 缓存、会话管理、限流
---
## 11. 后续扩展
### 11.1 OAuth 2.0 支持
- Google OAuth
- GitHub OAuth
- 企业 SSO
### 11.2 多因素认证 (MFA)
- TOTP (Time-based One-Time Password)
- 短信验证码
- 邮箱验证码
### 11.3 细粒度权限
- 资源级别的权限控制
- 动态权限分配
- 权限继承和覆盖
### 11.4 后台管理界面
- React/Vue 前端
- 实时监控面板
- 数据可视化
---
## 12. 风险评估
### 12.1 安全风险
- **Token 泄露**: 使用 HTTPS、Token 过期时间
- **密码泄露**: bcrypt 加密、密码复杂度要求
- **权限绕过**: 严格的权限检查、审计日志
### 12.2 性能风险
- **Token 验证性能**: Redis 缓存、JWT 验证优化
- **权限检查性能**: 权限缓存、批量检查
### 12.3 兼容性风险
- **现有 API 兼容**: 逐步迁移、版本控制
- **数据库迁移**: Alembic 迁移脚本
---
**文档版本**: v1.0
**最后更新**: 2025年12月22日
**下一步**: 开始实施第一阶段（基础认证）
services/mcp-server/app/audit.py
+400
View File
@@ -0,0 +1,400 @@
"""
合规审计日志模块
操作日志记录与审计查询
"""
from datetime import datetime, timedelta
from typing import Dict, List, Optional, Any
from sqlalchemy import select, func, and_, desc
from sqlalchemy.ext.asyncio import AsyncSession
from fastapi import Request
from models import AuditLog, User
# 审计操作类型定义
AUDIT_ACTIONS = {
# 认证相关
"auth.login": "用户登录",
"auth.logout": "用户登出",
"auth.password_change": "密码修改",
"auth.token_revoke": "Token撤销",
# 用户管理
"user.create": "创建用户",
"user.update": "更新用户",
"user.delete": "删除用户",
"user.recharge": "用户充值",
# 渠道管理
"channel.create": "创建渠道",
"channel.update": "更新渠道",
"channel.delete": "删除渠道",
"channel.allocate": "分配渠道资源",
# 管理员管理
"admin.create": "创建管理员",
"admin.delete": "删除管理员",
"admin.permission_change": "权限变更",
# Agent管理
"agent.create": "创建Agent",
"agent.update": "更新Agent",
"agent.delete": "删除Agent",
"agent.deploy": "部署Agent",
# 资源管理
"resource.allocate": "资源分配",
"resource.revoke": "资源撤销",
# 审批操作
"application.approve": "审批通过",
"application.reject": "审批拒绝",
# 计费操作
"billing.charge": "计费扣款",
"billing.refund": "退款",
"billing.adjust": "余额调整",
# 供应商管理
"provider.create": "创建供应商",
"provider.update": "更新供应商",
"provider.delete": "删除供应商",
# 系统操作
"system.config_change": "系统配置变更",
"system.maintenance": "系统维护",
}
async def log_audit_event(
action: str,
resource_type: str,
resource_id: Optional[str],
user_id: Optional[str],
success: bool,
details: Optional[Dict] = None,
error_message: Optional[str] = None,
request: Optional[Request] = None,
db: Optional[AsyncSession] = None
) -> Optional[AuditLog]:
"""
记录审计事件
Args:
action: 操作类型
resource_type: 资源类型
resource_id: 资源ID
user_id: 操作用户ID
success: 是否成功
details: 操作详情
error_message: 错误信息
request: HTTP请求对象
db: 数据库会话
Returns:
AuditLog记录
"""
if db is None:
return None
# 从请求中提取信息
ip_address = None
user_agent = None
if request:
ip_address = request.client.host if request.client else None
user_agent = request.headers.get("user-agent")
# 创建审计日志
audit_log = AuditLog(
action=action,
resource_type=resource_type,
resource_id=resource_id,
user_id=user_id,
success=success,
details=details or {},
error_message=error_message,
ip_address=ip_address,
user_agent=user_agent,
)
db.add(audit_log)
await db.commit()
await db.refresh(audit_log)
return audit_log
async def query_audit_logs(
user_id: Optional[str] = None,
action: Optional[str] = None,
resource_type: Optional[str] = None,
resource_id: Optional[str] = None,
success: Optional[bool] = None,
start_date: Optional[datetime] = None,
end_date: Optional[datetime] = None,
page: int = 1,
page_size: int = 20,
db: AsyncSession = None
) -> Dict:
"""
查询审计日志
Args:
user_id: 用户ID过滤
action: 操作类型过滤
resource_type: 资源类型过滤
resource_id: 资源ID过滤
success: 成功/失败过滤
start_date: 开始日期
end_date: 结束日期
page: 页码
page_size: 每页大小
db: 数据库会话
Returns:
审计日志查询结果
"""
query = select(AuditLog)
# 应用过滤条件
conditions = []
if user_id:
conditions.append(AuditLog.user_id == user_id)
if action:
conditions.append(AuditLog.action == action)
if resource_type:
conditions.append(AuditLog.resource_type == resource_type)
if resource_id:
conditions.append(AuditLog.resource_id == resource_id)
if success is not None:
conditions.append(AuditLog.success == success)
if start_date:
conditions.append(AuditLog.created_at >= start_date)
if end_date:
conditions.append(AuditLog.created_at <= end_date)
if conditions:
query = query.where(and_(*conditions))
# 计算总数
count_result = await db.execute(
select(func.count()).select_from(query.subquery())
)
total = count_result.scalar() or 0
# 分页查询
query = query.order_by(desc(AuditLog.created_at))
query = query.offset((page - 1) * page_size).limit(page_size)
result = await db.execute(query)
logs = result.scalars().all()
# 获取用户信息
user_ids = [str(log.user_id) for log in logs if log.user_id]
users_result = await db.execute(
select(User).where(User.id.in_(user_ids))
) if user_ids else None
users_map = {}
if users_result:
for user in users_result.scalars().all():
users_map[str(user.id)] = user.name or user.email
return {
"total": total,
"page": page,
"pageSize": page_size,
"totalPages": (total + page_size - 1) // page_size,
"logs": [
{
"id": str(log.id),
"action": log.action,
"actionName": AUDIT_ACTIONS.get(log.action, log.action),
"resourceType": log.resource_type,
"resourceId": log.resource_id,
"userId": str(log.user_id) if log.user_id else None,
"userName": users_map.get(str(log.user_id)) if log.user_id else None,
"success": log.success,
"details": log.details,
"errorMessage": log.error_message,
"ipAddress": log.ip_address,
"createdAt": log.created_at.isoformat(),
}
for log in logs
],
}
async def get_audit_summary(
start_date: datetime,
end_date: datetime,
db: AsyncSession
) -> Dict:
"""
获取审计日志汇总统计
Args:
start_date: 开始日期
end_date: 结束日期
db: 数据库会话
Returns:
汇总统计
"""
# 按操作类型统计
action_stats = await db.execute(
select(
AuditLog.action,
func.count(AuditLog.id).label("count"),
func.sum(func.cast(AuditLog.success, sa.Integer)).label("success_count"),
)
.where(
and_(
AuditLog.created_at >= start_date,
AuditLog.created_at <= end_date,
)
)
.group_by(AuditLog.action)
)
action_breakdown = [
{
"action": row.action,
"actionName": AUDIT_ACTIONS.get(row.action, row.action),
"count": row.count,
"successCount": int(row.success_count or 0),
"failCount": row.count - int(row.success_count or 0),
}
for row in action_stats.all()
]
# 按资源类型统计
resource_stats = await db.execute(
select(
AuditLog.resource_type,
func.count(AuditLog.id).label("count"),
)
.where(
and_(
AuditLog.created_at >= start_date,
AuditLog.created_at <= end_date,
)
)
.group_by(AuditLog.resource_type)
)
resource_breakdown = [
{
"resourceType": row.resource_type,
"count": row.count,
}
for row in resource_stats.all()
]
# 总计统计
total_stats = await db.execute(
select(
func.count(AuditLog.id).label("total"),
func.sum(func.cast(AuditLog.success, sa.Integer)).label("success_total"),
)
.where(
and_(
AuditLog.created_at >= start_date,
AuditLog.created_at <= end_date,
)
)
)
total_row = total_stats.first()
return {
"startDate": start_date.isoformat(),
"endDate": end_date.isoformat(),
"total": total_row.total or 0,
"successTotal": int(total_row.success_total or 0),
"failTotal": (total_row.total or 0) - int(total_row.success_total or 0),
"byAction": action_breakdown,
"byResourceType": resource_breakdown,
}
async def get_user_activity(
user_id: str,
days: int,
db: AsyncSession
) -> List[Dict]:
"""
获取用户活动历史
Args:
user_id: 用户ID
days: 天数
db: 数据库会话
Returns:
活动历史列表
"""
start_date = datetime.utcnow() - timedelta(days=days)
result = await db.execute(
select(AuditLog)
.where(
and_(
AuditLog.user_id == user_id,
AuditLog.created_at >= start_date,
)
)
.order_by(desc(AuditLog.created_at))
.limit(100)
)
logs = result.scalars().all()
return [
{
"action": log.action,
"actionName": AUDIT_ACTIONS.get(log.action, log.action),
"resourceType": log.resource_type,
"resourceId": log.resource_id,
"success": log.success,
"ipAddress": log.ip_address,
"createdAt": log.created_at.isoformat(),
}
for log in logs
]
async def export_audit_logs(
filters: Dict,
format: str,
db: AsyncSession
) -> Dict:
"""
导出审计日志
Args:
filters: 过滤条件
format: 导出格式 (csv, excel, json)
db: 数据库会话
Returns:
导出信息
"""
# 简化实现：生成导出URL
# 实际应该生成文件并上传到对象存储
import uuid
export_id = str(uuid.uuid4())
file_url = f"https://exports.taiji-ai.com/audit/{export_id}/audit_logs.{format}"
return {
"exportId": export_id,
"format": format,
"fileUrl": file_url,
"expiresAt": (datetime.utcnow() + timedelta(hours=24)).isoformat(),
"status": "processing",
}
# 需要导入sa模块
import sqlalchemy as sa
services/mcp-server/app/events.py
+412 -39
View File
@@ -1,53 +1,426 @@
"""NATS event handlers for the MCP server."""
from __future__ import annotations
"""
NATS事件采集模块
计费事件消息队列处理
"""
import json
import uuid
import asyncio
from datetime import datetime
from typing import Dict, List, Optional, Any, Callable
from sqlalchemy import select, update, and_
from sqlalchemy.ext.asyncio import AsyncSession
import structlog
from .state import get_state
from models import BillingEvent, User, Agent, Execution
logger = structlog.get_logger(__name__)
async def setup_nats_handlers() -> None:
"""Subscribe to NATS subjects once the client is connected."""
state = get_state()
client = state.nats_client
if not client:
return
await client.subscribe("agent.execution.*", cb=handle_agent_execution)
await client.subscribe("billing.*", cb=handle_billing_event)
await client.subscribe("system.*", cb=handle_system_event)
# 事件类型定义
EVENT_TYPES = {
"execution.start": "执行开始",
"execution.end": "执行结束",
"execution.error": "执行错误",
"quota.check": "配额检查",
"quota.exceeded": "配额超限",
"balance.deduct": "余额扣款",
"balance.recharge": "余额充值",
"rate_limit.hit": "速率限制触发",
}
async def handle_agent_execution(msg) -> None:
"""Broadcast agent execution events to WebSocket clients."""
state = get_state()
try:
payload = json.loads(msg.data.decode())
logger.info("收到Agent执行事件", payload=payload)
for websocket in list(state.active_websockets.values()):
await websocket.send_json({"type": "agent_execution", "data": payload})
except Exception as exc: # pragma: no cover - defensive logging
logger.error("处理Agent执行事件失败", error=str(exc))
class EventPublisher:
"""事件发布者"""
def __init__(self, nats_client=None):
self.nats = nats_client
self._handlers: Dict[str, List[Callable]] = {}
def register_handler(self, event_type: str, handler: Callable):
"""注册事件处理器"""
if event_type not in self._handlers:
self._handlers[event_type] = []
self._handlers[event_type].append(handler)
async def publish(
self,
event_type: str,
user_id: str,
payload: Dict,
agent_id: Optional[str] = None,
execution_id: Optional[str] = None,
db: Optional[AsyncSession] = None
) -> str:
"""
发布事件
Args:
event_type: 事件类型
user_id: 用户ID
payload: 事件数据
agent_id: Agent ID
execution_id: 执行ID
db: 数据库会话
Returns:
事件ID
"""
event_id = str(uuid.uuid4())
event_data = {
"event_id": event_id,
"event_type": event_type,
"user_id": user_id,
"agent_id": agent_id,
"execution_id": execution_id,
"payload": payload,
"timestamp": datetime.utcnow().isoformat(),
}
# 保存到数据库
if db:
billing_event = BillingEvent(
event_type=event_type,
event_id=event_id,
user_id=user_id,
agent_id=agent_id,
execution_id=execution_id,
payload=payload,
status="pending",
)
db.add(billing_event)
await db.commit()
# 发布到NATS（如果配置了）
if self.nats:
try:
subject = f"billing.{event_type.replace('.', '_')}"
await self.nats.publish(subject, json.dumps(event_data).encode())
logger.info("事件发布成功", event_id=event_id, event_type=event_type)
except Exception as e:
logger.error("NATS发布失败", error=str(e), event_id=event_id)
# 触发本地处理器
handlers = self._handlers.get(event_type, [])
for handler in handlers:
try:
await handler(event_data)
except Exception as e:
logger.error("事件处理器错误", error=str(e), event_id=event_id)
return event_id
async def publish_execution_start(
self,
user_id: str,
agent_id: str,
execution_id: str,
method: str,
params: Dict,
db: Optional[AsyncSession] = None
) -> str:
"""发布执行开始事件"""
return await self.publish(
event_type="execution.start",
user_id=user_id,
agent_id=agent_id,
execution_id=execution_id,
payload={
"method": method,
"params_keys": list(params.keys()) if params else [],
"started_at": datetime.utcnow().isoformat(),
},
db=db,
)
async def publish_execution_end(
self,
user_id: str,
agent_id: str,
execution_id: str,
success: bool,
duration_ms: int,
eu_consumed: float,
db: Optional[AsyncSession] = None
) -> str:
"""发布执行结束事件"""
return await self.publish(
event_type="execution.end",
user_id=user_id,
agent_id=agent_id,
execution_id=execution_id,
payload={
"success": success,
"duration_ms": duration_ms,
"eu_consumed": eu_consumed,
"ended_at": datetime.utcnow().isoformat(),
},
db=db,
)
async def publish_balance_event(
self,
user_id: str,
event_subtype: str,
amount: float,
balance_before: float,
balance_after: float,
description: str,
db: Optional[AsyncSession] = None
) -> str:
"""发布余额变动事件"""
return await self.publish(
event_type=f"balance.{event_subtype}",
user_id=user_id,
payload={
"amount": amount,
"balance_before": balance_before,
"balance_after": balance_after,
"description": description,
},
db=db,
)
async def publish_quota_event(
self,
user_id: str,
quota_type: str,
current_value: float,
limit_value: float,
db: Optional[AsyncSession] = None
) -> str:
"""发布配额事件"""
exceeded = current_value >= limit_value
return await self.publish(
event_type="quota.exceeded" if exceeded else "quota.check",
user_id=user_id,
payload={
"quota_type": quota_type,
"current_value": current_value,
"limit_value": limit_value,
"usage_percent": round(current_value / limit_value * 100, 2) if limit_value > 0 else 0,
},
db=db,
)
async def handle_billing_event(msg) -> None:
"""Log billing events published on NATS."""
try:
payload = json.loads(msg.data.decode())
logger.info("收到计费事件", payload=payload)
except Exception as exc:
logger.error("处理计费事件失败", error=str(exc))
class EventConsumer:
"""事件消费者"""
def __init__(self, nats_client=None, db_session_factory=None):
self.nats = nats_client
self.db_factory = db_session_factory
self._running = False
self._subscriptions = []
async def start(self):
"""启动事件消费"""
if not self.nats:
logger.warning("NATS未配置，跳过事件消费")
return
self._running = True
# 订阅计费相关事件
subjects = [
"billing.execution_start",
"billing.execution_end",
"billing.balance_*",
"billing.quota_*",
]
for subject in subjects:
try:
sub = await self.nats.subscribe(subject, cb=self._handle_message)
self._subscriptions.append(sub)
logger.info("订阅成功", subject=subject)
except Exception as e:
logger.error("订阅失败", subject=subject, error=str(e))
async def stop(self):
"""停止事件消费"""
self._running = False
for sub in self._subscriptions:
await sub.unsubscribe()
self._subscriptions.clear()
async def _handle_message(self, msg):
"""处理消息"""
try:
data = json.loads(msg.data.decode())
event_type = data.get("event_type")
event_id = data.get("event_id")
logger.info("收到事件", event_type=event_type, event_id=event_id)
# 更新事件状态
if self.db_factory:
async with self.db_factory() as db:
await self._mark_event_processed(event_id, db)
except Exception as e:
logger.error("消息处理失败", error=str(e))
async def _mark_event_processed(self, event_id: str, db: AsyncSession):
"""标记事件已处理"""
await db.execute(
update(BillingEvent)
.where(BillingEvent.event_id == event_id)
.values(status="completed", processed_at=datetime.utcnow())
)
await db.commit()
async def handle_system_event(msg) -> None:
"""Log system-wide events."""
try:
payload = json.loads(msg.data.decode())
logger.info("收到系统事件", payload=payload)
except Exception as exc:
logger.error("处理系统事件失败", error=str(exc))
async def get_pending_events(
limit: int,
db: AsyncSession
) -> List[Dict]:
"""
获取待处理的事件
Args:
limit: 数量限制
db: 数据库会话
Returns:
待处理事件列表
"""
result = await db.execute(
select(BillingEvent)
.where(BillingEvent.status == "pending")
.order_by(BillingEvent.created_at)
.limit(limit)
)
events = result.scalars().all()
return [
{
"id": str(event.id),
"eventId": event.event_id,
"eventType": event.event_type,
"userId": str(event.user_id),
"agentId": str(event.agent_id) if event.agent_id else None,
"payload": event.payload,
"status": event.status,
"createdAt": event.created_at.isoformat(),
}
for event in events
]
async def retry_failed_events(
max_retries: int,
db: AsyncSession
) -> int:
"""
重试失败的事件
Args:
max_retries: 最大重试次数
db: 数据库会话
Returns:
重试的事件数
"""
result = await db.execute(
select(BillingEvent)
.where(
and_(
BillingEvent.status == "failed",
BillingEvent.retry_count < max_retries,
)
)
.limit(100)
)
events = result.scalars().all()
count = 0
for event in events:
event.status = "pending"
event.retry_count += 1
count += 1
await db.commit()
return count
async def get_event_stats(
start_date: datetime,
end_date: datetime,
db: AsyncSession
) -> Dict:
"""
获取事件统计
Args:
start_date: 开始日期
end_date: 结束日期
db: 数据库会话
Returns:
事件统计
"""
from sqlalchemy import func
# 按状态统计
status_stats = await db.execute(
select(
BillingEvent.status,
func.count(BillingEvent.id).label("count"),
)
.where(
and_(
BillingEvent.created_at >= start_date,
BillingEvent.created_at <= end_date,
)
)
.group_by(BillingEvent.status)
)
status_breakdown = {row.status: row.count for row in status_stats.all()}
# 按事件类型统计
type_stats = await db.execute(
select(
BillingEvent.event_type,
func.count(BillingEvent.id).label("count"),
)
.where(
and_(
BillingEvent.created_at >= start_date,
BillingEvent.created_at <= end_date,
)
)
.group_by(BillingEvent.event_type)
)
type_breakdown = {row.event_type: row.count for row in type_stats.all()}
return {
"startDate": start_date.isoformat(),
"endDate": end_date.isoformat(),
"byStatus": status_breakdown,
"byType": type_breakdown,
"total": sum(status_breakdown.values()),
}
# 全局事件发布者实例
_event_publisher: Optional[EventPublisher] = None
def get_event_publisher() -> EventPublisher:
"""获取事件发布者实例"""
global _event_publisher
if _event_publisher is None:
_event_publisher = EventPublisher()
return _event_publisher
def set_event_publisher(publisher: EventPublisher):
"""设置事件发布者实例"""
global _event_publisher
_event_publisher = publisher
services/mcp-server/app/provider_health.py
+541
View File
@@ -0,0 +1,541 @@
"""
供应商健康检查模块
模型供应商可用性监控
"""
import asyncio
from datetime import datetime, timedelta
from typing import Dict, List, Optional, Tuple
from sqlalchemy import select, func, and_, desc
from sqlalchemy.ext.asyncio import AsyncSession
import httpx
import structlog
from models import ModelProvider, ProviderHealthCheck, ModelPricing
logger = structlog.get_logger(__name__)
# 健康检查配置
HEALTH_CHECK_TIMEOUT = 10 # 秒
HEALTH_CHECK_INTERVAL = 60 # 秒
UNHEALTHY_THRESHOLD = 3 # 连续失败次数
async def check_provider_health(
provider: ModelProvider,
check_type: str = "ping"
) -> Tuple[bool, int, Optional[int], Optional[str]]:
"""
检查单个供应商健康状态
Args:
provider: 模型供应商
check_type: 检查类型 (ping, model_list, completion)
Returns:
(是否健康, 响应时间ms, HTTP状态码, 错误信息)
"""
start_time = datetime.utcnow()
try:
async with httpx.AsyncClient(timeout=HEALTH_CHECK_TIMEOUT) as client:
if check_type == "ping":
# 简单的ping检查
response = await client.get(
f"{provider.api_url}/health",
headers={"Authorization": f"Bearer {provider.api_key_encrypted[:20]}..."}
)
elif check_type == "model_list":
# 检查模型列表API
response = await client.get(
f"{provider.api_url}/v1/models",
headers={"Authorization": f"Bearer {provider.api_key_encrypted[:20]}..."}
)
else:
# 默认ping
response = await client.head(provider.api_url)
end_time = datetime.utcnow()
response_time_ms = int((end_time - start_time).total_seconds() * 1000)
is_healthy = response.status_code < 500
return is_healthy, response_time_ms, response.status_code, None
except httpx.TimeoutException:
end_time = datetime.utcnow()
response_time_ms = int((end_time - start_time).total_seconds() * 1000)
return False, response_time_ms, None, "请求超时"
except httpx.ConnectError as e:
return False, 0, None, f"连接失败: {str(e)}"
except Exception as e:
return False, 0, None, f"检查失败: {str(e)}"
async def record_health_check(
provider_id: str,
is_healthy: bool,
response_time_ms: int,
status_code: Optional[int],
error_message: Optional[str],
check_type: str,
db: AsyncSession
) -> ProviderHealthCheck:
"""
记录健康检查结果
Args:
provider_id: 供应商ID
is_healthy: 是否健康
response_time_ms: 响应时间
status_code: HTTP状态码
error_message: 错误信息
check_type: 检查类型
db: 数据库会话
Returns:
ProviderHealthCheck记录
"""
health_check = ProviderHealthCheck(
provider_id=provider_id,
is_healthy=is_healthy,
response_time_ms=response_time_ms,
status_code=status_code,
error_message=error_message,
check_type=check_type,
)
db.add(health_check)
await db.commit()
await db.refresh(health_check)
return health_check
async def run_health_checks(db: AsyncSession) -> Dict:
"""
运行所有供应商的健康检查
Args:
db: 数据库会话
Returns:
检查结果汇总
"""
result = await db.execute(
select(ModelProvider).where(ModelProvider.is_active == True)
)
providers = result.scalars().all()
results = {
"timestamp": datetime.utcnow().isoformat(),
"totalProviders": len(providers),
"healthyCount": 0,
"unhealthyCount": 0,
"checks": [],
}
for provider in providers:
is_healthy, response_time_ms, status_code, error_message = await check_provider_health(
provider, "ping"
)
# 记录检查结果
await record_health_check(
provider_id=str(provider.id),
is_healthy=is_healthy,
response_time_ms=response_time_ms,
status_code=status_code,
error_message=error_message,
check_type="ping",
db=db,
)
if is_healthy:
results["healthyCount"] += 1
else:
results["unhealthyCount"] += 1
results["checks"].append({
"providerId": str(provider.id),
"providerName": provider.name,
"isHealthy": is_healthy,
"responseTimeMs": response_time_ms,
"statusCode": status_code,
"errorMessage": error_message,
})
# 更新供应商状态
if not is_healthy:
# 检查连续失败次数
consecutive_failures = await get_consecutive_failures(str(provider.id), db)
if consecutive_failures >= UNHEALTHY_THRESHOLD:
provider.status = "unhealthy"
await db.commit()
logger.warning(
"供应商标记为不健康",
provider_id=str(provider.id),
provider_name=provider.name,
consecutive_failures=consecutive_failures,
)
else:
if provider.status == "unhealthy":
provider.status = "active"
await db.commit()
logger.info(
"供应商恢复健康",
provider_id=str(provider.id),
provider_name=provider.name,
)
return results
async def get_consecutive_failures(
provider_id: str,
db: AsyncSession
) -> int:
"""
获取连续失败次数
Args:
provider_id: 供应商ID
db: 数据库会话
Returns:
连续失败次数
"""
result = await db.execute(
select(ProviderHealthCheck)
.where(ProviderHealthCheck.provider_id == provider_id)
.order_by(desc(ProviderHealthCheck.created_at))
.limit(UNHEALTHY_THRESHOLD)
)
checks = result.scalars().all()
failures = 0
for check in checks:
if not check.is_healthy:
failures += 1
else:
break
return failures
async def get_provider_health_summary(
provider_id: str,
hours: int,
db: AsyncSession
) -> Dict:
"""
获取供应商健康汇总
Args:
provider_id: 供应商ID
hours: 小时数
db: 数据库会话
Returns:
健康汇总
"""
start_time = datetime.utcnow() - timedelta(hours=hours)
# 获取供应商信息
provider_result = await db.execute(
select(ModelProvider).where(ModelProvider.id == provider_id)
)
provider = provider_result.scalar_one_or_none()
if not provider:
return {"error": "供应商不存在"}
# 统计健康检查
stats_result = await db.execute(
select(
func.count(ProviderHealthCheck.id).label("total_checks"),
func.sum(func.cast(ProviderHealthCheck.is_healthy, sa.Integer)).label("healthy_count"),
func.avg(ProviderHealthCheck.response_time_ms).label("avg_response_time"),
func.max(ProviderHealthCheck.response_time_ms).label("max_response_time"),
func.min(ProviderHealthCheck.response_time_ms).label("min_response_time"),
)
.where(
and_(
ProviderHealthCheck.provider_id == provider_id,
ProviderHealthCheck.created_at >= start_time,
)
)
)
stats = stats_result.first()
total_checks = stats.total_checks or 0
healthy_count = int(stats.healthy_count or 0)
uptime_percent = round(healthy_count / total_checks * 100, 2) if total_checks > 0 else 100
# 获取最近的检查记录
recent_result = await db.execute(
select(ProviderHealthCheck)
.where(ProviderHealthCheck.provider_id == provider_id)
.order_by(desc(ProviderHealthCheck.created_at))
.limit(10)
)
recent_checks = recent_result.scalars().all()
return {
"providerId": str(provider.id),
"providerName": provider.name,
"currentStatus": provider.status,
"period": f"{hours}h",
"totalChecks": total_checks,
"healthyCount": healthy_count,
"unhealthyCount": total_checks - healthy_count,
"uptimePercent": uptime_percent,
"avgResponseTimeMs": float(stats.avg_response_time or 0),
"maxResponseTimeMs": int(stats.max_response_time or 0),
"minResponseTimeMs": int(stats.min_response_time or 0),
"recentChecks": [
{
"isHealthy": check.is_healthy,
"responseTimeMs": check.response_time_ms,
"statusCode": check.status_code,
"errorMessage": check.error_message,
"createdAt": check.created_at.isoformat(),
}
for check in recent_checks
],
}
async def get_all_providers_health(db: AsyncSession) -> List[Dict]:
"""
获取所有供应商的健康状态
Args:
db: 数据库会话
Returns:
供应商健康状态列表
"""
result = await db.execute(
select(ModelProvider).where(ModelProvider.is_active == True)
)
providers = result.scalars().all()
health_list = []
for provider in providers:
# 获取最近一次检查
last_check_result = await db.execute(
select(ProviderHealthCheck)
.where(ProviderHealthCheck.provider_id == provider.id)
.order_by(desc(ProviderHealthCheck.created_at))
.limit(1)
)
last_check = last_check_result.scalar_one_or_none()
# 计算24小时可用率
twenty_four_hours_ago = datetime.utcnow() - timedelta(hours=24)
uptime_result = await db.execute(
select(
func.count(ProviderHealthCheck.id).label("total"),
func.sum(func.cast(ProviderHealthCheck.is_healthy, sa.Integer)).label("healthy"),
)
.where(
and_(
ProviderHealthCheck.provider_id == provider.id,
ProviderHealthCheck.created_at >= twenty_four_hours_ago,
)
)
)
uptime_stats = uptime_result.first()
total = uptime_stats.total or 0
healthy = int(uptime_stats.healthy or 0)
uptime_24h = round(healthy / total * 100, 2) if total > 0 else 100
health_list.append({
"providerId": str(provider.id),
"providerName": provider.name,
"provider": provider.provider,
"status": provider.status,
"isHealthy": last_check.is_healthy if last_check else True,
"lastResponseTimeMs": last_check.response_time_ms if last_check else None,
"lastCheckAt": last_check.created_at.isoformat() if last_check else None,
"uptime24h": uptime_24h,
})
return health_list
# ============= 模型定价管理 =============
async def create_model_pricing(
provider_id: str,
model_name: str,
input_price_per_1k: float,
output_price_per_1k: float,
eu_per_1k_tokens: float,
max_context_length: int,
max_output_tokens: int,
db: AsyncSession
) -> ModelPricing:
"""
创建模型定价
Args:
provider_id: 供应商ID
model_name: 模型名称
input_price_per_1k: 输入价格（每1K tokens）
output_price_per_1k: 输出价格（每1K tokens）
eu_per_1k_tokens: EU转换率
max_context_length: 最大上下文长度
max_output_tokens: 最大输出tokens
db: 数据库会话
Returns:
ModelPricing记录
"""
from decimal import Decimal
# 检查是否已存在
result = await db.execute(
select(ModelPricing).where(
and_(
ModelPricing.provider_id == provider_id,
ModelPricing.model_name == model_name,
)
)
)
existing = result.scalar_one_or_none()
if existing:
# 更新现有定价
existing.input_price_per_1k = Decimal(str(input_price_per_1k))
existing.output_price_per_1k = Decimal(str(output_price_per_1k))
existing.eu_per_1k_tokens = Decimal(str(eu_per_1k_tokens))
existing.max_context_length = max_context_length
existing.max_output_tokens = max_output_tokens
pricing = existing
else:
# 创建新定价
pricing = ModelPricing(
provider_id=provider_id,
model_name=model_name,
input_price_per_1k=Decimal(str(input_price_per_1k)),
output_price_per_1k=Decimal(str(output_price_per_1k)),
eu_per_1k_tokens=Decimal(str(eu_per_1k_tokens)),
max_context_length=max_context_length,
max_output_tokens=max_output_tokens,
)
db.add(pricing)
await db.commit()
await db.refresh(pricing)
return pricing
async def get_model_pricing(
provider_id: Optional[str],
model_name: Optional[str],
db: AsyncSession
) -> List[Dict]:
"""
获取模型定价列表
Args:
provider_id: 供应商ID过滤
model_name: 模型名称过滤
db: 数据库会话
Returns:
定价列表
"""
query = select(ModelPricing, ModelProvider).join(
ModelProvider, ModelPricing.provider_id == ModelProvider.id
)
if provider_id:
query = query.where(ModelPricing.provider_id == provider_id)
if model_name:
query = query.where(ModelPricing.model_name.ilike(f"%{model_name}%"))
query = query.where(ModelPricing.is_active == True)
result = await db.execute(query)
return [
{
"id": str(pricing.id),
"providerId": str(pricing.provider_id),
"providerName": provider.name,
"modelName": pricing.model_name,
"inputPricePer1k": float(pricing.input_price_per_1k),
"outputPricePer1k": float(pricing.output_price_per_1k),
"euPer1kTokens": float(pricing.eu_per_1k_tokens),
"maxContextLength": pricing.max_context_length,
"maxOutputTokens": pricing.max_output_tokens,
"isActive": pricing.is_active,
"effectiveFrom": pricing.effective_from.isoformat() if pricing.effective_from else None,
}
for pricing, provider in result.all()
]
async def calculate_model_cost(
model_name: str,
input_tokens: int,
output_tokens: int,
db: AsyncSession
) -> Dict:
"""
计算模型调用成本
Args:
model_name: 模型名称
input_tokens: 输入tokens
output_tokens: 输出tokens
db: 数据库会话
Returns:
成本详情
"""
from decimal import Decimal
# 查找定价
result = await db.execute(
select(ModelPricing)
.where(
and_(
ModelPricing.model_name == model_name,
ModelPricing.is_active == True,
)
)
.limit(1)
)
pricing = result.scalar_one_or_none()
if not pricing:
# 使用默认定价
input_cost = Decimal(input_tokens) / 1000 * Decimal("0.001")
output_cost = Decimal(output_tokens) / 1000 * Decimal("0.002")
eu_consumed = Decimal(input_tokens + output_tokens) / 1000 * Decimal("0.1")
else:
input_cost = Decimal(input_tokens) / 1000 * pricing.input_price_per_1k
output_cost = Decimal(output_tokens) / 1000 * pricing.output_price_per_1k
eu_consumed = Decimal(input_tokens + output_tokens) / 1000 * pricing.eu_per_1k_tokens
total_cost = input_cost + output_cost
return {
"modelName": model_name,
"inputTokens": input_tokens,
"outputTokens": output_tokens,
"inputCost": float(input_cost),
"outputCost": float(output_cost),
"totalCost": float(total_cost),
"euConsumed": float(eu_consumed),
}
# 需要导入sa模块
import sqlalchemy as sa
services/mcp-server/app/quota_manager.py
+383
View File
@@ -0,0 +1,383 @@
"""
预付费配额管理模块
配额预警与限制
"""
from datetime import datetime, timedelta
from decimal import Decimal
from typing import Dict, List, Optional, Tuple
from sqlalchemy import select, func, and_, update
from sqlalchemy.ext.asyncio import AsyncSession
from models import User, Channel, QuotaAlert, BillingRecord, ResourceAllocation
# 配额预警阈值配置
QUOTA_THRESHOLDS = {
"balance_warning": 20, # 余额低于20%时预警
"balance_critical": 10, # 余额低于10%时严重预警
"quota_warning": 80, # 配额使用超过80%时预警
"quota_critical": 95, # 配额使用超过95%时严重预警
}
async def check_user_balance_quota(
user_id: str,
db: AsyncSession
) -> Tuple[bool, Optional[str], Dict]:
"""
检查用户余额配额
Args:
user_id: 用户ID
db: 数据库会话
Returns:
(是否有足够配额, 预警类型, 详情)
"""
result = await db.execute(
select(User).where(User.id == user_id)
)
user = result.scalar_one_or_none()
if not user:
return False, "user_not_found", {"message": "用户不存在"}
balance = Decimal(str(user.balance))
credit_limit = Decimal(str(user.credit_limit))
available = balance + credit_limit
# 获取用户平均日消费
thirty_days_ago = datetime.utcnow() - timedelta(days=30)
cost_result = await db.execute(
select(func.sum(BillingRecord.cost))
.where(
and_(
BillingRecord.tenant_id == user_id,
BillingRecord.timestamp >= thirty_days_ago,
)
)
)
total_cost = cost_result.scalar() or 0
daily_avg = Decimal(str(total_cost)) / 30
# 预估可用天数
if daily_avg > 0:
estimated_days = float(available / daily_avg)
else:
estimated_days = 999
details = {
"balance": float(balance),
"creditLimit": float(credit_limit),
"available": float(available),
"dailyAvgCost": float(daily_avg),
"estimatedDays": round(estimated_days, 1),
}
# 判断是否需要预警
if available <= 0:
return False, "balance_exhausted", details
elif estimated_days <= 3:
return True, "balance_critical", details
elif estimated_days <= 7:
return True, "balance_warning", details
return True, None, details
async def check_channel_quota(
channel_id: str,
db: AsyncSession
) -> Tuple[bool, Optional[str], Dict]:
"""
检查渠道配额
Args:
channel_id: 渠道ID
db: 数据库会话
Returns:
(是否有足够配额, 预警类型, 详情)
"""
result = await db.execute(
select(Channel).where(Channel.id == channel_id)
)
channel = result.scalar_one_or_none()
if not channel:
return False, "channel_not_found", {"message": "渠道不存在"}
channel_credit = Decimal(str(channel.channel_credit))
# 获取渠道下所有租户的总消费
thirty_days_ago = datetime.utcnow() - timedelta(days=30)
cost_result = await db.execute(
select(func.sum(BillingRecord.cost))
.where(
and_(
BillingRecord.channel_id == channel_id,
BillingRecord.timestamp >= thirty_days_ago,
)
)
)
total_cost = cost_result.scalar() or 0
details = {
"channelCredit": float(channel_credit),
"monthlyUsage": float(total_cost),
"usagePercent": round(float(total_cost) / float(channel_credit) * 100, 2) if channel_credit > 0 else 0,
}
if channel_credit > 0:
usage_percent = float(total_cost) / float(channel_credit) * 100
if usage_percent >= QUOTA_THRESHOLDS["quota_critical"]:
return True, "quota_critical", details
elif usage_percent >= QUOTA_THRESHOLDS["quota_warning"]:
return True, "quota_warning", details
return True, None, details
async def create_quota_alert(
user_id: str,
channel_id: Optional[str],
alert_type: str,
threshold_percent: int,
current_value: Decimal,
threshold_value: Decimal,
db: AsyncSession
) -> QuotaAlert:
"""
创建配额预警记录
Args:
user_id: 用户ID
channel_id: 渠道ID
alert_type: 预警类型
threshold_percent: 阈值百分比
current_value: 当前值
threshold_value: 阈值
db: 数据库会话
Returns:
QuotaAlert记录
"""
# 检查是否已有相同的活跃预警
result = await db.execute(
select(QuotaAlert).where(
and_(
QuotaAlert.user_id == user_id,
QuotaAlert.alert_type == alert_type,
QuotaAlert.status == "active",
)
)
)
existing = result.scalar_one_or_none()
if existing:
# 更新现有预警
existing.current_value = current_value
existing.threshold_value = threshold_value
existing.updated_at = datetime.utcnow()
alert = existing
else:
# 创建新预警
alert = QuotaAlert(
user_id=user_id,
channel_id=channel_id,
alert_type=alert_type,
threshold_percent=threshold_percent,
current_value=current_value,
threshold_value=threshold_value,
status="active",
)
db.add(alert)
await db.commit()
await db.refresh(alert)
return alert
async def get_active_alerts(
user_id: Optional[str],
channel_id: Optional[str],
db: AsyncSession
) -> List[Dict]:
"""
获取活跃的配额预警
Args:
user_id: 用户ID
channel_id: 渠道ID
db: 数据库会话
Returns:
预警列表
"""
query = select(QuotaAlert).where(QuotaAlert.status == "active")
if user_id:
query = query.where(QuotaAlert.user_id == user_id)
if channel_id:
query = query.where(QuotaAlert.channel_id == channel_id)
result = await db.execute(query.order_by(QuotaAlert.created_at.desc()))
alerts = result.scalars().all()
return [
{
"id": str(alert.id),
"alertType": alert.alert_type,
"thresholdPercent": alert.threshold_percent,
"currentValue": float(alert.current_value) if alert.current_value else None,
"thresholdValue": float(alert.threshold_value) if alert.threshold_value else None,
"status": alert.status,
"createdAt": alert.created_at.isoformat(),
}
for alert in alerts
]
async def acknowledge_alert(
alert_id: str,
db: AsyncSession
) -> bool:
"""
确认预警
Args:
alert_id: 预警ID
db: 数据库会话
Returns:
是否成功
"""
result = await db.execute(
select(QuotaAlert).where(QuotaAlert.id == alert_id)
)
alert = result.scalar_one_or_none()
if not alert:
return False
alert.status = "acknowledged"
alert.acknowledged_at = datetime.utcnow()
await db.commit()
return True
async def resolve_alert(
alert_id: str,
db: AsyncSession
) -> bool:
"""
解决预警
Args:
alert_id: 预警ID
db: 数据库会话
Returns:
是否成功
"""
result = await db.execute(
select(QuotaAlert).where(QuotaAlert.id == alert_id)
)
alert = result.scalar_one_or_none()
if not alert:
return False
alert.status = "resolved"
alert.resolved_at = datetime.utcnow()
await db.commit()
return True
async def check_rate_limit(
user_id: str,
resource_type: str,
db: AsyncSession
) -> Tuple[bool, int, int]:
"""
检查速率限制
Args:
user_id: 用户ID
resource_type: 资源类型 (api, model)
db: 数据库会话
Returns:
(是否允许, 当前使用量, 限制量)
"""
# 获取最近1分钟的调用次数
one_minute_ago = datetime.utcnow() - timedelta(minutes=1)
result = await db.execute(
select(func.count(BillingRecord.id))
.where(
and_(
BillingRecord.tenant_id == user_id,
BillingRecord.timestamp >= one_minute_ago,
)
)
)
current_rpm = result.scalar() or 0
# 获取用户的RPM限制（从资源分配表）
limit_result = await db.execute(
select(ResourceAllocation.rpm)
.where(
and_(
ResourceAllocation.target_id == user_id,
ResourceAllocation.target_type == "tenant",
ResourceAllocation.resource_type == resource_type,
)
)
)
limit_row = limit_result.first()
rpm_limit = limit_row[0] if limit_row and limit_row[0] else 60 # 默认60 RPM
return current_rpm < rpm_limit, current_rpm, rpm_limit
async def get_quota_summary(
user_id: str,
db: AsyncSession
) -> Dict:
"""
获取配额汇总信息
Args:
user_id: 用户ID
db: 数据库会话
Returns:
配额汇总
"""
# 获取余额信息
has_quota, alert_type, balance_info = await check_user_balance_quota(user_id, db)
# 获取速率限制
rate_allowed, current_rpm, rpm_limit = await check_rate_limit(user_id, "api", db)
# 获取活跃预警数
alerts = await get_active_alerts(user_id, None, db)
return {
"hasQuota": has_quota,
"alertType": alert_type,
"balance": balance_info,
"rateLimit": {
"currentRpm": current_rpm,
"rpmLimit": rpm_limit,
"allowed": rate_allowed,
},
"activeAlerts": len(alerts),
"alerts": alerts,
}
services/mcp-server/app/resource_monitor.py
+361
View File
@@ -0,0 +1,361 @@
"""
资源使用监控模块
采集和统计用户资源使用情况
"""
from datetime import datetime, timedelta
from decimal import Decimal
from typing import Dict, List, Optional, Tuple
from sqlalchemy import select, func, and_
from sqlalchemy.ext.asyncio import AsyncSession
from models import ResourceUsage, User, Agent, BillingRecord, Execution
async def record_resource_usage(
user_id: str,
agent_id: Optional[str],
cpu_seconds: float,
memory_mb_seconds: float,
network_bytes: int,
storage_bytes: int,
api_calls: int,
period_start: datetime,
period_end: datetime,
granularity: str,
db: AsyncSession
) -> ResourceUsage:
"""
记录资源使用情况
Args:
user_id: 用户ID
agent_id: Agent ID
cpu_seconds: CPU使用秒数
memory_mb_seconds: 内存使用 (MB*秒)
network_bytes: 网络流量字节
storage_bytes: 存储使用字节
api_calls: API调用次数
period_start: 周期开始时间
period_end: 周期结束时间
granularity: 粒度 (hourly, daily, monthly)
db: 数据库会话
Returns:
ResourceUsage记录
"""
# 检查是否已有该周期的记录
result = await db.execute(
select(ResourceUsage).where(
and_(
ResourceUsage.user_id == user_id,
ResourceUsage.agent_id == agent_id if agent_id else True,
ResourceUsage.period_start == period_start,
ResourceUsage.granularity == granularity,
)
)
)
existing = result.scalar_one_or_none()
if existing:
# 更新现有记录
existing.cpu_seconds += cpu_seconds
existing.memory_mb_seconds += memory_mb_seconds
existing.network_bytes += network_bytes
existing.storage_bytes += storage_bytes
existing.api_calls += api_calls
usage = existing
else:
# 创建新记录
usage = ResourceUsage(
user_id=user_id,
agent_id=agent_id,
cpu_seconds=cpu_seconds,
memory_mb_seconds=memory_mb_seconds,
network_bytes=network_bytes,
storage_bytes=storage_bytes,
api_calls=api_calls,
period_start=period_start,
period_end=period_end,
granularity=granularity,
)
db.add(usage)
await db.commit()
await db.refresh(usage)
return usage
async def get_user_resource_summary(
user_id: str,
start_date: datetime,
end_date: datetime,
db: AsyncSession
) -> Dict:
"""
获取用户资源使用汇总
Args:
user_id: 用户ID
start_date: 开始日期
end_date: 结束日期
db: 数据库会话
Returns:
资源使用汇总
"""
result = await db.execute(
select(
func.sum(ResourceUsage.cpu_seconds).label("total_cpu_seconds"),
func.sum(ResourceUsage.memory_mb_seconds).label("total_memory_mb_seconds"),
func.sum(ResourceUsage.network_bytes).label("total_network_bytes"),
func.sum(ResourceUsage.storage_bytes).label("total_storage_bytes"),
func.sum(ResourceUsage.api_calls).label("total_api_calls"),
)
.where(
and_(
ResourceUsage.user_id == user_id,
ResourceUsage.period_start >= start_date,
ResourceUsage.period_end <= end_date,
)
)
)
row = result.first()
return {
"totalCpuSeconds": float(row.total_cpu_seconds or 0),
"totalMemoryMbSeconds": float(row.total_memory_mb_seconds or 0),
"totalNetworkBytes": int(row.total_network_bytes or 0),
"totalStorageBytes": int(row.total_storage_bytes or 0),
"totalApiCalls": int(row.total_api_calls or 0),
"startDate": start_date.isoformat(),
"endDate": end_date.isoformat(),
}
async def get_resource_trends(
user_id: str,
period: str,
granularity: str,
db: AsyncSession
) -> List[Dict]:
"""
获取资源使用趋势
Args:
user_id: 用户ID
period: 时间范围 (7d, 30d, 90d)
granularity: 粒度 (hourly, daily)
db: 数据库会话
Returns:
资源使用趋势列表
"""
days_map = {"7d": 7, "30d": 30, "90d": 90}
days = days_map.get(period, 7)
start_date = datetime.utcnow() - timedelta(days=days)
result = await db.execute(
select(ResourceUsage)
.where(
and_(
ResourceUsage.user_id == user_id,
ResourceUsage.period_start >= start_date,
ResourceUsage.granularity == granularity,
)
)
.order_by(ResourceUsage.period_start)
)
usages = result.scalars().all()
return [
{
"periodStart": usage.period_start.isoformat(),
"periodEnd": usage.period_end.isoformat(),
"cpuSeconds": float(usage.cpu_seconds),
"memoryMbSeconds": float(usage.memory_mb_seconds),
"networkBytes": int(usage.network_bytes),
"apiCalls": int(usage.api_calls),
}
for usage in usages
]
async def get_agent_resource_stats(
agent_id: str,
start_date: datetime,
end_date: datetime,
db: AsyncSession
) -> Dict:
"""
获取Agent资源统计
Args:
agent_id: Agent ID
start_date: 开始日期
end_date: 结束日期
db: 数据库会话
Returns:
Agent资源统计
"""
# 执行次数和时间统计
exec_result = await db.execute(
select(
func.count(Execution.id).label("total_executions"),
func.avg(Execution.execution_time).label("avg_execution_time"),
func.sum(Execution.eu_consumed).label("total_eu"),
)
.where(
and_(
Execution.agent_id == agent_id,
Execution.started_at >= start_date,
Execution.started_at <= end_date,
)
)
)
exec_row = exec_result.first()
# 成功率
success_result = await db.execute(
select(func.count(Execution.id))
.where(
and_(
Execution.agent_id == agent_id,
Execution.started_at >= start_date,
Execution.status == "completed",
)
)
)
success_count = success_result.scalar() or 0
total_count = exec_row.total_executions or 1
return {
"agentId": agent_id,
"totalExecutions": int(exec_row.total_executions or 0),
"avgExecutionTime": float(exec_row.avg_execution_time or 0),
"totalEuConsumed": float(exec_row.total_eu or 0),
"successRate": round(success_count / total_count * 100, 2) if total_count > 0 else 0,
"startDate": start_date.isoformat(),
"endDate": end_date.isoformat(),
}
async def get_platform_resource_overview(db: AsyncSession) -> Dict:
"""
获取平台资源概览（管理员视图）
Args:
db: 数据库会话
Returns:
平台资源概览
"""
now = datetime.utcnow()
today_start = now.replace(hour=0, minute=0, second=0, microsecond=0)
month_start = now.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
# 今日调用次数
today_calls = await db.execute(
select(func.count(BillingRecord.id))
.where(BillingRecord.timestamp >= today_start)
)
# 本月调用次数
month_calls = await db.execute(
select(func.count(BillingRecord.id))
.where(BillingRecord.timestamp >= month_start)
)
# 活跃用户数
active_users = await db.execute(
select(func.count(func.distinct(BillingRecord.tenant_id)))
.where(BillingRecord.timestamp >= today_start)
)
# 活跃Agent数
active_agents = await db.execute(
select(func.count(Agent.id))
.where(Agent.status == "active")
)
# 总EU消耗
total_eu = await db.execute(
select(func.sum(BillingRecord.eu))
.where(BillingRecord.timestamp >= month_start)
)
return {
"todayCalls": today_calls.scalar() or 0,
"monthCalls": month_calls.scalar() or 0,
"activeUsersToday": active_users.scalar() or 0,
"activeAgents": active_agents.scalar() or 0,
"monthTotalEu": int(total_eu.scalar() or 0),
"timestamp": now.isoformat(),
}
async def aggregate_hourly_usage(db: AsyncSession) -> int:
"""
聚合小时级资源使用数据（定时任务调用）
Args:
db: 数据库会话
Returns:
聚合的记录数
"""
# 获取上一小时的时间范围
now = datetime.utcnow()
period_end = now.replace(minute=0, second=0, microsecond=0)
period_start = period_end - timedelta(hours=1)
# 从执行记录聚合
result = await db.execute(
select(
Execution.agent_id,
func.count(Execution.id).label("api_calls"),
func.sum(Execution.execution_time).label("total_time"),
func.sum(Execution.cpu_usage).label("cpu_seconds"),
func.sum(Execution.memory_usage).label("memory_usage"),
)
.where(
and_(
Execution.started_at >= period_start,
Execution.started_at < period_end,
)
)
.group_by(Execution.agent_id)
)
count = 0
for row in result.all():
if row.agent_id:
# 获取Agent所有者
agent_result = await db.execute(
select(Agent.owner_id).where(Agent.id == row.agent_id)
)
owner = agent_result.scalar()
if owner:
await record_resource_usage(
user_id=str(owner),
agent_id=str(row.agent_id),
cpu_seconds=float(row.cpu_seconds or 0),
memory_mb_seconds=float(row.memory_usage or 0) * (float(row.total_time or 0) / 1000),
network_bytes=0,
storage_bytes=0,
api_calls=int(row.api_calls or 0),
period_start=period_start,
period_end=period_end,
granularity="hourly",
db=db,
)
count += 1
return count
services/mcp-server/app/routes/__init__.py
+2 -1
View File
@@ -4,7 +4,7 @@ from fastapi import FastAPI
from . import (
agents, frontend_integration, health, metrics, monitoring, tools, websocket,
auth, user, channel, admin, providers
auth, user, channel, admin, providers, billing_admin
)
@@ -16,6 +16,7 @@ def register_routes(app: FastAPI) -> None:
channel.router,
admin.router,
providers.router,
billing_admin.router, # 计费与资源管理路由
frontend_integration.router,
agents.router,
tools.router,
services/mcp-server/app/routes/auth.py
+32 -2
View File
@@ -161,11 +161,41 @@ async def login(req: LoginRequest, db: AsyncSession = Depends(get_db)):
@router.post("/logout", response_model=SuccessResponse)
async def logout(principal: dict = Depends(require_auth)):
async def logout(
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
用户登出
将当前Token加入黑名单
"""
# 实际应该将token加入黑名单，这里简化处理
from datetime import datetime, timedelta
from app.token_blacklist import add_token_to_blacklist
user_id = principal.get("user_id")
claims = principal.get("claims", {})
# 获取Token的过期时间（从claims中提取）
exp = claims.get("exp")
if exp:
expires_at = datetime.utcfromtimestamp(exp)
else:
# 默认7天后过期
expires_at = datetime.utcnow() + timedelta(days=7)
# 生成token的唯一标识（使用sub + 时间戳）
token_jti = f"logout_{user_id}_{datetime.utcnow().timestamp()}"
# 添加到黑名单
await add_token_to_blacklist(
token_jti=token_jti,
user_id=user_id,
expires_at=expires_at,
reason="logout",
db=db,
)
return SuccessResponse(message="登出成功")
services/mcp-server/app/routes/billing_admin.py
+573
View File
@@ -0,0 +1,573 @@
"""
计费与资源管理API路由
包含配额管理、资源监控、事件查询、审计日志、供应商健康等功能
"""
from datetime import datetime, timedelta
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
from sqlalchemy.ext.asyncio import AsyncSession
from pydantic import BaseModel, Field
from decimal import Decimal
from database import get_db
from app.auth import require_auth
from app.permissions import has_permission
from app.schemas import SuccessResponse
# 导入功能模块
from app.quota_manager import (
check_user_balance_quota,
check_channel_quota,
get_active_alerts,
acknowledge_alert,
resolve_alert,
get_quota_summary,
)
from app.resource_monitor import (
get_user_resource_summary,
get_resource_trends,
get_agent_resource_stats,
get_platform_resource_overview,
)
from app.events import (
get_pending_events,
retry_failed_events,
get_event_stats,
)
from app.tracing import (
get_execution_trace,
query_traces,
get_trace_stats,
)
from app.audit import (
query_audit_logs,
get_audit_summary,
get_user_activity,
log_audit_event,
)
from app.provider_health import (
run_health_checks,
get_provider_health_summary,
get_all_providers_health,
create_model_pricing,
get_model_pricing,
calculate_model_cost,
)
router = APIRouter(prefix="/api/billing-admin", tags=["计费与资源管理"])
def _get_role(principal: dict) -> str:
"""从principal获取角色"""
return principal.get("claims", {}).get("role", "")
def _verify_admin_permission(principal: dict):
"""验证管理员权限"""
role = _get_role(principal)
if role not in ["super_admin", "billing_admin", "operations_admin"]:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="需要管理员权限"
)
def _verify_write_permission(principal: dict):
"""验证写入权限"""
role = _get_role(principal)
if role not in ["super_admin", "billing_admin"]:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="需要写入权限"
)
# ============= 配额管理 =============
@router.get("/quota/user/{user_id}", response_model=SuccessResponse)
async def get_user_quota(
user_id: str,
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取用户配额信息
"""
_verify_admin_permission(principal)
summary = await get_quota_summary(user_id, db)
return SuccessResponse(data=summary)
@router.get("/quota/channel/{channel_id}", response_model=SuccessResponse)
async def get_channel_quota(
channel_id: str,
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取渠道配额信息
"""
_verify_admin_permission(principal)
has_quota, alert_type, details = await check_channel_quota(channel_id, db)
return SuccessResponse(data={
"hasQuota": has_quota,
"alertType": alert_type,
**details,
})
@router.get("/quota/alerts", response_model=SuccessResponse)
async def list_quota_alerts(
user_id: Optional[str] = Query(None),
channel_id: Optional[str] = Query(None),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取配额预警列表
"""
_verify_admin_permission(principal)
alerts = await get_active_alerts(user_id, channel_id, db)
return SuccessResponse(data={"alerts": alerts, "count": len(alerts)})
@router.put("/quota/alerts/{alert_id}/acknowledge", response_model=SuccessResponse)
async def ack_quota_alert(
alert_id: str,
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
确认配额预警
"""
_verify_admin_permission(principal)
success = await acknowledge_alert(alert_id, db)
if not success:
raise HTTPException(status_code=404, detail="预警不存在")
return SuccessResponse(message="预警已确认")
@router.put("/quota/alerts/{alert_id}/resolve", response_model=SuccessResponse)
async def resolve_quota_alert(
alert_id: str,
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
解决配额预警
"""
_verify_write_permission(principal)
success = await resolve_alert(alert_id, db)
if not success:
raise HTTPException(status_code=404, detail="预警不存在")
return SuccessResponse(message="预警已解决")
# ============= 资源监控 =============
@router.get("/resources/overview", response_model=SuccessResponse)
async def get_resources_overview(
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取平台资源概览
"""
_verify_admin_permission(principal)
overview = await get_platform_resource_overview(db)
return SuccessResponse(data=overview)
@router.get("/resources/user/{user_id}", response_model=SuccessResponse)
async def get_user_resources(
user_id: str,
start_date: str = Query(...),
end_date: str = Query(...),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取用户资源使用汇总
"""
_verify_admin_permission(principal)
start_dt = datetime.fromisoformat(start_date.replace("Z", "+00:00"))
end_dt = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
summary = await get_user_resource_summary(user_id, start_dt, end_dt, db)
return SuccessResponse(data=summary)
@router.get("/resources/trends", response_model=SuccessResponse)
async def get_resource_usage_trends(
user_id: str = Query(...),
period: str = Query("7d", pattern="^(7d|30d|90d)$"),
granularity: str = Query("daily", pattern="^(hourly|daily)$"),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取资源使用趋势
"""
_verify_admin_permission(principal)
trends = await get_resource_trends(user_id, period, granularity, db)
return SuccessResponse(data={"trends": trends})
@router.get("/resources/agent/{agent_id}", response_model=SuccessResponse)
async def get_agent_resources(
agent_id: str,
start_date: str = Query(...),
end_date: str = Query(...),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取Agent资源统计
"""
_verify_admin_permission(principal)
start_dt = datetime.fromisoformat(start_date.replace("Z", "+00:00"))
end_dt = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
stats = await get_agent_resource_stats(agent_id, start_dt, end_dt, db)
return SuccessResponse(data=stats)
# ============= 事件管理 =============
@router.get("/events/pending", response_model=SuccessResponse)
async def list_pending_events(
limit: int = Query(100, ge=1, le=1000),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取待处理的事件
"""
_verify_admin_permission(principal)
events = await get_pending_events(limit, db)
return SuccessResponse(data={"events": events, "count": len(events)})
@router.post("/events/retry-failed", response_model=SuccessResponse)
async def retry_failed_billing_events(
max_retries: int = Query(3, ge=1, le=10),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
重试失败的事件
"""
_verify_write_permission(principal)
count = await retry_failed_events(max_retries, db)
return SuccessResponse(data={"retriedCount": count}, message=f"已重试 {count} 个事件")
@router.get("/events/stats", response_model=SuccessResponse)
async def get_billing_event_stats(
start_date: str = Query(...),
end_date: str = Query(...),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取事件统计
"""
_verify_admin_permission(principal)
start_dt = datetime.fromisoformat(start_date.replace("Z", "+00:00"))
end_dt = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
stats = await get_event_stats(start_dt, end_dt, db)
return SuccessResponse(data=stats)
# ============= 追踪管理 =============
@router.get("/traces/execution/{execution_id}", response_model=SuccessResponse)
async def get_execution_traces(
execution_id: str,
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取执行追踪详情
"""
_verify_admin_permission(principal)
trace = await get_execution_trace(execution_id, db)
return SuccessResponse(data=trace)
@router.get("/traces", response_model=SuccessResponse)
async def list_traces(
user_id: Optional[str] = Query(None),
agent_id: Optional[str] = Query(None),
status: Optional[str] = Query(None),
start_date: Optional[str] = Query(None),
end_date: Optional[str] = Query(None),
page: int = Query(1, ge=1),
page_size: int = Query(20, ge=1, le=100),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
查询追踪记录
"""
_verify_admin_permission(principal)
start_dt = datetime.fromisoformat(start_date.replace("Z", "+00:00")) if start_date else None
end_dt = datetime.fromisoformat(end_date.replace("Z", "+00:00")) if end_date else None
result = await query_traces(
user_id=user_id,
agent_id=agent_id,
status=status,
start_date=start_dt,
end_date=end_dt,
page=page,
page_size=page_size,
db=db,
)
return SuccessResponse(data=result)
@router.get("/traces/stats", response_model=SuccessResponse)
async def get_traces_stats(
user_id: Optional[str] = Query(None),
start_date: str = Query(...),
end_date: str = Query(...),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取追踪统计
"""
_verify_admin_permission(principal)
start_dt = datetime.fromisoformat(start_date.replace("Z", "+00:00"))
end_dt = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
stats = await get_trace_stats(user_id, start_dt, end_dt, db)
return SuccessResponse(data=stats)
# ============= 审计日志 =============
@router.get("/audit/logs", response_model=SuccessResponse)
async def list_audit_logs(
user_id: Optional[str] = Query(None),
action: Optional[str] = Query(None),
resource_type: Optional[str] = Query(None),
success: Optional[bool] = Query(None),
start_date: Optional[str] = Query(None),
end_date: Optional[str] = Query(None),
page: int = Query(1, ge=1),
page_size: int = Query(20, ge=1, le=100),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
查询审计日志
"""
_verify_admin_permission(principal)
start_dt = datetime.fromisoformat(start_date.replace("Z", "+00:00")) if start_date else None
end_dt = datetime.fromisoformat(end_date.replace("Z", "+00:00")) if end_date else None
result = await query_audit_logs(
user_id=user_id,
action=action,
resource_type=resource_type,
success=success,
start_date=start_dt,
end_date=end_dt,
page=page,
page_size=page_size,
db=db,
)
return SuccessResponse(data=result)
@router.get("/audit/summary", response_model=SuccessResponse)
async def get_audit_logs_summary(
start_date: str = Query(...),
end_date: str = Query(...),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取审计日志汇总
"""
_verify_admin_permission(principal)
start_dt = datetime.fromisoformat(start_date.replace("Z", "+00:00"))
end_dt = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
summary = await get_audit_summary(start_dt, end_dt, db)
return SuccessResponse(data=summary)
@router.get("/audit/user/{user_id}/activity", response_model=SuccessResponse)
async def get_user_audit_activity(
user_id: str,
days: int = Query(30, ge=1, le=90),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取用户活动历史
"""
_verify_admin_permission(principal)
activity = await get_user_activity(user_id, days, db)
return SuccessResponse(data={"activity": activity})
# ============= 供应商健康检查 =============
@router.get("/providers/health", response_model=SuccessResponse)
async def get_providers_health(
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取所有供应商健康状态
"""
_verify_admin_permission(principal)
health_list = await get_all_providers_health(db)
return SuccessResponse(data={"providers": health_list})
@router.get("/providers/{provider_id}/health", response_model=SuccessResponse)
async def get_provider_health(
provider_id: str,
hours: int = Query(24, ge=1, le=168),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取单个供应商健康详情
"""
_verify_admin_permission(principal)
summary = await get_provider_health_summary(provider_id, hours, db)
return SuccessResponse(data=summary)
@router.post("/providers/health-check", response_model=SuccessResponse)
async def run_providers_health_check(
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
执行所有供应商健康检查
"""
_verify_write_permission(principal)
results = await run_health_checks(db)
return SuccessResponse(data=results)
# ============= 模型定价管理 =============
class ModelPricingRequest(BaseModel):
provider_id: str = Field(..., alias="providerId")
model_name: str = Field(..., alias="modelName")
input_price_per_1k: float = Field(..., alias="inputPricePer1k")
output_price_per_1k: float = Field(..., alias="outputPricePer1k")
eu_per_1k_tokens: float = Field(0.1, alias="euPer1kTokens")
max_context_length: int = Field(4096, alias="maxContextLength")
max_output_tokens: int = Field(2048, alias="maxOutputTokens")
class Config:
populate_by_name = True
@router.get("/pricing/models", response_model=SuccessResponse)
async def list_model_pricing(
provider_id: Optional[str] = Query(None),
model_name: Optional[str] = Query(None),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
获取模型定价列表
"""
_verify_admin_permission(principal)
pricing_list = await get_model_pricing(provider_id, model_name, db)
return SuccessResponse(data={"pricing": pricing_list})
@router.post("/pricing/models", response_model=SuccessResponse)
async def create_or_update_model_pricing(
req: ModelPricingRequest,
request: Request,
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
创建或更新模型定价
"""
_verify_write_permission(principal)
pricing = await create_model_pricing(
provider_id=req.provider_id,
model_name=req.model_name,
input_price_per_1k=req.input_price_per_1k,
output_price_per_1k=req.output_price_per_1k,
eu_per_1k_tokens=req.eu_per_1k_tokens,
max_context_length=req.max_context_length,
max_output_tokens=req.max_output_tokens,
db=db,
)
# 记录审计日志
await log_audit_event(
action="provider.pricing_update",
resource_type="model_pricing",
resource_id=str(pricing.id),
user_id=principal.get("user_id"),
success=True,
details={"modelName": req.model_name},
request=request,
db=db,
)
return SuccessResponse(
data={"id": str(pricing.id), "modelName": pricing.model_name},
message="模型定价已更新"
)
@router.post("/pricing/calculate", response_model=SuccessResponse)
async def calculate_pricing(
model_name: str = Query(...),
input_tokens: int = Query(..., ge=0),
output_tokens: int = Query(..., ge=0),
principal: dict = Depends(require_auth),
db: AsyncSession = Depends(get_db)
):
"""
计算模型调用成本
"""
_verify_admin_permission(principal)
result = await calculate_model_cost(model_name, input_tokens, output_tokens, db)
return SuccessResponse(data=result)
services/mcp-server/app/token_blacklist.py
+148
View File
@@ -0,0 +1,148 @@
"""
Token黑名单管理模块
用于JWT登出和Token失效控制
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import select, delete
from sqlalchemy.ext.asyncio import AsyncSession
from models import TokenBlacklist
async def add_token_to_blacklist(
token_jti: str,
user_id: str,
expires_at: datetime,
reason: str,
db: AsyncSession
) -> TokenBlacklist:
"""
将Token添加到黑名单
Args:
token_jti: JWT ID (jti claim)
user_id: 用户ID
expires_at: Token过期时间
reason: 黑名单原因 (logout, revoked, security)
db: 数据库会话
Returns:
TokenBlacklist记录
"""
blacklist_entry = TokenBlacklist(
token_jti=token_jti,
user_id=user_id,
expires_at=expires_at,
reason=reason,
)
db.add(blacklist_entry)
await db.commit()
await db.refresh(blacklist_entry)
return blacklist_entry
async def is_token_blacklisted(
token_jti: str,
db: AsyncSession
) -> bool:
"""
检查Token是否在黑名单中
Args:
token_jti: JWT ID
db: 数据库会话
Returns:
是否在黑名单中
"""
result = await db.execute(
select(TokenBlacklist).where(TokenBlacklist.token_jti == token_jti)
)
return result.scalar_one_or_none() is not None
async def revoke_all_user_tokens(
user_id: str,
reason: str,
db: AsyncSession
) -> int:
"""
撤销用户的所有Token
Args:
user_id: 用户ID
reason: 撤销原因
db: 数据库会话
Returns:
影响的Token数量
"""
# 这里需要配合JWT的jti claim使用
# 实际实现中，应该在生成JWT时记录jti，然后在这里批量加入黑名单
# 简化实现：直接标记用户的所有未过期Token
from datetime import timedelta
# 创建一个特殊的黑名单条目，表示该用户在此时间之前的所有Token都失效
blacklist_entry = TokenBlacklist(
token_jti=f"user_revoke_{user_id}_{datetime.utcnow().timestamp()}",
user_id=user_id,
expires_at=datetime.utcnow() + timedelta(days=7), # 7天后自动清理
reason=reason,
)
db.add(blacklist_entry)
await db.commit()
return 1
async def cleanup_expired_tokens(db: AsyncSession) -> int:
"""
清理过期的黑名单记录
Args:
db: 数据库会话
Returns:
清理的记录数
"""
result = await db.execute(
delete(TokenBlacklist).where(
TokenBlacklist.expires_at < datetime.utcnow()
)
)
await db.commit()
return result.rowcount
async def get_user_blacklist_count(
user_id: str,
db: AsyncSession
) -> int:
"""
获取用户的黑名单Token数量
Args:
user_id: 用户ID
db: 数据库会话
Returns:
黑名单数量
"""
from sqlalchemy import func
result = await db.execute(
select(func.count(TokenBlacklist.id)).where(
TokenBlacklist.user_id == user_id,
TokenBlacklist.expires_at > datetime.utcnow()
)
)
return result.scalar() or 0
services/mcp-server/app/tracing.py
+497
View File
@@ -0,0 +1,497 @@
"""
Agent轨迹追踪模块
执行轨迹记录与查询
"""
import uuid
from datetime import datetime, timedelta
from decimal import Decimal
from typing import Dict, List, Optional, Any
from sqlalchemy import select, func, and_, desc
from sqlalchemy.ext.asyncio import AsyncSession
import structlog
from models import AgentTrace, Execution, Agent, User
logger = structlog.get_logger(__name__)
class TraceContext:
"""追踪上下文"""
def __init__(self, trace_id: Optional[str] = None):
self.trace_id = trace_id or str(uuid.uuid4())
self.spans: List[Dict] = []
self._current_span_id: Optional[str] = None
def start_span(
self,
operation_name: str,
operation_type: str,
parent_span_id: Optional[str] = None
) -> str:
"""开始一个新的span"""
span_id = str(uuid.uuid4())
span = {
"span_id": span_id,
"parent_span_id": parent_span_id or self._current_span_id,
"operation_name": operation_name,
"operation_type": operation_type,
"started_at": datetime.utcnow(),
"ended_at": None,
"status": "running",
"input_data": None,
"output_data": None,
"error_type": None,
"error_message": None,
}
self.spans.append(span)
self._current_span_id = span_id
return span_id
def end_span(
self,
span_id: str,
status: str = "success",
output_data: Optional[Dict] = None,
error_type: Optional[str] = None,
error_message: Optional[str] = None
):
"""结束一个span"""
for span in self.spans:
if span["span_id"] == span_id:
span["ended_at"] = datetime.utcnow()
span["status"] = status
span["output_data"] = output_data
span["error_type"] = error_type
span["error_message"] = error_message
break
def set_span_input(self, span_id: str, input_data: Dict):
"""设置span的输入数据"""
for span in self.spans:
if span["span_id"] == span_id:
span["input_data"] = input_data
break
async def create_trace_record(
execution_id: str,
agent_id: str,
user_id: str,
trace_id: str,
span_id: str,
parent_span_id: Optional[str],
operation_name: str,
operation_type: str,
started_at: datetime,
ended_at: Optional[datetime],
status: str,
input_data: Optional[Dict],
output_data: Optional[Dict],
error_type: Optional[str],
error_message: Optional[str],
tokens_used: int,
eu_consumed: Decimal,
db: AsyncSession
) -> AgentTrace:
"""
创建追踪记录
Args:
execution_id: 执行ID
agent_id: Agent ID
user_id: 用户ID
trace_id: 追踪ID
span_id: Span ID
parent_span_id: 父Span ID
operation_name: 操作名称
operation_type: 操作类型
started_at: 开始时间
ended_at: 结束时间
status: 状态
input_data: 输入数据
output_data: 输出数据
error_type: 错误类型
error_message: 错误消息
tokens_used: 使用的Token数
eu_consumed: 消耗的EU
db: 数据库会话
Returns:
AgentTrace记录
"""
# 计算持续时间
duration_ms = None
if ended_at and started_at:
duration_ms = int((ended_at - started_at).total_seconds() * 1000)
# 对输入输出数据进行脱敏
sanitized_input = _sanitize_data(input_data) if input_data else None
sanitized_output = _sanitize_data(output_data) if output_data else None
trace = AgentTrace(
execution_id=execution_id,
agent_id=agent_id,
user_id=user_id,
trace_id=trace_id,
span_id=span_id,
parent_span_id=parent_span_id,
operation_name=operation_name,
operation_type=operation_type,
started_at=started_at,
ended_at=ended_at,
duration_ms=duration_ms,
input_data=sanitized_input,
output_data=sanitized_output,
status=status,
error_type=error_type,
error_message=error_message,
tokens_used=tokens_used,
eu_consumed=eu_consumed,
)
db.add(trace)
await db.commit()
await db.refresh(trace)
return trace
async def save_trace_context(
trace_context: TraceContext,
execution_id: str,
agent_id: str,
user_id: str,
db: AsyncSession
) -> int:
"""
保存完整的追踪上下文
Args:
trace_context: 追踪上下文
execution_id: 执行ID
agent_id: Agent ID
user_id: 用户ID
db: 数据库会话
Returns:
保存的span数量
"""
count = 0
for span in trace_context.spans:
await create_trace_record(
execution_id=execution_id,
agent_id=agent_id,
user_id=user_id,
trace_id=trace_context.trace_id,
span_id=span["span_id"],
parent_span_id=span["parent_span_id"],
operation_name=span["operation_name"],
operation_type=span["operation_type"],
started_at=span["started_at"],
ended_at=span["ended_at"],
status=span["status"],
input_data=span["input_data"],
output_data=span["output_data"],
error_type=span["error_type"],
error_message=span["error_message"],
tokens_used=0,
eu_consumed=Decimal(0),
db=db,
)
count += 1
return count
async def get_execution_trace(
execution_id: str,
db: AsyncSession
) -> Dict:
"""
获取执行的完整追踪
Args:
execution_id: 执行ID
db: 数据库会话
Returns:
执行追踪详情
"""
result = await db.execute(
select(AgentTrace)
.where(AgentTrace.execution_id == execution_id)
.order_by(AgentTrace.started_at)
)
traces = result.scalars().all()
if not traces:
return {"executionId": execution_id, "spans": []}
trace_id = traces[0].trace_id if traces else None
# 构建span树
spans = []
for trace in traces:
spans.append({
"spanId": trace.span_id,
"parentSpanId": trace.parent_span_id,
"operationName": trace.operation_name,
"operationType": trace.operation_type,
"startedAt": trace.started_at.isoformat(),
"endedAt": trace.ended_at.isoformat() if trace.ended_at else None,
"durationMs": trace.duration_ms,
"status": trace.status,
"inputData": trace.input_data,
"outputData": trace.output_data,
"errorType": trace.error_type,
"errorMessage": trace.error_message,
"tokensUsed": trace.tokens_used,
"euConsumed": float(trace.eu_consumed) if trace.eu_consumed else 0,
})
# 计算总持续时间和EU
total_duration = sum(s["durationMs"] or 0 for s in spans)
total_eu = sum(s["euConsumed"] for s in spans)
return {
"executionId": execution_id,
"traceId": trace_id,
"spans": spans,
"totalDurationMs": total_duration,
"totalEuConsumed": total_eu,
"spanCount": len(spans),
}
async def query_traces(
user_id: Optional[str] = None,
agent_id: Optional[str] = None,
status: Optional[str] = None,
start_date: Optional[datetime] = None,
end_date: Optional[datetime] = None,
page: int = 1,
page_size: int = 20,
db: AsyncSession = None
) -> Dict:
"""
查询追踪记录
Args:
user_id: 用户ID过滤
agent_id: Agent ID过滤
status: 状态过滤
start_date: 开始日期
end_date: 结束日期
page: 页码
page_size: 每页大小
db: 数据库会话
Returns:
查询结果
"""
# 按trace_id分组查询
query = select(
AgentTrace.trace_id,
AgentTrace.execution_id,
AgentTrace.agent_id,
AgentTrace.user_id,
func.min(AgentTrace.started_at).label("started_at"),
func.max(AgentTrace.ended_at).label("ended_at"),
func.count(AgentTrace.id).label("span_count"),
func.sum(AgentTrace.tokens_used).label("total_tokens"),
func.sum(AgentTrace.eu_consumed).label("total_eu"),
)
# 应用过滤条件
conditions = []
if user_id:
conditions.append(AgentTrace.user_id == user_id)
if agent_id:
conditions.append(AgentTrace.agent_id == agent_id)
if status:
conditions.append(AgentTrace.status == status)
if start_date:
conditions.append(AgentTrace.started_at >= start_date)
if end_date:
conditions.append(AgentTrace.started_at <= end_date)
if conditions:
query = query.where(and_(*conditions))
query = query.group_by(
AgentTrace.trace_id,
AgentTrace.execution_id,
AgentTrace.agent_id,
AgentTrace.user_id,
)
# 计算总数
count_query = select(func.count(func.distinct(AgentTrace.trace_id)))
if conditions:
count_query = count_query.where(and_(*conditions))
count_result = await db.execute(count_query)
total = count_result.scalar() or 0
# 分页
query = query.order_by(desc(func.min(AgentTrace.started_at)))
query = query.offset((page - 1) * page_size).limit(page_size)
result = await db.execute(query)
rows = result.all()
# 获取Agent名称
agent_ids = list(set(row.agent_id for row in rows if row.agent_id))
agents_map = {}
if agent_ids:
agents_result = await db.execute(
select(Agent).where(Agent.id.in_(agent_ids))
)
for agent in agents_result.scalars().all():
agents_map[str(agent.id)] = agent.name
traces = []
for row in rows:
duration_ms = None
if row.ended_at and row.started_at:
duration_ms = int((row.ended_at - row.started_at).total_seconds() * 1000)
traces.append({
"traceId": row.trace_id,
"executionId": str(row.execution_id),
"agentId": str(row.agent_id),
"agentName": agents_map.get(str(row.agent_id), "Unknown"),
"userId": str(row.user_id),
"startedAt": row.started_at.isoformat(),
"endedAt": row.ended_at.isoformat() if row.ended_at else None,
"durationMs": duration_ms,
"spanCount": row.span_count,
"totalTokens": int(row.total_tokens or 0),
"totalEu": float(row.total_eu or 0),
})
return {
"total": total,
"page": page,
"pageSize": page_size,
"totalPages": (total + page_size - 1) // page_size,
"traces": traces,
}
async def get_trace_stats(
user_id: Optional[str],
start_date: datetime,
end_date: datetime,
db: AsyncSession
) -> Dict:
"""
获取追踪统计
Args:
user_id: 用户ID
start_date: 开始日期
end_date: 结束日期
db: 数据库会话
Returns:
追踪统计
"""
conditions = [
AgentTrace.started_at >= start_date,
AgentTrace.started_at <= end_date,
]
if user_id:
conditions.append(AgentTrace.user_id == user_id)
# 总体统计
stats_result = await db.execute(
select(
func.count(func.distinct(AgentTrace.trace_id)).label("total_traces"),
func.count(AgentTrace.id).label("total_spans"),
func.sum(AgentTrace.duration_ms).label("total_duration"),
func.avg(AgentTrace.duration_ms).label("avg_duration"),
func.sum(AgentTrace.tokens_used).label("total_tokens"),
func.sum(AgentTrace.eu_consumed).label("total_eu"),
)
.where(and_(*conditions))
)
stats = stats_result.first()
# 按状态统计
status_stats = await db.execute(
select(
AgentTrace.status,
func.count(AgentTrace.id).label("count"),
)
.where(and_(*conditions))
.group_by(AgentTrace.status)
)
status_breakdown = {row.status: row.count for row in status_stats.all()}
# 按操作类型统计
type_stats = await db.execute(
select(
AgentTrace.operation_type,
func.count(AgentTrace.id).label("count"),
func.avg(AgentTrace.duration_ms).label("avg_duration"),
)
.where(and_(*conditions))
.group_by(AgentTrace.operation_type)
)
type_breakdown = [
{
"operationType": row.operation_type,
"count": row.count,
"avgDurationMs": float(row.avg_duration or 0),
}
for row in type_stats.all()
]
return {
"startDate": start_date.isoformat(),
"endDate": end_date.isoformat(),
"totalTraces": stats.total_traces or 0,
"totalSpans": stats.total_spans or 0,
"totalDurationMs": int(stats.total_duration or 0),
"avgDurationMs": float(stats.avg_duration or 0),
"totalTokens": int(stats.total_tokens or 0),
"totalEu": float(stats.total_eu or 0),
"byStatus": status_breakdown,
"byOperationType": type_breakdown,
}
def _sanitize_data(data: Dict) -> Dict:
"""
对敏感数据进行脱敏处理
Args:
data: 原始数据
Returns:
脱敏后的数据
"""
if not data:
return data
# 敏感字段列表
sensitive_fields = {
"password", "secret", "token", "api_key", "apikey",
"authorization", "auth", "credential", "key",
}
def _sanitize_value(key: str, value: Any) -> Any:
if isinstance(value, dict):
return {k: _sanitize_value(k, v) for k, v in value.items()}
elif isinstance(value, list):
return [_sanitize_value(key, v) for v in value]
elif key.lower() in sensitive_fields:
return "***REDACTED***"
elif isinstance(value, str) and len(value) > 1000:
return value[:1000] + "...[truncated]"
return value
return {k: _sanitize_value(k, v) for k, v in data.items()}
services/mcp-server/models.py
+222
View File
@@ -618,3 +618,225 @@ class ProviderModel(BaseModel, Base):
# 别名：Tenant 指向 User（租户即为user）
Tenant = User
class TokenBlacklist(BaseModel, Base):
"""Token黑名单（用于JWT登出和Token失效）"""
__tablename__ = "token_blacklist"
token_jti = Column(String(255), unique=True, nullable=False) # JWT ID
user_id = Column(GUID(), ForeignKey("users.id"), nullable=False)
expires_at = Column(DateTime, nullable=False) # Token过期时间
reason = Column(String(100)) # 黑名单原因：logout, revoked, security
# 关联关系
user = relationship("User")
__table_args__ = (
Index("idx_token_blacklist_jti", token_jti),
Index("idx_token_blacklist_expires", expires_at),
)
class ResourceUsage(BaseModel, Base):
"""资源使用记录（用于监控和统计）"""
__tablename__ = "resource_usage"
user_id = Column(GUID(), ForeignKey("users.id"), nullable=False)
agent_id = Column(GUID(), ForeignKey("agents.id"))
# 资源使用量
cpu_seconds = Column(sa.Float, default=0.0) # CPU使用秒数
memory_mb_seconds = Column(sa.Float, default=0.0) # 内存MB*秒
network_bytes = Column(sa.BigInteger, default=0) # 网络流量字节
storage_bytes = Column(sa.BigInteger, default=0) # 存储使用字节
api_calls = Column(Integer, default=0) # API调用次数
# 时间范围
period_start = Column(DateTime, nullable=False)
period_end = Column(DateTime, nullable=False)
granularity = Column(String(20), default="hourly") # hourly, daily, monthly
# 关联关系
user = relationship("User")
agent = relationship("Agent")
__table_args__ = (
Index("idx_resource_usage_user", user_id),
Index("idx_resource_usage_period", period_start, period_end),
Index("idx_resource_usage_granularity", granularity),
)
class QuotaAlert(BaseModel, Base):
"""配额预警记录"""
__tablename__ = "quota_alerts"
user_id = Column(GUID(), ForeignKey("users.id"), nullable=False)
channel_id = Column(GUID(), ForeignKey("channels.id"))
# 预警信息
alert_type = Column(String(50), nullable=False) # balance_low, quota_exceeded, rate_limit
threshold_percent = Column(Integer) # 触发阈值百分比
current_value = Column(sa.Numeric(12, 4)) # 当前值
threshold_value = Column(sa.Numeric(12, 4)) # 阈值
# 状态
status = Column(String(20), default="active") # active, acknowledged, resolved
acknowledged_at = Column(DateTime)
resolved_at = Column(DateTime)
# 通知
notification_sent = Column(Boolean, default=False)
notification_channel = Column(String(50)) # email, webhook, sms
# 关联关系
user = relationship("User")
channel = relationship("Channel")
__table_args__ = (
Index("idx_quota_alert_user", user_id),
Index("idx_quota_alert_status", status),
Index("idx_quota_alert_type", alert_type),
)
class ModelPricing(BaseModel, Base):
"""模型定价配置"""
__tablename__ = "model_pricing"
provider_id = Column(GUID(), ForeignKey("model_providers.id"), nullable=False)
model_name = Column(String(100), nullable=False)
# 定价（每1K tokens）
input_price_per_1k = Column(sa.Numeric(10, 6), nullable=False) # 输入价格
output_price_per_1k = Column(sa.Numeric(10, 6), nullable=False) # 输出价格
# EU转换
eu_per_1k_tokens = Column(sa.Numeric(8, 4), default=0.1) # 每1K tokens消耗的EU
# 限制
max_context_length = Column(Integer, default=4096)
max_output_tokens = Column(Integer, default=2048)
# 状态
is_active = Column(Boolean, default=True)
effective_from = Column(DateTime, default=datetime.utcnow)
effective_until = Column(DateTime)
# 关联关系
provider = relationship("ModelProvider")
__table_args__ = (
Index("idx_model_pricing_provider", provider_id),
Index("idx_model_pricing_model", model_name),
UniqueConstraint("provider_id", "model_name", name="uq_provider_model_pricing"),
)
class ProviderHealthCheck(BaseModel, Base):
"""供应商健康检查记录"""
__tablename__ = "provider_health_checks"
provider_id = Column(GUID(), ForeignKey("model_providers.id"), nullable=False)
# 检查结果
is_healthy = Column(Boolean, nullable=False)
response_time_ms = Column(Integer) # 响应时间毫秒
status_code = Column(Integer) # HTTP状态码
error_message = Column(Text)
# 检查类型
check_type = Column(String(50), default="ping") # ping, model_list, completion
# 关联关系
provider = relationship("ModelProvider")
__table_args__ = (
Index("idx_provider_health_provider", provider_id),
Index("idx_provider_health_time", created_at),
Index("idx_provider_health_status", is_healthy),
)
class AgentTrace(BaseModel, Base):
"""Agent执行轨迹（详细追踪记录）"""
__tablename__ = "agent_traces"
execution_id = Column(GUID(), ForeignKey("executions.id"), nullable=False)
agent_id = Column(GUID(), ForeignKey("agents.id"), nullable=False)
user_id = Column(GUID(), ForeignKey("users.id"), nullable=False)
# 轨迹信息
trace_id = Column(String(100), nullable=False) # 分布式追踪ID
span_id = Column(String(100), nullable=False) # 跨度ID
parent_span_id = Column(String(100)) # 父跨度ID
# 操作信息
operation_name = Column(String(200), nullable=False)
operation_type = Column(String(50)) # tool_call, llm_call, function_exec
# 时间信息
started_at = Column(DateTime, nullable=False)
ended_at = Column(DateTime)
duration_ms = Column(Integer)
# 输入输出
input_data = Column(JSON) # 输入数据（脱敏）
output_data = Column(JSON) # 输出数据（脱敏）
# 状态
status = Column(String(20), nullable=False) # running, success, error
error_type = Column(String(100))
error_message = Column(Text)
# 资源消耗
tokens_used = Column(Integer, default=0)
eu_consumed = Column(sa.Numeric(10, 4), default=0)
# 关联关系
execution = relationship("Execution")
agent = relationship("Agent")
user = relationship("User")
__table_args__ = (
Index("idx_agent_trace_execution", execution_id),
Index("idx_agent_trace_trace_id", trace_id),
Index("idx_agent_trace_user", user_id),
Index("idx_agent_trace_time", started_at),
)
class BillingEvent(BaseModel, Base):
"""计费事件（用于NATS消息队列）"""
__tablename__ = "billing_events"
# 事件信息
event_type = Column(String(50), nullable=False) # execution_start, execution_end, quota_check
event_id = Column(String(100), unique=True, nullable=False)
# 关联实体
user_id = Column(GUID(), ForeignKey("users.id"), nullable=False)
agent_id = Column(GUID(), ForeignKey("agents.id"))
execution_id = Column(GUID(), ForeignKey("executions.id"))
# 事件数据
payload = Column(JSON, nullable=False)
# 处理状态
status = Column(String(20), default="pending") # pending, processing, completed, failed
processed_at = Column(DateTime)
retry_count = Column(Integer, default=0)
error_message = Column(Text)
# 关联关系
user = relationship("User")
agent = relationship("Agent")
execution = relationship("Execution")
__table_args__ = (
Index("idx_billing_event_type", event_type),
Index("idx_billing_event_status", status),
Index("idx_billing_event_user", user_id),
Index("idx_billing_event_time", created_at),
)