forked from xiaohei/taiji-AI-PAD
生产备份
This commit is contained in:
zhanggangyong
@@ -1,279 +0,0 @@
|
||||
# Agent Manager 响应格式更新说明
|
||||
|
||||
> **更新日期**: 2026-03-04
|
||||
> **版本**: v2.0
|
||||
> **影响范围**: MCP-Server 所有平台 Agent 模板相关接口
|
||||
|
||||
---
|
||||
|
||||
## 📋 更新概述
|
||||
|
||||
Agent Manager 的 `/templates/platform` 和 `/templates/custom` 接口已更新，现在返回更丰富的模板信息：
|
||||
|
||||
### 新增字段
|
||||
|
||||
| 字段 | 类型 | 说明 | 示例 |
|
||||
|------|------|------|------|
|
||||
| `displayName` | string | 模板显示名称（人类可读） | "Echo 测试服务" |
|
||||
| `description` | string | 模板描述 | "简单的 Echo 服务，用于测试和调试" |
|
||||
| `category` | string | 模板分类 | "testing", "assistant", "database" |
|
||||
|
||||
### 原有字段
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `template` | string | 模板技术名称 |
|
||||
| `port` | integer | 服务端口 |
|
||||
| `env_info` | object | 环境变量配置 |
|
||||
|
||||
---
|
||||
|
||||
## 🔄 MCP-Server 更新内容
|
||||
|
||||
### 1. 数据模型更新
|
||||
|
||||
#### `agent_manager_client.py` - TemplateInfo 数据类
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class TemplateInfo:
|
||||
"""
|
||||
Template Information
|
||||
|
||||
Agent Manager now returns:
|
||||
- template: technical name (e.g., echo_agent)
|
||||
- displayName: human-readable name (e.g., "Echo 测试服务")
|
||||
- description: template description
|
||||
- category: template category (e.g., testing, assistant)
|
||||
- port: service port
|
||||
- env_info: environment variable configuration
|
||||
"""
|
||||
template: str
|
||||
display_name: Optional[str] = None
|
||||
description: Optional[str] = None
|
||||
category: Optional[str] = None
|
||||
port: Optional[int] = None
|
||||
env_info: Dict[str, Any] = field(default_factory=dict)
|
||||
template_type: Optional[str] = None # platform or custom
|
||||
```
|
||||
|
||||
#### `schemas.py` - TemplateInfo Pydantic 模型
|
||||
|
||||
```python
|
||||
class TemplateInfo(BaseModel):
|
||||
"""模板信息"""
|
||||
template: str
|
||||
displayName: Optional[str] = None
|
||||
description: Optional[str] = None
|
||||
category: Optional[str] = None
|
||||
port: Optional[int] = None
|
||||
env_info: Dict[str, Any] = {}
|
||||
```
|
||||
|
||||
### 2. 接口解析更新
|
||||
|
||||
所有调用 Agent Manager 的方法都已更新以解析新字段：
|
||||
|
||||
- ✅ `list_templates()`
|
||||
- ✅ `list_platform_templates()`
|
||||
- ✅ `list_custom_templates()`
|
||||
|
||||
### 3. 路由层优先级逻辑
|
||||
|
||||
更新了所有使用模板信息的路由，按以下优先级使用 displayName 和 description：
|
||||
|
||||
**优先级顺序**：
|
||||
1. **数据库管理员配置** (PlatformAgentTemplateConfig 表)
|
||||
2. **Agent Manager 返回值** (新增)
|
||||
3. **硬编码默认值** (TEMPLATE_DISPLAY_INFO，向后兼容)
|
||||
|
||||
#### 影响的路由文件
|
||||
|
||||
- ✅ `app/routes/admin.py` - 管理员接口
|
||||
- `_get_platform_templates_from_agent_manager()`
|
||||
- `/api/admin/platform-agents/templates`
|
||||
|
||||
- ✅ `app/routes/channel.py` - 渠道接口
|
||||
- `_get_platform_templates_from_agent_manager()`
|
||||
- `_get_custom_templates_from_agent_manager()`
|
||||
- `/api/channel/available-platform-agents`
|
||||
|
||||
- ✅ `app/routes/platform_agent_quota.py` - 平台 Agent 配额接口
|
||||
- `/api/channel/available-platform-agents` (旧版)
|
||||
|
||||
- ✅ `app/routes/agents.py` - Agent 模板接口
|
||||
- `GET /templates`
|
||||
- `GET /templates/platform`
|
||||
- `GET /templates/custom`
|
||||
|
||||
---
|
||||
|
||||
## 🎯 业务影响
|
||||
|
||||
### ✅ 优点
|
||||
|
||||
1. **无需硬编码** - 不再需要在 MCP-Server 中维护 `TEMPLATE_DISPLAY_INFO` 字典
|
||||
2. **动态更新** - Agent Manager 可以在运行时更新模板名称和描述
|
||||
3. **多语言支持** - Agent Manager 可以根据区域返回不同语言的 displayName 和 description
|
||||
4. **统一数据源** - 模板信息集中管理，避免数据不一致
|
||||
|
||||
### ⚠️ 向后兼容性
|
||||
|
||||
- **完全向后兼容** - 如果 Agent Manager 未返回 displayName/description/category，MCP-Server 会回退到使用硬编码的 `TEMPLATE_DISPLAY_INFO`
|
||||
- **渐进式升级** - Agent Manager 可以逐步为每个模板添加新字段，不影响已有功能
|
||||
|
||||
### 🔄 数据流向
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Agent Manager │
|
||||
│ │
|
||||
│ - template │
|
||||
│ - displayName │ ← 新增
|
||||
│ - description │ ← 新增
|
||||
│ - category │ ← 新增
|
||||
│ - port │
|
||||
│ - env_info │
|
||||
└────────┬────────┘
|
||||
│
|
||||
↓
|
||||
┌────────────────────────────┐
|
||||
│ MCP-Server │
|
||||
│ │
|
||||
│ 优先级: │
|
||||
│ 1. DB管理员配置 │
|
||||
│ 2. Agent Manager返回 ✨新 │
|
||||
│ 3. 硬编码默认值(兼容) │
|
||||
└────────┬───────────────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────┐
|
||||
│ 前端展示 │
|
||||
│ │
|
||||
│ - 管理端 │
|
||||
│ - 渠道端 │
|
||||
│ - 用户端 │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 示例对比
|
||||
|
||||
### Agent Manager 返回格式（旧）
|
||||
|
||||
```json
|
||||
{
|
||||
"templates": [
|
||||
{
|
||||
"template": "echo_agent",
|
||||
"port": 8000,
|
||||
"env_info": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Agent Manager 返回格式（新）
|
||||
|
||||
```json
|
||||
{
|
||||
"templates": [
|
||||
{
|
||||
"template": "echo_agent",
|
||||
"displayName": "Echo 测试服务",
|
||||
"description": "简单的 Echo 服务，用于测试和调试",
|
||||
"category": "testing",
|
||||
"port": 8000,
|
||||
"env_info": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### MCP-Server 响应格式
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"templates": [
|
||||
{
|
||||
"name": "echo_agent",
|
||||
"displayName": "Echo 测试服务",
|
||||
"description": "简单的 Echo 服务，用于测试和调试",
|
||||
"category": "testing",
|
||||
"version": "1.0.0",
|
||||
"port": 8000,
|
||||
"envInfo": {},
|
||||
"status": "available",
|
||||
"cpuRequest": "100m",
|
||||
"cpuLimit": "500m",
|
||||
"memoryRequest": "128Mi",
|
||||
"memoryLimit": "512Mi"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ 测试清单
|
||||
|
||||
### Agent Manager 端
|
||||
|
||||
- [ ] `/templates/platform` 返回 displayName、description、category
|
||||
- [ ] `/templates/custom` 返回 displayName、description、category
|
||||
- [ ] 所有现有模板都添加了这些字段
|
||||
- [ ] 新增模板自动包含这些字段
|
||||
|
||||
### MCP-Server 端
|
||||
|
||||
- [x] agent_manager_client.py 正确解析新字段
|
||||
- [x] schemas.py 包含新字段定义
|
||||
- [x] admin.py 优先使用 Agent Manager 返回的字段
|
||||
- [x] channel.py 优先使用 Agent Manager 返回的字段
|
||||
- [x] platform_agent_quota.py 使用新字段
|
||||
- [x] agents.py 返回新字段
|
||||
- [ ] 向后兼容测试（Agent Manager 未返回新字段时）
|
||||
- [ ] 前端接口测试（各端正确显示）
|
||||
|
||||
### 前端端
|
||||
|
||||
- [ ] 管理端正确显示 displayName 和 description
|
||||
- [ ] 渠道端正确显示 displayName 和 description
|
||||
- [ ] 用户端正确显示 displayName 和 description
|
||||
|
||||
---
|
||||
|
||||
## 🚀 部署建议
|
||||
|
||||
1. **先更新 Agent Manager** - 确保返回新字段
|
||||
2. **部署 MCP-Server** - 支持接收新字段（已完成）
|
||||
3. **验证接口** - 检查各端接口返回是否正确
|
||||
4. **前端适配** - 确保前端正确使用新字段
|
||||
|
||||
---
|
||||
|
||||
## 📝 注意事项
|
||||
|
||||
1. **硬编码的 TEMPLATE_DISPLAY_INFO 仍然保留** - 用于向后兼容和降级逻辑
|
||||
2. **数据库管理员配置优先级最高** - 管理员可以覆盖 Agent Manager 的默认值
|
||||
3. **category 分类建议** - testing, assistant, development, search, database, general
|
||||
|
||||
---
|
||||
|
||||
## 🔗 相关文件
|
||||
|
||||
- `services/mcp-server/app/agent_manager_client.py` - Agent Manager 客户端
|
||||
- `services/mcp-server/schemas.py` - API 响应模型
|
||||
- `services/mcp-server/app/routes/admin.py` - 管理员路由
|
||||
- `services/mcp-server/app/routes/channel.py` - 渠道路由
|
||||
- `services/mcp-server/app/routes/platform_agent_quota.py` - 平台 Agent 配额路由
|
||||
- `services/mcp-server/app/routes/agents.py` - Agent 模板路由
|
||||
- `services/mcp-server/models.py` - 数据库模型（PlatformAgentTemplateConfig）
|
||||
|
||||
---
|
||||
|
||||
**结论**: MCP-Server 已完全适配 Agent Manager 的新响应格式，并保持向后兼容性。✅
|
||||
@@ -1,115 +0,0 @@
|
||||
# Agent Manager 响应格式更新 - 快速参考
|
||||
|
||||
## ✅ 已完成的更新
|
||||
|
||||
### 1. 数据模型
|
||||
- ✅ `agent_manager_client.py::TemplateInfo` - 添加 `display_name`, `description`, `category` 字段
|
||||
- ✅ `schemas.py::TemplateInfo` - 添加 `displayName`, `description`, `category` 字段
|
||||
|
||||
### 2. 接口解析
|
||||
- ✅ `list_templates()` - 解析新字段
|
||||
- ✅ `list_platform_templates()` - 解析新字段
|
||||
- ✅ `list_custom_templates()` - 解析新字段
|
||||
|
||||
### 3. 路由层优先级逻辑
|
||||
|
||||
```
|
||||
优先级: DB配置 > Agent Manager返回 > 硬编码默认值
|
||||
```
|
||||
|
||||
- ✅ `admin.py::_get_platform_templates_from_agent_manager()`
|
||||
- ✅ `channel.py::_get_platform_templates_from_agent_manager()`
|
||||
- ✅ `channel.py::_get_custom_templates_from_agent_manager()`
|
||||
- ✅ `platform_agent_quota.py::get_available_platform_agents()`
|
||||
- ✅ `agents.py::list_templates()`
|
||||
- ✅ `agents.py::list_platform_templates()`
|
||||
- ✅ `agents.py::list_custom_templates()`
|
||||
|
||||
### 4. 向后兼容
|
||||
- ✅ 保留 `TEMPLATE_DISPLAY_INFO` 硬编码默认值
|
||||
- ✅ 降级逻辑：当 Agent Manager 未返回时使用默认值
|
||||
|
||||
---
|
||||
|
||||
## 📊 Agent Manager 预期返回格式
|
||||
|
||||
```json
|
||||
{
|
||||
"templates": [
|
||||
{
|
||||
"template": "echo_agent", // 必需
|
||||
"displayName": "Echo 测试服务", // 新增 ✨
|
||||
"description": "简单的测试服务", // 新增 ✨
|
||||
"category": "testing", // 新增 ✨
|
||||
"port": 8000, // 可选
|
||||
"env_info": {} // 可选
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 各端展示逻辑
|
||||
|
||||
### 管理端 (Admin)
|
||||
优先级: **DB配置 > Agent Manager > 硬编码**
|
||||
|
||||
### 渠道端 (Channel)
|
||||
优先级: **DB配置 > Agent Manager > 硬编码**
|
||||
|
||||
### 用户端 (User)
|
||||
优先级: **Agent Manager > 硬编码**
|
||||
|
||||
---
|
||||
|
||||
## ✅ 验证清单
|
||||
|
||||
### Agent Manager 端 (需要确认)
|
||||
- [ ] `GET /templates/platform` 返回 `displayName`
|
||||
- [ ] `GET /templates/platform` 返回 `description`
|
||||
- [ ] `GET /templates/platform` 返回 `category`
|
||||
- [ ] `GET /templates/custom` 返回新字段
|
||||
|
||||
### MCP-Server 端 (已完成)
|
||||
- [x] 数据模型包含新字段
|
||||
- [x] 接口解析新字段
|
||||
- [x] 路由优先使用 Agent Manager 返回值
|
||||
- [x] 保持向后兼容性
|
||||
- [x] 代码编译无错误
|
||||
|
||||
### 前端端 (需要测试)
|
||||
- [ ] 管理端正确显示 displayName
|
||||
- [ ] 渠道端正确显示 displayName
|
||||
- [ ] 用户端正确显示 displayName
|
||||
|
||||
---
|
||||
|
||||
## 🚀 测试命令
|
||||
|
||||
```bash
|
||||
# 1. 测试 Agent Manager 接口
|
||||
curl http://agent-manager:8000/templates/platform
|
||||
|
||||
# 2. 测试 MCP-Server 管理员接口
|
||||
curl http://localhost:8002/api/admin/platform-agents/templates \
|
||||
-H "Authorization: Bearer YOUR_TOKEN"
|
||||
|
||||
# 3. 测试 MCP-Server 渠道接口
|
||||
curl http://localhost:8002/api/channel/available-platform-agents \
|
||||
-H "Authorization: Bearer YOUR_TOKEN"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 重要说明
|
||||
|
||||
1. **Agent Manager 必须先更新** - 返回新字段后 MCP-Server 才会使用
|
||||
2. **降级逻辑自动生效** - 如果 Agent Manager 未返回，自动使用默认值
|
||||
3. **数据库配置优先** - 管理员可以在数据库中覆盖任何显示名称
|
||||
|
||||
---
|
||||
|
||||
## 📄 相关文档
|
||||
- [完整更新说明](./AGENT_MANAGER_RESPONSE_UPDATE.md)
|
||||
- [Agent Manager 接口规范](./Docs/Agent-Manager外部工具接口规范.md)
|
||||
@@ -124,7 +124,7 @@ http GET http://localhost:8002/api/user/dashboard/billing-overview \
|
||||
},
|
||||
"balance": {
|
||||
"eu": 50000.0,
|
||||
"cash": 5000.00
|
||||
"cash": 50000.0
|
||||
},
|
||||
"euHistory": [
|
||||
{
|
||||
|
||||
@@ -40,7 +40,7 @@
|
||||
},
|
||||
"balance": {
|
||||
"eu": 50000.0,
|
||||
"cash": 5000.00
|
||||
"cash": 50000.0
|
||||
},
|
||||
"euHistory": [
|
||||
{
|
||||
@@ -163,8 +163,8 @@
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| eu | float | 当前EU余额 |
|
||||
| cash | float | 当前现金余额（CNY） |
|
||||
| eu | float | 当前EU余额（1 EU = 1 美元） |
|
||||
| cash | float | 当前现金余额（美元），与 eu 相等（1 美元 = 1 EU） |
|
||||
|
||||
### euHistory (EU消费历史)
|
||||
|
||||
|
||||
@@ -0,0 +1,662 @@
|
||||
# 生产环境数据库同步指南
|
||||
|
||||
生成时间: 2026-03-12
|
||||
测试环境数据库: `taiji`
|
||||
生产环境数据库: `taiji_prod`
|
||||
|
||||
## 📋 迁移总览
|
||||
|
||||
测试环境在 2026年3月9-12日期间执行了 **8个数据库迁移**，需要同步到生产环境：
|
||||
|
||||
| 迁移编号 | 文件名 | 类型 | 影响范围 | 风险等级 |
|
||||
|---------|--------|------|----------|---------|
|
||||
| 017 | add_external_data_tools.sql | 新增表 | 外部数据工具功能 | 🟢 低 |
|
||||
| 018 | add_external_toolkits.sql | 新增表 | 外部工具集功能 | 🟢 低 |
|
||||
| 019 | add_agent_model_name.sql | 字段新增 | `agents`、`agent_billing_records` | 🟢 低 |
|
||||
| 020 | fix_quota_defaults.sql | 字段修改 | 配额表（3个表） | 🟡 中 |
|
||||
| 021 | fix_agent_type_length.sql | 字段修改 | `agent_billing_records` | 🟢 低 |
|
||||
| 022 | fix_eu_equals_cost.sql | 字段类型+数据修正 | `agent_billing_records`、`model_billing_records` | 🟡 中 |
|
||||
| 023 | fix_billing_channel_id.sql | 数据修正 | `agent_billing_records`、`model_billing_records` | 🟢 低 |
|
||||
| 024 | add_record_type.sql | 字段新增+数据修正 | `agent_billing_records` | 🟡 中 |
|
||||
|
||||
---
|
||||
|
||||
## 📝 详细迁移内容
|
||||
|
||||
### 迁移 017: 添加外部数据工具表
|
||||
|
||||
**文件**: `services/mcp-server/migrations/017_add_external_data_tools.sql`
|
||||
|
||||
**目的**: 支持用户创建和管理自定义的外部API数据工具
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
-- 创建表
|
||||
CREATE TABLE external_data_tools (
|
||||
id UUID PRIMARY KEY,
|
||||
name VARCHAR(100) NOT NULL,
|
||||
description TEXT,
|
||||
url VARCHAR(500) NOT NULL,
|
||||
method VARCHAR(10) DEFAULT 'POST',
|
||||
auth_type VARCHAR(20) DEFAULT 'none',
|
||||
tool_ref_id VARCHAR(100) UNIQUE,
|
||||
status VARCHAR(20) DEFAULT 'pending',
|
||||
error_message TEXT,
|
||||
owner_id UUID NOT NULL REFERENCES users(id),
|
||||
tenant_id UUID REFERENCES users(id),
|
||||
channel_id UUID REFERENCES channels(id),
|
||||
is_active BOOLEAN DEFAULT TRUE,
|
||||
usage_count INTEGER DEFAULT 0,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 索引
|
||||
CREATE INDEX idx_external_data_tool_owner ON external_data_tools(owner_id);
|
||||
CREATE INDEX idx_external_data_tool_ref ON external_data_tools(tool_ref_id);
|
||||
CREATE INDEX idx_external_data_tool_status ON external_data_tools(status);
|
||||
```
|
||||
|
||||
**影响**: 新功能，对现有数据无影响
|
||||
|
||||
**回滚方案**:
|
||||
```sql
|
||||
DROP TABLE IF EXISTS external_data_tools CASCADE;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 迁移 018: 添加外部数据工具集表
|
||||
|
||||
**文件**: `services/mcp-server/migrations/018_add_external_toolkits.sql`
|
||||
|
||||
**目的**: 允许用户将多个外部工具组合成工具集，方便部署自定义Agent
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
-- 创建表
|
||||
CREATE TABLE external_toolkits (
|
||||
id UUID PRIMARY KEY,
|
||||
name VARCHAR(100) NOT NULL,
|
||||
description TEXT,
|
||||
tool_ids JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
owner_id UUID NOT NULL REFERENCES users(id),
|
||||
tenant_id UUID REFERENCES users(id),
|
||||
channel_id UUID REFERENCES channels(id),
|
||||
is_active BOOLEAN DEFAULT TRUE,
|
||||
usage_count INTEGER DEFAULT 0,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
CONSTRAINT uq_toolkit_name_owner UNIQUE (name, owner_id)
|
||||
);
|
||||
|
||||
-- 索引
|
||||
CREATE INDEX idx_external_toolkit_owner ON external_toolkits(owner_id);
|
||||
```
|
||||
|
||||
**影响**: 新功能，对现有数据无影响
|
||||
|
||||
**回滚方案**:
|
||||
```sql
|
||||
DROP TABLE IF EXISTS external_toolkits CASCADE;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 迁移 019: 为Agent添加模型名称字段 ⭐ 重要
|
||||
|
||||
**文件**: `services/mcp-server/migrations/019_add_agent_model_name.sql`
|
||||
|
||||
**目的**:
|
||||
- 记录每个Agent使用的LLM模型（如 `azure/gpt-4`, `gemini/gemini-pro`）
|
||||
- 支持计费记录中追踪模型使用情况
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
-- 1. agents 表添加 model_name 字段
|
||||
ALTER TABLE agents ADD COLUMN IF NOT EXISTS model_name VARCHAR(100);
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_model_name ON agents(model_name);
|
||||
|
||||
-- 2. agent_billing_records 表添加 model_name 字段
|
||||
ALTER TABLE agent_billing_records ADD COLUMN IF NOT EXISTS model_name VARCHAR(100);
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_model_name ON agent_billing_records(model_name);
|
||||
```
|
||||
|
||||
**影响**:
|
||||
- 现有Agent的 `model_name` 为 NULL（可接受）
|
||||
- 新创建的Agent将记录模型信息
|
||||
|
||||
**注意事项**:
|
||||
- ⚠️ 如果生产环境已有该字段，使用 `IF NOT EXISTS` 将安全跳过
|
||||
- 该字段为 **可空**，不强制现有记录填充
|
||||
|
||||
**回滚方案**:
|
||||
```sql
|
||||
ALTER TABLE agents DROP COLUMN IF EXISTS model_name;
|
||||
ALTER TABLE agent_billing_records DROP COLUMN IF EXISTS model_name;
|
||||
DROP INDEX IF EXISTS idx_agent_model_name;
|
||||
DROP INDEX IF EXISTS idx_agent_billing_model_name;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 迁移 020: 修复配额字段默认值 ⚠️ 需要谨慎
|
||||
|
||||
**文件**: `services/mcp-server/migrations/020_fix_quota_defaults.sql`
|
||||
|
||||
**目的**:
|
||||
- 消除配额字段的 NULL 值，避免计算错误
|
||||
- 添加 NOT NULL 约束和检查约束
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
-- 1. tenant_custom_agent_quotas 表
|
||||
ALTER TABLE tenant_custom_agent_quotas
|
||||
ALTER COLUMN cpu_quota SET DEFAULT 0, ALTER COLUMN cpu_quota SET NOT NULL,
|
||||
ALTER COLUMN memory_quota SET DEFAULT 0, ALTER COLUMN memory_quota SET NOT NULL,
|
||||
ALTER COLUMN cpu_used SET DEFAULT 0, ALTER COLUMN cpu_used SET NOT NULL,
|
||||
ALTER COLUMN memory_used SET DEFAULT 0, ALTER COLUMN memory_used SET NOT NULL,
|
||||
ALTER COLUMN agent_count SET DEFAULT 0, ALTER COLUMN agent_count SET NOT NULL;
|
||||
|
||||
UPDATE tenant_custom_agent_quotas SET cpu_quota = 0 WHERE cpu_quota IS NULL;
|
||||
UPDATE tenant_custom_agent_quotas SET memory_quota = 0 WHERE memory_quota IS NULL;
|
||||
UPDATE tenant_custom_agent_quotas SET cpu_used = 0 WHERE cpu_used IS NULL;
|
||||
UPDATE tenant_custom_agent_quotas SET memory_used = 0 WHERE memory_used IS NULL;
|
||||
UPDATE tenant_custom_agent_quotas SET agent_count = 0 WHERE agent_count IS NULL;
|
||||
|
||||
-- 添加检查约束
|
||||
ALTER TABLE tenant_custom_agent_quotas
|
||||
ADD CONSTRAINT chk_tenant_cpu_used CHECK (cpu_used <= cpu_quota),
|
||||
ADD CONSTRAINT chk_tenant_memory_used CHECK (memory_used <= memory_quota);
|
||||
|
||||
-- 2. channel_custom_agent_quotas 表（类似操作）
|
||||
-- 3. platform_agent_quotas 表（类似操作）
|
||||
```
|
||||
|
||||
**影响**:
|
||||
- ⚠️ **数据修改**: 将所有 NULL 值更新为 0
|
||||
- ⚠️ **约束添加**: 新增检查约束，使用量不能超过配额
|
||||
|
||||
**⚠️ 执行前检查**:
|
||||
```sql
|
||||
-- 检查生产环境是否有 NULL 值
|
||||
SELECT
|
||||
COUNT(*) as total_records,
|
||||
COUNT(CASE WHEN cpu_quota IS NULL THEN 1 END) as null_cpu_quota,
|
||||
COUNT(CASE WHEN memory_quota IS NULL THEN 1 END) as null_memory_quota
|
||||
FROM tenant_custom_agent_quotas;
|
||||
|
||||
-- 检查是否有使用量超过配额的记录（会导致约束添加失败）
|
||||
SELECT * FROM tenant_custom_agent_quotas WHERE cpu_used > cpu_quota;
|
||||
SELECT * FROM tenant_custom_agent_quotas WHERE memory_used > memory_quota;
|
||||
```
|
||||
|
||||
**回滚方案**:
|
||||
```sql
|
||||
-- 删除约束（如果需要）
|
||||
ALTER TABLE tenant_custom_agent_quotas DROP CONSTRAINT IF EXISTS chk_tenant_cpu_used;
|
||||
ALTER TABLE tenant_custom_agent_quotas DROP CONSTRAINT IF EXISTS chk_tenant_memory_used;
|
||||
-- 注意：无法回滚 NOT NULL 约束和数据修改
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 迁移 021: 修复agent_type字段长度
|
||||
|
||||
**文件**: `services/mcp-server/migrations/021_fix_agent_type_length.sql`
|
||||
|
||||
**目的**: 修复 `microsoft_learn_agent`（22字符）超出 VARCHAR(20) 限制的问题
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
ALTER TABLE agent_billing_records ALTER COLUMN agent_type TYPE VARCHAR(100);
|
||||
```
|
||||
|
||||
**影响**: 扩展字段长度，对现有数据无负面影响
|
||||
|
||||
**测试建议**:
|
||||
```sql
|
||||
-- 检查是否有被截断的数据
|
||||
SELECT agent_type, LENGTH(agent_type) as len, COUNT(*)
|
||||
FROM agent_billing_records
|
||||
GROUP BY agent_type
|
||||
ORDER BY len DESC;
|
||||
```
|
||||
|
||||
**回滚方案**:
|
||||
```sql
|
||||
-- 仅在确认所有值都 ≤20 字符时才能回滚
|
||||
ALTER TABLE agent_billing_records ALTER COLUMN agent_type TYPE VARCHAR(20);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 迁移 022: 修复EU计算逻辑 ⚠️ 重要数据修正
|
||||
|
||||
**文件**: `services/mcp-server/migrations/022_fix_eu_equals_cost.sql`
|
||||
|
||||
**目的**:
|
||||
- **旧逻辑**: `EU = ceil(duration_seconds / 10)`，即 1 EU = 10秒
|
||||
- **新逻辑**: `EU = Cost（美元）`，即 1 EU = 1 美元
|
||||
- 将历史数据的 `eu_consumed` 修正为 `cost` 的值
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
-- 1. 修改字段类型以支持小数
|
||||
ALTER TABLE agent_billing_records
|
||||
ALTER COLUMN eu_consumed TYPE NUMERIC(12, 4);
|
||||
|
||||
-- 2. 更新历史数据
|
||||
UPDATE agent_billing_records
|
||||
SET eu_consumed = cost
|
||||
WHERE cost IS NOT NULL AND cost > 0;
|
||||
|
||||
UPDATE model_billing_records
|
||||
SET eu_consumed = total_cost
|
||||
WHERE total_cost IS NOT NULL AND total_cost > 0;
|
||||
|
||||
-- 3. 添加注释说明
|
||||
COMMENT ON COLUMN agent_billing_records.eu_consumed IS 'EU 消耗（1 EU = 1 美元，EU = Cost）';
|
||||
```
|
||||
|
||||
**影响**:
|
||||
- ⚠️ **修改所有历史计费记录**的 `eu_consumed` 值
|
||||
- 会导致历史数据的EU统计发生变化
|
||||
|
||||
**⚠️ 执行前备份**:
|
||||
```sql
|
||||
-- 备份历史数据
|
||||
CREATE TABLE agent_billing_records_backup_20260312 AS
|
||||
SELECT * FROM agent_billing_records;
|
||||
|
||||
CREATE TABLE model_billing_records_backup_20260312 AS
|
||||
SELECT * FROM model_billing_records;
|
||||
```
|
||||
|
||||
**验证**:
|
||||
```sql
|
||||
-- 检查 EU 和 Cost 是否一致
|
||||
SELECT id, agent_name, eu_consumed, cost,
|
||||
CASE WHEN ABS(eu_consumed - cost) < 0.0001 THEN 'OK' ELSE 'MISMATCH' END as status
|
||||
FROM agent_billing_records
|
||||
WHERE cost > 0
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
**回滚方案**:
|
||||
```sql
|
||||
-- 从备份表恢复
|
||||
UPDATE agent_billing_records abr
|
||||
SET eu_consumed = backup.eu_consumed
|
||||
FROM agent_billing_records_backup_20260312 backup
|
||||
WHERE abr.id = backup.id;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 迁移 023: 修复计费记录中缺失的channel_id
|
||||
|
||||
**文件**: `services/mcp-server/migrations/023_fix_billing_channel_id.sql`
|
||||
|
||||
**目的**: 修复计费记录创建时未正确设置 `channel_id` 的问题
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
-- 1. 更新 AgentBillingRecord
|
||||
UPDATE agent_billing_records abr
|
||||
SET channel_id = u.channel_id
|
||||
FROM users u
|
||||
WHERE abr.user_id = u.id
|
||||
AND abr.channel_id IS NULL
|
||||
AND u.channel_id IS NOT NULL;
|
||||
|
||||
-- 2. 更新 ModelBillingRecord
|
||||
UPDATE model_billing_records mbr
|
||||
SET channel_id = u.channel_id
|
||||
FROM users u
|
||||
WHERE mbr.tenant_id = u.id
|
||||
AND mbr.channel_id IS NULL
|
||||
AND u.channel_id IS NOT NULL;
|
||||
```
|
||||
|
||||
**影响**: 补充缺失的 `channel_id`，提高数据完整性
|
||||
|
||||
**执行前检查**:
|
||||
```sql
|
||||
-- 检查有多少记录缺失 channel_id
|
||||
SELECT
|
||||
'AgentBillingRecord' as table_name,
|
||||
COUNT(*) as total,
|
||||
COUNT(channel_id) as with_channel,
|
||||
COUNT(*) - COUNT(channel_id) as without_channel
|
||||
FROM agent_billing_records
|
||||
UNION ALL
|
||||
SELECT
|
||||
'ModelBillingRecord' as table_name,
|
||||
COUNT(*) as total,
|
||||
COUNT(channel_id) as with_channel,
|
||||
COUNT(*) - COUNT(channel_id) as without_channel
|
||||
FROM model_billing_records;
|
||||
```
|
||||
|
||||
**回滚方案**: 无需回滚（数据修正）
|
||||
|
||||
---
|
||||
|
||||
### 迁移 024: 添加record_type字段 ⚠️ 重要业务逻辑变更
|
||||
|
||||
**文件**: `services/mcp-server/migrations/024_add_record_type.sql`
|
||||
|
||||
**目的**: 区分两种计费方式
|
||||
- `vm_runtime`: VM运行时间计费（一个Agent = 一条记录，周期性更新）
|
||||
- `api_call`: API调用计费（每次调用 = 一条新记录）
|
||||
|
||||
**操作**:
|
||||
```sql
|
||||
-- 1. 添加字段
|
||||
ALTER TABLE agent_billing_records
|
||||
ADD COLUMN IF NOT EXISTS record_type VARCHAR(20) NOT NULL DEFAULT 'vm_runtime';
|
||||
|
||||
-- 2. 推断现有记录类型
|
||||
UPDATE agent_billing_records
|
||||
SET record_type = 'api_call'
|
||||
WHERE request_id IS NOT NULL
|
||||
AND request_id != ''
|
||||
AND end_time IS NOT NULL;
|
||||
|
||||
-- 3. 添加索引
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_record_type
|
||||
ON agent_billing_records(record_type);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_vm_runtime
|
||||
ON agent_billing_records(record_type, end_time)
|
||||
WHERE record_type = 'vm_runtime' AND end_time IS NULL;
|
||||
```
|
||||
|
||||
**影响**:
|
||||
- 新增业务逻辑字段
|
||||
- 周期计费任务将只更新 `record_type='vm_runtime'` 的记录
|
||||
|
||||
**验证**:
|
||||
```sql
|
||||
SELECT
|
||||
record_type,
|
||||
COUNT(*) as count,
|
||||
COUNT(CASE WHEN end_time IS NULL THEN 1 END) as running_count,
|
||||
COUNT(CASE WHEN end_time IS NOT NULL THEN 1 END) as completed_count
|
||||
FROM agent_billing_records
|
||||
GROUP BY record_type;
|
||||
```
|
||||
|
||||
**回滚方案**:
|
||||
```sql
|
||||
ALTER TABLE agent_billing_records DROP COLUMN IF EXISTS record_type;
|
||||
DROP INDEX IF EXISTS idx_agent_billing_record_type;
|
||||
DROP INDEX IF EXISTS idx_agent_billing_vm_runtime;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 执行顺序和依赖关系
|
||||
|
||||
迁移必须按以下顺序执行（有依赖关系）：
|
||||
|
||||
```
|
||||
017 ─────┐
|
||||
├──→ 可并行执行
|
||||
018 ─────┘
|
||||
|
||||
019 ─→ 020 ─→ 021 ─→ 022 ─→ 023 ─→ 024
|
||||
↑ ↑
|
||||
│ │
|
||||
修改配额约束 修改计费逻辑（最关键）
|
||||
```
|
||||
|
||||
**推荐分批执行**:
|
||||
- **第一批（新功能）**: 017, 018, 019
|
||||
- **第二批（数据修正）**: 020, 021, 023
|
||||
- **第三批（核心逻辑）**: 022, 024
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ 执行步骤
|
||||
|
||||
### 步骤 1: 备份生产数据库 🔴 必做
|
||||
|
||||
```bash
|
||||
# 使用项目提供的备份脚本
|
||||
cd /home/taiji/tools/taiji-AI-PAD
|
||||
bash scripts/backup_postgres.sh
|
||||
|
||||
# 或手动备份
|
||||
kubectl exec -n taiji-ai-pad <postgres-pod> -- \
|
||||
pg_dump -U postgres taiji_prod > backup_taiji_prod_$(date +%Y%m%d_%H%M%S).dump
|
||||
```
|
||||
|
||||
### 步骤 2: 检查生产环境状态
|
||||
|
||||
```bash
|
||||
# 检查数据库连接
|
||||
python services/mcp-server/check_prod_database_status.py
|
||||
|
||||
# 检查计费健康状态
|
||||
python services/mcp-server/check_billing_health.py
|
||||
|
||||
# 检查配额数据
|
||||
python services/mcp-server/check_quota_data.py
|
||||
```
|
||||
|
||||
### 步骤 3: 使用自动同步脚本（推荐）
|
||||
|
||||
```bash
|
||||
# 使用交互式同步脚本
|
||||
bash scripts/sync_prod_database.sh
|
||||
```
|
||||
|
||||
该脚本会：
|
||||
- ✅ 自动检查K8s集群连接
|
||||
- ✅ 显示当前数据库配置
|
||||
- ✅ 逐个执行迁移，每步都需要确认
|
||||
- ✅ 记录执行日志
|
||||
|
||||
### 步骤 4: 手动执行（如果需要更细粒度控制）
|
||||
|
||||
```bash
|
||||
# 进入mcp-server Pod
|
||||
POD_NAME=$(kubectl get pods -n taiji-ai-pad -l app=mcp-server -o jsonpath='{.items[0].metadata.name}')
|
||||
kubectl exec -it -n taiji-ai-pad $POD_NAME -- bash
|
||||
|
||||
# 切换到migrations目录
|
||||
cd /app/migrations
|
||||
|
||||
# 执行迁移（按顺序）
|
||||
python run_017_migration.py
|
||||
python run_018_migration.py
|
||||
python run_019_migration.py
|
||||
python run_020_fix_quota_defaults.py
|
||||
# ... 依次执行
|
||||
```
|
||||
|
||||
### 步骤 5: 验证结果
|
||||
|
||||
```bash
|
||||
# 检查表结构
|
||||
kubectl exec -n taiji-ai-pad <postgres-pod> -- \
|
||||
psql -U postgres -d taiji_prod -c "\d agent_billing_records"
|
||||
|
||||
# 检查数据完整性
|
||||
kubectl exec -n taiji-ai-pad <postgres-pod> -- \
|
||||
psql -U postgres -d taiji_prod -c "
|
||||
SELECT
|
||||
COUNT(*) as total_records,
|
||||
COUNT(record_type) as with_record_type,
|
||||
COUNT(model_name) as with_model_name,
|
||||
COUNT(channel_id) as with_channel_id
|
||||
FROM agent_billing_records;
|
||||
"
|
||||
|
||||
# 检查EU和Cost的一致性
|
||||
kubectl exec -n taiji-ai-pad <postgres-pod> -- \
|
||||
psql -U postgres -d taiji_prod -c "
|
||||
SELECT
|
||||
COUNT(*) as total,
|
||||
COUNT(CASE WHEN ABS(eu_consumed - cost) < 0.0001 THEN 1 END) as consistent,
|
||||
COUNT(CASE WHEN ABS(eu_consumed - cost) >= 0.0001 THEN 1 END) as inconsistent
|
||||
FROM agent_billing_records
|
||||
WHERE cost > 0;
|
||||
"
|
||||
```
|
||||
|
||||
### 步骤 6: 重启服务（如果需要）
|
||||
|
||||
```bash
|
||||
# 重启mcp-server以应用新配置
|
||||
kubectl rollout restart deployment/mcp-server -n taiji-ai-pad
|
||||
|
||||
# 等待Pod就绪
|
||||
kubectl rollout status deployment/mcp-server -n taiji-ai-pad
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 风险和注意事项
|
||||
|
||||
### 🔴 高风险操作
|
||||
|
||||
1. **迁移 022（EU计算逻辑修改）**
|
||||
- 会修改所有历史计费记录
|
||||
- 建议在业务低峰期执行
|
||||
- 必须先备份
|
||||
|
||||
2. **迁移 020（配额约束）**
|
||||
- 会添加检查约束
|
||||
- 如果有数据不一致（使用量>配额），迁移会失败
|
||||
- 需要先修复数据
|
||||
|
||||
### 🟡 中风险操作
|
||||
|
||||
1. **迁移 024（record_type字段）**
|
||||
- 修改了计费逻辑
|
||||
- 需要确保周期计费任务已更新代码
|
||||
|
||||
### 🟢 低风险操作
|
||||
|
||||
- 迁移 017, 018, 019, 021, 023
|
||||
- 这些主要是新增字段/表，对现有业务无影响
|
||||
|
||||
---
|
||||
|
||||
## 🔄 回滚计划
|
||||
|
||||
如果迁移后发现问题，按以下步骤回滚：
|
||||
|
||||
### 快速回滚（恢复备份）
|
||||
|
||||
```bash
|
||||
# 停止服务
|
||||
kubectl scale deployment/mcp-server -n taiji-ai-pad --replicas=0
|
||||
|
||||
# 恢复数据库
|
||||
kubectl exec -n taiji-ai-pad <postgres-pod> -- \
|
||||
psql -U postgres -c "DROP DATABASE taiji_prod;"
|
||||
kubectl exec -n taiji-ai-pad <postgres-pod> -- \
|
||||
psql -U postgres -c "CREATE DATABASE taiji_prod;"
|
||||
kubectl exec -i -n taiji-ai-pad <postgres-pod> -- \
|
||||
psql -U postgres taiji_prod < backup_taiji_prod_20260312_HHMMSS.dump
|
||||
|
||||
# 重启服务
|
||||
kubectl scale deployment/mcp-server -n taiji-ai-pad --replicas=1
|
||||
```
|
||||
|
||||
### 部分回滚（单个迁移）
|
||||
|
||||
参考每个迁移的"回滚方案"章节，执行对应的 SQL 语句。
|
||||
|
||||
---
|
||||
|
||||
## 📊 预期影响评估
|
||||
|
||||
### 数据量影响
|
||||
|
||||
```sql
|
||||
-- 评估受影响的记录数
|
||||
SELECT
|
||||
'agent_billing_records' as table_name,
|
||||
COUNT(*) as total_records,
|
||||
pg_size_pretty(pg_total_relation_size('agent_billing_records')) as table_size
|
||||
FROM agent_billing_records
|
||||
UNION ALL
|
||||
SELECT
|
||||
'model_billing_records' as table_name,
|
||||
COUNT(*) as total_records,
|
||||
pg_size_pretty(pg_total_relation_size('model_billing_records')) as table_size
|
||||
FROM model_billing_records;
|
||||
```
|
||||
|
||||
### 停机时间估算
|
||||
|
||||
- **新表创建（017, 018）**: < 1秒
|
||||
- **字段添加（019, 021, 024）**: 1-5秒（取决于表大小）
|
||||
- **数据修正（020, 022, 023）**: 1-10分钟（取决于记录数）
|
||||
|
||||
**建议总停机时间**: 15-30分钟（保守估计）
|
||||
|
||||
---
|
||||
|
||||
## ✅ 完成检查清单
|
||||
|
||||
执行完成后，请确认：
|
||||
|
||||
- [ ] 所有迁移脚本执行成功，无错误
|
||||
- [ ] 新表已创建：`external_data_tools`, `external_toolkits`
|
||||
- [ ] 新字段已添加：`model_name`, `record_type`
|
||||
- [ ] `eu_consumed` 和 `cost` 数据一致
|
||||
- [ ] `channel_id` 缺失值已修复
|
||||
- [ ] 配额约束已添加且无冲突
|
||||
- [ ] 服务正常启动，无报错
|
||||
- [ ] 健康检查通过
|
||||
- [ ] 计费任务正常运行
|
||||
- [ ] 备份文件已妥善保存
|
||||
|
||||
---
|
||||
|
||||
## 📞 问题反馈
|
||||
|
||||
如果迁移过程中遇到问题，请检查：
|
||||
|
||||
1. **Pod日志**:
|
||||
```bash
|
||||
kubectl logs -n taiji-ai-pad deployment/mcp-server --tail=100
|
||||
```
|
||||
|
||||
2. **数据库日志**:
|
||||
```bash
|
||||
kubectl logs -n taiji-ai-pad <postgres-pod> --tail=100
|
||||
```
|
||||
|
||||
3. **运行健康检查**:
|
||||
```bash
|
||||
python services/mcp-server/check_billing_health.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 相关文档
|
||||
|
||||
- [计费系统安全性分析](./BILLING_SECURITY_ANALYSIS.md)
|
||||
- [数据库配置文档](./config/database-config.md)
|
||||
- [计费系统架构文档](./Docs/项目文档/计费管理三维度接口文档.md)
|
||||
|
||||
---
|
||||
|
||||
## 📅 变更记录
|
||||
|
||||
| 日期 | 操作人 | 环境 | 迁移 | 结果 | 备注 |
|
||||
|------|--------|------|------|------|------|
|
||||
| 2026-03-09~12 | - | 测试环境 | 017-024 | ✅ 成功 | 初次执行 |
|
||||
| _待填写_ | _待填写_ | 生产环境 | 017-024 | _待填写_ | _待填写_ |
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2026-03-12
|
||||
**文档版本**: v1.0
|
||||
Binary file not shown.
Executable
+320
@@ -0,0 +1,320 @@
|
||||
#!/bin/bash
|
||||
|
||||
###############################################################################
|
||||
# 生产环境数据库同步脚本
|
||||
#
|
||||
# 用途：将测试环境（taiji数据库）在3月9-12日期间的数据库迁移同步到生产环境（taiji_prod）
|
||||
#
|
||||
# 使用方法：
|
||||
# 1. 确保已连接到生产环境K8s集群
|
||||
# 2. 运行： bash scripts/sync_prod_database.sh
|
||||
# 3. 按照提示逐步执行或跳过每个迁移步骤
|
||||
#
|
||||
# 注意事项：
|
||||
# - 执行前请确保已备份生产数据库
|
||||
# - 建议在业务低峰期执行
|
||||
# - 每步执行后都会暂停等待确认
|
||||
###############################################################################
|
||||
|
||||
set -e # 遇到错误立即退出
|
||||
|
||||
# 颜色定义
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 打印带颜色的消息
|
||||
print_info() {
|
||||
echo -e "${BLUE}[INFO]${NC} $1"
|
||||
}
|
||||
|
||||
print_success() {
|
||||
echo -e "${GREEN}[SUCCESS]${NC} $1"
|
||||
}
|
||||
|
||||
print_warning() {
|
||||
echo -e "${YELLOW}[WARNING]${NC} $1"
|
||||
}
|
||||
|
||||
print_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
# 确认函数
|
||||
confirm() {
|
||||
local message=$1
|
||||
local default=${2:-n}
|
||||
|
||||
if [[ $default == "y" ]]; then
|
||||
prompt="[Y/n]"
|
||||
else
|
||||
prompt="[y/N]"
|
||||
fi
|
||||
|
||||
read -p "$(echo -e ${YELLOW}$message $prompt:${NC} )" response
|
||||
response=${response:-$default}
|
||||
|
||||
[[ "$response" =~ ^[Yy]$ ]]
|
||||
}
|
||||
|
||||
# 获取mcp-server Pod名称
|
||||
get_mcp_pod() {
|
||||
kubectl get pods -n taiji-ai-pad -l app=mcp-server -o jsonpath='{.items[0].metadata.name}' 2>/dev/null
|
||||
}
|
||||
|
||||
# 检查K8s连接
|
||||
check_k8s_connection() {
|
||||
print_info "检查K8s集群连接..."
|
||||
if ! kubectl cluster-info &>/dev/null; then
|
||||
print_error "无法连接到K8s集群，请确认kubectl配置正确"
|
||||
exit 1
|
||||
fi
|
||||
print_success "K8s集群连接正常"
|
||||
}
|
||||
|
||||
# 检查mcp-server pod
|
||||
check_mcp_pod() {
|
||||
print_info "检查mcp-server Pod..."
|
||||
POD_NAME=$(get_mcp_pod)
|
||||
|
||||
if [[ -z "$POD_NAME" ]]; then
|
||||
print_error "未找到mcp-server Pod"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
print_success "找到mcp-server Pod: $POD_NAME"
|
||||
}
|
||||
|
||||
# 显示当前数据库配置
|
||||
show_database_config() {
|
||||
print_info "当前数据库配置："
|
||||
kubectl exec -n taiji-ai-pad $POD_NAME -- printenv | grep DATABASE_URL || true
|
||||
}
|
||||
|
||||
# 执行迁移
|
||||
run_migration() {
|
||||
local migration_name=$1
|
||||
local migration_file=$2
|
||||
local description=$3
|
||||
|
||||
echo ""
|
||||
echo "========================================================================"
|
||||
print_info "迁移: $migration_name"
|
||||
print_info "描述: $description"
|
||||
echo "========================================================================"
|
||||
|
||||
if ! confirm "是否执行此迁移？" "n"; then
|
||||
print_warning "跳过迁移: $migration_name"
|
||||
return 0
|
||||
fi
|
||||
|
||||
print_info "开始执行迁移..."
|
||||
|
||||
if kubectl exec -n taiji-ai-pad $POD_NAME -- python $migration_file; then
|
||||
print_success "迁移执行成功: $migration_name"
|
||||
|
||||
if confirm "是否继续下一个迁移？" "y"; then
|
||||
return 0
|
||||
else
|
||||
print_warning "用户选择停止执行"
|
||||
exit 0
|
||||
fi
|
||||
else
|
||||
print_error "迁移执行失败: $migration_name"
|
||||
if confirm "是否继续下一个迁移？" "n"; then
|
||||
return 0
|
||||
else
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# 执行数据修复脚本
|
||||
run_fix_script() {
|
||||
local script_name=$1
|
||||
local script_file=$2
|
||||
local description=$3
|
||||
|
||||
echo ""
|
||||
echo "========================================================================"
|
||||
print_info "数据修复脚本: $script_name"
|
||||
print_info "描述: $description"
|
||||
echo "========================================================================"
|
||||
|
||||
if ! confirm "是否执行此脚本？" "n"; then
|
||||
print_warning "跳过脚本: $script_name"
|
||||
return 0
|
||||
fi
|
||||
|
||||
print_info "开始执行脚本..."
|
||||
|
||||
if kubectl exec -n taiji-ai-pad $POD_NAME -- python $script_file; then
|
||||
print_success "脚本执行成功: $script_name"
|
||||
|
||||
if confirm "是否继续？" "y"; then
|
||||
return 0
|
||||
else
|
||||
print_warning "用户选择停止执行"
|
||||
exit 0
|
||||
fi
|
||||
else
|
||||
print_error "脚本执行失败: $script_name"
|
||||
if confirm "是否继续？" "n"; then
|
||||
return 0
|
||||
else
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# 主流程
|
||||
main() {
|
||||
echo ""
|
||||
echo "========================================================================"
|
||||
echo " 生产环境数据库同步脚本 (taiji -> taiji_prod) "
|
||||
echo "========================================================================"
|
||||
echo ""
|
||||
|
||||
print_warning "⚠️ 重要提醒："
|
||||
echo " 1. 此脚本将在生产环境（taiji_prod）执行数据库迁移"
|
||||
echo " 2. 请确保已经备份生产数据库"
|
||||
echo " 3. 建议在业务低峰期执行"
|
||||
echo " 4. 每步执行前都会询问确认"
|
||||
echo ""
|
||||
|
||||
if ! confirm "确认已阅读上述提醒并继续？" "n"; then
|
||||
print_warning "用户取消操作"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 前置检查
|
||||
check_k8s_connection
|
||||
check_mcp_pod
|
||||
show_database_config
|
||||
|
||||
echo ""
|
||||
print_warning "请再次确认数据库配置中使用的是 taiji_prod 数据库"
|
||||
if ! confirm "确认数据库配置正确？" "n"; then
|
||||
print_error "请检查数据库配置后重试"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 备份确认
|
||||
echo ""
|
||||
print_warning "开始执行前，请确认已完成数据库备份"
|
||||
if ! confirm "已完成数据库备份？" "n"; then
|
||||
print_error "请先备份数据库"
|
||||
print_info "可使用命令： bash scripts/backup_postgres.sh taiji_prod"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 执行迁移
|
||||
echo ""
|
||||
print_info "==================== 第一批：新功能表创建（低风险） ===================="
|
||||
|
||||
run_migration \
|
||||
"Migration 017 - 添加外部数据工具表" \
|
||||
"migrations/run_017_migration.py" \
|
||||
"创建external_data_tools表，支持用户自定义外部API工具"
|
||||
|
||||
run_migration \
|
||||
"Migration 018 - 添加外部工具集表" \
|
||||
"migrations/run_018_migration.py" \
|
||||
"创建external_toolkits表，支持用户组合多个工具"
|
||||
|
||||
echo ""
|
||||
print_info "==================== 第二批：字段新增和修改（低-中风险） ===================="
|
||||
|
||||
run_migration \
|
||||
"Migration 019 - 添加模型名称字段" \
|
||||
"migrations/run_019_migration.py" \
|
||||
"为agents和agent_billing_records表添加model_name字段"
|
||||
|
||||
run_migration \
|
||||
"Migration 020 - 修复配额字段默认值" \
|
||||
"migrations/run_020_fix_quota_defaults.py" \
|
||||
"修复配额表字段的默认值和NULL约束，添加检查约束"
|
||||
|
||||
run_migration \
|
||||
"Migration 021 - 修复agent_type字段长度" \
|
||||
"migrations/run_021_fix_agent_type_length.py" \
|
||||
"扩展agent_type字段从VARCHAR(20)到VARCHAR(100)"
|
||||
|
||||
echo ""
|
||||
print_info "==================== 第三批：数据修正和业务逻辑变更（中-高风险） ===================="
|
||||
print_warning "⚠️ 以下迁移会修改历史数据，请确保已完成备份！"
|
||||
echo ""
|
||||
|
||||
run_migration \
|
||||
"Migration 022 - 修复EU计算逻辑" \
|
||||
"migrations/run_022_fix_eu_equals_cost.py" \
|
||||
"⚠️ 将EU计算方式从时间改为成本（1 EU = 1 美元），会修改所有历史记录"
|
||||
|
||||
run_migration \
|
||||
"Migration 023 - 修复计费channel_id" \
|
||||
"migrations/run_023_fix_billing_channel_id.py" \
|
||||
"填充计费记录中缺失的channel_id字段"
|
||||
|
||||
run_migration \
|
||||
"Migration 024 - 添加计费记录类型字段" \
|
||||
"migrations/run_024_add_record_type.py" \
|
||||
"添加record_type字段，区分VM运行计费和API调用计费"
|
||||
|
||||
# 数据修复脚本（可选）
|
||||
echo ""
|
||||
print_info "==================== 第四批：数据修复脚本（可选） ===================="
|
||||
print_warning "以下脚本为可选执行，请根据生产环境实际情况决定是否需要"
|
||||
|
||||
run_fix_script \
|
||||
"修复渠道配额记录" \
|
||||
"fix_channel_quota_records.py" \
|
||||
"为已使用资源但缺少配额记录的渠道创建记录"
|
||||
|
||||
run_fix_script \
|
||||
"修复Agent资源配置" \
|
||||
"fix_agent_quotas.py" \
|
||||
"修复自定义Agent的K8s资源配置格式"
|
||||
|
||||
print_warning "注意：fix_fake_quota.py 脚本会修改配额数据，建议单独执行并充分测试"
|
||||
if confirm "是否执行 fix_fake_quota.py？" "n"; then
|
||||
run_fix_script \
|
||||
"清理假用量数据" \
|
||||
"fix_fake_quota.py" \
|
||||
"重新统计真实配额使用量"
|
||||
fi
|
||||
|
||||
# 完成
|
||||
echo ""
|
||||
echo "========================================================================"
|
||||
print_success "所有迁移执行完成！"
|
||||
echo "========================================================================"
|
||||
|
||||
print_info "后续步骤："
|
||||
echo " 1. 验证数据库字段和数据是否正确"
|
||||
echo " 2. 执行功能测试"
|
||||
echo " 3. 部署最新代码到生产环境"
|
||||
echo " 4. 监控应用运行状态"
|
||||
echo ""
|
||||
|
||||
print_info "验证命令参考："
|
||||
echo " 查看详细验证步骤，请参考："
|
||||
echo " - DATABASE_MIGRATION_QUICK_START.md（快速开始）"
|
||||
echo " - PRODUCTION_DATABASE_SYNC_GUIDE.md（完整指南）"
|
||||
echo ""
|
||||
|
||||
print_info "快速验证："
|
||||
echo " # 检查新表"
|
||||
echo " kubectl exec -n taiji-ai-pad \$POD_NAME -- python -c \\"
|
||||
echo " 'from database import get_db; import asyncio; \\"
|
||||
echo " async def check(): \\"
|
||||
echo " async for db in get_db(): \\"
|
||||
echo " result = await db.execute(\"SELECT COUNT(*) FROM external_data_tools\"); \\"
|
||||
echo " print(f\"external_data_tools: {result.scalar()}\"); \\"
|
||||
echo " asyncio.run(check())'"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# 执行主流程
|
||||
main "$@"
|
||||
@@ -0,0 +1,69 @@
|
||||
"""
|
||||
添加 record_type 列到 agent_billing_records 表
|
||||
|
||||
这个脚本用于数据库迁移，添加用于区分 VM 运行时间计费和 API 调用计费的字段。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from sqlalchemy import text
|
||||
from database import AsyncSessionLocal
|
||||
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def add_record_type_column():
|
||||
"""添加 record_type 列"""
|
||||
|
||||
async with AsyncSessionLocal() as db:
|
||||
try:
|
||||
# 检查列是否已存在
|
||||
check_query = text("""
|
||||
SELECT column_name
|
||||
FROM information_schema.columns
|
||||
WHERE table_name='agent_billing_records'
|
||||
AND column_name='record_type';
|
||||
""")
|
||||
result = await db.execute(check_query)
|
||||
exists = result.fetchone()
|
||||
|
||||
if exists:
|
||||
logger.info("✅ record_type 列已存在，无需添加")
|
||||
return
|
||||
|
||||
logger.info("🔧 开始添加 record_type 列...")
|
||||
|
||||
# 添加列（默认值为 'vm_runtime'，表示现有记录都是 VM 运行时间计费）
|
||||
alter_query = text("""
|
||||
ALTER TABLE agent_billing_records
|
||||
ADD COLUMN record_type VARCHAR(20) NOT NULL DEFAULT 'vm_runtime';
|
||||
""")
|
||||
await db.execute(alter_query)
|
||||
await db.commit()
|
||||
|
||||
logger.info("✅ 成功添加 record_type 列")
|
||||
logger.info(" - 列名: record_type")
|
||||
logger.info(" - 类型: VARCHAR(20)")
|
||||
logger.info(" - 默认值: 'vm_runtime'")
|
||||
logger.info(" - 可选值: 'vm_runtime' (VM运行时间计费) 或 'api_call' (API调用计费)")
|
||||
|
||||
# 创建索引以提高查询性能
|
||||
logger.info("🔧 创建索引...")
|
||||
index_query = text("""
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_record_type
|
||||
ON agent_billing_records(record_type);
|
||||
""")
|
||||
await db.execute(index_query)
|
||||
await db.commit()
|
||||
|
||||
logger.info("✅ 成功创建索引: idx_agent_billing_record_type")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"❌ 迁移失败: {e}")
|
||||
await db.rollback()
|
||||
raise
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(add_record_type_column())
|
||||
@@ -28,6 +28,7 @@ eu_consumed 字段的值应该等于 cost 字段的值。
|
||||
"""
|
||||
|
||||
import math
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from typing import Optional, Tuple
|
||||
@@ -36,6 +37,8 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from models import User, Channel, BillingRecord, Agent, Balance, AgentBillingRecord
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ============= EU定价配置（已废弃） =============
|
||||
#
|
||||
@@ -190,34 +193,32 @@ async def deduct_balance(
|
||||
amount: Decimal,
|
||||
db: AsyncSession,
|
||||
description: str = "消费",
|
||||
auto_commit: bool = False
|
||||
auto_commit: bool = True # 🔧 修改：默认自动提交，确保扣款立即生效
|
||||
) -> Tuple[bool, str]:
|
||||
"""
|
||||
扣除余额（优先扣除账户余额，不足时使用授信额度）
|
||||
|
||||
使用行锁保护并发扣款操作，防止超扣。
|
||||
🔧 安全修复：使用真正的原子操作，在一条SQL中完成余额检查和扣款，
|
||||
彻底避免竞态条件。
|
||||
|
||||
Args:
|
||||
user_id: 用户ID
|
||||
amount: 扣除金额
|
||||
amount: 扣除金额（自动统一精度到6位小数）
|
||||
db: 数据库会话
|
||||
description: 描述
|
||||
auto_commit: 是否自动提交（默认False，由调用者管理事务）
|
||||
auto_commit: 是否自动提交（默认True，确保扣款立即生效）
|
||||
|
||||
Returns:
|
||||
(是否成功, 消息)
|
||||
|
||||
Note:
|
||||
默认不会 commit，由调用者统一管理事务。
|
||||
如需独立提交，请设置 auto_commit=True。
|
||||
修复后默认 auto_commit=True，确保扣款立即生效，避免长时间事务导致的问题。
|
||||
如需批量操作，请明确设置 auto_commit=False。
|
||||
"""
|
||||
# 使用 FOR UPDATE 锁定余额行，防止并发扣款
|
||||
balance_result = await db.execute(
|
||||
select(Balance)
|
||||
.where(Balance.user_id == user_id)
|
||||
.with_for_update()
|
||||
)
|
||||
balance_obj = balance_result.scalar_one_or_none()
|
||||
from sqlalchemy import text
|
||||
|
||||
# 🔧 修复1: 统一精度到6位小数，避免累积误差
|
||||
amount = amount.quantize(Decimal('0.000001'))
|
||||
|
||||
# 从 User 表获取授信额度
|
||||
user_result = await db.execute(
|
||||
@@ -228,28 +229,74 @@ async def deduct_balance(
|
||||
if not user:
|
||||
return False, "用户不存在"
|
||||
|
||||
if balance_obj is None:
|
||||
# 如果余额记录不存在，创建一个新的（初始余额为0）
|
||||
balance_obj = Balance(user_id=user_id, eu_balance=0.0)
|
||||
db.add(balance_obj)
|
||||
await db.flush() # 确保记录创建后再继续
|
||||
|
||||
balance = Decimal(str(balance_obj.eu_balance))
|
||||
credit_limit = Decimal(str(user.credit_limit))
|
||||
available = balance + credit_limit
|
||||
amount_str = str(amount)
|
||||
credit_str = str(credit_limit)
|
||||
|
||||
if available < amount:
|
||||
return False, f"余额不足，当前可用额度: {available}, 需要: {amount}"
|
||||
# 🔧 修复2: 真正的原子操作 - 分步执行但在同一事务中
|
||||
# Step 1: 确保余额记录存在（生成UUID）
|
||||
await db.execute(
|
||||
text("""
|
||||
INSERT INTO balances (id, user_id, eu_balance, created_at, updated_at)
|
||||
VALUES (gen_random_uuid(), :uid, 0.0, NOW(), NOW())
|
||||
ON CONFLICT (user_id) DO NOTHING
|
||||
"""),
|
||||
{"uid": user_id}
|
||||
)
|
||||
|
||||
# 优先扣除账户余额，允许透支到授信额度
|
||||
new_balance = balance - amount
|
||||
balance_obj.eu_balance = float(new_balance)
|
||||
# Step 2: 原子更新 - 只有在余额充足时才扣款
|
||||
result = await db.execute(
|
||||
text("""
|
||||
WITH current AS (
|
||||
SELECT eu_balance FROM balances WHERE user_id = :uid FOR UPDATE
|
||||
)
|
||||
UPDATE balances
|
||||
SET eu_balance = eu_balance - CAST(:amount AS NUMERIC(15, 6)),
|
||||
updated_at = NOW()
|
||||
WHERE user_id = :uid
|
||||
AND (SELECT eu_balance FROM current) + CAST(:credit AS NUMERIC(15, 6)) >= CAST(:amount AS NUMERIC(15, 6))
|
||||
RETURNING
|
||||
eu_balance as new_balance,
|
||||
eu_balance + CAST(:amount AS NUMERIC(15, 6)) as old_balance
|
||||
"""),
|
||||
{"uid": user_id, "amount": amount_str, "credit": credit_str}
|
||||
)
|
||||
|
||||
row = result.fetchone()
|
||||
|
||||
if row is None:
|
||||
# UPDATE 影响了0行，说明余额不足
|
||||
# 查询当前余额用于错误信息
|
||||
balance_result = await db.execute(
|
||||
text("SELECT eu_balance FROM balances WHERE user_id = :uid"),
|
||||
{"uid": user_id}
|
||||
)
|
||||
balance_row = balance_result.fetchone()
|
||||
current = Decimal(str(balance_row[0])) if balance_row else Decimal("0")
|
||||
available = current + credit_limit
|
||||
|
||||
if auto_commit:
|
||||
await db.commit() # 即使失败也提交（没有实际修改）
|
||||
|
||||
return False, f"余额不足，当前可用: {available:.6f}, 需要: {amount:.6f}"
|
||||
|
||||
new_balance = Decimal(str(row[0]))
|
||||
old_balance = Decimal(str(row[1]))
|
||||
|
||||
# 可选：自动提交
|
||||
if auto_commit:
|
||||
await db.commit()
|
||||
logger.info(
|
||||
f"💰 扣款成功并已提交: user={user_id[:8]}, "
|
||||
f"金额={amount:.6f}, 余额: {old_balance:.6f} → {new_balance:.6f}"
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"💰 扣款成功待提交: user={user_id[:8]}, "
|
||||
f"金额={amount:.6f}, 余额: {old_balance:.6f} → {new_balance:.6f}"
|
||||
)
|
||||
|
||||
return True, f"成功扣除 {amount} 元"
|
||||
return True, f"成功扣除 {amount:.6f} 元，余额: {old_balance:.6f} → {new_balance:.6f}"
|
||||
|
||||
|
||||
async def add_balance(
|
||||
@@ -257,27 +304,31 @@ async def add_balance(
|
||||
amount: Decimal,
|
||||
db: AsyncSession,
|
||||
description: str = "充值",
|
||||
auto_commit: bool = False
|
||||
auto_commit: bool = True # 🔧 修改：默认自动提交，与deduct_balance保持一致
|
||||
) -> Tuple[bool, str]:
|
||||
"""
|
||||
增加余额
|
||||
|
||||
使用行锁保护并发充值操作，确保余额准确。
|
||||
🔧 安全修复：使用原子操作，统一精度处理
|
||||
|
||||
Args:
|
||||
user_id: 用户ID
|
||||
amount: 充值金额
|
||||
amount: 充值金额（自动统一精度到6位小数）
|
||||
db: 数据库会话
|
||||
description: 描述
|
||||
auto_commit: 是否自动提交（默认False，由调用者管理事务）
|
||||
auto_commit: 是否自动提交（默认True）
|
||||
|
||||
Returns:
|
||||
(是否成功, 消息)
|
||||
|
||||
Note:
|
||||
默认不会 commit，由调用者统一管理事务。
|
||||
如需独立提交，请设置 auto_commit=True。
|
||||
修复后默认 auto_commit=True，与 deduct_balance 保持一致。
|
||||
"""
|
||||
from sqlalchemy import text
|
||||
|
||||
# 🔧 修复：统一精度到6位小数
|
||||
amount = amount.quantize(Decimal('0.000001'))
|
||||
|
||||
# 检查用户是否存在
|
||||
user_result = await db.execute(
|
||||
select(User).where(User.id == user_id)
|
||||
@@ -287,27 +338,51 @@ async def add_balance(
|
||||
if not user:
|
||||
return False, "用户不存在"
|
||||
|
||||
# 使用 FOR UPDATE 锁定余额行，防止并发更新
|
||||
balance_result = await db.execute(
|
||||
select(Balance)
|
||||
.where(Balance.user_id == user_id)
|
||||
.with_for_update()
|
||||
amount_str = str(amount)
|
||||
|
||||
# 🔧 修复：使用原子操作 - 分步执行
|
||||
# Step 1: 确保余额记录存在（生成UUID）
|
||||
await db.execute(
|
||||
text("""
|
||||
INSERT INTO balances (id, user_id, eu_balance, created_at, updated_at)
|
||||
VALUES (gen_random_uuid(), :uid, 0.0, NOW(), NOW())
|
||||
ON CONFLICT (user_id) DO NOTHING
|
||||
"""),
|
||||
{"uid": user_id}
|
||||
)
|
||||
balance_obj = balance_result.scalar_one_or_none()
|
||||
|
||||
if balance_obj is None:
|
||||
# 如果余额记录不存在，创建一个新的
|
||||
balance_obj = Balance(user_id=user_id, eu_balance=0.0)
|
||||
db.add(balance_obj)
|
||||
await db.flush() # 确保记录创建后再继续
|
||||
# Step 2: 原子更新 - 增加余额
|
||||
result = await db.execute(
|
||||
text("""
|
||||
UPDATE balances
|
||||
SET eu_balance = eu_balance + CAST(:amount AS NUMERIC(15, 6)),
|
||||
updated_at = NOW()
|
||||
WHERE user_id = :uid
|
||||
RETURNING
|
||||
eu_balance as new_balance,
|
||||
eu_balance - CAST(:amount AS NUMERIC(15, 6)) as old_balance
|
||||
"""),
|
||||
{"uid": user_id, "amount": amount_str}
|
||||
)
|
||||
|
||||
old_balance = Decimal(str(balance_obj.eu_balance))
|
||||
new_balance = old_balance + amount
|
||||
balance_obj.eu_balance = float(new_balance)
|
||||
row = result.fetchone()
|
||||
new_balance = Decimal(str(row[0])) if row else amount
|
||||
old_balance = Decimal(str(row[1])) if row else Decimal("0")
|
||||
|
||||
# 可选：自动提交
|
||||
if auto_commit:
|
||||
await db.commit()
|
||||
logger.info(
|
||||
f"💰 充值成功并已提交: user={user_id[:8]}, "
|
||||
f"金额={amount:.6f}, 余额: {old_balance:.6f} → {new_balance:.6f}"
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"💰 充值成功待提交: user={user_id[:8]}, "
|
||||
f"金额={amount:.6f}, 余额: {old_balance:.6f} → {new_balance:.6f}"
|
||||
)
|
||||
|
||||
return True, f"成功充值 {amount:.6f} 元，余额: {old_balance:.6f} → {new_balance:.6f}"
|
||||
|
||||
return True, f"成功充值 {amount} 元，当前余额: {new_balance}"
|
||||
|
||||
@@ -614,6 +689,11 @@ PLATFORM_AGENT_PRICING = {
|
||||
"default": Decimal("0.10"), # 默认每小时 $0.10
|
||||
}
|
||||
|
||||
# API 调用计费配置（Agent Manager 回调）
|
||||
API_CALL_PRICING = {
|
||||
"per_call": Decimal("0.01"), # 每次 API 调用 $0.01（0.01 EU/call）
|
||||
}
|
||||
|
||||
|
||||
def get_platform_agent_hourly_price(template_name: str) -> Decimal:
|
||||
"""
|
||||
@@ -672,6 +752,19 @@ def calculate_platform_agent_cost(
|
||||
return hours * hourly_price
|
||||
|
||||
|
||||
def calculate_api_call_cost() -> Decimal:
|
||||
"""
|
||||
计算 API 调用成本（固定费用）
|
||||
|
||||
用于 Agent Manager 回调的 API 调用计费。
|
||||
每次调用固定收费 0.001 EU（= $0.001），与运行时间无关。
|
||||
|
||||
Returns:
|
||||
成本金额（USD）：0.001
|
||||
"""
|
||||
return API_CALL_PRICING["per_call"]
|
||||
|
||||
|
||||
# ============= Agent 计费记录创建 =============
|
||||
|
||||
async def create_agent_billing_record(
|
||||
|
||||
@@ -48,7 +48,6 @@ logger = logging.getLogger(__name__)
|
||||
# 计费周期配置
|
||||
BILLING_INTERVAL_SECONDS = 3600 # 每小时执行一次
|
||||
QUOTA_CHECK_INTERVAL_HOURS = 24 # 配额一致性检查周期：每24小时一次
|
||||
VM_EU_PER_HOUR = Decimal("0.5") # VM计算 0.5 EU/hour
|
||||
|
||||
# 上次配额检查时间
|
||||
_last_quota_check: Optional[datetime] = None
|
||||
@@ -86,12 +85,13 @@ async def stop_user_agents(user_id: str, db: AsyncSession) -> list:
|
||||
stopped_agents = []
|
||||
now = datetime.utcnow()
|
||||
|
||||
# 查询该用户所有运行中的Agent
|
||||
# 查询该用户所有运行中的Agent（只查询vm_runtime类型）
|
||||
result = await db.execute(
|
||||
select(AgentBillingRecord).where(
|
||||
and_(
|
||||
AgentBillingRecord.user_id == user_id,
|
||||
AgentBillingRecord.end_time == None
|
||||
AgentBillingRecord.end_time == None,
|
||||
AgentBillingRecord.record_type == "vm_runtime" # 只查询VM运行时间计费
|
||||
)
|
||||
)
|
||||
)
|
||||
@@ -224,9 +224,11 @@ async def update_running_agent_billing(db: AsyncSession) -> dict:
|
||||
"""
|
||||
更新所有运行中 Agent 的计费记录
|
||||
|
||||
🔧 安全修复：每个Agent使用独立事务，避免批量回滚导致收入损失
|
||||
|
||||
功能：
|
||||
1. 更新EU消耗和成本
|
||||
2. 增量扣款
|
||||
2. 增量扣款（每个Agent独立事务）
|
||||
3. 余额不足时自动停止Agent
|
||||
|
||||
Returns:
|
||||
@@ -243,37 +245,63 @@ async def update_running_agent_billing(db: AsyncSession) -> dict:
|
||||
"stopped_agents": [], # 因余额不足而停止的Agent
|
||||
}
|
||||
|
||||
# 🔧 修复：先查询所有运行中的Agent（使用只读查询）
|
||||
try:
|
||||
# 查询所有运行中的 Agent（end_time 为空）
|
||||
result = await db.execute(
|
||||
select(AgentBillingRecord).where(
|
||||
AgentBillingRecord.end_time == None
|
||||
and_(
|
||||
AgentBillingRecord.end_time == None,
|
||||
AgentBillingRecord.record_type == "vm_runtime" # 只更新VM运行时间计费
|
||||
)
|
||||
)
|
||||
)
|
||||
running_agents = result.scalars().all()
|
||||
|
||||
now = datetime.utcnow()
|
||||
|
||||
for record in running_agents:
|
||||
# 提取Agent ID列表，避免在循环中持有长时间事务
|
||||
agent_records = [
|
||||
{
|
||||
'id': str(record.id),
|
||||
'agent_name': record.agent_name,
|
||||
'user_id': str(record.user_id),
|
||||
'start_time': record.start_time,
|
||||
'is_platform_agent': record.is_platform_agent,
|
||||
'agent_type': record.agent_type,
|
||||
'cpu_used': record.cpu_used,
|
||||
'memory_used': record.memory_used,
|
||||
'cost': record.cost or 0
|
||||
}
|
||||
for record in running_agents
|
||||
]
|
||||
except Exception as e:
|
||||
logger.error(f"查询运行中Agent失败: {e}")
|
||||
stats["errors"].append(f"查询错误: {str(e)}")
|
||||
return stats
|
||||
|
||||
now = datetime.utcnow()
|
||||
|
||||
# 🔧 修复：为每个Agent创建独立事务
|
||||
for agent_info in agent_records:
|
||||
# 为每个Agent创建独立的数据库会话
|
||||
async with AsyncSessionLocal() as agent_db:
|
||||
try:
|
||||
# 跳过没有开始时间的记录
|
||||
if record.start_time is None:
|
||||
logger.warning(f"Agent {record.agent_name} 缺少 start_time，跳过计费")
|
||||
if agent_info['start_time'] is None:
|
||||
logger.warning(f"Agent {agent_info['agent_name']} 缺少 start_time，跳过计费")
|
||||
stats["failed"] += 1
|
||||
stats["errors"].append(f"Agent {record.agent_name}: start_time 为空")
|
||||
stats["errors"].append(f"Agent {agent_info['agent_name']}: start_time 为空")
|
||||
continue
|
||||
|
||||
# 计算从开始到现在的运行时长
|
||||
duration = (now - record.start_time).total_seconds()
|
||||
duration = (now - agent_info['start_time']).total_seconds()
|
||||
duration_seconds = int(duration)
|
||||
|
||||
# 计算成本
|
||||
if record.is_platform_agent:
|
||||
cost = calculate_platform_agent_cost(record.agent_type, duration_seconds)
|
||||
if agent_info['is_platform_agent']:
|
||||
cost = calculate_platform_agent_cost(agent_info['agent_type'], duration_seconds)
|
||||
stats["platform_agents"] += 1
|
||||
else:
|
||||
cpu_cores = _parse_cpu_to_cores(record.cpu_used) if record.cpu_used else 0.1
|
||||
memory_gb = _parse_memory_to_gb(record.memory_used) if record.memory_used else 0.125
|
||||
cpu_cores = _parse_cpu_to_cores(agent_info['cpu_used']) if agent_info['cpu_used'] else 0.1
|
||||
memory_gb = _parse_memory_to_gb(agent_info['memory_used']) if agent_info['memory_used'] else 0.125
|
||||
cost = calculate_agent_cost_by_resources(cpu_cores, memory_gb, duration_seconds)
|
||||
stats["custom_agents"] += 1
|
||||
|
||||
@@ -281,52 +309,60 @@ async def update_running_agent_billing(db: AsyncSession) -> dict:
|
||||
eu_consumed = float(cost)
|
||||
|
||||
# 计算本次周期需要扣除的增量
|
||||
previous_cost = Decimal(str(record.cost or 0))
|
||||
previous_cost = Decimal(str(agent_info['cost']))
|
||||
cost_increment = cost - previous_cost
|
||||
|
||||
# 更新记录
|
||||
record.duration_seconds = duration_seconds
|
||||
record.eu_consumed = eu_consumed # EU = Cost（美元）
|
||||
record.cost = float(cost)
|
||||
|
||||
# 如果有增量，进行扣款
|
||||
# 如果有增量，进行扣款（使用独立会话，自动提交）
|
||||
if cost_increment > 0:
|
||||
success, message = await deduct_balance(
|
||||
str(record.user_id), cost_increment, db,
|
||||
f"Agent周期计费: {record.agent_name}"
|
||||
agent_info['user_id'],
|
||||
cost_increment,
|
||||
agent_db, # 使用独立会话
|
||||
f"Agent周期计费: {agent_info['agent_name']}",
|
||||
auto_commit=True # 🔧 关键：自动提交，确保扣款立即生效
|
||||
)
|
||||
if not success:
|
||||
logger.warning(
|
||||
f"周期计费扣款失败: {message}, "
|
||||
f"用户: {record.user_id}, Agent: {record.agent_name}"
|
||||
f"用户: {agent_info['user_id']}, Agent: {agent_info['agent_name']}"
|
||||
)
|
||||
stats["failed"] += 1
|
||||
stats["errors"].append(f"{agent_info['agent_name']}: 扣款失败 - {message}")
|
||||
continue # 扣款失败则跳过更新记录
|
||||
|
||||
# 🔧 重新查询并更新计费记录（在同一个独立会话中）
|
||||
record_result = await agent_db.execute(
|
||||
select(AgentBillingRecord).where(AgentBillingRecord.id == agent_info['id'])
|
||||
)
|
||||
record = record_result.scalar_one_or_none()
|
||||
|
||||
if record:
|
||||
record.duration_seconds = duration_seconds
|
||||
record.eu_consumed = eu_consumed # EU = Cost（美元）
|
||||
record.cost = float(cost)
|
||||
await agent_db.commit() # 提交记录更新
|
||||
|
||||
# 检查用户余额，如果透支则停止该用户的所有Agent
|
||||
balance, credit_limit, available = await get_available_balance(
|
||||
str(record.user_id), db
|
||||
agent_info['user_id'], agent_db
|
||||
)
|
||||
if available < 0:
|
||||
logger.warning(
|
||||
f"用户 {record.user_id} 余额不足 (可用: {available})，将停止所有Agent"
|
||||
f"用户 {agent_info['user_id']} 余额不足 (可用: {available})，将停止所有Agent"
|
||||
)
|
||||
stopped = await stop_user_agents(str(record.user_id), db)
|
||||
stopped = await stop_user_agents(agent_info['user_id'], agent_db)
|
||||
stats["stopped_agents"].extend(stopped)
|
||||
await agent_db.commit() # 提交停止操作
|
||||
|
||||
stats["processed"] += 1
|
||||
stats["total_eu_consumed"] += Decimal(str(eu_consumed))
|
||||
stats["total_cost"] += cost
|
||||
|
||||
except Exception as e:
|
||||
await agent_db.rollback()
|
||||
stats["failed"] += 1
|
||||
stats["errors"].append(f"Agent {record.agent_name}: {str(e)}")
|
||||
logger.error(f"处理 Agent {record.agent_name} 计费失败: {e}")
|
||||
|
||||
await db.commit()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"周期性计费任务失败: {e}")
|
||||
stats["errors"].append(f"全局错误: {str(e)}")
|
||||
await db.rollback()
|
||||
stats["errors"].append(f"Agent {agent_info['agent_name']}: {str(e)}")
|
||||
logger.error(f"处理 Agent {agent_info['agent_name']} 计费失败: {e}")
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
@@ -292,10 +292,13 @@ class ResourceController:
|
||||
user = user_result.scalar_one_or_none()
|
||||
user_channel_id = user.channel_id if user else None
|
||||
|
||||
# ✅ 确保 agent_name 不为空字符串
|
||||
agent_name = resource_id if resource_id else "unknown"
|
||||
|
||||
billing_record = AgentBillingRecord(
|
||||
user_id=user_id,
|
||||
channel_id=user_channel_id, # 从用户表获取 channel_id
|
||||
agent_name=resource_id or "unknown",
|
||||
agent_name=agent_name, # ✅ 使用验证后的名称
|
||||
agent_type="custom", # 默认为自定义 Agent
|
||||
is_platform_agent=False,
|
||||
duration_seconds=int(execution_time_ms / 1000),
|
||||
|
||||
@@ -2841,8 +2841,8 @@ async def get_running_agents_billing(
|
||||
|
||||
@router.get("/billing/overview", response_model=SuccessResponse)
|
||||
async def get_billing_overview(
|
||||
startTime: str = Query(...),
|
||||
endTime: str = Query(...),
|
||||
startTime: Optional[str] = Query(None, description="开始时间（ISO 8601格式），不传则默认为当前时间前1个月"),
|
||||
endTime: Optional[str] = Query(None, description="结束时间（ISO 8601格式），不传则默认为当前时间"),
|
||||
channelName: Optional[str] = Query(None),
|
||||
tenantName: Optional[str] = Query(None),
|
||||
minCalls: Optional[int] = Query(None),
|
||||
@@ -2857,13 +2857,24 @@ async def get_billing_overview(
|
||||
从 AgentBillingRecord 和 ModelBillingRecord 表查询计费数据，
|
||||
返回渠道维度、租户维度和调用记录三个维度的统计。
|
||||
|
||||
时间参数：
|
||||
- 不传参数时，默认查询最近1个月（endTime=当前时间，startTime=当前时间-30天）
|
||||
- 传参时按照传入的时间范围查询，格式为 ISO 8601（如：2026-02-10T09:43:48.048Z）
|
||||
|
||||
EU 计算规则：1 EU = 10 秒运行时间（向上取整）
|
||||
"""
|
||||
_verify_read_permission(principal)
|
||||
|
||||
# 解析时间，移除时区信息以匹配数据库中的 naive datetime
|
||||
start_dt = datetime.fromisoformat(startTime.replace("Z", "+00:00"))
|
||||
end_dt = datetime.fromisoformat(endTime.replace("Z", "+00:00"))
|
||||
# 处理时间参数：不传则使用默认值（最近1个月）
|
||||
if endTime is None:
|
||||
end_dt = datetime.utcnow()
|
||||
else:
|
||||
end_dt = datetime.fromisoformat(endTime.replace("Z", "+00:00"))
|
||||
|
||||
if startTime is None:
|
||||
start_dt = end_dt - timedelta(days=30)
|
||||
else:
|
||||
start_dt = datetime.fromisoformat(startTime.replace("Z", "+00:00"))
|
||||
|
||||
# 转换为 naive datetime（移除时区信息）
|
||||
if start_dt.tzinfo is not None:
|
||||
@@ -3048,8 +3059,8 @@ async def get_billing_overview(
|
||||
"timestamp": record.start_time.isoformat() if record.start_time else None,
|
||||
"channelName": channel_name or "无渠道",
|
||||
"tenantName": user_name,
|
||||
"agentName": record.agent_name,
|
||||
"modelName": record.model_name,
|
||||
"agentName": record.agent_name or "unknown", # ✅ 提供默认值
|
||||
"modelName": record.model_name or "N/A", # ✅ 提供默认值
|
||||
"duration": record.duration_seconds or 0,
|
||||
"eu": record.eu_consumed or 0,
|
||||
"cost": float(record.cost or 0),
|
||||
|
||||
@@ -9,13 +9,17 @@ LiteLLM Callback Webhook路由
|
||||
- EU计算：基于Token数量和模型类型
|
||||
- 公式：EU = total_tokens * MODEL_EU_RATE[model_name]
|
||||
- 存储表：model_billing_records
|
||||
- 扣款时机：立即扣款
|
||||
|
||||
2. Agent Manager 回调（/agent-callback）
|
||||
- 数据来源：Agent Manager 的运行结束通知
|
||||
- EU计算：基于Pod运行时长（调用 app.billing.calculate_eu）
|
||||
- 公式：EU = ceil(duration_seconds / 10)
|
||||
- 数据来源：Agent Manager 的 API 调用通知
|
||||
- 计费类型：API 调用计费（record_type="api_call"）
|
||||
- 计费标准：每次调用固定 0.001 EU（= $0.001），与运行时间无关
|
||||
- 公式：Cost = 0.001 USD/call，EU = Cost
|
||||
- 存储表：agent_billing_records
|
||||
- 注意：周期性计费任务会预先扣款，此回调只扣除增量部分
|
||||
- 扣款时机：每次 API 调用立即扣款
|
||||
- 注意：这与 VM 运行时间计费（record_type="vm_runtime"）不同，
|
||||
VM 运行时间计费按小时费率由周期任务处理
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
@@ -32,6 +36,7 @@ from database import get_db
|
||||
from models import ModelBillingRecord, TenantModelKey, Balance, User
|
||||
from app.schemas import AgentManagerCallbackData, AgentManagerCallbackResponse
|
||||
from app.db_utils import ensure_idempotent, atomic_eu_consume, with_retry
|
||||
from app.billing import calculate_api_call_cost
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/api/v1/billing", tags=["计费Webhook"])
|
||||
@@ -382,16 +387,19 @@ async def agent_manager_callback(
|
||||
接收 Agent Manager 的回调数据
|
||||
|
||||
Agent Manager 在用户调用 agent 后返回运行时信息：
|
||||
- Pod 运行时间（必填）
|
||||
- Pod 运行时间（记录但不用于计费）
|
||||
- 使用的工具列表（可选）
|
||||
|
||||
此接口用于更新或创建 Agent 的计费记录：
|
||||
- 如果存在运行中的计费记录（end_time=None），则更新该记录
|
||||
- 如果不存在运行中的记录，则创建新记录（异常情况的兜底）
|
||||
此接口用于创建 Agent 的 API 调用计费记录：
|
||||
- record_type="api_call"（API 调用计费）
|
||||
- 每次调用固定费用：0.001 EU（= $0.001）
|
||||
- 与运行时间无关，按调用次数计费
|
||||
|
||||
计费逻辑：
|
||||
- 周期计费任务会对运行中的 Agent 进行增量扣款
|
||||
- 此回调负责结算最终费用，只扣除增量部分，避免重复扣款
|
||||
- API 调用计费：每次调用立即扣款 0.001 EU
|
||||
- VM 运行时间计费（record_type="vm_runtime"）：由周期计费任务处理，按小时费率计费
|
||||
|
||||
注意：API 调用计费与 VM 运行时间计费是两种不同的计费方式
|
||||
"""
|
||||
from models import AgentBillingRecord
|
||||
from sqlalchemy import and_
|
||||
@@ -432,96 +440,109 @@ async def agent_manager_callback(
|
||||
except Exception as e:
|
||||
logger.warning(f"结束时间解析失败: {e}")
|
||||
|
||||
# 计算成本
|
||||
duration_seconds = callback_data.podRunningTimeSeconds
|
||||
# ✅ 验证必填字段，防止空值
|
||||
agent_name = callback_data.agentName or "unknown"
|
||||
if not callback_data.agentName:
|
||||
logger.warning(
|
||||
f"⚠️ Agent Manager回调缺少agentName: "
|
||||
f"user_id={callback_data.userId}, request_id={callback_data.requestId}"
|
||||
)
|
||||
|
||||
# 假设是平台Agent（可以根据agent_name前缀判断）
|
||||
# ✅ 计算 API 调用成本：每次调用固定 0.001 EU（= $0.001）
|
||||
# API 调用计费与运行时间无关，每次调用固定费用
|
||||
duration_seconds = callback_data.podRunningTimeSeconds # 仅记录，不用于计费
|
||||
|
||||
# 假设是平台Agent（可以从agent_name前缀判断）
|
||||
is_platform_agent = True
|
||||
agent_type = "platform" # 可以从agent名称中提取
|
||||
new_cost = calculate_platform_agent_cost(agent_type, duration_seconds)
|
||||
|
||||
# ✅ 使用 API 调用计费函数：固定 0.001 EU/call
|
||||
cost = calculate_api_call_cost()
|
||||
|
||||
# 调试日志 - 使用 print 确保输出
|
||||
print(f"\n⭐️⭐️⭐️ Agent Manager 回调 ⭐️⭐️⭐️")
|
||||
print(f"User: {callback_data.userId}")
|
||||
print(f"Agent: {agent_name}")
|
||||
print(f"Cost: {cost} (type: {type(cost)})")
|
||||
print(f"Duration: {duration_seconds}秒")
|
||||
print("⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️\n", flush=True)
|
||||
logger.info(f"📝 Agent Manager 回调收到: user={callback_data.userId}, agent={agent_name}, cost={cost}, type={type(cost)}")
|
||||
|
||||
# EU = Cost（1 EU = 1 美元）
|
||||
eu_consumed = float(new_cost)
|
||||
eu_consumed = float(cost)
|
||||
|
||||
# ✅ 先查找现有的运行中计费记录（end_time == None）
|
||||
existing_result = await db.execute(
|
||||
select(AgentBillingRecord).where(
|
||||
and_(
|
||||
AgentBillingRecord.agent_name == callback_data.agentName,
|
||||
AgentBillingRecord.user_id == callback_data.userId,
|
||||
AgentBillingRecord.end_time == None # 运行中的记录
|
||||
)
|
||||
)
|
||||
# ✅ 新逻辑：每次调用都创建新的 API 调用记录
|
||||
# 这是 API 调用计费，不是 VM 运行时间计费
|
||||
# VM 运行时间计费由周期任务处理
|
||||
|
||||
billing_record = AgentBillingRecord(
|
||||
record_type="api_call", # ✅ API 调用计费
|
||||
user_id=callback_data.userId,
|
||||
channel_id=user.channel_id if user.channel_id else None,
|
||||
agent_name=agent_name,
|
||||
agent_type=agent_type,
|
||||
is_platform_agent=is_platform_agent,
|
||||
duration_seconds=duration_seconds, # 本次调用的处理时间
|
||||
request_count=1, # 这是一次调用
|
||||
eu_consumed=eu_consumed,
|
||||
cost=float(cost),
|
||||
start_time=start_time or datetime.utcnow(),
|
||||
end_time=end_time or datetime.utcnow(), # API 调用已完成
|
||||
period_start=start_time or datetime.utcnow(),
|
||||
period_end=end_time or datetime.utcnow(),
|
||||
tools_used=callback_data.toolsUsed, # 本次调用使用的工具
|
||||
request_id=callback_data.requestId, # 本次调用的唯一ID
|
||||
)
|
||||
existing_record = existing_result.scalar_one_or_none()
|
||||
|
||||
if existing_record:
|
||||
# ✅ 更新现有记录（避免重复创建）
|
||||
# ⚠️ 不在此处扣款，由周期计费（periodic_billing.py）统一处理扣款
|
||||
|
||||
existing_record.end_time = end_time or datetime.utcnow()
|
||||
existing_record.duration_seconds = duration_seconds
|
||||
existing_record.eu_consumed = eu_consumed
|
||||
existing_record.cost = float(new_cost)
|
||||
existing_record.period_end = end_time or datetime.utcnow()
|
||||
existing_record.tools_used = callback_data.toolsUsed
|
||||
existing_record.request_id = callback_data.requestId
|
||||
|
||||
billing_record = existing_record
|
||||
|
||||
logger.info(
|
||||
f"📝 更新Agent计费记录（不扣款）: agent={callback_data.agentName}, "
|
||||
f"最终成本={new_cost}，扣款由周期计费处理"
|
||||
db.add(billing_record)
|
||||
|
||||
# ✅ API 调用立即扣款
|
||||
print(f"\n🔍 准备扣款: cost={cost}, type={type(cost)}, cost>0={cost > 0}", flush=True)
|
||||
logger.info(f"🔍 准备扣款: cost={cost}, type={type(cost)}, cost>0={cost > 0}")
|
||||
if cost > 0:
|
||||
print(f"✅ Cost > 0, 开始调用 deduct_balance", flush=True)
|
||||
logger.info(f"🔍 开始调用 deduct_balance: user_id={callback_data.userId}, cost={cost}")
|
||||
success, message = await deduct_balance(
|
||||
callback_data.userId, cost, db,
|
||||
f"Agent API调用: {agent_name}",
|
||||
auto_commit=False # 🔧 修复：与billing_record在同一事务中
|
||||
)
|
||||
|
||||
print(f"✅ deduct_balance 返回: success={success}, message={message}", flush=True)
|
||||
logger.info(f"🔍 deduct_balance 返回: success={success}, message={message}")
|
||||
if success:
|
||||
logger.info(
|
||||
f"💰 Agent API调用计费成功: agent={agent_name}, "
|
||||
f"request_id={callback_data.requestId}, duration={duration_seconds}秒, "
|
||||
f"tools={callback_data.toolsUsed}, cost={cost}"
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
f"⚠️ Agent API调用扣款失败: agent={agent_name}, "
|
||||
f"request_id={callback_data.requestId}, cost={cost}, 原因={message}"
|
||||
)
|
||||
else:
|
||||
# ⚠️ 没有现有记录，创建新记录（异常情况的兜底）
|
||||
logger.warning(
|
||||
f"⚠️ 未找到运行中的计费记录，将创建新记录: "
|
||||
f"agent={callback_data.agentName}, user={callback_data.userId}"
|
||||
)
|
||||
|
||||
# ⚠️ 创建新记录（异常情况的兜底）
|
||||
# ⚠️ 不在此处扣款，由周期计费（periodic_billing.py）统一处理扣款
|
||||
billing_record = AgentBillingRecord(
|
||||
user_id=callback_data.userId,
|
||||
channel_id=user.channel_id if user.channel_id else None,
|
||||
agent_name=callback_data.agentName,
|
||||
agent_type=agent_type,
|
||||
is_platform_agent=is_platform_agent,
|
||||
duration_seconds=duration_seconds,
|
||||
eu_consumed=eu_consumed,
|
||||
cost=float(new_cost),
|
||||
start_time=start_time or datetime.utcnow(),
|
||||
end_time=end_time or datetime.utcnow(),
|
||||
period_start=start_time or datetime.utcnow(),
|
||||
period_end=end_time or datetime.utcnow(),
|
||||
tools_used=callback_data.toolsUsed,
|
||||
request_id=callback_data.requestId,
|
||||
)
|
||||
|
||||
db.add(billing_record)
|
||||
|
||||
logger.info(
|
||||
f"📝 创建Agent计费记录（不扣款）: agent={callback_data.agentName}, "
|
||||
f"最终成本={new_cost}，扣款由周期计费处理"
|
||||
f"📝 创建Agent API调用记录（费用为0）: agent={agent_name}, "
|
||||
f"request_id={callback_data.requestId}"
|
||||
)
|
||||
|
||||
print(f"\n🔥 准备 commit...", flush=True)
|
||||
await db.commit()
|
||||
print(f"🔥 commit 完成！\n", flush=True)
|
||||
await db.refresh(billing_record)
|
||||
|
||||
logger.info(
|
||||
f"✅ Agent 计费记录{'更新' if existing_record else '创建'}成功: "
|
||||
f"agent={callback_data.agentName}, duration={duration_seconds}秒, "
|
||||
f"EU={eu_consumed}, cost={new_cost}, tools={callback_data.toolsUsed}"
|
||||
f"✅ Agent API调用计费记录创建成功: "
|
||||
f"agent={agent_name}, request_id={callback_data.requestId}, "
|
||||
f"duration={duration_seconds}秒, EU={eu_consumed}, cost={cost}, "
|
||||
f"tools={callback_data.toolsUsed}"
|
||||
)
|
||||
|
||||
return AgentManagerCallbackResponse(
|
||||
success=True,
|
||||
message=f"Agent 计费记录{'更新' if existing_record else '创建'}成功",
|
||||
message="Agent API调用计费记录创建成功",
|
||||
recordId=str(billing_record.id)
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
|
||||
@@ -794,10 +794,10 @@ async def get_billing_overview(
|
||||
"percentage": round(cost_comparison, 1),
|
||||
"direction": "up" if cost_comparison > 0 else "down" if cost_comparison < 0 else "stable"
|
||||
},
|
||||
# 余额信息
|
||||
# 余额信息（1 美元 = 1 EU，cash 和 eu 返回相同的值）
|
||||
"balance": {
|
||||
"eu": float(balance.eu_balance),
|
||||
"cash": 0 # cash_balance 字段已移除，保留字段以兼容前端
|
||||
"cash": float(balance.eu_balance) # 1 美元 = 1 EU，保持前端兼容
|
||||
},
|
||||
# EU消费历史（图表数据）
|
||||
"euHistory": eu_history,
|
||||
@@ -1481,6 +1481,7 @@ async def deploy_agent(
|
||||
# 记录计费（包含访问信息）
|
||||
access_info = result.access_info or {}
|
||||
billing_record = AgentBillingRecord(
|
||||
record_type="vm_runtime", # VM运行时间计费
|
||||
user_id=user_id,
|
||||
channel_id=channel_id,
|
||||
agent_type=quota.template_name,
|
||||
@@ -2602,6 +2603,7 @@ async def deploy_platform_agent(
|
||||
# 记录计费（包含访问信息）
|
||||
access_info = result.access_info or {}
|
||||
billing_record = AgentBillingRecord(
|
||||
record_type="vm_runtime", # VM运行时间计费
|
||||
user_id=user_id,
|
||||
channel_id=channel_id,
|
||||
agent_type=req.agentType,
|
||||
@@ -2789,6 +2791,7 @@ async def use_platform_agent(
|
||||
# 记录计费（包含访问信息）
|
||||
access_info = result.access_info or {}
|
||||
billing_record = AgentBillingRecord(
|
||||
record_type="vm_runtime", # VM运行时间计费
|
||||
user_id=user_id,
|
||||
channel_id=channel_id,
|
||||
agent_type=req.agentType,
|
||||
@@ -3386,6 +3389,7 @@ async def create_custom_agent(
|
||||
# 记录计费（包含访问信息）
|
||||
access_info = result.access_info or {}
|
||||
billing_record = AgentBillingRecord(
|
||||
record_type="vm_runtime", # VM运行时间计费
|
||||
user_id=user_id,
|
||||
channel_id=channel_id,
|
||||
agent_type=template_name, # template 默认和 name 一致
|
||||
@@ -3470,26 +3474,18 @@ async def delete_custom_agent(
|
||||
|
||||
user_id = principal.get("user_id")
|
||||
|
||||
# 查找计费记录
|
||||
# 查找计费记录（只查询vm_runtime类型）
|
||||
billing_result = await db.execute(
|
||||
select(AgentBillingRecord).where(
|
||||
and_(
|
||||
AgentBillingRecord.agent_name == name,
|
||||
AgentBillingRecord.user_id == user_id,
|
||||
AgentBillingRecord.is_platform_agent == False,
|
||||
AgentBillingRecord.end_time == None
|
||||
AgentBillingRecord.end_time == None,
|
||||
AgentBillingRecord.record_type == "vm_runtime" # 只查询VM运行时间计费记录
|
||||
)
|
||||
)
|
||||
)
|
||||
billing_record = billing_result.scalar_one_or_none()
|
||||
|
||||
if not billing_record:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"未找到 Agent {name} 或该 Agent 不属于您"
|
||||
)
|
||||
|
||||
# ✅ Step 1: 先删除 Pod（如果失败，整个操作终止，不修改数据库）
|
||||
# 从 namespace 提取完整的 agent 名称（namespace 格式如 "agent-123123123-674ddd"）
|
||||
agent_full_name = name
|
||||
if billing_record.namespace and billing_record.namespace.startswith("agent-"):
|
||||
@@ -3532,7 +3528,8 @@ async def delete_custom_agent(
|
||||
# 扣除用户余额
|
||||
success, message = await deduct_balance(
|
||||
user_id, cost, db,
|
||||
f"自定义Agent使用: {name} (运行{int(duration)}秒)"
|
||||
f"自定义Agent使用: {name} (运行{int(duration)}秒)",
|
||||
auto_commit=False # 🔧 修复：与billing_record和quota更新在同一事务中
|
||||
)
|
||||
if not success:
|
||||
logger.warning(f"扣款失败: {message}, 用户: {user_id}, Agent: {name}")
|
||||
@@ -3588,26 +3585,18 @@ async def scale_custom_agent_api(
|
||||
|
||||
user_id = principal.get("user_id")
|
||||
|
||||
# 查找计费记录
|
||||
# 查找计费记录（只查询vm_runtime类型）
|
||||
billing_result = await db.execute(
|
||||
select(AgentBillingRecord).where(
|
||||
and_(
|
||||
AgentBillingRecord.agent_name == name,
|
||||
AgentBillingRecord.user_id == user_id,
|
||||
AgentBillingRecord.is_platform_agent == False,
|
||||
AgentBillingRecord.end_time == None
|
||||
AgentBillingRecord.end_time == None,
|
||||
AgentBillingRecord.record_type == "vm_runtime" # 只查询VM运行时间计费记录
|
||||
)
|
||||
)
|
||||
)
|
||||
billing_record = billing_result.scalar_one_or_none()
|
||||
|
||||
if not billing_record:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"未找到 Agent {name} 或该 Agent 不属于您"
|
||||
)
|
||||
|
||||
# 获取用户配额（使用行锁防止并发问题）
|
||||
quota_result = await db.execute(
|
||||
select(TenantCustomAgentQuota)
|
||||
.where(TenantCustomAgentQuota.tenant_id == user_id)
|
||||
|
||||
@@ -1,420 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
计费系统健康检查脚本
|
||||
|
||||
功能：
|
||||
1. 检测假用量问题（配额数据与实际运行Agent不一致）
|
||||
2. 检查余额不足用户
|
||||
3. 测试余额不足自动停止功能
|
||||
4. 生成详细健康报告
|
||||
|
||||
使用方法：
|
||||
在mcp-server容器内运行：
|
||||
|
||||
# 完整检查
|
||||
python check_billing_health.py
|
||||
|
||||
# 只检查假用量
|
||||
python check_billing_health.py --check-quota-only
|
||||
|
||||
# 只检查余额
|
||||
python check_billing_health.py --check-balance-only
|
||||
|
||||
# 测试自动停止（需要指定用户ID）
|
||||
python check_billing_health.py --test-auto-stop <user_id>
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Any, Optional
|
||||
from decimal import Decimal
|
||||
|
||||
from sqlalchemy import select, func, and_
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from database import get_db
|
||||
from models import (
|
||||
Agent,
|
||||
TenantCustomAgentQuota,
|
||||
AgentBillingRecord,
|
||||
Balance,
|
||||
User
|
||||
)
|
||||
|
||||
# 配置日志
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class BillingHealthChecker:
|
||||
"""计费系统健康检查器"""
|
||||
|
||||
def __init__(self):
|
||||
self.report_file = Path("/app/logs/billing_health_report.json")
|
||||
self.report_file.parent.mkdir(exist_ok=True)
|
||||
|
||||
async def check_quota_consistency(self, db: AsyncSession) -> Dict[str, Any]:
|
||||
"""
|
||||
检查配额一致性（假用量检测）
|
||||
|
||||
对比：
|
||||
- 配额表中的使用量
|
||||
- 实际运行的Agent数量和资源
|
||||
"""
|
||||
logger.info("=" * 60)
|
||||
logger.info("🔍 开始检查配额一致性（假用量检测）")
|
||||
logger.info("=" * 60)
|
||||
|
||||
# 获取所有配额记录
|
||||
result = await db.execute(select(TenantCustomAgentQuota))
|
||||
quotas = result.scalars().all()
|
||||
|
||||
inconsistencies = []
|
||||
total_fake_cpu = 0.0
|
||||
total_fake_memory = 0.0
|
||||
total_fake_agents = 0
|
||||
|
||||
for quota in quotas:
|
||||
tenant_id = str(quota.tenant_id)
|
||||
|
||||
# 查询配额使用量
|
||||
quota_usage = {
|
||||
'cpu_used': float(quota.cpu_used or 0),
|
||||
'memory_used': float(quota.memory_used or 0),
|
||||
'agent_count': quota.agent_count or 0
|
||||
}
|
||||
|
||||
# 查询实际运行的Agent
|
||||
real_agents_result = await db.execute(
|
||||
select(Agent)
|
||||
.where(Agent.owner_id == tenant_id)
|
||||
.where(Agent.type == 'custom')
|
||||
.where(Agent.status == 'active')
|
||||
)
|
||||
real_agents = real_agents_result.scalars().all()
|
||||
|
||||
# 计算实际使用量
|
||||
real_usage = {
|
||||
'cpu_used': sum(float(a.cpu or 0) for a in real_agents),
|
||||
'memory_used': sum(float(a.memory or 0) for a in real_agents),
|
||||
'agent_count': len(real_agents)
|
||||
}
|
||||
|
||||
# 计算差异
|
||||
cpu_diff = quota_usage['cpu_used'] - real_usage['cpu_used']
|
||||
memory_diff = quota_usage['memory_used'] - real_usage['memory_used']
|
||||
agent_diff = quota_usage['agent_count'] - real_usage['agent_count']
|
||||
|
||||
# 允许小误差（0.01核，0.01GB）
|
||||
has_inconsistency = (
|
||||
abs(cpu_diff) > 0.01 or
|
||||
abs(memory_diff) > 0.01 or
|
||||
agent_diff != 0
|
||||
)
|
||||
|
||||
if has_inconsistency:
|
||||
inconsistency = {
|
||||
'tenant_id': tenant_id,
|
||||
'quota_usage': quota_usage,
|
||||
'real_usage': real_usage,
|
||||
'diff': {
|
||||
'cpu': cpu_diff,
|
||||
'memory': memory_diff,
|
||||
'agent_count': agent_diff
|
||||
},
|
||||
'severity': 'high' if (cpu_diff > 1 or memory_diff > 1 or agent_diff > 2) else 'medium'
|
||||
}
|
||||
inconsistencies.append(inconsistency)
|
||||
|
||||
total_fake_cpu += cpu_diff
|
||||
total_fake_memory += memory_diff
|
||||
total_fake_agents += agent_diff
|
||||
|
||||
logger.warning(
|
||||
f"⚠️ 检测到配额不一致: 租户={tenant_id}\n"
|
||||
f" 配额显示: CPU={quota_usage['cpu_used']:.2f}核, "
|
||||
f"内存={quota_usage['memory_used']:.2f}GB, Agent={quota_usage['agent_count']}个\n"
|
||||
f" 实际运行: CPU={real_usage['cpu_used']:.2f}核, "
|
||||
f"内存={real_usage['memory_used']:.2f}GB, Agent={real_usage['agent_count']}个\n"
|
||||
f" 假用量: CPU={cpu_diff:.2f}核, "
|
||||
f"内存={memory_diff:.2f}GB, Agent={agent_diff}个"
|
||||
)
|
||||
|
||||
summary = {
|
||||
'total_tenants': len(quotas),
|
||||
'inconsistent_tenants': len(inconsistencies),
|
||||
'consistency_rate': (len(quotas) - len(inconsistencies)) / len(quotas) * 100 if quotas else 100,
|
||||
'total_fake_cpu': total_fake_cpu,
|
||||
'total_fake_memory': total_fake_memory,
|
||||
'total_fake_agents': total_fake_agents,
|
||||
'details': inconsistencies
|
||||
}
|
||||
|
||||
if inconsistencies:
|
||||
logger.error(
|
||||
f"❌ 发现 {len(inconsistencies)} 个租户存在配额不一致问题！\n"
|
||||
f" 假用量汇总: CPU={total_fake_cpu:.2f}核, "
|
||||
f"内存={total_fake_memory:.2f}GB, Agent={total_fake_agents}个\n"
|
||||
f" 建议运行: python fix_fake_quota.py"
|
||||
)
|
||||
else:
|
||||
logger.info("✅ 所有租户配额数据一致，无假用量问题")
|
||||
|
||||
return summary
|
||||
|
||||
async def check_balance_status(self, db: AsyncSession) -> Dict[str, Any]:
|
||||
"""
|
||||
检查用户余额状态
|
||||
|
||||
识别：
|
||||
- 余额不足的用户
|
||||
- 透支用户
|
||||
- 有运行Agent但余额不足的用户（风险）
|
||||
"""
|
||||
logger.info("=" * 60)
|
||||
logger.info("💰 开始检查用户余额状态")
|
||||
logger.info("=" * 60)
|
||||
|
||||
# 查询所有用户及其余额
|
||||
result = await db.execute(
|
||||
select(User, Balance)
|
||||
.outerjoin(Balance, User.id == Balance.user_id)
|
||||
)
|
||||
users_data = result.all()
|
||||
|
||||
low_balance_users = []
|
||||
overdraft_users = []
|
||||
risky_users = [] # 有运行Agent但余额不足
|
||||
|
||||
for user, balance in users_data:
|
||||
user_id = str(user.id)
|
||||
eu_balance = float(balance.eu_balance) if balance else 0.0
|
||||
credit_limit = float(user.credit_limit or 0)
|
||||
available = eu_balance + credit_limit
|
||||
|
||||
# 检查是否有运行中的Agent
|
||||
running_agents_result = await db.execute(
|
||||
select(func.count(AgentBillingRecord.id))
|
||||
.where(AgentBillingRecord.user_id == user_id)
|
||||
.where(AgentBillingRecord.end_time == None)
|
||||
)
|
||||
running_agent_count = running_agents_result.scalar() or 0
|
||||
|
||||
user_info = {
|
||||
'user_id': user_id,
|
||||
'email': user.email,
|
||||
'eu_balance': eu_balance,
|
||||
'credit_limit': credit_limit,
|
||||
'available_balance': available,
|
||||
'running_agents': running_agent_count
|
||||
}
|
||||
|
||||
# 透支（可用余额为负）
|
||||
if available < 0:
|
||||
overdraft_users.append(user_info)
|
||||
logger.error(
|
||||
f"💥 透支用户: {user.email} (ID: {user_id})\n"
|
||||
f" 账户余额: {eu_balance:.2f} EU\n"
|
||||
f" 授信额度: {credit_limit:.2f} EU\n"
|
||||
f" 可用余额: {available:.2f} EU\n"
|
||||
f" 运行Agent数: {running_agent_count}"
|
||||
)
|
||||
|
||||
# 余额不足（低于10 EU）
|
||||
elif available < 10:
|
||||
low_balance_users.append(user_info)
|
||||
logger.warning(
|
||||
f"⚠️ 余额不足: {user.email} (ID: {user_id})\n"
|
||||
f" 可用余额: {available:.2f} EU\n"
|
||||
f" 运行Agent数: {running_agent_count}"
|
||||
)
|
||||
|
||||
# 风险用户：有运行Agent但余额很低（< 5 EU）
|
||||
if running_agent_count > 0 and available < 5:
|
||||
risky_users.append(user_info)
|
||||
logger.warning(
|
||||
f"🚨 风险用户: {user.email} (ID: {user_id})\n"
|
||||
f" 有 {running_agent_count} 个Agent运行但余额仅剩 {available:.2f} EU"
|
||||
)
|
||||
|
||||
summary = {
|
||||
'total_users': len(users_data),
|
||||
'overdraft_users': len(overdraft_users),
|
||||
'low_balance_users': len(low_balance_users),
|
||||
'risky_users': len(risky_users),
|
||||
'overdraft_details': overdraft_users,
|
||||
'low_balance_details': low_balance_users,
|
||||
'risky_users_details': risky_users
|
||||
}
|
||||
|
||||
logger.info(
|
||||
f"\n📊 余额状态汇总:\n"
|
||||
f" 总用户数: {len(users_data)}\n"
|
||||
f" 透支用户: {len(overdraft_users)}\n"
|
||||
f" 余额不足: {len(low_balance_users)}\n"
|
||||
f" 风险用户: {len(risky_users)}"
|
||||
)
|
||||
|
||||
return summary
|
||||
|
||||
async def test_auto_stop(self, db: AsyncSession, user_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
测试余额不足自动停止功能
|
||||
|
||||
模拟周期计费检测到余额不足时的行为
|
||||
"""
|
||||
logger.info("=" * 60)
|
||||
logger.info(f"🧪 测试自动停止功能: user_id={user_id}")
|
||||
logger.info("=" * 60)
|
||||
|
||||
from app.billing import get_available_balance
|
||||
from app.periodic_billing import stop_user_agents
|
||||
|
||||
# 检查用户余额
|
||||
balance, credit_limit, available = await get_available_balance(user_id, db)
|
||||
|
||||
logger.info(
|
||||
f"用户余额状态:\n"
|
||||
f" 账户余额: {balance:.2f} EU\n"
|
||||
f" 授信额度: {credit_limit:.2f} EU\n"
|
||||
f" 可用余额: {available:.2f} EU"
|
||||
)
|
||||
|
||||
# 检查运行中的Agent
|
||||
running_agents_result = await db.execute(
|
||||
select(AgentBillingRecord)
|
||||
.where(AgentBillingRecord.user_id == user_id)
|
||||
.where(AgentBillingRecord.end_time == None)
|
||||
)
|
||||
running_agents = running_agents_result.scalars().all()
|
||||
|
||||
logger.info(f"运行中的Agent数量: {len(running_agents)}")
|
||||
for agent in running_agents:
|
||||
logger.info(f" - {agent.agent_name} (启动时间: {agent.start_time})")
|
||||
|
||||
result = {
|
||||
'user_id': user_id,
|
||||
'balance': float(balance),
|
||||
'credit_limit': float(credit_limit),
|
||||
'available': float(available),
|
||||
'running_agents_before': len(running_agents),
|
||||
'stopped_agents': [],
|
||||
'action_taken': 'none'
|
||||
}
|
||||
|
||||
# 如果余额不足，执行停止
|
||||
if available < 0:
|
||||
logger.warning(f"⚠️ 可用余额为负 ({available:.2f} EU)，将停止所有Agent...")
|
||||
|
||||
stopped = await stop_user_agents(user_id, db)
|
||||
result['stopped_agents'] = stopped
|
||||
result['action_taken'] = 'stopped'
|
||||
|
||||
logger.info(
|
||||
f"✅ 已停止 {len(stopped)} 个Agent:\n" +
|
||||
"\n".join(f" - {name}" for name in stopped)
|
||||
)
|
||||
else:
|
||||
logger.info(f"✅ 余额充足 ({available:.2f} EU)，无需停止Agent")
|
||||
result['action_taken'] = 'no_action_needed'
|
||||
|
||||
return result
|
||||
|
||||
async def generate_full_report(self, db: AsyncSession) -> Dict[str, Any]:
|
||||
"""生成完整的健康检查报告"""
|
||||
logger.info("\n" + "=" * 60)
|
||||
logger.info("📋 生成计费系统健康检查报告")
|
||||
logger.info("=" * 60 + "\n")
|
||||
|
||||
report = {
|
||||
'check_time': datetime.utcnow().isoformat(),
|
||||
'quota_consistency': await self.check_quota_consistency(db),
|
||||
'balance_status': await self.check_balance_status(db)
|
||||
}
|
||||
|
||||
# 生成健康评分
|
||||
quota_ok = report['quota_consistency']['inconsistent_tenants'] == 0
|
||||
no_overdraft = report['balance_status']['overdraft_users'] == 0
|
||||
few_risky = report['balance_status']['risky_users'] < 5
|
||||
|
||||
health_score = 0
|
||||
if quota_ok:
|
||||
health_score += 40
|
||||
if no_overdraft:
|
||||
health_score += 40
|
||||
if few_risky:
|
||||
health_score += 20
|
||||
|
||||
report['health_score'] = health_score
|
||||
report['health_status'] = (
|
||||
'healthy' if health_score >= 90 else
|
||||
'warning' if health_score >= 70 else
|
||||
'critical'
|
||||
)
|
||||
|
||||
# 生成建议
|
||||
recommendations = []
|
||||
if not quota_ok:
|
||||
recommendations.append("运行 fix_fake_quota.py 修复配额不一致问题")
|
||||
if not no_overdraft:
|
||||
recommendations.append(f"有 {report['balance_status']['overdraft_users']} 个用户透支，建议停止其Agent或充值")
|
||||
if not few_risky:
|
||||
recommendations.append(f"有 {report['balance_status']['risky_users']} 个风险用户，建议提醒充值")
|
||||
|
||||
report['recommendations'] = recommendations
|
||||
|
||||
# 保存报告
|
||||
with open(self.report_file, 'w', encoding='utf-8') as f:
|
||||
json.dump(report, f, indent=2, ensure_ascii=False)
|
||||
|
||||
logger.info(f"\n{'=' * 60}")
|
||||
logger.info(f"📊 健康检查完成")
|
||||
logger.info(f"{'=' * 60}")
|
||||
logger.info(f"健康评分: {health_score}/100 ({report['health_status'].upper()})")
|
||||
logger.info(f"报告保存: {self.report_file}")
|
||||
|
||||
if recommendations:
|
||||
logger.info(f"\n💡 建议:")
|
||||
for i, rec in enumerate(recommendations, 1):
|
||||
logger.info(f" {i}. {rec}")
|
||||
|
||||
return report
|
||||
|
||||
|
||||
async def main():
|
||||
"""主函数"""
|
||||
parser = argparse.ArgumentParser(description='计费系统健康检查')
|
||||
parser.add_argument('--check-quota-only', action='store_true', help='只检查配额一致性')
|
||||
parser.add_argument('--check-balance-only', action='store_true', help='只检查余额状态')
|
||||
parser.add_argument('--test-auto-stop', type=str, metavar='USER_ID', help='测试自动停止功能（指定用户ID）')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
checker = BillingHealthChecker()
|
||||
|
||||
async for db in get_db():
|
||||
try:
|
||||
if args.check_quota_only:
|
||||
await checker.check_quota_consistency(db)
|
||||
elif args.check_balance_only:
|
||||
await checker.check_balance_status(db)
|
||||
elif args.test_auto_stop:
|
||||
await checker.test_auto_stop(db, args.test_auto_stop)
|
||||
else:
|
||||
await checker.generate_full_report(db)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"执行失败: {str(e)}", exc_info=True)
|
||||
finally:
|
||||
break
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -1,117 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
检查配额数据是否正确写入数据库
|
||||
"""
|
||||
import asyncio
|
||||
import sys
|
||||
from sqlalchemy import select, text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
# 添加项目路径
|
||||
sys.path.insert(0, '/app')
|
||||
|
||||
from database import engine
|
||||
from models import TenantCustomAgentQuota, ChannelCustomAgentQuota, User
|
||||
|
||||
async def check_quota_data():
|
||||
"""检查配额数据"""
|
||||
|
||||
async with AsyncSession(engine) as db:
|
||||
print("=" * 80)
|
||||
print("检查租户自定义 Agent 配额数据")
|
||||
print("=" * 80)
|
||||
|
||||
# 查询所有租户配额
|
||||
result = await db.execute(
|
||||
select(TenantCustomAgentQuota)
|
||||
)
|
||||
tenant_quotas = result.scalars().all()
|
||||
|
||||
print(f"\n找到 {len(tenant_quotas)} 条租户配额记录:\n")
|
||||
|
||||
for quota in tenant_quotas:
|
||||
# 获取租户信息
|
||||
user_result = await db.execute(
|
||||
select(User).where(User.id == quota.tenant_id)
|
||||
)
|
||||
user = user_result.scalar_one_or_none()
|
||||
|
||||
print(f"租户ID: {quota.tenant_id}")
|
||||
print(f" - 租户名称: {user.name if user else 'N/A'}")
|
||||
print(f" - 租户邮箱: {user.email if user else 'N/A'}")
|
||||
print(f" - CPU 配额: {quota.cpu_quota} 核")
|
||||
print(f" - 内存配额: {quota.memory_quota} GB")
|
||||
print(f" - CPU 已使用: {quota.cpu_used} 核")
|
||||
print(f" - 内存已使用: {quota.memory_used} GB")
|
||||
print(f" - Agent 数量: {quota.agent_count}")
|
||||
print(f" - CPU 剩余: {float(quota.cpu_quota) - float(quota.cpu_used)} 核")
|
||||
print(f" - 内存剩余: {float(quota.memory_quota) - float(quota.memory_used)} GB")
|
||||
print()
|
||||
|
||||
print("=" * 80)
|
||||
print("检查渠道自定义 Agent 配额数据")
|
||||
print("=" * 80)
|
||||
|
||||
# 查询所有渠道配额
|
||||
result = await db.execute(
|
||||
select(ChannelCustomAgentQuota)
|
||||
)
|
||||
channel_quotas = result.scalars().all()
|
||||
|
||||
print(f"\n找到 {len(channel_quotas)} 条渠道配额记录:\n")
|
||||
|
||||
for quota in channel_quotas:
|
||||
print(f"渠道ID: {quota.channel_id}")
|
||||
print(f" - CPU 配额: {quota.cpu_quota} 核")
|
||||
print(f" - 内存配额: {quota.memory_quota} GB")
|
||||
print(f" - CPU 已分配: {quota.cpu_allocated} 核")
|
||||
print(f" - 内存已分配: {quota.memory_allocated} GB")
|
||||
print(f" - CPU 剩余: {float(quota.cpu_quota) - float(quota.cpu_allocated)} 核")
|
||||
print(f" - 内存剩余: {float(quota.memory_quota) - float(quota.memory_allocated)} GB")
|
||||
print()
|
||||
|
||||
# 特别检查指定的租户ID
|
||||
target_tenant_id = "b00a7b8e-9e8b-463d-9593-a3b4d0006778"
|
||||
print("=" * 80)
|
||||
print(f"检查目标租户: {target_tenant_id}")
|
||||
print("=" * 80)
|
||||
|
||||
result = await db.execute(
|
||||
select(TenantCustomAgentQuota).where(
|
||||
TenantCustomAgentQuota.tenant_id == target_tenant_id
|
||||
)
|
||||
)
|
||||
target_quota = result.scalar_one_or_none()
|
||||
|
||||
if target_quota:
|
||||
print(f"\n找到配额记录:")
|
||||
print(f" - CPU 配额: {target_quota.cpu_quota} 核")
|
||||
print(f" - 内存配额: {target_quota.memory_quota} GB")
|
||||
print(f" - CPU 已使用: {target_quota.cpu_used} 核")
|
||||
print(f" - 内存已使用: {target_quota.memory_used} GB")
|
||||
print(f" - Agent 数量: {target_quota.agent_count}")
|
||||
print(f" - CPU 剩余: {float(target_quota.cpu_quota) - float(target_quota.cpu_used)} 核")
|
||||
print(f" - 内存剩余: {float(target_quota.memory_quota) - float(target_quota.memory_used)} GB")
|
||||
else:
|
||||
print(f"\n❌ 未找到该租户的配额记录！")
|
||||
print(f"这就是为什么会出现 '剩余: 0.00 核' 的原因")
|
||||
|
||||
# 检查该租户是否存在
|
||||
user_result = await db.execute(
|
||||
select(User).where(User.id == target_tenant_id)
|
||||
)
|
||||
user = user_result.scalar_one_or_none()
|
||||
|
||||
if user:
|
||||
print(f"\n租户信息:")
|
||||
print(f" - 名称: {user.name}")
|
||||
print(f" - 邮箱: {user.email}")
|
||||
print(f" - 角色: {user.role}")
|
||||
print(f" - 渠道ID: {user.channel_id}")
|
||||
else:
|
||||
print(f"\n❌ 该租户不存在！")
|
||||
|
||||
await engine.dispose()
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(check_quota_data())
|
||||
@@ -1,177 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""修复Agent资源配置和配额记录
|
||||
|
||||
修复两个问题：
|
||||
1. 旧的自定义Agent缺少K8s格式的资源配置
|
||||
2. 租户配额使用量与实际Agent资源不匹配
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)) + '/../services/mcp-server')
|
||||
|
||||
from sqlalchemy import select, func
|
||||
from database import AsyncSessionLocal
|
||||
from models import Agent, TenantCustomAgentQuota
|
||||
import asyncio
|
||||
|
||||
def parse_cpu(cpu_str):
|
||||
"""解析 CPU 字符串为浮点数（核心数）"""
|
||||
if not cpu_str:
|
||||
return 0.0
|
||||
cpu_str = str(cpu_str)
|
||||
if cpu_str.endswith('m'):
|
||||
return float(cpu_str[:-1]) / 1000
|
||||
return float(cpu_str)
|
||||
|
||||
def parse_memory(mem_str):
|
||||
"""解析内存字符串为浮点数（GB）"""
|
||||
if not mem_str:
|
||||
return 0.0
|
||||
mem_str = str(mem_str)
|
||||
if mem_str.endswith('Gi'):
|
||||
return float(mem_str[:-2])
|
||||
elif mem_str.endswith('G'):
|
||||
return float(mem_str[:-1])
|
||||
elif mem_str.endswith('Mi'):
|
||||
return float(mem_str[:-2]) / 1024
|
||||
return float(mem_str)
|
||||
|
||||
async def fix_agent_resources():
|
||||
"""修复Agent资源配置"""
|
||||
async with AsyncSessionLocal() as db:
|
||||
# 查询所有自定义Agent
|
||||
result = await db.execute(
|
||||
select(Agent).where(Agent.type == 'custom')
|
||||
)
|
||||
agents = result.scalars().all()
|
||||
|
||||
print(f"找到 {len(agents)} 个自定义Agent")
|
||||
print("=" * 60)
|
||||
|
||||
fixed_count = 0
|
||||
for agent in agents:
|
||||
# 检查是否需要修复（cpu_limit 不匹配 cpu）
|
||||
current_cpu_limit = parse_cpu(agent.cpu_limit)
|
||||
expected_cpu = float(agent.cpu or 0)
|
||||
|
||||
current_memory_limit = parse_memory(agent.memory_limit)
|
||||
expected_memory = float(agent.memory or 0)
|
||||
|
||||
# 判断是否需要更新
|
||||
needs_update = False
|
||||
if abs(current_cpu_limit - expected_cpu) > 0.01:
|
||||
needs_update = True
|
||||
if abs(current_memory_limit - expected_memory) > 0.01:
|
||||
needs_update = True
|
||||
|
||||
if not needs_update:
|
||||
continue
|
||||
|
||||
print(f"\n🔧 修复Agent: {agent.name} ({agent.id})")
|
||||
print(f" 所有者: {agent.owner_id}")
|
||||
print(f" 当前 CPU: {agent.cpu} 核 -> limit: {agent.cpu_limit} ({current_cpu_limit} 核)")
|
||||
print(f" 当前 Memory: {agent.memory} GB -> limit: {agent.memory_limit} ({current_memory_limit} GB)")
|
||||
|
||||
# 更新 K8s 格式资源配置
|
||||
cpu_limit_k8s = f"{int(expected_cpu * 1000)}m"
|
||||
cpu_request_k8s = f"{max(100, int(expected_cpu * 100))}m"
|
||||
memory_limit_k8s = f"{expected_memory}Gi"
|
||||
memory_request_k8s = f"{max(0.128, expected_memory * 0.25):.3f}Gi"
|
||||
|
||||
agent.cpu_limit = cpu_limit_k8s
|
||||
agent.cpu_request = cpu_request_k8s
|
||||
agent.memory_limit = memory_limit_k8s
|
||||
agent.memory_request = memory_request_k8s
|
||||
|
||||
print(f" 新 CPU limit: {cpu_limit_k8s}, request: {cpu_request_k8s}")
|
||||
print(f" 新 Memory limit: {memory_limit_k8s}, request: {memory_request_k8s}")
|
||||
|
||||
fixed_count += 1
|
||||
|
||||
if fixed_count > 0:
|
||||
await db.commit()
|
||||
print("\n" + "=" * 60)
|
||||
print(f"✅ Agent资源配置修复完成，共修复 {fixed_count} 个Agent")
|
||||
else:
|
||||
print("\n" + "=" * 60)
|
||||
print("✅ 所有Agent资源配置正常，无需修复")
|
||||
|
||||
print("=" * 60)
|
||||
|
||||
async def recalculate_quotas():
|
||||
"""重新计算租户配额使用量"""
|
||||
async with AsyncSessionLocal() as db:
|
||||
# 查询所有租户配额记录
|
||||
result = await db.execute(select(TenantCustomAgentQuota))
|
||||
quotas = result.scalars().all()
|
||||
|
||||
print(f"\n找到 {len(quotas)} 个租户配额记录")
|
||||
print("=" * 60)
|
||||
|
||||
for quota in quotas:
|
||||
tenant_id = quota.tenant_id
|
||||
|
||||
# 查询该租户的所有Agent
|
||||
agent_result = await db.execute(
|
||||
select(Agent).where(
|
||||
Agent.owner_id == tenant_id,
|
||||
Agent.type == 'custom'
|
||||
)
|
||||
)
|
||||
agents = agent_result.scalars().all()
|
||||
|
||||
# 计算实际使用量
|
||||
total_cpu = 0.0
|
||||
total_memory = 0.0
|
||||
for agent in agents:
|
||||
total_cpu += float(agent.cpu or 0)
|
||||
total_memory += float(agent.memory or 0)
|
||||
|
||||
# 获取当前记录的使用量
|
||||
current_cpu = float(quota.cpu_used or 0)
|
||||
current_memory = float(quota.memory_used or 0)
|
||||
|
||||
# 检查是否需要更新
|
||||
if abs(current_cpu - total_cpu) < 0.01 and abs(current_memory - total_memory) < 0.01:
|
||||
continue
|
||||
|
||||
print(f"\n🔧 修复租户配额: {tenant_id}")
|
||||
print(f" Agent数量: {len(agents)}")
|
||||
print(f" 当前记录: CPU={current_cpu} 核, Memory={current_memory} GB")
|
||||
print(f" 实际使用: CPU={total_cpu} 核, Memory={total_memory} GB")
|
||||
|
||||
# 更新配额使用量
|
||||
quota.cpu_used = total_cpu
|
||||
quota.memory_used = total_memory
|
||||
quota.agent_count = len(agents)
|
||||
|
||||
print(f" ✅ 已更新配额使用量")
|
||||
|
||||
await db.commit()
|
||||
print("\n" + "=" * 60)
|
||||
print("✅ 配额使用量重新计算完成")
|
||||
print("=" * 60)
|
||||
|
||||
async def main():
|
||||
"""主流程"""
|
||||
print("=" * 60)
|
||||
print("修复Agent资源配置和配额记录")
|
||||
print("=" * 60)
|
||||
|
||||
# 1. 修复Agent资源配置
|
||||
await fix_agent_resources()
|
||||
|
||||
# 2. 重新计算配额使用量
|
||||
await recalculate_quotas()
|
||||
|
||||
print("\n✅ 所有修复完成！")
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
asyncio.run(main())
|
||||
except Exception as e:
|
||||
print(f"\n❌ 修复失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
sys.exit(1)
|
||||
@@ -1,91 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""修复已存在的渠道配额记录
|
||||
|
||||
对于已经使用 customAgentResources 分配过资源的渠道，
|
||||
但还没有 ChannelCustomAgentQuota 记录的，创建对应的记录。
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)) + '/../services/mcp-server')
|
||||
|
||||
from sqlalchemy import select
|
||||
from database import engine, AsyncSessionLocal
|
||||
from models import Channel, ChannelCustomAgentQuota
|
||||
import asyncio
|
||||
|
||||
async def fix_channel_quotas():
|
||||
"""修复渠道配额记录"""
|
||||
async with AsyncSessionLocal() as db:
|
||||
# 查询所有渠道
|
||||
result = await db.execute(select(Channel))
|
||||
channels = result.scalars().all()
|
||||
|
||||
print(f"找到 {len(channels)} 个渠道")
|
||||
print("=" * 60)
|
||||
|
||||
fixed_count = 0
|
||||
for channel in channels:
|
||||
# 检查是否有旧格式的配额（custom_agent_cpu 或 custom_agent_memory > 0）
|
||||
has_old_quota = (
|
||||
(channel.custom_agent_cpu and channel.custom_agent_cpu > 0) or
|
||||
(channel.custom_agent_memory and channel.custom_agent_memory > 0)
|
||||
)
|
||||
|
||||
if not has_old_quota:
|
||||
continue
|
||||
|
||||
# 检查是否已经有 ChannelCustomAgentQuota 记录
|
||||
quota_result = await db.execute(
|
||||
select(ChannelCustomAgentQuota).where(
|
||||
ChannelCustomAgentQuota.channel_id == channel.id
|
||||
)
|
||||
)
|
||||
existing_quota = quota_result.scalar_one_or_none()
|
||||
|
||||
if existing_quota:
|
||||
print(f"✅ 渠道 {channel.name} ({channel.id}) 已有配额记录，跳过")
|
||||
continue
|
||||
|
||||
# 创建新的配额记录
|
||||
cpu_quota = float(channel.custom_agent_cpu or 0)
|
||||
memory_quota = float(channel.custom_agent_memory or 0)
|
||||
|
||||
print(f"\n🔧 修复渠道 {channel.name} ({channel.id})")
|
||||
print(f" 旧格式配额: CPU={cpu_quota}核, Memory={memory_quota}GB")
|
||||
|
||||
# 同时更新新格式字段
|
||||
channel.custom_agent_cpu_quota = cpu_quota
|
||||
channel.custom_agent_memory_quota = memory_quota
|
||||
|
||||
# 创建配额记录
|
||||
new_quota = ChannelCustomAgentQuota(
|
||||
channel_id=channel.id,
|
||||
cpu_quota=cpu_quota,
|
||||
memory_quota=memory_quota,
|
||||
cpu_allocated=0,
|
||||
memory_allocated=0,
|
||||
)
|
||||
db.add(new_quota)
|
||||
|
||||
print(f" ✅ 创建 ChannelCustomAgentQuota 记录")
|
||||
fixed_count += 1
|
||||
|
||||
if fixed_count > 0:
|
||||
await db.commit()
|
||||
print("\n" + "=" * 60)
|
||||
print(f"✅ 修复完成，共修复 {fixed_count} 个渠道")
|
||||
else:
|
||||
print("\n" + "=" * 60)
|
||||
print("✅ 没有需要修复的渠道")
|
||||
|
||||
print("=" * 60)
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
asyncio.run(fix_channel_quotas())
|
||||
except Exception as e:
|
||||
print(f"\n❌ 修复失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
sys.exit(1)
|
||||
@@ -1,228 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
修复脚本：清理历史假用量数据
|
||||
|
||||
此脚本用于清理因旧接口（/api/user/tools/generate）产生的假用量数据。
|
||||
旧接口会在数据库中创建Agent记录并扣除配额，但不实际部署到K8s，导致quota表中记录了用量但实际没有Pod运行。
|
||||
|
||||
脚本逻辑：
|
||||
1. 备份当前quota数据到JSON文件
|
||||
2. 遍历所有租户的TenantCustomAgentQuota记录
|
||||
3. 对每个租户，重新统计agents表中type='custom'且status='active'的真实运行Agent
|
||||
4. 更新quota表的cpu_used、memory_used、agent_count为真实值
|
||||
5. 生成修复报告
|
||||
|
||||
使用方法：
|
||||
在mcp-server容器内运行：
|
||||
python fix_fake_quota.py
|
||||
|
||||
注意：此脚本会修改数据库，请务必先备份数据库！
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Any
|
||||
|
||||
from sqlalchemy import select, func
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from database import get_db
|
||||
from models import Agent, TenantCustomAgentQuota
|
||||
|
||||
# 配置日志
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class QuotaFixer:
|
||||
"""配额修复器"""
|
||||
|
||||
def __init__(self):
|
||||
self.backup_file = Path("/app/logs/quota_backup.json")
|
||||
self.report_file = Path("/app/logs/quota_fix_report.json")
|
||||
|
||||
async def backup_current_quota(self, db: AsyncSession) -> Dict[str, Any]:
|
||||
"""备份当前quota数据"""
|
||||
logger.info("开始备份当前quota数据...")
|
||||
|
||||
result = await db.execute(
|
||||
select(TenantCustomAgentQuota)
|
||||
)
|
||||
quotas = result.scalars().all()
|
||||
|
||||
backup_data = {
|
||||
"backup_time": datetime.utcnow().isoformat(),
|
||||
"quotas": []
|
||||
}
|
||||
|
||||
for quota in quotas:
|
||||
backup_data["quotas"].append({
|
||||
"tenant_id": str(quota.tenant_id),
|
||||
"cpu_quota": float(quota.cpu_quota or 0),
|
||||
"memory_quota": float(quota.memory_quota or 0),
|
||||
"cpu_used": float(quota.cpu_used or 0),
|
||||
"memory_used": float(quota.memory_used or 0),
|
||||
"agent_count": quota.agent_count or 0
|
||||
})
|
||||
|
||||
# 确保logs目录存在
|
||||
self.backup_file.parent.mkdir(exist_ok=True)
|
||||
|
||||
# 保存备份
|
||||
with open(self.backup_file, 'w', encoding='utf-8') as f:
|
||||
json.dump(backup_data, f, indent=2, ensure_ascii=False)
|
||||
|
||||
logger.info(f"备份完成，已保存到: {self.backup_file}")
|
||||
return backup_data
|
||||
|
||||
async def calculate_real_usage(self, db: AsyncSession, tenant_id: str) -> Dict[str, float]:
|
||||
"""计算租户的真实资源使用量
|
||||
|
||||
只统计type='custom'且status='active'的Agent
|
||||
"""
|
||||
# 查询该租户所有运行中的自定义Agent
|
||||
result = await db.execute(
|
||||
select(
|
||||
func.count(Agent.id).label('agent_count'),
|
||||
func.sum(Agent.cpu).label('total_cpu'),
|
||||
func.sum(Agent.memory).label('total_memory')
|
||||
)
|
||||
.where(Agent.owner_id == tenant_id)
|
||||
.where(Agent.type == 'custom')
|
||||
.where(Agent.status == 'active')
|
||||
)
|
||||
|
||||
row = result.first()
|
||||
|
||||
real_usage = {
|
||||
'agent_count': row.agent_count or 0,
|
||||
'cpu_used': float(row.total_cpu or 0),
|
||||
'memory_used': float(row.total_memory or 0)
|
||||
}
|
||||
|
||||
logger.info(f"租户 {tenant_id} 真实用量: {real_usage}")
|
||||
return real_usage
|
||||
|
||||
async def fix_tenant_quota(self, db: AsyncSession, quota: TenantCustomAgentQuota) -> Dict[str, Any]:
|
||||
"""修复单个租户的quota"""
|
||||
tenant_id = str(quota.tenant_id)
|
||||
|
||||
# 获取当前记录的用量
|
||||
current_usage = {
|
||||
'cpu_used': float(quota.cpu_used or 0),
|
||||
'memory_used': float(quota.memory_used or 0),
|
||||
'agent_count': quota.agent_count or 0
|
||||
}
|
||||
|
||||
# 计算真实用量
|
||||
real_usage = await self.calculate_real_usage(db, tenant_id)
|
||||
|
||||
# 计算差异
|
||||
diff = {
|
||||
'cpu_diff': real_usage['cpu_used'] - current_usage['cpu_used'],
|
||||
'memory_diff': real_usage['memory_used'] - current_usage['memory_used'],
|
||||
'count_diff': real_usage['agent_count'] - current_usage['agent_count']
|
||||
}
|
||||
|
||||
# 更新quota
|
||||
quota.cpu_used = real_usage['cpu_used']
|
||||
quota.memory_used = real_usage['memory_used']
|
||||
quota.agent_count = real_usage['agent_count']
|
||||
|
||||
await db.commit()
|
||||
await db.refresh(quota)
|
||||
|
||||
logger.info(f"租户 {tenant_id} quota已更新: {current_usage} -> {real_usage}")
|
||||
|
||||
return {
|
||||
'tenant_id': tenant_id,
|
||||
'before': current_usage,
|
||||
'after': real_usage,
|
||||
'diff': diff
|
||||
}
|
||||
|
||||
async def fix_all_quotas(self, db: AsyncSession) -> Dict[str, Any]:
|
||||
"""修复所有租户的quota"""
|
||||
logger.info("开始修复所有租户的quota...")
|
||||
|
||||
# 获取所有quota记录
|
||||
result = await db.execute(
|
||||
select(TenantCustomAgentQuota)
|
||||
)
|
||||
quotas = result.scalars().all()
|
||||
|
||||
logger.info(f"找到 {len(quotas)} 个租户的quota记录")
|
||||
|
||||
# 备份数据
|
||||
backup_data = await self.backup_current_quota(db)
|
||||
|
||||
# 修复报告
|
||||
fix_report = {
|
||||
"fix_time": datetime.utcnow().isoformat(),
|
||||
"total_tenants": len(quotas),
|
||||
"fixed_tenants": [],
|
||||
"summary": {
|
||||
"total_fake_cpu": 0.0,
|
||||
"total_fake_memory": 0.0,
|
||||
"total_fake_agents": 0
|
||||
}
|
||||
}
|
||||
|
||||
for quota in quotas:
|
||||
try:
|
||||
fix_result = await self.fix_tenant_quota(db, quota)
|
||||
fix_report["fixed_tenants"].append(fix_result)
|
||||
|
||||
# 累加假用量
|
||||
fix_report["summary"]["total_fake_cpu"] += fix_result["diff"]["cpu_diff"]
|
||||
fix_report["summary"]["total_fake_memory"] += fix_result["diff"]["memory_diff"]
|
||||
fix_report["summary"]["total_fake_agents"] += fix_result["diff"]["count_diff"]
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"修复租户 {quota.tenant_id} 失败: {str(e)}")
|
||||
fix_report["fixed_tenants"].append({
|
||||
"tenant_id": str(quota.tenant_id),
|
||||
"error": str(e)
|
||||
})
|
||||
|
||||
# 保存修复报告
|
||||
with open(self.report_file, 'w', encoding='utf-8') as f:
|
||||
json.dump(fix_report, f, indent=2, ensure_ascii=False)
|
||||
|
||||
logger.info(f"修复完成！报告已保存到: {self.report_file}")
|
||||
logger.info(f"总共清理假用量: CPU {fix_report['summary']['total_fake_cpu']:.2f}核, "
|
||||
f"内存 {fix_report['summary']['total_fake_memory']:.2f}GB, "
|
||||
f"Agent {fix_report['summary']['total_fake_agents']}个")
|
||||
|
||||
return fix_report
|
||||
|
||||
async def run(self):
|
||||
"""运行修复任务"""
|
||||
logger.info("开始执行配额修复任务...")
|
||||
|
||||
async for db in get_db():
|
||||
try:
|
||||
report = await self.fix_all_quotas(db)
|
||||
logger.info("配额修复任务完成！")
|
||||
return report
|
||||
except Exception as e:
|
||||
logger.error(f"修复任务失败: {str(e)}")
|
||||
raise
|
||||
finally:
|
||||
await db.close()
|
||||
|
||||
|
||||
async def main():
|
||||
"""主函数"""
|
||||
fixer = QuotaFixer()
|
||||
await fixer.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,52 @@
|
||||
-- Migration 024: 添加 record_type 字段区分 VM 运行计费和 API 调用计费
|
||||
--
|
||||
-- 背景：
|
||||
-- 原逻辑混淆了两种计费方式：
|
||||
-- 1. VM 运行时间计费：Agent 创建时创建记录（end_time=NULL），周期任务持续更新并增量扣款
|
||||
-- 2. API 调用计费：每次调用 Agent 时 Agent Manager 回调创建新记录，记录单次调用详情
|
||||
--
|
||||
-- 修改：
|
||||
-- 添加 record_type 字段：
|
||||
-- - "vm_runtime": VM 运行时间计费（一个 Agent = 一条记录，持续更新）
|
||||
-- - "api_call": API 调用计费（每次调用 = 一条新记录）
|
||||
--
|
||||
-- 影响：
|
||||
-- - 现有记录默认设为 "vm_runtime"（兼容旧逻辑）
|
||||
-- - Agent Manager 回调将创建 "api_call" 类型的新记录
|
||||
-- - 周期计费任务只更新 "vm_runtime" 记录
|
||||
|
||||
-- 1. 添加 record_type 字段
|
||||
ALTER TABLE agent_billing_records
|
||||
ADD COLUMN IF NOT EXISTS record_type VARCHAR(20) NOT NULL DEFAULT 'vm_runtime';
|
||||
|
||||
-- 2. 为现有记录设置类型（根据业务逻辑推断）
|
||||
-- 如果有 request_id 且 end_time 不为空，可能是 API 调用记录
|
||||
UPDATE agent_billing_records
|
||||
SET record_type = 'api_call'
|
||||
WHERE request_id IS NOT NULL
|
||||
AND request_id != ''
|
||||
AND end_time IS NOT NULL;
|
||||
|
||||
-- 其余记录保持为 vm_runtime（默认值已设置）
|
||||
|
||||
-- 3. 添加索引以优化查询性能
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_record_type
|
||||
ON agent_billing_records(record_type);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_vm_runtime
|
||||
ON agent_billing_records(record_type, end_time)
|
||||
WHERE record_type = 'vm_runtime' AND end_time IS NULL;
|
||||
|
||||
-- 4. 添加注释
|
||||
COMMENT ON COLUMN agent_billing_records.record_type IS '计费记录类型：vm_runtime=VM运行时间计费，api_call=API调用计费';
|
||||
|
||||
-- 验证结果
|
||||
-- SELECT
|
||||
-- record_type,
|
||||
-- COUNT(*) as count,
|
||||
-- COUNT(CASE WHEN end_time IS NULL THEN 1 END) as running_count,
|
||||
-- COUNT(CASE WHEN end_time IS NOT NULL THEN 1 END) as completed_count,
|
||||
-- COUNT(request_id) as with_request_id
|
||||
-- FROM agent_billing_records
|
||||
-- GROUP BY record_type
|
||||
-- ORDER BY record_type;
|
||||
@@ -0,0 +1,183 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Migration 021: 修复 agent_type 字段长度
|
||||
|
||||
问题：
|
||||
microsoft_learn_agent (22字符) 超出 VARCHAR(20) 限制
|
||||
|
||||
修改：
|
||||
将 agent_billing_records.agent_type 字段从 VARCHAR(20) 扩展到 VARCHAR(100)
|
||||
|
||||
使用方法：
|
||||
cd services/mcp-server
|
||||
python migrations/run_021_fix_agent_type_length.py
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
# 添加项目根目录到 Python 路径
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
|
||||
async def run_migration():
|
||||
"""执行迁移"""
|
||||
# 从环境变量获取数据库连接
|
||||
database_url = os.environ.get("DATABASE_URL")
|
||||
if not database_url:
|
||||
print("❌ 错误：未设置 DATABASE_URL 环境变量")
|
||||
return False
|
||||
|
||||
# 转换为异步 URL
|
||||
if database_url.startswith("postgresql://"):
|
||||
database_url = database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
||||
|
||||
# 移除 sslmode 参数（asyncpg 不支持）
|
||||
if "sslmode=" in database_url:
|
||||
import re
|
||||
database_url = re.sub(r'[?&]sslmode=[^&]*', '', database_url)
|
||||
database_url = database_url.rstrip('?&')
|
||||
|
||||
print(f"📦 连接数据库...")
|
||||
|
||||
# 创建异步引擎
|
||||
engine = create_async_engine(database_url, echo=False)
|
||||
async_session = sessionmaker(
|
||||
engine, class_=AsyncSession, expire_on_commit=False
|
||||
)
|
||||
|
||||
try:
|
||||
async with async_session() as session:
|
||||
print("✅ 数据库连接成功")
|
||||
print("\n" + "="*80)
|
||||
print("开始执行 Migration 021: 修复 agent_type 字段长度")
|
||||
print("="*80 + "\n")
|
||||
|
||||
# 步骤1: 检查当前字段类型
|
||||
print("📊 步骤1: 检查当前字段类型...")
|
||||
result = await session.execute(text("""
|
||||
SELECT
|
||||
column_name,
|
||||
data_type,
|
||||
character_maximum_length
|
||||
FROM information_schema.columns
|
||||
WHERE table_name = 'agent_billing_records'
|
||||
AND column_name = 'agent_type'
|
||||
"""))
|
||||
column_info = result.fetchone()
|
||||
|
||||
if column_info:
|
||||
print(f" 当前字段: {column_info[0]}")
|
||||
print(f" 数据类型: {column_info[1]}")
|
||||
print(f" 当前长度: {column_info[2]}")
|
||||
else:
|
||||
print(" ⚠️ 未找到 agent_type 字段！")
|
||||
return False
|
||||
|
||||
# 步骤2: 检查是否有超长数据
|
||||
print("\n📊 步骤2: 检查是否有超长的 agent_type 值...")
|
||||
result = await session.execute(text("""
|
||||
SELECT DISTINCT
|
||||
agent_type,
|
||||
LENGTH(agent_type) as len,
|
||||
COUNT(*) as count
|
||||
FROM agent_billing_records
|
||||
WHERE LENGTH(agent_type) > 20
|
||||
GROUP BY agent_type
|
||||
ORDER BY len DESC
|
||||
LIMIT 10
|
||||
"""))
|
||||
long_types = result.fetchall()
|
||||
|
||||
if long_types:
|
||||
print(f" ⚠️ 发现 {len(long_types)} 种超过20字符的 agent_type：")
|
||||
for agent_type, length, count in long_types:
|
||||
print(f" - {agent_type!r} (长度={length}, 数量={count})")
|
||||
else:
|
||||
print(" ✅ 没有发现超过20字符的 agent_type")
|
||||
|
||||
# 步骤3: 修改字段长度
|
||||
print("\n🔧 步骤3: 修改字段长度...")
|
||||
print(" 执行: ALTER TABLE agent_billing_records ALTER COLUMN agent_type TYPE VARCHAR(100)")
|
||||
|
||||
await session.execute(text("""
|
||||
ALTER TABLE agent_billing_records
|
||||
ALTER COLUMN agent_type TYPE VARCHAR(100)
|
||||
"""))
|
||||
|
||||
await session.commit()
|
||||
print(" ✅ 字段长度修改成功")
|
||||
|
||||
# 步骤4: 验证修改结果
|
||||
print("\n✅ 步骤4: 验证修改结果...")
|
||||
result = await session.execute(text("""
|
||||
SELECT
|
||||
column_name,
|
||||
data_type,
|
||||
character_maximum_length
|
||||
FROM information_schema.columns
|
||||
WHERE table_name = 'agent_billing_records'
|
||||
AND column_name = 'agent_type'
|
||||
"""))
|
||||
column_info = result.fetchone()
|
||||
|
||||
if column_info:
|
||||
print(f" 字段: {column_info[0]}")
|
||||
print(f" 数据类型: {column_info[1]}")
|
||||
print(f" 新长度: {column_info[2]}")
|
||||
|
||||
if column_info[2] == 100:
|
||||
print(" ✅ 验证通过：字段长度已成功更新为 100")
|
||||
else:
|
||||
print(f" ⚠️ 字段长度为 {column_info[2]}，期望为 100")
|
||||
|
||||
# 步骤5: 统计 agent_type 分布
|
||||
print("\n📊 步骤5: 统计 agent_type 长度分布...")
|
||||
result = await session.execute(text("""
|
||||
SELECT
|
||||
LENGTH(agent_type) as len,
|
||||
COUNT(*) as count
|
||||
FROM agent_billing_records
|
||||
GROUP BY LENGTH(agent_type)
|
||||
ORDER BY len DESC
|
||||
LIMIT 10
|
||||
"""))
|
||||
length_distribution = result.fetchall()
|
||||
|
||||
if length_distribution:
|
||||
print(" agent_type 长度分布（Top 10）：")
|
||||
for length, count in length_distribution:
|
||||
print(f" 长度 {length:3d}: {count:6d} 条记录")
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✅ Migration 021 执行完成！")
|
||||
print("="*80 + "\n")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n❌ 迁移执行失败：{str(e)}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print("""
|
||||
╔════════════════════════════════════════════════════════════════════════════╗
|
||||
║ Migration 021: 修复 agent_type 字段长度 ║
|
||||
║ ║
|
||||
║ 操作：将 agent_billing_records.agent_type 从 VARCHAR(20) 扩展到 VARCHAR(100) ║
|
||||
║ 风险：低（只扩展字段长度，不影响现有数据） ║
|
||||
╚════════════════════════════════════════════════════════════════════════════╝
|
||||
""")
|
||||
|
||||
success = asyncio.run(run_migration())
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -0,0 +1,158 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Migration 024: 添加 record_type 字段区分 VM 运行计费和 API 调用计费
|
||||
|
||||
背景：
|
||||
原逻辑混淆了两种计费方式：
|
||||
1. VM 运行时间计费：Agent 创建时创建记录（end_time=NULL），周期任务持续更新并增量扣款
|
||||
2. API 调用计费：每次调用 Agent 时 Agent Manager 回调创建新记录，记录单次调用详情
|
||||
|
||||
修改：
|
||||
添加 record_type 字段：
|
||||
- "vm_runtime": VM 运行时间计费（一个 Agent = 一条记录，持续更新）
|
||||
- "api_call": API 调用计费（每次调用 = 一条新记录）
|
||||
|
||||
使用方法：
|
||||
cd services/mcp-server
|
||||
python migrations/run_024_add_record_type.py
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
# 添加项目根目录到 Python 路径
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
|
||||
async def run_migration():
|
||||
"""执行迁移"""
|
||||
# 从环境变量获取数据库连接
|
||||
database_url = os.environ.get("DATABASE_URL")
|
||||
if not database_url:
|
||||
print("❌ 错误：未设置 DATABASE_URL 环境变量")
|
||||
return False
|
||||
|
||||
# 转换为异步 URL
|
||||
if database_url.startswith("postgresql://"):
|
||||
database_url = database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
||||
|
||||
# 移除 sslmode 参数（asyncpg 不支持）
|
||||
if "sslmode=" in database_url:
|
||||
import re
|
||||
database_url = re.sub(r'[?&]sslmode=[^&]*', '', database_url)
|
||||
database_url = database_url.rstrip('?&')
|
||||
|
||||
print(f"📦 连接数据库...")
|
||||
engine = create_async_engine(database_url, echo=False)
|
||||
async_session = sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
async with async_session() as session:
|
||||
try:
|
||||
# 1. 检查字段是否已存在
|
||||
print("\n🔍 检查 record_type 字段是否已存在...")
|
||||
result = await session.execute(text("""
|
||||
SELECT column_name
|
||||
FROM information_schema.columns
|
||||
WHERE table_name='agent_billing_records'
|
||||
AND column_name='record_type'
|
||||
"""))
|
||||
if result.fetchone():
|
||||
print("✅ record_type 字段已存在，跳过迁移")
|
||||
return True
|
||||
|
||||
# 2. 检查当前记录数
|
||||
print("\n📊 检查当前记录数...")
|
||||
result = await session.execute(text("""
|
||||
SELECT COUNT(*) as total FROM agent_billing_records
|
||||
"""))
|
||||
total_records = result.scalar()
|
||||
print(f" 总记录数: {total_records}")
|
||||
|
||||
# 3. 添加 record_type 字段
|
||||
print("\n🔄 添加 record_type 字段...")
|
||||
await session.execute(text("""
|
||||
ALTER TABLE agent_billing_records
|
||||
ADD COLUMN IF NOT EXISTS record_type VARCHAR(20) NOT NULL DEFAULT 'vm_runtime'
|
||||
"""))
|
||||
print("✅ record_type 字段添加成功")
|
||||
|
||||
# 4. 为现有记录设置类型（根据业务逻辑推断）
|
||||
print("\n🔄 推断并设置现有记录的类型...")
|
||||
result = await session.execute(text("""
|
||||
UPDATE agent_billing_records
|
||||
SET record_type = 'api_call'
|
||||
WHERE request_id IS NOT NULL
|
||||
AND request_id != ''
|
||||
AND end_time IS NOT NULL
|
||||
"""))
|
||||
api_call_count = result.rowcount
|
||||
print(f" 标记为 api_call: {api_call_count} 条（有 request_id 且已完成）")
|
||||
print(f" 保持为 vm_runtime: {total_records - api_call_count} 条")
|
||||
|
||||
# 5. 添加索引
|
||||
print("\n🔄 创建索引...")
|
||||
await session.execute(text("""
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_record_type
|
||||
ON agent_billing_records(record_type)
|
||||
"""))
|
||||
print("✅ 创建索引: idx_agent_billing_record_type")
|
||||
|
||||
await session.execute(text("""
|
||||
CREATE INDEX IF NOT EXISTS idx_agent_billing_vm_runtime
|
||||
ON agent_billing_records(record_type, end_time)
|
||||
WHERE record_type = 'vm_runtime' AND end_time IS NULL
|
||||
"""))
|
||||
print("✅ 创建部分索引: idx_agent_billing_vm_runtime")
|
||||
|
||||
# 6. 提交事务
|
||||
await session.commit()
|
||||
|
||||
# 7. 验证结果
|
||||
print("\n📊 验证迁移结果...")
|
||||
result = await session.execute(text("""
|
||||
SELECT
|
||||
record_type,
|
||||
COUNT(*) as count,
|
||||
COUNT(CASE WHEN end_time IS NULL THEN 1 END) as running_count,
|
||||
COUNT(CASE WHEN end_time IS NOT NULL THEN 1 END) as completed_count,
|
||||
COUNT(request_id) as with_request_id
|
||||
FROM agent_billing_records
|
||||
GROUP BY record_type
|
||||
ORDER BY record_type
|
||||
"""))
|
||||
|
||||
print("\n 记录类型统计:")
|
||||
print(" ┌─────────────┬───────┬──────────┬────────────┬────────────┐")
|
||||
print(" │ 记录类型 │ 总数 │ 运行中 │ 已完成 │ 有请求ID │")
|
||||
print(" ├─────────────┼───────┼──────────┼────────────┼────────────┤")
|
||||
for row in result:
|
||||
print(f" │ {row[0]:<11} │ {row[1]:5} │ {row[2]:8} │ {row[3]:10} │ {row[4]:10} │")
|
||||
print(" └─────────────┴───────┴──────────┴────────────┴────────────┘")
|
||||
|
||||
print("\n✅ 迁移成功完成！")
|
||||
print("\n💡 说明:")
|
||||
print(" - vm_runtime: VM 运行时间计费（周期任务持续更新）")
|
||||
print(" - api_call: API 调用计费（每次调用创建新记录）")
|
||||
print("\n🔧 后续步骤:")
|
||||
print(" 1. 修改 billing_webhook.py: Agent Manager 回调创建 api_call 记录")
|
||||
print(" 2. 修改 periodic_billing.py: 周期任务只更新 vm_runtime 记录")
|
||||
print(" 3. 修改 user.py: 创建/删除 Agent 时设置正确的 record_type")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
await session.rollback()
|
||||
print(f"\n❌ 迁移失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = asyncio.run(run_migration())
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -1187,6 +1187,9 @@ class AgentBillingRecord(BaseModel, Base):
|
||||
is_platform_agent = Column(Boolean, default=True) # 是否为平台 Agent
|
||||
template_name = Column(String(100), nullable=True) # 模板名称（可选）
|
||||
|
||||
# 计费记录类型：区分 VM 运行时间计费 和 API 调用计费
|
||||
record_type = Column(String(20), nullable=False, default="vm_runtime") # "vm_runtime" 或 "api_call"
|
||||
|
||||
# 使用量
|
||||
duration_seconds = Column(Integer, nullable=False, default=0) # 运行时长（秒）
|
||||
cpu_seconds = Column(sa.Float, default=0) # CPU 秒数
|
||||
|
||||
Reference in New Issue
Block a user