Initial commit
This commit is contained in:
@@ -0,0 +1,12 @@
|
||||
# CODEOWNERS — 代码所有权与 PR 审查责任绑定
|
||||
# 这些条目要求对应路径的改动必须经所有者 review。
|
||||
|
||||
# 默认:整个 Swarm 仓库
|
||||
* @Songhaoz666
|
||||
|
||||
# 文档 / 标准
|
||||
/docs/ @Songhaoz666
|
||||
/README.md @Songhaoz666
|
||||
|
||||
# Manager ↔ Swarm 契约相关(如本仓后续新增 docs/integration)
|
||||
/docs/integration/ @Songhaoz666
|
||||
@@ -0,0 +1,43 @@
|
||||
## 变更说明
|
||||
|
||||
<!-- 简述本次改动做了什么、为什么 -->
|
||||
|
||||
## 影响范围
|
||||
|
||||
- [ ] Client
|
||||
- [ ] Manager
|
||||
- [ ] Agent / Swarm
|
||||
- [ ] CodeGW
|
||||
- [ ] 计费
|
||||
- [ ] 密钥 / secret_ref
|
||||
- [ ] 审计
|
||||
- [ ] 发布链路
|
||||
- [ ] 文档 / 标准
|
||||
|
||||
## 是否读取标准
|
||||
|
||||
- [ ] 已读取当前仓 CLAUDE.md
|
||||
- [ ] 已读取当前仓 PROJECT_STANDARD.md
|
||||
- [ ] 已读取 heicodeDocs 相关标准
|
||||
|
||||
## 是否涉及接口契约
|
||||
|
||||
- [ ] 不涉及
|
||||
- [ ] 涉及,已更新 docs/integration 或相关文档
|
||||
|
||||
## 验收方式
|
||||
|
||||
<!-- 请说明如何验证,例如:
|
||||
python scripts/test-runtime-contract.py
|
||||
python scripts/test-merge-smoke.py
|
||||
python scripts/test-workflow-e2e.py
|
||||
-->
|
||||
|
||||
## TODO / Mock
|
||||
|
||||
- [ ] 没有 TODO / mock
|
||||
- [ ] 有,说明原因和后续计划
|
||||
|
||||
## Release 仓库确认
|
||||
|
||||
- [ ] 本 PR 不涉及 release 仓库业务代码
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
# Secrets / credentials — never commit
|
||||
.env
|
||||
.env.*
|
||||
secrets/
|
||||
*.pem
|
||||
*.key
|
||||
*.p12
|
||||
*.pfx
|
||||
id_rsa
|
||||
id_ed25519
|
||||
*_rsa
|
||||
*_ed25519
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.pyd
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
.coverage
|
||||
.coverage.*
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
ENV/
|
||||
|
||||
# Node / desktop-client
|
||||
node_modules/
|
||||
desktop-client/dist/
|
||||
desktop-client/out/
|
||||
|
||||
# Build / dist
|
||||
build/
|
||||
dist/
|
||||
*.egg-info/
|
||||
|
||||
# Runtime / temp outputs
|
||||
tmp-workspace/
|
||||
*.log
|
||||
|
||||
# OS / editor
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
.idea/
|
||||
.vscode/
|
||||
@@ -0,0 +1,36 @@
|
||||
# CLAUDE.md — HeiCode Swarm(执行面 / 运行时)
|
||||
|
||||
本仓库(`agent_swarm_v5`,对应 **HeiCode-Swarm**)是蜂群执行面 / 回调 / Swarm Runtime。整体介绍见 [README.md](README.md),工程标准见 [PROJECT_STANDARD.md](PROJECT_STANDARD.md),交付说明见 [docs/DELIVERY.md](docs/DELIVERY.md)。
|
||||
|
||||
## 改动前必读
|
||||
1. 先读本 `CLAUDE.md` 与本仓 `PROJECT_STANDARD.md`。
|
||||
2. **heicodeDocs 是唯一标准源**;涉及产品/工程/Agent/Swarm/计费/安全/交付标准时,先读取 heicodeDocs 对应标准文件。
|
||||
3. 跨端或动 Manager ↔ Swarm 接口时,必须先读 `heicode-mananger/docs/heicode.md` 与 `docs/integration/` 契约。
|
||||
4. 不确定标准时,**先停止修改并要求提供标准路径**;发现文档与代码冲突时,**输出冲突点**,不得自行选择一方。
|
||||
|
||||
## 仓库边界
|
||||
- 只允许修改 Swarm(orchestrator + agent)范围;不得跨仓改 Manager、客户端、release 仓库或 heicodeDocs。
|
||||
- 一个任务只允许修改任务声明范围内的模块。
|
||||
- 禁止直接 push / force push `main`;所有改动走 PR,并按 PR 模板声明影响范围。
|
||||
|
||||
## 安全与合规
|
||||
- 禁止把密钥、Token、云凭据、`.env`、证书、私钥写入代码、日志、Markdown 或提交记录。
|
||||
- 凭据一律经部署环境 / `secret_ref` / Key Vault 注入;`.env` 已被 `.gitignore` 忽略,仅供本地开发。
|
||||
- `secret_ref`、审批链、鉴权、计费、审计相关改动必须遵循 Manager 审批链与 heicodeDocs 安全规则。
|
||||
- 禁止只写 TODO / mock / 示例代码就声称完成。
|
||||
|
||||
## 架构与关键约束(便于定位)
|
||||
- **orchestrator/**:FastAPI 编排器。Manager 面接口、HMAC 签名回调、审批链**必须保持契约**。Redis 为权威存储;内存回退仅限 `REDIS_FAKE` / `ALLOW_MEMORY_STORE`(开发/CI)。
|
||||
- **agent/**:执行单元,**OpenAI 兼容**模型;保留计费/审计归属(`usage` 与 `X-Agent/X-Agnet` 头)。
|
||||
- **工作流开关默认关闭**:`ENABLE_PLANNER_FALLBACK`、`ENABLE_REVIEW_LOOP`、`ENABLE_SUBTASK_HANDOFF`。
|
||||
- 提交前必须本地通过:
|
||||
```
|
||||
python scripts/test-runtime-contract.py
|
||||
python scripts/test-merge-smoke.py
|
||||
python scripts/test-workflow-e2e.py
|
||||
```
|
||||
|
||||
## Agent / Teammate 协作
|
||||
- 每个 teammate / agent 必须遵循本 `CLAUDE.md` 与 heicodeDocs。
|
||||
- 不允许 teammate 跨仓自行修改未声明范围内的代码,或绕过 Manager ↔ Swarm 契约、计费、审计、审批链。
|
||||
- 多 agent 协作的最终总结需说明各 agent 负责范围、修改文件、风险与未完成项。
|
||||
@@ -0,0 +1,26 @@
|
||||
FROM python:3.11-slim
|
||||
|
||||
# Install Git
|
||||
RUN apt-get update && \
|
||||
apt-get install -y git && \
|
||||
apt-get clean && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Copy agent code
|
||||
COPY agent/ /app/agent/
|
||||
|
||||
# Install Python dependencies
|
||||
RUN pip install --no-cache-dir -r /app/agent/requirements.txt
|
||||
|
||||
# Create workspace directory
|
||||
RUN mkdir -p /workspace
|
||||
|
||||
# Set environment variables
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
ENV WORKSPACE_DIR=/workspace
|
||||
|
||||
# Run agent
|
||||
CMD ["python", "-m", "agent.main"]
|
||||
@@ -0,0 +1,16 @@
|
||||
FROM python:3.11-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install dependencies
|
||||
COPY orchestrator/requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
# Copy orchestrator code
|
||||
COPY orchestrator/ ./orchestrator/
|
||||
|
||||
# Expose port
|
||||
EXPOSE 8000
|
||||
|
||||
# Run orchestrator
|
||||
CMD ["python", "-m", "uvicorn", "orchestrator.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
@@ -0,0 +1,37 @@
|
||||
# PROJECT_STANDARD.md — HeiCode Swarm 工程标准
|
||||
|
||||
本文件约定本仓(`agent_swarm_v5` / HeiCode-Swarm)的工程标准。与 heicodeDocs 冲突时,以 **heicodeDocs** 为准;本文件仅作本仓落地补充。配套:[CLAUDE.md](CLAUDE.md)、[README.md](README.md)、[docs/DELIVERY.md](docs/DELIVERY.md)。
|
||||
|
||||
## 运行时与依赖
|
||||
- Python 3.13+;依赖见 `orchestrator/requirements.txt`、`agent/requirements.txt`。
|
||||
- 模型后端:**OpenAI 兼容** Chat Completions(`OPENAI_API_KEY` / `OPENAI_API_BASE` / `OPENAI_MODEL`,可指向自定义端点)。
|
||||
- 持久化:**Redis 为权威存储**。`fakeredis` 内存回退仅限开发/CI,且必须由 `REDIS_FAKE` / `ALLOW_MEMORY_STORE` 显式开启;生产在 Redis 不可用时快速失败。
|
||||
|
||||
## 功能开关(默认关闭,保持契约)
|
||||
- `ENABLE_PLANNER_FALLBACK`:无 Manager 分工时启用 LLM 规划回退。
|
||||
- `ENABLE_REVIEW_LOOP` / `MAX_REVIEW_CYCLES`:主控评审/重做循环 + 结果汇总。
|
||||
- `ENABLE_SUBTASK_HANDOFF`:动态子任务移交。
|
||||
- 开启上述开关不得改变 Manager 面接口、回调、审批链与计费/审计字段语义。
|
||||
|
||||
## 安全
|
||||
- 不得提交密钥/Token/凭据/`.env`/证书/私钥;通过环境 / `secret_ref` / Key Vault 注入。
|
||||
- K8s 部署:Agent 模型密钥使用 Secret `openai-secret`(Pod 模板)或 `openai-api-key`(Deployment 清单)。
|
||||
- 保留模型用量归属(`usage`、`X-Agent/X-Agnet` 头)以满足计费/审计。
|
||||
|
||||
## 测试与验收(PR 前必须通过)
|
||||
```
|
||||
python scripts/test-runtime-contract.py # Manager 契约校验
|
||||
python scripts/test-merge-smoke.py # 机制级冒烟(评审/协作/汇总等)
|
||||
python scripts/test-workflow-e2e.py # 端到端工作流(无需密钥)
|
||||
```
|
||||
- 新增/修改逻辑应补充对应断言;不得以 TODO / mock 充当完成。
|
||||
- 涉及 Manager 契约的改动,必须先通过 `test-runtime-contract.py`。
|
||||
|
||||
## 部署
|
||||
- 正式部署走**镜像**:`Dockerfile.orchestrator` / `Dockerfile.agent` + `k8s/` 清单(`orchestrator-deployment.yaml` 使用 `swarm-orchestrator:latest`)。
|
||||
- `k8s/orchestrator-source-configmap.yaml` 为陈旧快照,请勿用于当前部署(见 `docs/DELIVERY.md` §九)。
|
||||
|
||||
## 提交与分支
|
||||
- 禁止直接 push / force push `main`;所有改动经 PR 合并,至少 1 人 review。
|
||||
- PR 必须按 `.github/pull_request_template.md` 声明影响范围(Client / Manager / Agent-Swarm / CodeGW / 计费 / 密钥 / 审计 / 发布链路 / 文档)。
|
||||
- 提交信息清晰描述改动;保持与现有代码风格一致。
|
||||
@@ -0,0 +1,93 @@
|
||||
# Agent Swarm(HeiCode Swarm)
|
||||
|
||||
一个多智能体「蜂群」系统:用户提出需求后,由主控逻辑自动将其分解为多个子任务,分发给擅长不同领域的专家 Agent 并行完成;专家之间可就重叠领域相互协作;产出汇总后由主控评审是否达标,未达标则退回重做,循环直至生成满意的最终回答。
|
||||
|
||||
## 系统架构
|
||||
|
||||
```
|
||||
用户
|
||||
│
|
||||
桌面客户端(desktop-client)
|
||||
│ 提交需求 / 查看状态与结果
|
||||
▼
|
||||
Heicode Manager(控制面,外部)
|
||||
│ 下发编排方案 / 接收带签名回调(审批、计费、审计)
|
||||
▼
|
||||
Orchestrator(编排器,FastAPI)
|
||||
分解 · 派发 · 协作路由 · 评审/重做 · 汇总
|
||||
├─ Redis(权威状态存储)
|
||||
└─ WebSocket ┐
|
||||
▼
|
||||
Agent · Agent · Agent …(执行单元 / K8s Pod,调用大模型完成任务)
|
||||
```
|
||||
|
||||
## 工作流
|
||||
|
||||
```
|
||||
分解(Plan) → 派发给专家(Dispatch) → 专家执行(Execute)
|
||||
→ 重叠领域协作 / 移交(Collaborate & Handoff)
|
||||
→ 主控评审(Review)──不达标──▶ 退回相关任务重做(循环,受上限约束)
|
||||
└──达标──▶ 汇总为统一回答并交付(Deliver)
|
||||
```
|
||||
|
||||
- **分解**:优先采用 Manager 提供的编排方案;若未提供且开启规划回退(`ENABLE_PLANNER_FALLBACK`),由编排器用大模型把目标拆解为「实现 → 测试 → 文档」等专家子任务。
|
||||
- **派发**:按「能力匹配 + 剩余容量」将就绪任务下发给已连接的 Agent;派发时把已完成依赖的产物与同伴信息注入上下文。
|
||||
- **协作 / 移交**:测试、文档等角色可向实现角色咨询以保持语义一致;复杂子任务可移交给更合适的专家。
|
||||
- **评审 / 重做**:所有任务完成后由评审者判断是否达标(开启 `ENABLE_REVIEW_LOOP`),不达标则退回相关任务重做,受 `MAX_REVIEW_CYCLES` 约束。
|
||||
- **汇总交付**:通过后将各专家产出汇总为统一、面向用户的最终回答。
|
||||
|
||||
## 仓库结构与子文档
|
||||
|
||||
| 目录 | 角色 | 文档 |
|
||||
|---|---|---|
|
||||
| `orchestrator/` | 编排器:分解、派发、协作、评审、汇总、Manager 对接 | [orchestrator/README.md](orchestrator/README.md) |
|
||||
| `agent/` | 执行单元:连接编排器、调用大模型完成任务、提交结果 | [agent/README.md](agent/README.md) |
|
||||
| `desktop-client/` | 桌面客户端:任务提交、复杂度分析、状态展示、结果聚合 | [desktop-client/README.md](desktop-client/README.md) |
|
||||
| `desktop-client/src/ai/` | 复杂度分析模块:评估规模、推荐 Agent 数量 | [desktop-client/src/ai/README.md](desktop-client/src/ai/README.md) |
|
||||
| `docs/` | 交付说明(交付物、构建/部署、配置、验收、影响与合规) | [docs/DELIVERY.md](docs/DELIVERY.md) |
|
||||
| `k8s/` | Kubernetes 部署清单(编排器、Agent、Redis、RBAC、监控等) | — |
|
||||
| `scripts/` | 部署、契约校验、冒烟与端到端工作流测试脚本 | — |
|
||||
| `test-data/` | 复杂度分析的标注数据集 | — |
|
||||
|
||||
## 快速开始(本地)
|
||||
|
||||
需要 Python 3.13+ 环境。本地开发可用受控的内存存储(`REDIS_FAKE=1`,免装 Redis)。
|
||||
|
||||
```bash
|
||||
# 1) 编排器(在本目录 agent_swarm_v5 下运行,开启规划回退与评审循环)
|
||||
set "REDIS_FAKE=1"
|
||||
set "ENABLE_PLANNER_FALLBACK=1"
|
||||
set "ENABLE_REVIEW_LOOP=1"
|
||||
python -m uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
|
||||
|
||||
# 2) 一个 Agent(凭据放在本目录 .env:OPENAI_API_KEY 等)
|
||||
set "ORCHESTRATOR_URL=ws://localhost:8000"
|
||||
set "AGENT_ID=worker-1"
|
||||
set "AGENT_CAPABILITIES=python,code_generation,testing,pytest,technical-writing,general"
|
||||
set "WORKSPACE_DIR=..\tmp-workspace\worker-1"
|
||||
python -m agent.main
|
||||
|
||||
# 3) 提交一个需求
|
||||
curl -s -X POST http://localhost:8000/api/swarms -H "Content-Type: application/json" ^
|
||||
-d "{\"mode\":\"swarm\",\"requirement\":{\"objective\":\"实现 add(a,b) 并补充测试与说明\"},\"callback\":{\"url\":\"http://localhost:9999/cb\",\"subscribed_events\":[]},\"metadata\":{\"manager_deployment_id\":\"dev-1\"}}"
|
||||
```
|
||||
|
||||
随后可通过 `GET /api/swarms/{deployment_id}/workflow` 与 `/logs` 观察分解、派发、评审与汇总过程。各组件的环境变量与接口详见对应子文档。
|
||||
|
||||
## 关键设计
|
||||
|
||||
- **存储**:Redis 为权威状态存储;仅 `REDIS_FAKE` / `ALLOW_MEMORY_STORE` 开启时才使用进程内回退,生产环境在 Redis 不可用时快速失败。
|
||||
- **模型**:Agent 与编排器规划/评审均使用 **OpenAI 兼容** API,可指向自定义端点;未配置密钥时规划/评审退化为静态分解与启发式判定。
|
||||
- **Manager 契约**:面向 Manager 的接口、带签名回调、审批链与计费/审计字段均予以保留。
|
||||
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
python scripts/test-runtime-contract.py # Manager 契约校验
|
||||
python scripts/test-merge-smoke.py # 工作流冒烟测试(评审 / 协作 / 汇总等)
|
||||
```
|
||||
|
||||
## 安全与合规
|
||||
|
||||
- **禁止**将密钥、令牌、云凭据写入代码、日志或提交记录;凭据通过 `.env`(已忽略)或部署环境 / `secret_ref` 注入。
|
||||
- 涉及鉴权、审批链、计费与审计的改动需遵循 Manager 安全规则。
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
# Agent 执行器(Agent Runtime)
|
||||
|
||||
Agent 是蜂群系统中的执行单元(worker)。每个 Agent 作为独立进程运行(在 Kubernetes 中通常是一个 Pod),通过 WebSocket 连接到 Orchestrator,领取任务、调用大模型完成编码工作,并把结果写回 Git 分支。
|
||||
|
||||
## 职责概览
|
||||
|
||||
- 通过 WebSocket 连接 Orchestrator,并以自身**能力(capabilities)**注册,例如 `python`、`testing`、`technical-writing`。
|
||||
- 接收 Orchestrator 下发的任务,使用 **OpenAI 兼容**的大模型完成子任务。
|
||||
- 在隔离的「按任务工作目录」中生成/修改文件,并提交到独立的结果分支。
|
||||
- 支持与其它 Agent **协作(peer collaboration)**与**移交(handoff)**。
|
||||
- 持续上报心跳、状态与执行结果,并暴露 Prometheus 指标。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
agent/
|
||||
├── main.py # Agent 主循环:连接、注册、心跳、消息处理、并发与重连
|
||||
├── task_executor.py # 任务执行引擎:调用 OpenAI 兼容模型,应用文件改动
|
||||
├── handoff_logic.py # 移交决策逻辑(复杂度 / 能力 / 预估时长)
|
||||
├── git_operations.py # Git 克隆、建分支、提交、推送
|
||||
└── requirements.txt # Python 依赖
|
||||
```
|
||||
|
||||
## 核心模块
|
||||
|
||||
### 主循环(`main.py`)
|
||||
- **连接与重连**:断线后按指数退避自动重连并重新注册。
|
||||
- **有界并发**:通过信号量限制同时执行的任务数(`MAX_CONCURRENT_TASKS`),并在注册/心跳中上报剩余容量 `available_slots`。
|
||||
- **任务受理**:对每个分配先做校验,重复任务返回 `task_accepted(status=duplicate)`,超出容量返回 `task_rejected`,由 Orchestrator 重新入队。
|
||||
- **执行保护**:单任务超时(`TASK_TIMEOUT_SECONDS`)与任务取消(`cancel_task`)。
|
||||
- **心跳**:每 15 秒上报一次存活与容量信息。
|
||||
- **优雅退出**:收到 SIGINT/SIGTERM 时停止接单并清理在执行的任务。
|
||||
|
||||
### 任务执行器(`task_executor.py`)
|
||||
- 使用 **OpenAI 兼容 API**(可指向自定义 `base_url`)完成子任务。
|
||||
- 根据上下文中的**依赖产物(dependency_artifacts)**与**专家角色(specialist_role)**对齐输出:测试与文档以实现产物的行为/异常语义为准。
|
||||
- 将模型返回的文件改动安全地写入工作目录(带路径越界校验)。
|
||||
- 记录用量并附带计费/审计归属(`usage` 与 `X-Agent/X-Agnet` 模型归属头),供 Orchestrator 上报。
|
||||
|
||||
### 移交逻辑(`handoff_logic.py`)
|
||||
当子任务满足以下条件时建议移交给更合适的 Agent:
|
||||
- 复杂度为 `high`;
|
||||
- 需要当前 Agent 不具备的专业能力;
|
||||
- 预估耗时超过 60 分钟。
|
||||
|
||||
### Git 操作(`git_operations.py`)
|
||||
- 自动识别仓库根目录(`repo_root`),使按任务子目录也能在父级 Git 仓库中正确执行。
|
||||
- 为每个任务创建结果分支(形如 `agent/{agent-id}/{task}-{timestamp}`),提交并推送。
|
||||
|
||||
## 配置(环境变量)
|
||||
|
||||
凭据通过同目录的 `.env`(已被 `.gitignore` 忽略)加载,亦可由部署环境/`secret_ref` 注入。**请勿将密钥写入代码或提交记录。**
|
||||
|
||||
```bash
|
||||
# 模型(OpenAI 兼容)
|
||||
OPENAI_API_KEY=... # 或 MODEL_API_KEY
|
||||
OPENAI_API_BASE=https://api.openai.com/v1 # 或 MODEL_API_BASE,可指向自定义端点
|
||||
OPENAI_MODEL=gpt-4o-mini # 或 MODEL_NAME / MODEL_ID
|
||||
|
||||
# 连接与身份
|
||||
ORCHESTRATOR_URL=ws://localhost:8000 # Orchestrator 的 WebSocket 地址
|
||||
AGENT_ID=worker-1 # 不填则自动生成
|
||||
AGENT_CAPABILITIES=python,testing # 逗号分隔的能力列表
|
||||
WORKSPACE_DIR=/workspace # 工作目录
|
||||
|
||||
# 可选
|
||||
GIT_REPO_URL=https://... # 需在仓库内工作时设置
|
||||
GIT_USERNAME / GIT_PASSWORD / GIT_TOKEN# 推送凭据
|
||||
GIT_BASE_BRANCH=main # 结果分支的基线
|
||||
MAX_CONCURRENT_TASKS=4 # 最大并发任务数
|
||||
TASK_TIMEOUT_SECONDS=60 # 单任务超时
|
||||
METRICS_PORT=9000 # 设置后暴露 Prometheus 指标
|
||||
ENABLE_SUBTASK_HANDOFF=false # 是否启用动态子任务移交
|
||||
```
|
||||
|
||||
## 本地运行
|
||||
|
||||
```bash
|
||||
pip install -r agent/requirements.txt
|
||||
|
||||
# 准备 .env(含 OPENAI_API_KEY 等),然后从仓库根目录运行:
|
||||
set "ORCHESTRATOR_URL=ws://localhost:8000"
|
||||
set "AGENT_ID=worker-1"
|
||||
set "AGENT_CAPABILITIES=python,code_generation,testing,pytest,technical-writing,general"
|
||||
set "WORKSPACE_DIR=...\tmp-workspace\worker-1"
|
||||
python -m agent.main
|
||||
```
|
||||
|
||||
## 任务执行流程
|
||||
|
||||
1. **注册**:连接 Orchestrator 并上报能力与可用容量。
|
||||
2. **心跳**:每 15 秒上报存活与剩余容量。
|
||||
3. **受理**:校验分配,必要时拒绝(容量不足)或忽略(重复)。
|
||||
4. **执行**:在独立的按任务工作目录中调用模型完成子任务;如启用,可向同伴 Agent 咨询或移交。
|
||||
5. **提交**:若为 Git 工作区,提交并推送到结果分支。
|
||||
6. **回报**:将结果(含用量)发送回 Orchestrator,并将状态置为空闲。
|
||||
|
||||
## 协作(Peer Collaboration)
|
||||
|
||||
当上下文提供了 `peer_agents` 时,测试/文档等角色可向实现角色发起咨询;被咨询方会回复自己最近一次完成任务的摘要,帮助各专家在重叠领域保持一致。
|
||||
|
||||
## 监控
|
||||
|
||||
Agent 暴露的 Prometheus 指标包括:已执行/失败任务数、任务时长、移交次数、重连次数、被拒/重复任务数、当前活跃任务数与 Agent 状态。
|
||||
@@ -0,0 +1,3 @@
|
||||
"""Agent package for K8s-based swarm mode."""
|
||||
|
||||
__all__: list[str] = []
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Git operations for the agent workspace.
|
||||
|
||||
Merged from agent_swarm_v4:
|
||||
- Adds repository-root discovery (`repo_root`) so per-task child workspaces can still
|
||||
operate inside a parent Git checkout (all git commands run at the repo toplevel)
|
||||
- Keeps the original branch/commit/push API so the protocol stays compatible
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from urllib.parse import quote, urlsplit, urlunsplit
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GitOperations:
|
||||
"""Handles Git operations for agent workspace."""
|
||||
|
||||
def __init__(self, workspace_dir: str, agent_id: str):
|
||||
self.workspace_dir = str(Path(workspace_dir))
|
||||
self.agent_id = agent_id
|
||||
self.result_branch = f"agent-{agent_id}-results"
|
||||
self.base_branch = os.getenv("GIT_BASE_BRANCH", "main")
|
||||
|
||||
def _authenticated_repo_url(self, repo_url: str) -> str:
|
||||
username = os.getenv("GIT_USERNAME")
|
||||
password = os.getenv("GIT_PASSWORD") or os.getenv("GIT_TOKEN")
|
||||
if not username or not password:
|
||||
return repo_url
|
||||
|
||||
parsed = urlsplit(repo_url)
|
||||
if parsed.scheme not in {"http", "https"} or "@" in parsed.netloc:
|
||||
return repo_url
|
||||
|
||||
userinfo = f"{quote(username)}:{quote(password)}"
|
||||
return urlunsplit((
|
||||
parsed.scheme,
|
||||
f"{userinfo}@{parsed.netloc}",
|
||||
parsed.path,
|
||||
parsed.query,
|
||||
parsed.fragment,
|
||||
))
|
||||
|
||||
async def repo_root(self) -> Optional[str]:
|
||||
result = await self._run_git_command(
|
||||
["git", "rev-parse", "--show-toplevel"],
|
||||
cwd=self.workspace_dir,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return None
|
||||
return result.stdout.strip() or None
|
||||
|
||||
async def is_git_workspace(self) -> bool:
|
||||
result = await self._run_git_command(
|
||||
["git", "rev-parse", "--is-inside-work-tree"],
|
||||
cwd=self.workspace_dir,
|
||||
)
|
||||
return result.returncode == 0 and result.stdout.strip() == "true"
|
||||
|
||||
async def clone_workspace(self, repo_url: str) -> bool:
|
||||
try:
|
||||
logger.info(f"Cloning repository from {repo_url}")
|
||||
|
||||
workspace = Path(self.workspace_dir)
|
||||
workspace.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if workspace.exists():
|
||||
if await self.is_git_workspace():
|
||||
logger.info("Workspace already contains a Git checkout; refreshing existing clone")
|
||||
return await self._refresh_existing_workspace(repo_url)
|
||||
|
||||
if any(workspace.iterdir()):
|
||||
logger.warning("Workspace directory exists and is not empty; clearing before clone")
|
||||
self._clear_workspace_dir(workspace)
|
||||
|
||||
clone_url = self._authenticated_repo_url(repo_url)
|
||||
result = await self._run_git_command([
|
||||
"git", "clone", clone_url, self.workspace_dir
|
||||
])
|
||||
|
||||
if result.returncode != 0:
|
||||
logger.error(f"Git clone failed: {result.stderr}")
|
||||
return False
|
||||
|
||||
logger.info(f"Successfully cloned repository to {self.workspace_dir}")
|
||||
await self._configure_git_user()
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error cloning workspace: {e}")
|
||||
return False
|
||||
|
||||
def _clear_workspace_dir(self, workspace: Path):
|
||||
for child in workspace.iterdir():
|
||||
if child.is_dir() and not child.is_symlink():
|
||||
shutil.rmtree(child)
|
||||
else:
|
||||
child.unlink()
|
||||
|
||||
async def _configure_git_user(self):
|
||||
cwd = await self.repo_root() or self.workspace_dir
|
||||
await self._run_git_command([
|
||||
"git", "config", "user.name", f"Agent {self.agent_id}"
|
||||
], cwd=cwd)
|
||||
await self._run_git_command([
|
||||
"git", "config", "user.email", f"{self.agent_id}@agent.local"
|
||||
], cwd=cwd)
|
||||
|
||||
async def _refresh_existing_workspace(self, repo_url: str) -> bool:
|
||||
cwd = await self.repo_root() or self.workspace_dir
|
||||
clone_url = self._authenticated_repo_url(repo_url)
|
||||
await self._run_git_command(["git", "remote", "set-url", "origin", clone_url], cwd=cwd)
|
||||
await self._configure_git_user()
|
||||
|
||||
fetch = await self._run_git_command(["git", "fetch", "origin"], cwd=cwd)
|
||||
if fetch.returncode != 0:
|
||||
logger.error(f"Git fetch failed: {fetch.stderr}")
|
||||
return False
|
||||
|
||||
base_ref = f"origin/{self.base_branch}"
|
||||
base_check = await self._run_git_command(["git", "rev-parse", "--verify", base_ref], cwd=cwd)
|
||||
if base_check.returncode == 0:
|
||||
reset = await self._run_git_command(["git", "reset", "--hard", base_ref], cwd=cwd)
|
||||
if reset.returncode != 0:
|
||||
logger.error(f"Git reset failed: {reset.stderr}")
|
||||
return False
|
||||
|
||||
clean = await self._run_git_command(["git", "clean", "-fd"], cwd=cwd)
|
||||
if clean.returncode != 0:
|
||||
logger.error(f"Git clean failed: {clean.stderr}")
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
async def create_result_branch(self, task_id: Optional[str] = None) -> bool:
|
||||
try:
|
||||
cwd = await self.repo_root() or self.workspace_dir
|
||||
if task_id:
|
||||
safe_task_id = task_id.replace("/", "-")[:12]
|
||||
timestamp = int(time.time())
|
||||
self.result_branch = f"agent/{self.agent_id}/{safe_task_id}-{timestamp}"
|
||||
|
||||
await self._run_git_command(["git", "fetch", "origin"], cwd=cwd)
|
||||
|
||||
base_ref = f"origin/{self.base_branch}"
|
||||
base_check = await self._run_git_command(["git", "rev-parse", "--verify", base_ref], cwd=cwd)
|
||||
if base_check.returncode != 0:
|
||||
base_ref = "HEAD"
|
||||
logger.warning(f"Base branch origin/{self.base_branch} not found; creating result branch from HEAD")
|
||||
|
||||
result = await self._run_git_command(["git", "checkout", "-B", self.result_branch, base_ref], cwd=cwd)
|
||||
if result.returncode != 0:
|
||||
logger.error(f"Failed to create branch: {result.stderr}")
|
||||
return False
|
||||
|
||||
logger.info(f"Created result branch: {self.result_branch}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating result branch: {e}")
|
||||
return False
|
||||
|
||||
async def commit_changes(self, message: str) -> Optional[str]:
|
||||
try:
|
||||
cwd = await self.repo_root() or self.workspace_dir
|
||||
status_result = await self._run_git_command(["git", "status", "--porcelain"], cwd=cwd)
|
||||
if not status_result.stdout.strip():
|
||||
logger.info("No changes to commit")
|
||||
return None
|
||||
|
||||
await self._run_git_command(["git", "add", "-A"], cwd=cwd)
|
||||
commit_result = await self._run_git_command(["git", "commit", "-m", message], cwd=cwd)
|
||||
if commit_result.returncode != 0:
|
||||
logger.error(f"Git commit failed: {commit_result.stderr}")
|
||||
return None
|
||||
|
||||
sha_result = await self._run_git_command(["git", "rev-parse", "HEAD"], cwd=cwd)
|
||||
commit_sha = sha_result.stdout.strip()
|
||||
logger.info(f"Committed changes: {commit_sha[:8]} - {message}")
|
||||
return commit_sha
|
||||
except Exception as e:
|
||||
logger.error(f"Error committing changes: {e}")
|
||||
return None
|
||||
|
||||
async def push_results(self) -> Optional[str]:
|
||||
try:
|
||||
cwd = await self.repo_root() or self.workspace_dir
|
||||
result = await self._run_git_command(["git", "push", "-u", "origin", self.result_branch], cwd=cwd)
|
||||
if result.returncode != 0:
|
||||
logger.error(f"Git push failed: {result.stderr}")
|
||||
return None
|
||||
logger.info(f"Pushed results to branch: {self.result_branch}")
|
||||
return self.result_branch
|
||||
except Exception as e:
|
||||
logger.error(f"Error pushing results: {e}")
|
||||
return None
|
||||
|
||||
async def _run_git_command(
|
||||
self,
|
||||
command: list[str],
|
||||
cwd: Optional[str] = None,
|
||||
) -> subprocess.CompletedProcess:
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*command,
|
||||
cwd=cwd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
stdout, stderr = await process.communicate()
|
||||
return subprocess.CompletedProcess(
|
||||
args=command,
|
||||
returncode=process.returncode,
|
||||
stdout=stdout.decode(),
|
||||
stderr=stderr.decode(),
|
||||
)
|
||||
@@ -0,0 +1,224 @@
|
||||
"""Handoff decision logic for determining when to delegate tasks."""
|
||||
import logging
|
||||
from typing import Iterable, List, Optional
|
||||
from dataclasses import dataclass
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class HandoffDecision:
|
||||
"""Result of handoff decision analysis."""
|
||||
should_handoff: bool
|
||||
reason: str
|
||||
target_capabilities: List[str]
|
||||
|
||||
|
||||
CAPABILITY_ALIASES = {
|
||||
"code_generation": {
|
||||
"code-reading",
|
||||
"code-editing",
|
||||
"command-line",
|
||||
"credentials-handling",
|
||||
"editing",
|
||||
"environment-inspection",
|
||||
"environment-setup",
|
||||
"file-editing",
|
||||
"file-operations",
|
||||
"file-system",
|
||||
"filesystem",
|
||||
"read-only-operations",
|
||||
"reporting",
|
||||
"repository-inspection",
|
||||
"shell",
|
||||
"testing",
|
||||
"pytest",
|
||||
"quality-checking",
|
||||
"attention-to-detail",
|
||||
"error-handling",
|
||||
"technical-writing",
|
||||
"validation",
|
||||
"verification",
|
||||
},
|
||||
"python": {
|
||||
"pytest",
|
||||
"testing",
|
||||
"code-reading",
|
||||
"code-editing",
|
||||
"command-line",
|
||||
"editing",
|
||||
"file-editing",
|
||||
"file-operations",
|
||||
"file-system",
|
||||
"filesystem",
|
||||
"reporting",
|
||||
"shell",
|
||||
"quality-checking",
|
||||
"error-handling",
|
||||
"validation",
|
||||
"verification",
|
||||
},
|
||||
"general": {
|
||||
"general",
|
||||
"code-reading",
|
||||
"file-system",
|
||||
"filesystem",
|
||||
"quality-checking",
|
||||
"attention-to-detail",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _expanded_capabilities(capabilities: Iterable[str]) -> set[str]:
|
||||
"""Return capabilities plus local aliases supported by this agent."""
|
||||
expanded = {"general", *capabilities}
|
||||
for capability in capabilities:
|
||||
expanded.update(CAPABILITY_ALIASES.get(capability, set()))
|
||||
return expanded
|
||||
|
||||
|
||||
def should_handoff(
|
||||
subtask: dict,
|
||||
agent_id: str,
|
||||
agent_capabilities: Optional[List[str]] = None,
|
||||
) -> HandoffDecision:
|
||||
"""Determine if a subtask should be handed off to another agent.
|
||||
|
||||
Args:
|
||||
subtask: Subtask definition with complexity, capabilities, etc.
|
||||
agent_id: Current agent ID
|
||||
agent_capabilities: Capabilities advertised by the current agent
|
||||
|
||||
Returns:
|
||||
HandoffDecision: Decision with reasoning
|
||||
"""
|
||||
complexity = subtask.get("complexity", "medium")
|
||||
required_capabilities = subtask.get("required_capabilities", ["general"])
|
||||
estimated_time = subtask.get("estimated_time", 30)
|
||||
current_capabilities = _expanded_capabilities(agent_capabilities or ["general"])
|
||||
|
||||
# Decision criteria:
|
||||
# 1. High complexity tasks should be handed off to specialist agents
|
||||
# 2. Tasks requiring specialized capabilities should go to specialists
|
||||
# 3. Tasks estimated to take > 60 minutes should be broken down further
|
||||
|
||||
# Check complexity threshold
|
||||
if complexity == "high":
|
||||
logger.info(f"High complexity subtask detected: {subtask['description']}")
|
||||
return HandoffDecision(
|
||||
should_handoff=True,
|
||||
reason="Task complexity exceeds agent capability threshold",
|
||||
target_capabilities=required_capabilities
|
||||
)
|
||||
|
||||
# Check for specialized capabilities
|
||||
specialized_capabilities = [
|
||||
cap for cap in required_capabilities
|
||||
if cap not in current_capabilities
|
||||
]
|
||||
|
||||
if specialized_capabilities:
|
||||
logger.info(f"Specialized capabilities required: {specialized_capabilities}")
|
||||
return HandoffDecision(
|
||||
should_handoff=True,
|
||||
reason=f"Requires specialized capabilities: {', '.join(specialized_capabilities)}",
|
||||
target_capabilities=specialized_capabilities
|
||||
)
|
||||
|
||||
# Check estimated time
|
||||
if estimated_time > 60:
|
||||
logger.info(f"Long-running task detected: {estimated_time} minutes")
|
||||
return HandoffDecision(
|
||||
should_handoff=True,
|
||||
reason=f"Task estimated to take {estimated_time} minutes (threshold: 60)",
|
||||
target_capabilities=required_capabilities
|
||||
)
|
||||
|
||||
# No handoff needed
|
||||
return HandoffDecision(
|
||||
should_handoff=False,
|
||||
reason="Task within agent capability",
|
||||
target_capabilities=[]
|
||||
)
|
||||
|
||||
|
||||
def select_target_agent(
|
||||
available_agents: List[dict],
|
||||
required_capabilities: List[str]
|
||||
) -> str:
|
||||
"""Select the best agent for a handoff based on capabilities.
|
||||
|
||||
Args:
|
||||
available_agents: List of available agent metadata
|
||||
required_capabilities: Required capabilities for the task
|
||||
|
||||
Returns:
|
||||
str: Selected agent ID or None if no suitable agent found
|
||||
"""
|
||||
# Score each agent based on capability match
|
||||
best_agent = None
|
||||
best_score = -1
|
||||
|
||||
for agent in available_agents:
|
||||
agent_capabilities = set(agent.get("capabilities", []))
|
||||
required_set = set(required_capabilities)
|
||||
|
||||
# Calculate match score
|
||||
matches = len(agent_capabilities.intersection(required_set))
|
||||
total_required = len(required_set)
|
||||
|
||||
if total_required > 0:
|
||||
score = matches / total_required
|
||||
else:
|
||||
score = 0
|
||||
|
||||
# Prefer agents with exact capability match
|
||||
if score > best_score:
|
||||
best_score = score
|
||||
best_agent = agent
|
||||
|
||||
if best_agent:
|
||||
logger.info(
|
||||
f"Selected agent {best_agent['agent_id']} "
|
||||
f"with score {best_score:.2f} for capabilities {required_capabilities}"
|
||||
)
|
||||
return best_agent["agent_id"]
|
||||
|
||||
logger.warning(f"No suitable agent found for capabilities: {required_capabilities}")
|
||||
return None
|
||||
|
||||
|
||||
def estimate_task_complexity(description: str) -> str:
|
||||
"""Estimate task complexity based on description keywords.
|
||||
|
||||
Args:
|
||||
description: Task description
|
||||
|
||||
Returns:
|
||||
str: "low", "medium", or "high"
|
||||
"""
|
||||
description_lower = description.lower()
|
||||
|
||||
# High complexity indicators
|
||||
high_complexity_keywords = [
|
||||
"refactor", "redesign", "architecture", "migrate",
|
||||
"optimize", "performance", "security", "scale",
|
||||
"distributed", "concurrent", "async", "parallel"
|
||||
]
|
||||
|
||||
# Low complexity indicators
|
||||
low_complexity_keywords = [
|
||||
"fix typo", "update comment", "rename", "format",
|
||||
"add log", "simple", "trivial", "quick"
|
||||
]
|
||||
|
||||
# Check for high complexity
|
||||
if any(keyword in description_lower for keyword in high_complexity_keywords):
|
||||
return "high"
|
||||
|
||||
# Check for low complexity
|
||||
if any(keyword in description_lower for keyword in low_complexity_keywords):
|
||||
return "low"
|
||||
|
||||
# Default to medium
|
||||
return "medium"
|
||||
+629
@@ -0,0 +1,629 @@
|
||||
"""Agent main loop with WebSocket connection to orchestrator.
|
||||
|
||||
Merged from agent_swarm_v4:
|
||||
- Bounded concurrency, duplicate protection, capacity reporting (available_slots)
|
||||
- Serialized websocket sends (safe_send) and reconnect-with-reregister logic
|
||||
- Per-task timeout and graceful task cancellation
|
||||
- Separate repository root from per-task execution workspace (see git_operations)
|
||||
- Peer collaboration: request/reply routing fixed relative to v4 (peer_waiters is now
|
||||
initialized and inbound peer replies are routed back to the waiting coroutine; inbound
|
||||
peer queries are answered with a lightweight, cost-free acknowledgement)
|
||||
- OpenAI-only task executor (see task_executor)
|
||||
- Preserves heicode handoff wiring, Manager-facing metrics, and env-var entrypoint
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import websockets
|
||||
from dotenv import load_dotenv
|
||||
from pydantic import BaseModel, Field, ValidationError
|
||||
from prometheus_client import Counter, Gauge, Histogram, start_http_server
|
||||
from websockets.exceptions import ConnectionClosed
|
||||
|
||||
from .git_operations import GitOperations
|
||||
from .task_executor import TaskExecutor
|
||||
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
# load_dotenv() above reads a local .env (gitignored) so credentials need not be passed on
|
||||
# the command line. These defaults just guarantee the vars exist; an empty OPENAI_API_KEY is
|
||||
# still falsy, so TaskExecutor raises a clear error rather than silently using no key.
|
||||
# Real deployments supply the key via the environment / secret_ref, NOT a committed file.
|
||||
os.environ.setdefault("OPENAI_API_KEY", "")
|
||||
os.environ.setdefault("OPENAI_API_BASE", "https://api.openai.com/v1")
|
||||
os.environ.setdefault("OPENAI_MODEL", "gpt-4o-mini")
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TASKS_EXECUTED = Counter("agent_tasks_executed_total", "Total tasks executed")
|
||||
TASKS_FAILED = Counter("agent_tasks_failed_total", "Total tasks failed")
|
||||
TASK_DURATION = Histogram("agent_task_duration_seconds", "Task execution duration")
|
||||
HANDOFFS_INITIATED = Counter("agent_handoffs_initiated_total", "Total handoffs initiated")
|
||||
AGENT_STATUS = Gauge("agent_status", "Agent status (0=idle, 1=busy, 2=failed)")
|
||||
WEBSOCKET_RECONNECTS = Counter("agent_websocket_reconnects_total", "WebSocket reconnection attempts")
|
||||
TASKS_REJECTED = Counter("agent_tasks_rejected_total", "Total tasks rejected due to capacity")
|
||||
TASKS_DUPLICATE = Counter("agent_tasks_duplicate_total", "Total duplicate task assignments")
|
||||
ACTIVE_TASKS = Gauge("agent_active_tasks", "Current number of active tasks")
|
||||
|
||||
|
||||
class TaskAssignment(BaseModel):
|
||||
task_id: str
|
||||
description: str
|
||||
context: dict = Field(default_factory=dict)
|
||||
|
||||
|
||||
class AgentRuntimeDisconnected(RuntimeError):
|
||||
"""Raised when the runtime attempts to send without an active websocket."""
|
||||
|
||||
|
||||
class Agent:
|
||||
"""Agent that connects to orchestrator and executes tasks."""
|
||||
|
||||
MAX_CONCURRENT_TASKS = int(os.getenv("MAX_CONCURRENT_TASKS", "4"))
|
||||
TASK_TIMEOUT_SECONDS = int(os.getenv("TASK_TIMEOUT_SECONDS", "60"))
|
||||
HEARTBEAT_INTERVAL_SECONDS = 15
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
orchestrator_url: str,
|
||||
agent_id: Optional[str] = None,
|
||||
capabilities: Optional[list[str]] = None,
|
||||
workspace_dir: str = "/workspace",
|
||||
git_repo_url: Optional[str] = None,
|
||||
):
|
||||
# Initialize connection state, execution limits, and workspace helpers for this agent runtime.
|
||||
self.agent_id = agent_id or f"agent-{uuid.uuid4().hex[:8]}"
|
||||
self.orchestrator_url = orchestrator_url.rstrip("/")
|
||||
self.capabilities = capabilities or ["general"]
|
||||
self.workspace_dir = Path(workspace_dir)
|
||||
self.git_repo_url = git_repo_url
|
||||
|
||||
self.websocket: Optional[websockets.WebSocketClientProtocol] = None
|
||||
self.running = False
|
||||
self.current_task_id: Optional[str] = None
|
||||
self.send_lock = asyncio.Lock()
|
||||
self.task_semaphore = asyncio.Semaphore(self.MAX_CONCURRENT_TASKS)
|
||||
self.active_tasks: dict[str, asyncio.Task] = {}
|
||||
self.heartbeat_task: Optional[asyncio.Task] = None
|
||||
# Outstanding peer-collaboration requests awaiting a reply, keyed by correlation_id.
|
||||
self.peer_waiters: dict[str, asyncio.Future] = {}
|
||||
# Summary of this agent's most recently completed task, shared when peers consult it.
|
||||
self.last_summary: Optional[str] = None
|
||||
|
||||
self.workspace_git = GitOperations(str(self.workspace_dir), self.agent_id)
|
||||
|
||||
def available_slots(self) -> int:
|
||||
# Return how many additional tasks this agent can currently accept.
|
||||
return max(0, self.MAX_CONCURRENT_TASKS - len(self.active_tasks))
|
||||
|
||||
def task_workspace(self, task_id: str) -> Path:
|
||||
# Compute the dedicated per-task working directory inside the agent workspace.
|
||||
return self.workspace_dir / ".agent_tasks" / task_id
|
||||
|
||||
async def safe_send(self, payload: dict):
|
||||
# Serialize and send a websocket message while holding a lock to prevent concurrent writes.
|
||||
if not self.websocket:
|
||||
raise AgentRuntimeDisconnected("websocket unavailable")
|
||||
async with self.send_lock:
|
||||
await self.websocket.send(json.dumps(payload))
|
||||
|
||||
async def connect(self) -> bool:
|
||||
# Open a websocket connection to the orchestrator and cache the live socket on success.
|
||||
try:
|
||||
self.websocket = await websockets.connect(f"{self.orchestrator_url}/ws/{self.agent_id}")
|
||||
logger.info(f"Connected to orchestrator at {self.orchestrator_url}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to connect to orchestrator: {e}")
|
||||
return False
|
||||
|
||||
async def connect_with_retry(self) -> bool:
|
||||
# Repeatedly attempt to connect with exponential backoff until connected or shutting down.
|
||||
delay = 1
|
||||
while self.running:
|
||||
if await self.connect():
|
||||
return True
|
||||
WEBSOCKET_RECONNECTS.inc()
|
||||
await asyncio.sleep(min(delay, 60))
|
||||
delay = min(delay * 2, 60)
|
||||
return False
|
||||
|
||||
async def register(self):
|
||||
# Announce this agent and its current capacity to the orchestrator after connecting.
|
||||
try:
|
||||
await self.safe_send({
|
||||
"type": "register",
|
||||
"agent_id": self.agent_id,
|
||||
"capabilities": self.capabilities,
|
||||
# Backward-compatible extra fields; older orchestrators ignore them.
|
||||
"available_slots": self.available_slots(),
|
||||
"active_task_ids": list(self.active_tasks.keys()),
|
||||
})
|
||||
logger.info(f"Registered agent {self.agent_id} with capabilities: {self.capabilities}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to register: {e}")
|
||||
return False
|
||||
|
||||
async def send_heartbeat(self):
|
||||
# Send a periodic liveness and capacity update to the orchestrator.
|
||||
await self.safe_send({
|
||||
"type": "heartbeat",
|
||||
"agent_id": self.agent_id,
|
||||
"timestamp": time.time(),
|
||||
"active_tasks": len(self.active_tasks),
|
||||
"available_slots": self.available_slots(),
|
||||
})
|
||||
|
||||
async def heartbeat_loop(self):
|
||||
# Keep sending heartbeat messages until the connection closes or the agent stops running.
|
||||
while self.running:
|
||||
try:
|
||||
await self.send_heartbeat()
|
||||
except (ConnectionClosed, AgentRuntimeDisconnected):
|
||||
return
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to send heartbeat: {e}")
|
||||
return
|
||||
await asyncio.sleep(self.HEARTBEAT_INTERVAL_SECONDS)
|
||||
|
||||
async def send_status_update(self, status: str, task_id: Optional[str] = None, message: str = ""):
|
||||
# Report a human-readable task or agent status transition to the orchestrator.
|
||||
try:
|
||||
await self.safe_send({
|
||||
"type": "status_update",
|
||||
"agent_id": self.agent_id,
|
||||
"status": status,
|
||||
"task_id": task_id,
|
||||
"message": message,
|
||||
"timestamp": time.time(),
|
||||
})
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to send status update: {e}")
|
||||
|
||||
async def request_peer_collaboration(
|
||||
self,
|
||||
task_id: str,
|
||||
target_agent_id: str,
|
||||
content: str,
|
||||
timeout_seconds: float = 20.0,
|
||||
):
|
||||
# Ask a peer agent (via the orchestrator) for guidance and wait for its reply.
|
||||
correlation_id = f"peer-{task_id}-{uuid.uuid4().hex[:8]}"
|
||||
loop = asyncio.get_running_loop()
|
||||
waiter = loop.create_future()
|
||||
self.peer_waiters[correlation_id] = waiter
|
||||
try:
|
||||
await self.safe_send({
|
||||
"type": "peer_message",
|
||||
"agent_id": self.agent_id,
|
||||
"target_agent_id": target_agent_id,
|
||||
"task_id": task_id,
|
||||
"content": content,
|
||||
"correlation_id": correlation_id,
|
||||
"is_reply": False,
|
||||
"timestamp": time.time(),
|
||||
})
|
||||
return await asyncio.wait_for(waiter, timeout=timeout_seconds)
|
||||
finally:
|
||||
self.peer_waiters.pop(correlation_id, None)
|
||||
|
||||
@staticmethod
|
||||
def _summarize_result(result: dict) -> Optional[str]:
|
||||
# Extract a short, human-readable summary from an execution result.
|
||||
if not isinstance(result, dict):
|
||||
return None
|
||||
for subtask_result in reversed(result.get("subtasks", []) or []):
|
||||
summary = subtask_result.get("summary")
|
||||
if isinstance(summary, str) and summary.strip():
|
||||
return summary.strip()
|
||||
summary = result.get("summary")
|
||||
return summary.strip() if isinstance(summary, str) and summary.strip() else None
|
||||
|
||||
async def answer_peer_query(self, message: dict):
|
||||
# Respond to an inbound peer query, sharing what this agent has actually done so far.
|
||||
requester = message.get("from_agent_id") or message.get("agent_id")
|
||||
correlation_id = message.get("correlation_id")
|
||||
if not requester or not correlation_id:
|
||||
return
|
||||
shared = (
|
||||
f"My latest result: {self.last_summary}"
|
||||
if self.last_summary
|
||||
else "I have no completed result to share yet."
|
||||
)
|
||||
reply_content = (
|
||||
f"From {self.agent_id} (capabilities: {', '.join(self.capabilities)}). {shared} "
|
||||
"Treat implementation artifacts as the source of truth for behavior and exception semantics."
|
||||
)
|
||||
try:
|
||||
await self.safe_send({
|
||||
"type": "peer_message",
|
||||
"agent_id": self.agent_id,
|
||||
"target_agent_id": requester,
|
||||
"task_id": message.get("task_id"),
|
||||
"content": reply_content,
|
||||
"correlation_id": correlation_id,
|
||||
"is_reply": True,
|
||||
"timestamp": time.time(),
|
||||
})
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to answer peer query: {e}")
|
||||
|
||||
async def send_task_result(self, task_id: str, success: bool, result: dict):
|
||||
# Send a completion or failure payload for a finished task execution.
|
||||
try:
|
||||
payload = {
|
||||
"type": "task_complete" if success else "task_failed",
|
||||
"agent_id": self.agent_id,
|
||||
"task_id": task_id,
|
||||
"timestamp": time.time(),
|
||||
}
|
||||
if success:
|
||||
payload["result"] = result
|
||||
else:
|
||||
payload["reason"] = result.get("error", "Task failed")
|
||||
payload["result"] = result
|
||||
await self.safe_send(payload)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to send task result: {e}")
|
||||
|
||||
async def send_blocked_on_handoff(self, task_id: str, result: dict):
|
||||
# Report that a task is paused because execution delegated work to a child handoff task.
|
||||
try:
|
||||
child_task_id = None
|
||||
for subtask_result in result.get("subtasks", []):
|
||||
if subtask_result.get("status") == "handed_off":
|
||||
child_task_id = subtask_result.get("child_task_id")
|
||||
break
|
||||
await self.safe_send({
|
||||
"type": "blocked_on_handoff",
|
||||
"agent_id": self.agent_id,
|
||||
"task_id": task_id,
|
||||
"child_task_id": child_task_id,
|
||||
"reason": "Waiting on delegated child task",
|
||||
"result": result,
|
||||
"timestamp": time.time(),
|
||||
})
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to send blocked_on_handoff: {e}")
|
||||
|
||||
async def request_handoff(self, task_id: str, subtask: dict, target_capabilities: list[str]):
|
||||
# Ask the orchestrator to delegate a discovered subtask to another capable agent.
|
||||
try:
|
||||
await self.safe_send({
|
||||
"type": "handoff_request",
|
||||
"agent_id": self.agent_id,
|
||||
"task_id": task_id,
|
||||
"subtask": subtask,
|
||||
"target_capabilities": target_capabilities,
|
||||
"timestamp": time.time(),
|
||||
})
|
||||
HANDOFFS_INITIATED.inc()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to request handoff: {e}")
|
||||
|
||||
async def execute_assignment(self, assignment: TaskAssignment):
|
||||
# Execute one accepted assignment, manage workspace/git flow, and publish lifecycle updates.
|
||||
async with self.task_semaphore:
|
||||
task_id = assignment.task_id
|
||||
description = assignment.description
|
||||
context = assignment.context or {}
|
||||
self.current_task_id = task_id
|
||||
ACTIVE_TASKS.set(len(self.active_tasks))
|
||||
AGENT_STATUS.set(1)
|
||||
|
||||
await self.safe_send({
|
||||
"type": "task_start",
|
||||
"agent_id": self.agent_id,
|
||||
"task_id": task_id,
|
||||
"timestamp": time.time(),
|
||||
})
|
||||
await self.send_status_update("busy", task_id, "Starting task execution")
|
||||
|
||||
start_time = time.time()
|
||||
task_workspace = self.task_workspace(task_id)
|
||||
task_workspace.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
git_enabled = False
|
||||
try:
|
||||
executor = TaskExecutor(agent_id=self.agent_id, workspace_dir=str(task_workspace))
|
||||
git_enabled = await self.workspace_git.is_git_workspace()
|
||||
if git_enabled:
|
||||
branch_created = await self.workspace_git.create_result_branch(task_id)
|
||||
if not branch_created:
|
||||
logger.warning("Failed to create task result branch; continuing without git push")
|
||||
git_enabled = False
|
||||
|
||||
result = await asyncio.wait_for(
|
||||
executor.execute_task(
|
||||
task_id=task_id,
|
||||
description=description,
|
||||
context={
|
||||
**context,
|
||||
"workspace_dir": str(task_workspace),
|
||||
"repo_workspace_dir": str(self.workspace_dir),
|
||||
"git_repo_url": self.git_repo_url,
|
||||
"agent_id": self.agent_id,
|
||||
},
|
||||
handoff_callback=self.request_handoff,
|
||||
agent_capabilities=self.capabilities,
|
||||
peer_collaboration_callback=self.request_peer_collaboration,
|
||||
),
|
||||
timeout=self.TASK_TIMEOUT_SECONDS,
|
||||
)
|
||||
|
||||
awaiting_handoff = result.get("awaiting_handoff", False)
|
||||
if result.get("success"):
|
||||
self.last_summary = self._summarize_result(result) or self.last_summary
|
||||
if result.get("success") and not awaiting_handoff:
|
||||
if git_enabled:
|
||||
commit_sha = await self.workspace_git.commit_changes(
|
||||
message=f"Task {task_id}: {description[:50]}"
|
||||
)
|
||||
if commit_sha:
|
||||
branch_name = await self.workspace_git.push_results()
|
||||
result["git_branch"] = branch_name
|
||||
result["commit_sha"] = commit_sha
|
||||
else:
|
||||
result["git_skipped"] = "No workspace changes to commit"
|
||||
else:
|
||||
result["git_skipped"] = "Workspace is not a Git checkout"
|
||||
|
||||
if awaiting_handoff:
|
||||
await self.send_blocked_on_handoff(task_id, result)
|
||||
else:
|
||||
await self.send_task_result(task_id, result.get("success", False), result)
|
||||
|
||||
duration = time.time() - start_time
|
||||
TASK_DURATION.observe(duration)
|
||||
if result.get("success"):
|
||||
TASKS_EXECUTED.inc()
|
||||
else:
|
||||
TASKS_FAILED.inc()
|
||||
|
||||
AGENT_STATUS.set(0)
|
||||
if awaiting_handoff:
|
||||
await self.send_status_update("handoff-pending", task_id, "Waiting for delegated child task")
|
||||
await self.send_status_update("idle", None, "Delegated child task created")
|
||||
else:
|
||||
await self.send_status_update("idle", None, "Task completed")
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
TASKS_FAILED.inc()
|
||||
AGENT_STATUS.set(2)
|
||||
await self.send_task_result(task_id, False, {"error": "timeout", "success": False})
|
||||
await self.send_status_update("idle", None, "Task failed: timeout")
|
||||
except asyncio.CancelledError:
|
||||
await self.send_task_result(task_id, False, {"error": "cancelled", "success": False})
|
||||
await self.send_status_update("idle", None, "Task cancelled")
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Error executing task {task_id}: {e}")
|
||||
TASKS_FAILED.inc()
|
||||
AGENT_STATUS.set(2)
|
||||
await self.send_task_result(task_id, False, {"error": str(e), "success": False})
|
||||
await self.send_status_update("idle", None, f"Task failed: {e}")
|
||||
finally:
|
||||
self.active_tasks.pop(task_id, None)
|
||||
ACTIVE_TASKS.set(len(self.active_tasks))
|
||||
self.current_task_id = None
|
||||
if not self.active_tasks:
|
||||
AGENT_STATUS.set(0)
|
||||
|
||||
async def cancel_task(self, task_id: str):
|
||||
# Cancel an actively running asyncio task if the orchestrator requests termination.
|
||||
task = self.active_tasks.get(task_id)
|
||||
if task:
|
||||
task.cancel()
|
||||
|
||||
async def handle_task_assignment(self, message: dict):
|
||||
# Validate an incoming assignment, reject duplicates/capacity overflow, and start execution.
|
||||
try:
|
||||
assignment = TaskAssignment(**message)
|
||||
except ValidationError:
|
||||
logger.warning(f"invalid task assignment: {message}")
|
||||
return
|
||||
|
||||
if assignment.task_id in self.active_tasks:
|
||||
TASKS_DUPLICATE.inc()
|
||||
await self.safe_send({
|
||||
"type": "task_accepted",
|
||||
"task_id": assignment.task_id,
|
||||
"status": "duplicate",
|
||||
})
|
||||
return
|
||||
|
||||
if len(self.active_tasks) >= self.MAX_CONCURRENT_TASKS:
|
||||
TASKS_REJECTED.inc()
|
||||
await self.safe_send({
|
||||
"type": "task_rejected",
|
||||
"task_id": assignment.task_id,
|
||||
"reason": "at_capacity",
|
||||
"available_slots": self.available_slots(),
|
||||
})
|
||||
return
|
||||
|
||||
await self.safe_send({
|
||||
"type": "task_accepted",
|
||||
"task_id": assignment.task_id,
|
||||
"available_slots": self.available_slots() - 1,
|
||||
})
|
||||
|
||||
task = asyncio.create_task(self.execute_assignment(assignment))
|
||||
self.active_tasks[assignment.task_id] = task
|
||||
ACTIVE_TASKS.set(len(self.active_tasks))
|
||||
|
||||
def _resolve_peer_reply(self, message: dict) -> bool:
|
||||
# Resolve the waiting future for an inbound peer reply; return True if it was a reply.
|
||||
correlation_id = message.get("correlation_id")
|
||||
if not correlation_id:
|
||||
return False
|
||||
waiter = self.peer_waiters.get(correlation_id)
|
||||
if waiter and not waiter.done():
|
||||
waiter.set_result(message)
|
||||
return True
|
||||
# A correlation we own but already resolved/timed out: treat as handled reply.
|
||||
return bool(message.get("is_reply"))
|
||||
|
||||
async def handle_peer_message(self, message: dict):
|
||||
# Route a peer message: resolve our own pending request, or answer an inbound query.
|
||||
if message.get("is_reply") or message.get("correlation_id") in self.peer_waiters:
|
||||
handled = self._resolve_peer_reply(message)
|
||||
if handled:
|
||||
return
|
||||
logger.info(
|
||||
"Received peer query for task %s from %s",
|
||||
message.get("task_id"),
|
||||
message.get("from_agent_id") or message.get("agent_id"),
|
||||
)
|
||||
await self.answer_peer_query(message)
|
||||
|
||||
async def handle_message(self, message: dict):
|
||||
# Route each inbound orchestrator message to the appropriate handler.
|
||||
msg_type = message.get("type")
|
||||
control_messages = {"registered", "heartbeat_ack", "task_completed", "task_failed_ack", "task_blocked_ack"}
|
||||
|
||||
if msg_type == "task_assignment":
|
||||
await self.handle_task_assignment(message)
|
||||
elif msg_type == "handoff_response":
|
||||
logger.info(f"Handoff accepted for task {message.get('task_id')}")
|
||||
elif msg_type == "cancel_task":
|
||||
await self.cancel_task(message["task_id"])
|
||||
elif msg_type == "ping":
|
||||
await self.safe_send({"type": "pong"})
|
||||
elif msg_type == "peer_message":
|
||||
await self.handle_peer_message(message)
|
||||
elif msg_type in control_messages:
|
||||
logger.debug(f"Received control message: {msg_type}")
|
||||
else:
|
||||
logger.warning(f"Unknown message type: {msg_type}")
|
||||
|
||||
async def message_loop(self):
|
||||
# Continuously receive websocket messages and dispatch them until the connection ends.
|
||||
async for raw in self.websocket:
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
await self.handle_message(data)
|
||||
except json.JSONDecodeError as e:
|
||||
logger.error(f"Failed to parse message: {e}")
|
||||
except Exception as e:
|
||||
logger.error(f"Error handling message: {e}")
|
||||
|
||||
async def shutdown_active_tasks(self):
|
||||
# Cancel and await all active task coroutines during agent shutdown.
|
||||
tasks = list(self.active_tasks.values())
|
||||
for task in tasks:
|
||||
task.cancel()
|
||||
if tasks:
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
def install_signal_handlers(self):
|
||||
# Register process signal handlers that trigger a graceful runtime shutdown.
|
||||
def _handle_shutdown():
|
||||
# Flip the runtime into shutdown mode and close the websocket asynchronously.
|
||||
logger.info("shutdown signal received")
|
||||
self.running = False
|
||||
if self.websocket:
|
||||
asyncio.create_task(self.websocket.close())
|
||||
|
||||
for sig in (signal.SIGINT, signal.SIGTERM):
|
||||
try:
|
||||
asyncio.get_running_loop().add_signal_handler(sig, _handle_shutdown)
|
||||
except NotImplementedError:
|
||||
# add_signal_handler is unsupported on Windows event loops; skip gracefully.
|
||||
pass
|
||||
|
||||
async def run(self):
|
||||
# Run the full agent lifecycle: startup, registration, message processing, reconnect, and cleanup.
|
||||
self.running = True
|
||||
self.install_signal_handlers()
|
||||
|
||||
if os.getenv("METRICS_PORT"):
|
||||
start_http_server(int(os.getenv("METRICS_PORT", "9000")))
|
||||
|
||||
if self.git_repo_url:
|
||||
logger.info(f"Cloning workspace from {self.git_repo_url}")
|
||||
if not await self.workspace_git.clone_workspace(self.git_repo_url):
|
||||
logger.error("Failed to clone workspace, exiting")
|
||||
return
|
||||
|
||||
while self.running:
|
||||
connected = await self.connect_with_retry()
|
||||
if not connected:
|
||||
break
|
||||
|
||||
if not await self.register():
|
||||
logger.error("Failed to register with orchestrator, exiting")
|
||||
return
|
||||
|
||||
self.heartbeat_task = asyncio.create_task(self.heartbeat_loop())
|
||||
try:
|
||||
await self.message_loop()
|
||||
except ConnectionClosed:
|
||||
logger.warning("connection closed; reconnecting")
|
||||
except Exception as e:
|
||||
logger.error(f"Error in message loop: {e}")
|
||||
finally:
|
||||
if self.heartbeat_task:
|
||||
self.heartbeat_task.cancel()
|
||||
try:
|
||||
await self.heartbeat_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self.heartbeat_task = None
|
||||
|
||||
if self.websocket:
|
||||
try:
|
||||
await self.safe_send({"type": "deregister", "agent_id": self.agent_id})
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
await self.websocket.close()
|
||||
except Exception:
|
||||
pass
|
||||
self.websocket = None
|
||||
|
||||
if self.running:
|
||||
await asyncio.sleep(1)
|
||||
|
||||
await self.shutdown_active_tasks()
|
||||
logger.info("Agent shutdown complete")
|
||||
|
||||
|
||||
async def main():
|
||||
# Build an agent instance from environment configuration and start its runtime loop.
|
||||
orchestrator_url = os.getenv("ORCHESTRATOR_URL", "ws://localhost:8000")
|
||||
agent_id = os.getenv("AGENT_ID")
|
||||
capabilities = os.getenv("AGENT_CAPABILITIES", "general").split(",")
|
||||
workspace_dir = os.getenv("WORKSPACE_DIR", "/workspace")
|
||||
git_repo_url = os.getenv("GIT_REPO_URL")
|
||||
|
||||
logger.info(f"Starting agent with ID: {agent_id or 'auto-generated'}")
|
||||
logger.info(f"Capabilities: {capabilities}")
|
||||
logger.info(f"Orchestrator URL: {orchestrator_url}")
|
||||
|
||||
agent = Agent(
|
||||
orchestrator_url=orchestrator_url,
|
||||
agent_id=agent_id,
|
||||
capabilities=capabilities,
|
||||
workspace_dir=workspace_dir,
|
||||
git_repo_url=git_repo_url,
|
||||
)
|
||||
await agent.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,5 @@
|
||||
openai==1.55.3
|
||||
websockets==13.1
|
||||
pydantic==2.9.2
|
||||
python-dotenv==1.0.1
|
||||
prometheus-client==0.20.0
|
||||
@@ -0,0 +1,480 @@
|
||||
"""Task execution engine with OpenAI-compatible LLM provider integration.
|
||||
|
||||
Merged from agent_swarm_v4:
|
||||
- OpenAI-only provider configuration (OPENAI_*/MODEL_* env vars and custom base URLs)
|
||||
- Peer-collaboration hook and specialist-role alignment prompting
|
||||
- Preserves model invocation, handoff decision hooks, workspace summarization,
|
||||
file-application behavior, and the Manager billing/audit usage attribution
|
||||
(X-Agent/X-Agnet headers, usage payload, billing_source)
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
|
||||
from openai import AsyncOpenAI
|
||||
|
||||
from .handoff_logic import should_handoff
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TaskExecutor:
|
||||
"""Executes tasks using a configurable OpenAI-compatible API."""
|
||||
|
||||
def __init__(self, agent_id: str, workspace_dir: str):
|
||||
self.agent_id = agent_id
|
||||
self.workspace_dir = workspace_dir
|
||||
|
||||
api_key = (
|
||||
os.getenv("OPENAI_API_KEY")
|
||||
or os.getenv("MODEL_API_KEY")
|
||||
)
|
||||
if not api_key:
|
||||
raise ValueError("OPENAI_API_KEY environment variable not set")
|
||||
|
||||
self.model = (
|
||||
os.getenv("OPENAI_MODEL")
|
||||
or os.getenv("MODEL_NAME")
|
||||
or os.getenv("MODEL_ID")
|
||||
or "gpt-4o-mini"
|
||||
)
|
||||
self.api_base = (
|
||||
os.getenv("OPENAI_API_BASE")
|
||||
or os.getenv("MODEL_API_BASE")
|
||||
or "https://api.openai.com/v1"
|
||||
)
|
||||
self.api_mode = "openai"
|
||||
self.client = AsyncOpenAI(api_key=api_key, base_url=self.api_base)
|
||||
self.client_type = "openai"
|
||||
|
||||
self.usage = self._empty_usage()
|
||||
self.current_context: dict = {}
|
||||
|
||||
async def execute_task(
|
||||
self,
|
||||
task_id: str,
|
||||
description: str,
|
||||
context: dict,
|
||||
handoff_callback: Optional[Callable] = None,
|
||||
agent_capabilities: Optional[list[str]] = None,
|
||||
peer_collaboration_callback: Optional[Callable] = None,
|
||||
) -> dict:
|
||||
logger.info(f"Executing task {task_id}: {description}")
|
||||
|
||||
try:
|
||||
started_at = time.time()
|
||||
self.current_context = context or {}
|
||||
self.usage = self._empty_usage(context)
|
||||
multi_agent_leaf_mode = self._multi_agent_leaf_mode(context)
|
||||
allow_handoff = self._allow_dynamic_handoff(context)
|
||||
|
||||
if multi_agent_leaf_mode:
|
||||
subtasks = [{
|
||||
"description": description,
|
||||
"complexity": "medium",
|
||||
"estimated_time": 30,
|
||||
"dependencies": [],
|
||||
"required_capabilities": context.get("required_capabilities") or ["general"],
|
||||
}]
|
||||
elif self._subtask_handoff_enabled():
|
||||
subtasks = await self._parse_task(description, context)
|
||||
else:
|
||||
subtasks = [{
|
||||
"description": description,
|
||||
"complexity": "medium",
|
||||
"estimated_time": 30,
|
||||
"dependencies": [],
|
||||
"required_capabilities": ["general"],
|
||||
}]
|
||||
|
||||
results = []
|
||||
for subtask in subtasks:
|
||||
if handoff_callback and self._subtask_handoff_enabled() and allow_handoff:
|
||||
decision = should_handoff(
|
||||
subtask,
|
||||
self.agent_id,
|
||||
agent_capabilities=agent_capabilities,
|
||||
)
|
||||
if decision.should_handoff:
|
||||
await handoff_callback(
|
||||
task_id=task_id,
|
||||
subtask=subtask,
|
||||
target_capabilities=decision.target_capabilities,
|
||||
)
|
||||
results.append({
|
||||
"subtask": subtask,
|
||||
"status": "handed_off",
|
||||
"reason": decision.reason,
|
||||
"target_capabilities": decision.target_capabilities,
|
||||
})
|
||||
continue
|
||||
|
||||
results.append(await self._execute_subtask(subtask, task_id, context, peer_collaboration_callback))
|
||||
|
||||
success = all(r["status"] in ["completed", "handed_off"] for r in results)
|
||||
awaiting_handoff = any(r["status"] == "handed_off" for r in results)
|
||||
return {
|
||||
"success": success,
|
||||
"task_id": task_id,
|
||||
"subtasks": results,
|
||||
"awaiting_handoff": awaiting_handoff,
|
||||
"agent_id": self.agent_id,
|
||||
"usage": self._usage_payload(time.time() - started_at),
|
||||
}
|
||||
except Exception as e:
|
||||
logger.error(f"Error executing task {task_id}: {e}")
|
||||
return {
|
||||
"success": False,
|
||||
"task_id": task_id,
|
||||
"error": str(e),
|
||||
"agent_id": self.agent_id,
|
||||
"usage": self._usage_payload(time.time() - started_at if "started_at" in locals() else 0),
|
||||
}
|
||||
finally:
|
||||
self.current_context = {}
|
||||
|
||||
async def _parse_task(self, description: str, context: dict) -> list[dict]:
|
||||
try:
|
||||
prompt = f"""You are a task planning assistant. Break down the following programming task into concrete, actionable subtasks.
|
||||
|
||||
Task: {description}
|
||||
|
||||
Context: {json.dumps(context, indent=2)}
|
||||
|
||||
Return a JSON array of subtasks, where each subtask has:
|
||||
- description: Clear description of what needs to be done
|
||||
- complexity: \"low\", \"medium\", or \"high\"
|
||||
- estimated_time: Estimated time in minutes
|
||||
- dependencies: List of subtask indices this depends on (empty if none)
|
||||
- required_capabilities: List of capabilities needed
|
||||
|
||||
Return ONLY the JSON array, no other text."""
|
||||
content = await self._complete(prompt, max_tokens=2000)
|
||||
return json.loads(self._strip_json_fence(content))
|
||||
except Exception as e:
|
||||
logger.error(f"Error parsing task: {e}")
|
||||
return [{
|
||||
"description": description,
|
||||
"complexity": "medium",
|
||||
"estimated_time": 30,
|
||||
"dependencies": [],
|
||||
"required_capabilities": ["general"],
|
||||
}]
|
||||
|
||||
async def _execute_subtask(self, subtask: dict, task_id: str, context: dict, peer_collaboration_callback: Optional[Callable]) -> dict:
|
||||
try:
|
||||
description = subtask["description"]
|
||||
workspace_files = self._summarize_workspace()
|
||||
workspace_context = self._collect_workspace_context()
|
||||
user_prompt = context.get("user_prompt") or context.get("run_goal") or context.get("root_task_description") or ""
|
||||
specialist_role = context.get("specialist_role", "general")
|
||||
dependency_artifacts = context.get("dependency_artifacts") or []
|
||||
implementation_artifacts = [
|
||||
artifact for artifact in dependency_artifacts
|
||||
if "implementation" in (artifact.get("task_id") or "")
|
||||
]
|
||||
testing_artifacts = [
|
||||
artifact for artifact in dependency_artifacts
|
||||
if "testing" in (artifact.get("task_id") or "")
|
||||
]
|
||||
peer_context = []
|
||||
peer_agents = context.get("peer_agents") or []
|
||||
should_consult_peers = specialist_role in {"testing", "documentation"}
|
||||
if peer_collaboration_callback and peer_agents and should_consult_peers:
|
||||
max_peer_consults = int((context or {}).get("max_peer_consults", 2) or 2)
|
||||
preferred_roles = []
|
||||
if specialist_role in {"testing", "documentation"}:
|
||||
preferred_roles = ["implementation"]
|
||||
|
||||
ordered_peers = sorted(
|
||||
peer_agents,
|
||||
key=lambda peer: 0 if peer.get("role") in preferred_roles else 1,
|
||||
)
|
||||
|
||||
for peer in ordered_peers[:max_peer_consults]:
|
||||
try:
|
||||
reply = await peer_collaboration_callback(
|
||||
task_id=task_id,
|
||||
target_agent_id=peer.get("agent_id"),
|
||||
content=(
|
||||
f"Specialist role: {specialist_role}. "
|
||||
f"Please provide guidance and confirm behavior for: {description}. "
|
||||
f"Original user request: {user_prompt}"
|
||||
),
|
||||
timeout_seconds=float((context or {}).get("peer_timeout_seconds", 10.0) or 10.0),
|
||||
)
|
||||
peer_context.append(
|
||||
{
|
||||
"agent_id": peer.get("agent_id"),
|
||||
"role": peer.get("role"),
|
||||
"content": reply.get("content", ""),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(f"Peer collaboration failed with {peer.get('agent_id')}: {exc}")
|
||||
prompt = f"""You are a programming assistant working in a collaborative agent system.
|
||||
|
||||
Original user request:
|
||||
{user_prompt}
|
||||
|
||||
Your specialist role:
|
||||
{specialist_role}
|
||||
|
||||
Task: {description}
|
||||
|
||||
Dependency artifacts from other specialists:
|
||||
{json.dumps(dependency_artifacts, indent=2)}
|
||||
|
||||
Implementation artifacts relevant to alignment:
|
||||
{json.dumps(implementation_artifacts, indent=2)}
|
||||
|
||||
Testing artifacts relevant to alignment:
|
||||
{json.dumps(testing_artifacts, indent=2)}
|
||||
|
||||
Workspace directory: {self.workspace_dir}
|
||||
Current workspace files:
|
||||
{json.dumps(workspace_files, indent=2)}
|
||||
Relevant workspace file contents:
|
||||
{json.dumps(workspace_context, indent=2)}
|
||||
Peer specialist input:
|
||||
{json.dumps(peer_context, indent=2)}
|
||||
|
||||
Alignment requirements:
|
||||
- If your role is testing, align your tests with the implementation artifacts and their stated error semantics.
|
||||
- If your role is documentation, align your docs with both implementation and testing artifacts.
|
||||
- Do not invent behavior that conflicts with dependency artifacts unless you explicitly surface an error.
|
||||
- Treat implementation artifacts as the source of truth for API behavior and exception semantics.
|
||||
- If peer specialist input conflicts with implementation artifacts, prefer implementation semantics and explain the correction in your changes summary.
|
||||
- If your role is documentation or testing, update only your specialist outputs to converge on implementation behavior unless the implementation artifact is clearly missing or contradictory.
|
||||
|
||||
Return your response as JSON with this structure:
|
||||
{{
|
||||
\"status\": \"completed\" or \"failed\",
|
||||
\"summary\": \"Brief description of what was done\",
|
||||
\"files\": [{{\"path\": \"relative/path.py\", \"action\": \"write\", \"content\": \"complete file content\"}}],
|
||||
\"changes\": \"Detailed description of changes\",
|
||||
\"error\": \"Error message if failed, null otherwise\"
|
||||
}}
|
||||
|
||||
Return ONLY the JSON, no other text."""
|
||||
content = await self._complete(prompt, max_tokens=4000)
|
||||
result = self._parse_json_response(content)
|
||||
if result.get("status") == "completed":
|
||||
apply_result = await self._apply_file_changes(result.get("files", []))
|
||||
result["files_modified"] = apply_result["files_modified"]
|
||||
result["files_deleted"] = apply_result["files_deleted"]
|
||||
if apply_result["errors"]:
|
||||
result["status"] = "failed"
|
||||
result["error"] = "; ".join(apply_result["errors"])
|
||||
result["subtask"] = subtask
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.error(f"Error executing subtask: {e}")
|
||||
return {
|
||||
"subtask": subtask,
|
||||
"status": "failed",
|
||||
"error": str(e),
|
||||
"summary": f"Failed to execute: {e}",
|
||||
}
|
||||
|
||||
async def _complete(self, prompt: str, max_tokens: int) -> str:
|
||||
extra_headers = self._model_attribution_headers()
|
||||
response = await self.client.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
max_tokens=max_tokens,
|
||||
extra_headers=extra_headers or None,
|
||||
)
|
||||
self._record_openai_usage(response)
|
||||
return response.choices[0].message.content or ""
|
||||
|
||||
def _empty_usage(self, context: Optional[dict] = None) -> dict:
|
||||
plan = ((context or {}).get("orchestration_plan") or {})
|
||||
billing = plan.get("billing_context") or {}
|
||||
return {
|
||||
"model_id": self.model,
|
||||
"model_tokens": 0,
|
||||
"prompt_tokens": 0,
|
||||
"completion_tokens": 0,
|
||||
"model_cost_usd": 0.0,
|
||||
"runtime_seconds": 0.0,
|
||||
"billing_source": billing.get("provider") or os.getenv("BILLING_SOURCE", "unknown"),
|
||||
}
|
||||
|
||||
def _record_openai_usage(self, response):
|
||||
usage = getattr(response, "usage", None)
|
||||
if not usage:
|
||||
return
|
||||
prompt_tokens = int(getattr(usage, "prompt_tokens", 0) or 0)
|
||||
completion_tokens = int(getattr(usage, "completion_tokens", 0) or 0)
|
||||
total_tokens = int(getattr(usage, "total_tokens", 0) or prompt_tokens + completion_tokens)
|
||||
self._add_usage(prompt_tokens, completion_tokens, total_tokens)
|
||||
|
||||
def _add_usage(self, prompt_tokens: int, completion_tokens: int, total_tokens: int):
|
||||
self.usage["prompt_tokens"] += prompt_tokens
|
||||
self.usage["completion_tokens"] += completion_tokens
|
||||
self.usage["model_tokens"] += total_tokens
|
||||
input_cost = float(os.getenv("MODEL_INPUT_COST_PER_1M", "0") or 0)
|
||||
output_cost = float(os.getenv("MODEL_OUTPUT_COST_PER_1M", "0") or 0)
|
||||
self.usage["model_cost_usd"] += (
|
||||
prompt_tokens * input_cost / 1_000_000
|
||||
+ completion_tokens * output_cost / 1_000_000
|
||||
)
|
||||
|
||||
def _usage_payload(self, runtime_seconds: float) -> dict:
|
||||
usage = dict(self.usage)
|
||||
usage["runtime_seconds"] = runtime_seconds
|
||||
return usage
|
||||
|
||||
def _model_attribution_headers(self) -> dict:
|
||||
headers = {}
|
||||
mapping = {
|
||||
"manager_deployment_id": ["X-Agent-Manager-Deployment-ID", "X-Agnet-Manager-Deployment-ID"],
|
||||
"swarm_id": ["X-Agent-Swarm-ID", "X-Agnet-Swarm-ID"],
|
||||
"task_id": ["X-Agent-Task-ID", "X-Agnet-Task-ID"],
|
||||
"agent_role": ["X-Agent-Agent-Role", "X-Agnet-Agent-Role"],
|
||||
"correlation_id": ["X-Correlation-ID"],
|
||||
"model_id": ["X-Agent-Model-ID", "X-Agnet-Model-ID"],
|
||||
}
|
||||
for key, header_names in mapping.items():
|
||||
value = self.current_context.get(key)
|
||||
if value:
|
||||
for header in header_names:
|
||||
headers[header] = str(value)
|
||||
headers.setdefault("X-Agent-Model-ID", self.model)
|
||||
headers.setdefault("X-Agnet-Model-ID", self.model)
|
||||
return headers
|
||||
|
||||
def _subtask_handoff_enabled(self) -> bool:
|
||||
return os.getenv("ENABLE_SUBTASK_HANDOFF", "false").lower() in {"1", "true", "yes"}
|
||||
|
||||
def _multi_agent_leaf_mode(self, context: Optional[dict]) -> bool:
|
||||
return self._subtask_handoff_enabled() and (context or {}).get("workflow_mode") == "multi_agent"
|
||||
|
||||
def _allow_dynamic_handoff(self, context: Optional[dict]) -> bool:
|
||||
if (context or {}).get("workflow_mode") != "multi_agent":
|
||||
return True
|
||||
return bool((context or {}).get("allow_handoff", False))
|
||||
|
||||
def _strip_json_fence(self, content: str) -> str:
|
||||
content = content.strip()
|
||||
if not content.startswith("```"):
|
||||
return content
|
||||
lines = content.split("\n")
|
||||
if lines and lines[0].startswith("```"):
|
||||
lines = lines[1:]
|
||||
if lines and lines[-1].strip() == "```":
|
||||
lines = lines[:-1]
|
||||
return "\n".join(lines).strip()
|
||||
|
||||
def _parse_json_response(self, content: str) -> dict:
|
||||
content = self._strip_json_fence(content)
|
||||
try:
|
||||
return json.loads(content)
|
||||
except json.JSONDecodeError:
|
||||
start = content.find("{")
|
||||
end = content.rfind("}")
|
||||
if start == -1 or end == -1 or end <= start:
|
||||
raise
|
||||
return json.loads(content[start:end + 1])
|
||||
|
||||
def _summarize_workspace(self, max_files: int = 80) -> list[str]:
|
||||
root = Path(self.workspace_dir)
|
||||
if not root.exists():
|
||||
return []
|
||||
ignored_dirs = {".git", "__pycache__", "node_modules", ".venv", "venv"}
|
||||
files = []
|
||||
for path in root.rglob("*"):
|
||||
if len(files) >= max_files:
|
||||
break
|
||||
if not path.is_file():
|
||||
continue
|
||||
if any(part in ignored_dirs for part in path.relative_to(root).parts):
|
||||
continue
|
||||
files.append(str(path.relative_to(root)))
|
||||
return sorted(files)
|
||||
|
||||
def _collect_workspace_context(self, max_files: int = 20, max_bytes_per_file: int = 6000) -> dict[str, str]:
|
||||
root = Path(self.workspace_dir)
|
||||
if not root.exists():
|
||||
return {}
|
||||
ignored_dirs = {".git", "__pycache__", "node_modules", ".venv", "venv"}
|
||||
allowed_suffixes = {".py", ".js", ".ts", ".tsx", ".jsx", ".json", ".md", ".txt", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".sh"}
|
||||
context = {}
|
||||
for path in sorted(root.rglob("*")):
|
||||
if len(context) >= max_files:
|
||||
break
|
||||
if not path.is_file():
|
||||
continue
|
||||
relative = path.relative_to(root)
|
||||
if any(part in ignored_dirs for part in relative.parts):
|
||||
continue
|
||||
if path.suffix and path.suffix.lower() not in allowed_suffixes:
|
||||
continue
|
||||
try:
|
||||
data = path.read_bytes()[:max_bytes_per_file]
|
||||
context[str(relative)] = data.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
continue
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to read workspace file {relative}: {e}")
|
||||
return context
|
||||
|
||||
def _resolve_workspace_path(self, file_path: str) -> Path:
|
||||
if not file_path or os.path.isabs(file_path):
|
||||
raise ValueError(f"Invalid relative path: {file_path}")
|
||||
root = Path(self.workspace_dir).resolve()
|
||||
resolved = (root / file_path).resolve()
|
||||
if root != resolved and root not in resolved.parents:
|
||||
raise ValueError(f"Path escapes workspace: {file_path}")
|
||||
return resolved
|
||||
|
||||
async def _apply_file_changes(self, files: list[dict]) -> dict:
|
||||
result = {"files_modified": [], "files_deleted": [], "errors": []}
|
||||
for file_change in files:
|
||||
path = file_change.get("path")
|
||||
action = file_change.get("action", "write")
|
||||
try:
|
||||
target = self._resolve_workspace_path(path)
|
||||
if action == "write":
|
||||
content = file_change.get("content")
|
||||
if content is None:
|
||||
raise ValueError(f"Missing content for {path}")
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(content, encoding="utf-8")
|
||||
result["files_modified"].append(path)
|
||||
elif action == "delete":
|
||||
if target.exists():
|
||||
target.unlink()
|
||||
result["files_deleted"].append(path)
|
||||
else:
|
||||
raise ValueError(f"Unsupported file action for {path}: {action}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to apply file change {path}: {e}")
|
||||
result["errors"].append(f"{path}: {e}")
|
||||
return result
|
||||
|
||||
async def read_file(self, file_path: str) -> Optional[str]:
|
||||
try:
|
||||
full_path = os.path.join(self.workspace_dir, file_path)
|
||||
with open(full_path, "r", encoding="utf-8") as f:
|
||||
return f.read()
|
||||
except Exception as e:
|
||||
logger.error(f"Error reading file {file_path}: {e}")
|
||||
return None
|
||||
|
||||
async def write_file(self, file_path: str, content: str) -> bool:
|
||||
try:
|
||||
full_path = self._resolve_workspace_path(file_path)
|
||||
os.makedirs(os.path.dirname(full_path), exist_ok=True)
|
||||
with open(full_path, "w", encoding="utf-8") as f:
|
||||
f.write(content)
|
||||
logger.info(f"Wrote file: {file_path}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Error writing file {file_path}: {e}")
|
||||
return False
|
||||
@@ -0,0 +1,115 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Swarm Terminal Snapshot</title>
|
||||
<style>
|
||||
body {
|
||||
margin: 0;
|
||||
background:
|
||||
radial-gradient(circle at top right, rgba(74, 222, 128, 0.10), transparent 20%),
|
||||
radial-gradient(circle at top left, rgba(96, 165, 250, 0.10), transparent 24%),
|
||||
#0a0f14;
|
||||
color: #d7e3ee;
|
||||
font-family: Menlo, Monaco, "SFMono-Regular", "JetBrains Mono", monospace;
|
||||
}
|
||||
.frame {
|
||||
width: 1600px;
|
||||
min-height: 900px;
|
||||
margin: 0 auto;
|
||||
padding: 48px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
.window {
|
||||
background: #0b1220;
|
||||
border: 1px solid rgba(148, 163, 184, 0.20);
|
||||
border-radius: 18px;
|
||||
overflow: hidden;
|
||||
box-shadow: 0 30px 80px rgba(0, 0, 0, 0.45);
|
||||
}
|
||||
.topbar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 14px 18px;
|
||||
background: #121a2a;
|
||||
border-bottom: 1px solid rgba(148, 163, 184, 0.12);
|
||||
}
|
||||
.dots {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}
|
||||
.dot {
|
||||
width: 12px;
|
||||
height: 12px;
|
||||
border-radius: 50%;
|
||||
}
|
||||
.dot.red { background: #fb7185; }
|
||||
.dot.yellow { background: #fbbf24; }
|
||||
.dot.green { background: #4ade80; }
|
||||
.title {
|
||||
color: #93a4b8;
|
||||
font-size: 15px;
|
||||
letter-spacing: 0.02em;
|
||||
}
|
||||
.terminal {
|
||||
padding: 28px 32px 36px;
|
||||
font-size: 24px;
|
||||
line-height: 1.7;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
.line:nth-child(1),
|
||||
.line:nth-child(8),
|
||||
.line:nth-child(13) {
|
||||
color: #7dd3fc;
|
||||
}
|
||||
.line:nth-last-child(2),
|
||||
.line:nth-last-child(1) {
|
||||
color: #bef264;
|
||||
font-weight: 600;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="frame">
|
||||
<div class="window">
|
||||
<div class="topbar">
|
||||
<div class="dots">
|
||||
<span class="dot red"></span>
|
||||
<span class="dot yellow"></span>
|
||||
<span class="dot green"></span>
|
||||
</div>
|
||||
<div class="title">swarm-terminal-snapshot</div>
|
||||
<div class="title">swarm-300589052dd0</div>
|
||||
</div>
|
||||
<div class="terminal">
|
||||
<div class='line'>$ python3 scripts/run_swarm_poster_demo.py</div>
|
||||
<div class='line'>[swarm] deployment_id=runtime-dep-90b592c531e3</div>
|
||||
<div class='line'>[swarm] swarm_id=swarm-300589052dd0</div>
|
||||
<div class='line'>[task] task_id=swarm-300589052dd0-task-1</div>
|
||||
<div class='line'>[task] status=in_progress</div>
|
||||
<div class='line'>[task] assigned_agent=agent-full-5d6968f886-7cm2l</div>
|
||||
<div class='line'></div>
|
||||
<div class='line'>$ curl -H "Authorization: Bearer ***" /api/swarms/{swarm_id}/tasks</div>
|
||||
<div class='line'>task.started_at = 2026-05-29 10:12:36 UTC</div>
|
||||
<div class='line'>task.halfway_at = 2026-05-29 10:12:56 UTC</div>
|
||||
<div class='line'>halfway.observed_at = 2026-05-29 10:12:58 UTC</div>
|
||||
<div class='line'></div>
|
||||
<div class='line'>$ curl -H "Authorization: Bearer ***" /api/swarms/{swarm_id}/metrics</div>
|
||||
<div class='line'>runtime.status = running</div>
|
||||
<div class='line'>runtime.tasks_total = 1</div>
|
||||
<div class='line'>runtime.tasks_by_status = {"in_progress": 1}</div>
|
||||
<div class='line'>runtime.agents_connected = 1</div>
|
||||
<div class='line'>runtime.budget.duration_seconds = 40</div>
|
||||
<div class='line'>runtime.budget.duration_ratio = 495.7%</div>
|
||||
<div class='line'></div>
|
||||
<div class='line'># milestone</div>
|
||||
<div class='line'>> 任务开始: 2026-05-29 10:12:36 UTC</div>
|
||||
<div class='line'>> 任务过半: 2026-05-29 10:12:58 UTC</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 223 KiB |
@@ -0,0 +1,285 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Swarm Blog MVP Live Demo</title>
|
||||
<style>
|
||||
:root {
|
||||
--bg: #07111f;
|
||||
--panel: rgba(10, 25, 47, 0.84);
|
||||
--panel-strong: rgba(12, 32, 60, 0.95);
|
||||
--line: rgba(125, 211, 252, 0.25);
|
||||
--cyan: #7dd3fc;
|
||||
--teal: #5eead4;
|
||||
--lime: #bef264;
|
||||
--text: #e6f0ff;
|
||||
--muted: #9bb1c8;
|
||||
--warn: #fbbf24;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: "Avenir Next", "PingFang SC", "Helvetica Neue", sans-serif;
|
||||
color: var(--text);
|
||||
background:
|
||||
radial-gradient(circle at top left, rgba(45, 212, 191, 0.18), transparent 32%),
|
||||
radial-gradient(circle at top right, rgba(125, 211, 252, 0.24), transparent 28%),
|
||||
linear-gradient(135deg, #050c16 0%, #07111f 42%, #0d1d35 100%);
|
||||
min-height: 100vh;
|
||||
}
|
||||
.canvas {
|
||||
width: 1600px;
|
||||
min-height: 900px;
|
||||
margin: 0 auto;
|
||||
padding: 56px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.grid {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
background-image:
|
||||
linear-gradient(rgba(125, 211, 252, 0.05) 1px, transparent 1px),
|
||||
linear-gradient(90deg, rgba(125, 211, 252, 0.05) 1px, transparent 1px);
|
||||
background-size: 56px 56px;
|
||||
mask-image: linear-gradient(to bottom, rgba(0,0,0,.75), transparent);
|
||||
pointer-events: none;
|
||||
}
|
||||
.hero {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 28px;
|
||||
align-items: flex-start;
|
||||
margin-bottom: 28px;
|
||||
}
|
||||
.hero h1 {
|
||||
margin: 0 0 12px 0;
|
||||
font-size: 68px;
|
||||
line-height: 0.95;
|
||||
letter-spacing: -2px;
|
||||
}
|
||||
.hero p {
|
||||
margin: 0;
|
||||
max-width: 800px;
|
||||
color: var(--muted);
|
||||
font-size: 24px;
|
||||
line-height: 1.5;
|
||||
}
|
||||
.badge {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
border: 1px solid var(--line);
|
||||
background: rgba(6, 19, 36, 0.72);
|
||||
border-radius: 999px;
|
||||
padding: 12px 18px;
|
||||
color: var(--cyan);
|
||||
font-size: 16px;
|
||||
letter-spacing: 0.08em;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
.layout {
|
||||
display: grid;
|
||||
grid-template-columns: 1.25fr 0.75fr;
|
||||
gap: 28px;
|
||||
}
|
||||
.panel {
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 28px;
|
||||
padding: 28px;
|
||||
box-shadow: 0 25px 60px rgba(0, 0, 0, 0.32);
|
||||
backdrop-filter: blur(16px);
|
||||
}
|
||||
.panel strong {
|
||||
display: block;
|
||||
font-size: 16px;
|
||||
letter-spacing: 0.08em;
|
||||
text-transform: uppercase;
|
||||
color: var(--cyan);
|
||||
margin-bottom: 14px;
|
||||
}
|
||||
.cards {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(3, 1fr);
|
||||
gap: 18px;
|
||||
margin-bottom: 28px;
|
||||
}
|
||||
.metric {
|
||||
background: var(--panel-strong);
|
||||
border: 1px solid rgba(190, 242, 100, 0.16);
|
||||
border-radius: 22px;
|
||||
padding: 20px;
|
||||
}
|
||||
.metric .label {
|
||||
color: var(--muted);
|
||||
font-size: 15px;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.metric .value {
|
||||
font-size: 34px;
|
||||
font-weight: 700;
|
||||
letter-spacing: -1px;
|
||||
}
|
||||
.timeline {
|
||||
display: grid;
|
||||
gap: 16px;
|
||||
margin-top: 8px;
|
||||
}
|
||||
.step {
|
||||
position: relative;
|
||||
padding: 18px 18px 18px 68px;
|
||||
border-radius: 22px;
|
||||
background: rgba(7, 18, 33, 0.9);
|
||||
border: 1px solid rgba(125, 211, 252, 0.12);
|
||||
}
|
||||
.step::before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
left: 28px;
|
||||
top: 26px;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border-radius: 50%;
|
||||
background: linear-gradient(135deg, var(--teal), var(--cyan));
|
||||
box-shadow: 0 0 0 8px rgba(94, 234, 212, 0.12);
|
||||
}
|
||||
.step h3 {
|
||||
margin: 0 0 8px 0;
|
||||
font-size: 24px;
|
||||
}
|
||||
.step p {
|
||||
margin: 0;
|
||||
color: var(--muted);
|
||||
font-size: 16px;
|
||||
line-height: 1.5;
|
||||
}
|
||||
.code {
|
||||
font-family: "SF Mono", "JetBrains Mono", monospace;
|
||||
color: var(--lime);
|
||||
word-break: break-all;
|
||||
}
|
||||
.sidebar {
|
||||
display: grid;
|
||||
gap: 18px;
|
||||
}
|
||||
.status {
|
||||
border-radius: 24px;
|
||||
padding: 22px;
|
||||
background: linear-gradient(160deg, rgba(94, 234, 212, 0.14), rgba(125, 211, 252, 0.08));
|
||||
border: 1px solid rgba(94, 234, 212, 0.25);
|
||||
}
|
||||
.status .headline {
|
||||
font-size: 18px;
|
||||
color: var(--muted);
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.status .value {
|
||||
font-size: 44px;
|
||||
font-weight: 700;
|
||||
letter-spacing: -1px;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
.status .sub {
|
||||
color: var(--muted);
|
||||
font-size: 16px;
|
||||
line-height: 1.5;
|
||||
}
|
||||
.meta-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
padding: 10px 0;
|
||||
border-bottom: 1px solid rgba(125, 211, 252, 0.08);
|
||||
font-size: 15px;
|
||||
}
|
||||
.meta-row:last-child { border-bottom: none; }
|
||||
.meta-label { color: var(--muted); }
|
||||
.meta-value { text-align: right; max-width: 56%; }
|
||||
.footer {
|
||||
margin-top: 22px;
|
||||
color: var(--warn);
|
||||
font-size: 15px;
|
||||
line-height: 1.5;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="canvas">
|
||||
<div class="grid"></div>
|
||||
<div class="hero">
|
||||
<div>
|
||||
<div class="badge">Live Swarm Poster</div>
|
||||
<h1>Swarm Blog MVP Live Demo</h1>
|
||||
<p>真实蜂群任务已成功创建,并已记录到“任务开始”和“进行过半”两段里程碑。下面的内容全部来自当前运行中的 swarm 状态,而不是手工拼接。</p>
|
||||
</div>
|
||||
<div class="panel" style="min-width: 360px;">
|
||||
<strong>Demo Goal</strong>
|
||||
<p style="font-size: 18px; color: var(--text); line-height: 1.55;">在目标仓库里开发一个博客系统 MVP,包括文章列表、详情和基础增删改能力。</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="layout">
|
||||
<div class="panel">
|
||||
<div class="cards">
|
||||
<div class="metric">
|
||||
<div class="label">Swarm</div>
|
||||
<div class="value">swarm-300589052dd0</div>
|
||||
</div>
|
||||
<div class="metric">
|
||||
<div class="label">Task Status</div>
|
||||
<div class="value">in_progress</div>
|
||||
</div>
|
||||
<div class="metric">
|
||||
<div class="label">Budget Progress</div>
|
||||
<div class="value">496%</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<strong>Timeline</strong>
|
||||
<div class="timeline">
|
||||
<div class="step">
|
||||
<h3>任务开始</h3>
|
||||
<p>任务已被 agent 领取并进入运行态。开始时间:<span class="code">2026-05-29 10:12:36 UTC</span></p>
|
||||
</div>
|
||||
<div class="step">
|
||||
<h3>进行到一半</h3>
|
||||
<p>脚本按 budget 的 50% 自动确认里程碑。预算过半时间:<span class="code">2026-05-29 10:12:56 UTC</span>,实际记录时间:<span class="code">2026-05-29 10:12:58 UTC</span></p>
|
||||
</div>
|
||||
<div class="step">
|
||||
<h3>当前执行中</h3>
|
||||
<p>任务仍处于 <span class="code">in_progress</span>,最近心跳:<span class="code">2026-05-29T10:12:49.319420Z</span></p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="sidebar">
|
||||
<div class="status">
|
||||
<div class="headline">Poster Snapshot</div>
|
||||
<div class="value">Mid-Run</div>
|
||||
<div class="sub">这张海报展示的是蜂群任务已经真正启动,并且已经跑到预算半程时的现场状态。</div>
|
||||
</div>
|
||||
|
||||
<div class="panel">
|
||||
<strong>Swarm Meta</strong>
|
||||
<div class="meta-row"><span class="meta-label">Deployment</span><span class="meta-value code">runtime-dep-90b592c531e3</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Task</span><span class="meta-value code">swarm-300589052dd0-task-1</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Agent</span><span class="meta-value code">agent-full-5d6968f886-7cm2l</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Attempts</span><span class="meta-value">0</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Budget</span><span class="meta-value">40s</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Output File</span><span class="meta-value code">swarm-poster-300589052dd0.html</span></div>
|
||||
</div>
|
||||
|
||||
<div class="panel">
|
||||
<strong>Latest Events</strong>
|
||||
<div class="meta-row"><span class="meta-label">1</span><span class="meta-value code">task.running</span></div>
|
||||
<div class="meta-row"><span class="meta-label">2</span><span class="meta-value code">task.heartbeat</span></div>
|
||||
<div class="meta-row"><span class="meta-label">3</span><span class="meta-value code">budget.alert</span></div>
|
||||
<div class="footer">注:如果后续继续执行,日志与状态还会变化;这张图锁定的是“开始后已过半”的那一刻。</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"swarm": {
|
||||
"deployment_id": "runtime-dep-90b592c531e3",
|
||||
"runtime_deployment_id": "runtime-dep-90b592c531e3",
|
||||
"manager_deployment_id": "dep_poster_demo_20260529T100940Z",
|
||||
"swarm_id": "swarm-300589052dd0",
|
||||
"status": "running",
|
||||
"created": true
|
||||
},
|
||||
"task": {
|
||||
"task_id": "swarm-300589052dd0-task-1",
|
||||
"title": "Swarm objective",
|
||||
"description": "Build a blog system MVP in the configured workspace repository. Deliver a runnable implementation with a simple backend and frontend for posts: list posts, view a post, create, edit, and delete. Keep setup minimal and include short run instructions.",
|
||||
"status": "in_progress",
|
||||
"agent_role": "general",
|
||||
"required_capabilities": [
|
||||
"general"
|
||||
],
|
||||
"depends_on": [],
|
||||
"parent_task_id": null,
|
||||
"root_task_id": "swarm-300589052dd0-task-1",
|
||||
"source": "runtime_bridge",
|
||||
"task_graph_id": "task-1",
|
||||
"assigned_agent_id": "agent-full-5d6968f886-7cm2l",
|
||||
"created_at": 1780049380.5639093,
|
||||
"started_at": 1780049556.1651127,
|
||||
"completed_at": null,
|
||||
"attempt": 0,
|
||||
"max_retries": 3,
|
||||
"retry_count": 0,
|
||||
"blocked_reason": null,
|
||||
"context": {
|
||||
"orchestration_plan": {
|
||||
"objective": "Build a blog system MVP in the configured workspace repository. Deliver a runnable implementation with a simple backend and frontend for posts: list posts, view a post, create, edit, and delete. Keep setup minimal and include short run instructions.",
|
||||
"sub_mode": "code",
|
||||
"risk_level": "low",
|
||||
"budget": {
|
||||
"duration_seconds": 40,
|
||||
"token_limit": 120000
|
||||
},
|
||||
"agents": [
|
||||
{
|
||||
"task_id": "poster-blog-20260529t100940z",
|
||||
"role": "fullstack",
|
||||
"title": "Implement blog system MVP",
|
||||
"description": "Create a runnable blog system MVP in the target repository with post list, post detail, and create/edit/delete flows. Keep the implementation pragmatic and coherent with the existing repo. Commit and push the result branch when changes are ready.",
|
||||
"depends_on": []
|
||||
}
|
||||
]
|
||||
},
|
||||
"resource_grants": [],
|
||||
"sub_mode": "code",
|
||||
"runtime_deployment_id": "runtime-dep-90b592c531e3",
|
||||
"swarm_id": "swarm-300589052dd0",
|
||||
"manager_deployment_id": "dep_poster_demo_20260529T100940Z",
|
||||
"correlation_id": "corr_poster_demo_20260529T100940Z",
|
||||
"task_graph_id": "task-1",
|
||||
"task_title": "Swarm objective",
|
||||
"depends_on": [],
|
||||
"agent_role": "general",
|
||||
"required_capabilities": [
|
||||
"general"
|
||||
],
|
||||
"parent_task_id": null,
|
||||
"root_task_id": "swarm-300589052dd0-task-1",
|
||||
"source": "runtime_bridge",
|
||||
"workflow_mode": "single_agent",
|
||||
"allow_handoff": false,
|
||||
"model_id": null
|
||||
}
|
||||
},
|
||||
"metrics": {
|
||||
"swarm_id": "swarm-300589052dd0",
|
||||
"window": "15m",
|
||||
"step": "60s",
|
||||
"status": "running",
|
||||
"runtime_duration_seconds": 198.26700401306152,
|
||||
"tasks_total": 1,
|
||||
"tasks_by_status": {
|
||||
"in_progress": 1
|
||||
},
|
||||
"agents_connected": 1,
|
||||
"agents_registered": 1,
|
||||
"average_task_duration_seconds": 0,
|
||||
"budget": {
|
||||
"duration_seconds": 40,
|
||||
"duration_ratio": 4.956675100326538
|
||||
}
|
||||
},
|
||||
"started_at": 1780049556.1651127,
|
||||
"halfway_at": 1780049576.1651127,
|
||||
"halfway_observed_at": 1780049578.828649,
|
||||
"poster_html": "/Users/mac/Projects/HeiCode-Sawrm/artifacts/swarm-posters/swarm-poster-300589052dd0.html"
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 643 KiB |
+109
@@ -0,0 +1,109 @@
|
||||
#!/bin/bash
|
||||
# K8s 蜂群系统 - 快速状态检查
|
||||
|
||||
echo "======================================"
|
||||
echo " K8s 蜂群系统 - 状态检查"
|
||||
echo "======================================"
|
||||
echo ""
|
||||
|
||||
# 颜色定义
|
||||
GREEN='\033[0;32m'
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 检查 kubectl
|
||||
if ! command -v kubectl &> /dev/null; then
|
||||
echo -e "${RED}❌ kubectl 未安装${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 检查集群连接
|
||||
echo "1. 检查集群连接..."
|
||||
if kubectl cluster-info &> /dev/null; then
|
||||
echo -e "${GREEN}✅ 集群连接正常${NC}"
|
||||
else
|
||||
echo -e "${RED}❌ 无法连接到集群${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 检查命名空间
|
||||
echo ""
|
||||
echo "2. 检查命名空间..."
|
||||
if kubectl get namespace swarm-system &> /dev/null; then
|
||||
echo -e "${GREEN}✅ swarm-system 命名空间存在${NC}"
|
||||
else
|
||||
echo -e "${RED}❌ swarm-system 命名空间不存在${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 检查 Redis
|
||||
echo ""
|
||||
echo "3. 检查 Redis..."
|
||||
REDIS_STATUS=$(kubectl get pod redis-0 -n swarm-system -o jsonpath='{.status.phase}' 2>/dev/null)
|
||||
if [ "$REDIS_STATUS" = "Running" ]; then
|
||||
echo -e "${GREEN}✅ Redis 运行中${NC}"
|
||||
else
|
||||
echo -e "${RED}❌ Redis 状态: $REDIS_STATUS${NC}"
|
||||
fi
|
||||
|
||||
# 检查 Orchestrator
|
||||
echo ""
|
||||
echo "4. 检查 Orchestrator..."
|
||||
ORCH_READY=$(kubectl get deployment orchestrator -n swarm-system -o jsonpath='{.status.readyReplicas}' 2>/dev/null)
|
||||
if [ "$ORCH_READY" = "1" ]; then
|
||||
echo -e "${GREEN}✅ Orchestrator 运行中${NC}"
|
||||
else
|
||||
echo -e "${RED}❌ Orchestrator 未就绪${NC}"
|
||||
fi
|
||||
|
||||
# 获取 Orchestrator 外部 IP
|
||||
echo ""
|
||||
echo "5. 获取访问地址..."
|
||||
EXTERNAL_IP=$(kubectl get svc orchestrator-service -n swarm-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}' 2>/dev/null)
|
||||
if [ -n "$EXTERNAL_IP" ]; then
|
||||
echo -e "${GREEN}✅ Orchestrator 地址: http://$EXTERNAL_IP:8000${NC}"
|
||||
|
||||
# 测试健康检查
|
||||
echo ""
|
||||
echo "6. 测试 API 健康检查..."
|
||||
if curl -s -f "http://$EXTERNAL_IP:8000/health" > /dev/null 2>&1; then
|
||||
echo -e "${GREEN}✅ API 健康检查通过${NC}"
|
||||
|
||||
# 显示健康状态详情
|
||||
HEALTH=$(curl -s "http://$EXTERNAL_IP:8000/health")
|
||||
echo " 详情: $HEALTH"
|
||||
else
|
||||
echo -e "${YELLOW}⚠️ API 健康检查失败(可能还在启动中)${NC}"
|
||||
fi
|
||||
else
|
||||
echo -e "${YELLOW}⚠️ 等待 LoadBalancer 分配外部 IP...${NC}"
|
||||
fi
|
||||
|
||||
# 显示所有 Pod
|
||||
echo ""
|
||||
echo "7. 所有 Pod 状态:"
|
||||
echo "-----------------------------------"
|
||||
kubectl get pods -n swarm-system
|
||||
|
||||
# 显示服务
|
||||
echo ""
|
||||
echo "8. 服务状态:"
|
||||
echo "-----------------------------------"
|
||||
kubectl get svc -n swarm-system
|
||||
|
||||
# 总结
|
||||
echo ""
|
||||
echo "======================================"
|
||||
echo " 状态检查完成"
|
||||
echo "======================================"
|
||||
echo ""
|
||||
echo "快速命令:"
|
||||
echo " 查看日志: kubectl logs -n swarm-system deployment/orchestrator -f"
|
||||
echo " 查看 Pod: kubectl get pods -n swarm-system"
|
||||
echo " 运行测试: python3 test_system.py"
|
||||
echo ""
|
||||
echo "文档:"
|
||||
echo " 快速入门: 快速入门指南.md"
|
||||
echo " 完整文档: 部署和使用文档.md"
|
||||
echo ""
|
||||
@@ -0,0 +1,71 @@
|
||||
# HeiCode Swarm 桌面客户端
|
||||
|
||||
基于 **Electron + React + TypeScript** 的桌面客户端,是蜂群系统面向用户的入口。它负责提交任务、连接并展示 Orchestrator 的运行状态,分析任务复杂度以推荐 Agent 数量,并对生成结果进行聚合与查看。
|
||||
|
||||
## 环境要求
|
||||
|
||||
- Node.js 18+ 与 npm
|
||||
- 可访问的 Kubernetes 集群与有效的 kubeconfig(如使用 K8s 调度)
|
||||
- 已正确配置的 kubectl
|
||||
|
||||
## 安装与开发
|
||||
|
||||
```bash
|
||||
npm install # 安装依赖
|
||||
npm run build # 编译 TypeScript
|
||||
npm run dev # 开发模式启动
|
||||
npm run type-check
|
||||
npm run lint
|
||||
```
|
||||
|
||||
## 配置
|
||||
|
||||
应用按以下顺序加载 kubeconfig:
|
||||
1. 环境变量 `KUBECONFIG`(若设置)
|
||||
2. 默认位置 `~/.kube/config`
|
||||
|
||||
请确保 kubeconfig 指向正确的集群且凭据有效。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
desktop-client/
|
||||
├── src/
|
||||
│ ├── main.ts # Electron 主进程
|
||||
│ ├── App.tsx # React 根组件
|
||||
│ ├── index.html # 入口页面
|
||||
│ ├── api/
|
||||
│ │ └── orchestrator-client.ts # 调用 Orchestrator REST 接口
|
||||
│ ├── ai/ # 任务复杂度分析(见该目录 README)
|
||||
│ ├── components/ # 界面组件
|
||||
│ │ ├── TaskSubmissionForm.tsx # 任务提交
|
||||
│ │ ├── AgentDashboard.tsx # Agent 与运行状态看板
|
||||
│ │ ├── ResultViewer.tsx # 结果查看
|
||||
│ │ └── CostControlDialog.tsx # 成本/规模确认
|
||||
│ ├── analytics/usage-tracker.ts # 用量统计
|
||||
│ ├── git/ # Git 工作区与分支合并
|
||||
│ │ ├── workspace-manager.ts
|
||||
│ │ └── merge-manager.ts
|
||||
│ └── k8s/ # K8s 接入
|
||||
│ ├── kubeconfig-loader.ts # 鉴权/集群配置加载
|
||||
│ └── auto-scaler.ts # 按复杂度自动伸缩 Agent 规模
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── tailwind.config.js
|
||||
```
|
||||
|
||||
## 主要能力
|
||||
|
||||
- **任务提交**:用户输入需求后,通过 `orchestrator-client` 创建蜂群部署。
|
||||
- **复杂度分析**:`ai/` 模块评估任务规模并推荐 Agent 数量(详见 `src/ai/README.md`)。
|
||||
- **成本控制**:超过阈值时通过 `CostControlDialog` 提示并要求确认,避免规模失控。
|
||||
- **状态展示**:`AgentDashboard` 实时展示 Agent、任务与工作流阶段。
|
||||
- **结果聚合**:`git/merge-manager` 合并各 Agent 的结果分支,`ResultViewer` 统一查看。
|
||||
- **弹性伸缩**:`k8s/auto-scaler` 依据复杂度分析结果调整 Agent 数量。
|
||||
|
||||
## 工作方式
|
||||
|
||||
1. 用户在客户端提交需求并(可选)查看复杂度分析与推荐规模。
|
||||
2. 客户端调用 Orchestrator 创建部署,Orchestrator 分解任务并调度 Agent 执行。
|
||||
3. 客户端轮询/订阅运行状态,展示阶段进度与各 Agent 产出。
|
||||
4. 运行完成后,聚合各结果分支并在结果视图中呈现最终交付物。
|
||||
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"name": "heicode-swarm-desktop",
|
||||
"version": "0.1.0",
|
||||
"description": "Desktop client for HeiCode Swarm - K8s-based AI agent orchestration",
|
||||
"main": "dist/main.js",
|
||||
"scripts": {
|
||||
"start": "electron .",
|
||||
"dev": "concurrently \"npm run build:watch\" \"npm run electron:dev\"",
|
||||
"build": "tsc",
|
||||
"build:watch": "tsc --watch",
|
||||
"electron:dev": "wait-on dist/main.js && electron .",
|
||||
"package": "electron-builder",
|
||||
"lint": "eslint src --ext .ts,.tsx",
|
||||
"type-check": "tsc --noEmit",
|
||||
"validate-analyzer": "tsc && node dist/ai/validate-analyzer.js"
|
||||
},
|
||||
"keywords": [
|
||||
"electron",
|
||||
"kubernetes",
|
||||
"ai",
|
||||
"swarm",
|
||||
"orchestration"
|
||||
],
|
||||
"author": "HeiCode",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@kubernetes/client-node": "^0.20.0",
|
||||
"@anthropic-ai/sdk": "^0.20.0",
|
||||
"isomorphic-git": "^1.25.0",
|
||||
"zustand": "^4.5.0",
|
||||
"react": "^18.2.0",
|
||||
"react-dom": "^18.2.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.11.0",
|
||||
"@types/react": "^18.2.0",
|
||||
"@types/react-dom": "^18.2.0",
|
||||
"@typescript-eslint/eslint-plugin": "^6.19.0",
|
||||
"@typescript-eslint/parser": "^6.19.0",
|
||||
"concurrently": "^8.2.2",
|
||||
"electron": "^28.0.0",
|
||||
"electron-builder": "^24.9.1",
|
||||
"eslint": "^8.56.0",
|
||||
"typescript": "^5.3.3",
|
||||
"wait-on": "^7.2.0",
|
||||
"tailwindcss": "^3.4.0",
|
||||
"autoprefixer": "^10.4.17",
|
||||
"postcss": "^8.4.33"
|
||||
},
|
||||
"build": {
|
||||
"appId": "com.heicode.swarm",
|
||||
"productName": "HeiCode Swarm",
|
||||
"directories": {
|
||||
"output": "release"
|
||||
},
|
||||
"files": [
|
||||
"dist/**/*",
|
||||
"package.json"
|
||||
],
|
||||
"mac": {
|
||||
"category": "public.app-category.developer-tools",
|
||||
"target": ["dmg", "zip"]
|
||||
},
|
||||
"linux": {
|
||||
"target": ["AppImage", "deb"],
|
||||
"category": "Development"
|
||||
},
|
||||
"win": {
|
||||
"target": ["nsis", "portable"]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,319 @@
|
||||
import React, { useState, useEffect } from 'react';
|
||||
import { createRoot } from 'react-dom/client';
|
||||
import { TaskSubmissionForm } from './components/TaskSubmissionForm';
|
||||
import { AgentDashboard } from './components/AgentDashboard';
|
||||
import { ResultViewer } from './components/ResultViewer';
|
||||
import { ComplexityAnalysis, getAnalyzer } from './ai/complexity-analyzer';
|
||||
import { getOrchestratorClient } from './api/orchestrator-client';
|
||||
import { AutoScaler } from './k8s/auto-scaler';
|
||||
import { MergeManager, MergeConflict, ConflictResolution } from './git/merge-manager';
|
||||
import { WorkspaceManager } from './git/workspace-manager';
|
||||
import { KubeConfigLoader } from './k8s/kubeconfig-loader';
|
||||
|
||||
interface AppState {
|
||||
status: 'idle' | 'loading' | 'connected' | 'error';
|
||||
message: string;
|
||||
currentWorkspaceId: string | null;
|
||||
mergeConflicts: MergeConflict[];
|
||||
mergedBranches: string[];
|
||||
showConflictResolution: boolean;
|
||||
}
|
||||
|
||||
const App: React.FC = () => {
|
||||
const [state, setState] = useState<AppState>({
|
||||
status: 'idle',
|
||||
message: 'Initializing HeiCode Swarm...',
|
||||
currentWorkspaceId: null,
|
||||
mergeConflicts: [],
|
||||
mergedBranches: [],
|
||||
showConflictResolution: false,
|
||||
});
|
||||
|
||||
const [orchestratorUrl] = useState('http://localhost:8000');
|
||||
const [autoScaler, setAutoScaler] = useState<AutoScaler | null>(null);
|
||||
const [workspaceManager] = useState(new WorkspaceManager());
|
||||
|
||||
useEffect(() => {
|
||||
initializeApp();
|
||||
}, []);
|
||||
|
||||
const initializeApp = async () => {
|
||||
setState(prev => ({ ...prev, status: 'loading', message: 'Connecting to Kubernetes...' }));
|
||||
|
||||
try {
|
||||
// Load kubeconfig
|
||||
const kubeLoader = new KubeConfigLoader();
|
||||
kubeLoader.load();
|
||||
|
||||
const validation = kubeLoader.validate();
|
||||
if (!validation.valid) {
|
||||
throw new Error(`Kubeconfig validation failed: ${validation.error}`);
|
||||
}
|
||||
|
||||
// Test connection
|
||||
const connectionTest = await kubeLoader.testConnection();
|
||||
if (!connectionTest.success) {
|
||||
throw new Error(`K8s connection failed: ${connectionTest.error}`);
|
||||
}
|
||||
|
||||
// Initialize auto-scaler
|
||||
const scaler = new AutoScaler(kubeLoader.getKubeConfig(), 'default');
|
||||
setAutoScaler(scaler);
|
||||
|
||||
// Initialize complexity analyzer
|
||||
const apiKey = process.env.ANTHROPIC_API_KEY || '';
|
||||
if (!apiKey) {
|
||||
console.warn('ANTHROPIC_API_KEY not set. Complexity analysis will fail.');
|
||||
}
|
||||
getAnalyzer({ apiKey });
|
||||
|
||||
// Initialize orchestrator client
|
||||
getOrchestratorClient({ baseUrl: orchestratorUrl });
|
||||
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'connected',
|
||||
message: `Connected to ${validation.cluster}`,
|
||||
}));
|
||||
} catch (error) {
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'error',
|
||||
message: error instanceof Error ? error.message : 'Initialization failed',
|
||||
}));
|
||||
}
|
||||
};
|
||||
|
||||
const handleTaskSubmit = async (taskDescription: string, complexity: ComplexityAnalysis) => {
|
||||
if (!autoScaler) {
|
||||
alert('Auto-scaler not initialized');
|
||||
return;
|
||||
}
|
||||
|
||||
setState(prev => ({ ...prev, status: 'loading', message: 'Creating workspace...' }));
|
||||
|
||||
try {
|
||||
// Create workspace
|
||||
const workspaceId = `ws-${Date.now()}`;
|
||||
const workspaceResult = await workspaceManager.initWorkspace(workspaceId);
|
||||
|
||||
if (!workspaceResult.success) {
|
||||
throw new Error(`Failed to create workspace: ${workspaceResult.error}`);
|
||||
}
|
||||
|
||||
const { gitUrl } = workspaceResult.data;
|
||||
|
||||
setState(prev => ({ ...prev, message: 'Creating agent pods...' }));
|
||||
|
||||
// Scale up agents
|
||||
const scaleResult = await autoScaler.scaleUp(complexity, {
|
||||
namespace: 'default',
|
||||
image: 'heicode-swarm-agent:latest',
|
||||
orchestratorUrl,
|
||||
gitUrl,
|
||||
workspaceId,
|
||||
});
|
||||
|
||||
if (!scaleResult.success) {
|
||||
throw new Error(`Failed to create agents: ${scaleResult.errors.join(', ')}`);
|
||||
}
|
||||
|
||||
setState(prev => ({ ...prev, message: 'Submitting task to orchestrator...' }));
|
||||
|
||||
// Create task in orchestrator
|
||||
const client = getOrchestratorClient();
|
||||
await client.createTask(taskDescription, {
|
||||
workspaceId,
|
||||
complexity: complexity.complexity,
|
||||
agentCount: complexity.agentCount,
|
||||
});
|
||||
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'connected',
|
||||
message: `Task submitted with ${scaleResult.createdPods.length} agents`,
|
||||
currentWorkspaceId: workspaceId,
|
||||
}));
|
||||
} catch (error) {
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'error',
|
||||
message: error instanceof Error ? error.message : 'Task submission failed',
|
||||
}));
|
||||
}
|
||||
};
|
||||
|
||||
const handleMergeResults = async () => {
|
||||
if (!state.currentWorkspaceId) {
|
||||
alert('No active workspace');
|
||||
return;
|
||||
}
|
||||
|
||||
setState(prev => ({ ...prev, status: 'loading', message: 'Merging agent results...' }));
|
||||
|
||||
try {
|
||||
const repoPath = workspaceManager.getWorkspacePath(state.currentWorkspaceId);
|
||||
const mergeManager = new MergeManager(repoPath);
|
||||
|
||||
// List agent branches
|
||||
const agentBranches = await mergeManager.listAgentBranches(state.currentWorkspaceId);
|
||||
|
||||
if (agentBranches.length === 0) {
|
||||
throw new Error('No agent branches found');
|
||||
}
|
||||
|
||||
// Merge branches
|
||||
const mergeResult = await mergeManager.mergeAgentResults('main', agentBranches);
|
||||
|
||||
if (mergeResult.conflicts.length > 0) {
|
||||
// Show conflict resolution UI
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'connected',
|
||||
message: `${mergeResult.conflicts.length} conflicts detected`,
|
||||
mergeConflicts: mergeResult.conflicts,
|
||||
mergedBranches: mergeResult.mergedBranches,
|
||||
showConflictResolution: true,
|
||||
}));
|
||||
} else {
|
||||
// Merge successful
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'connected',
|
||||
message: 'Merge completed successfully',
|
||||
mergedBranches: mergeResult.mergedBranches,
|
||||
showConflictResolution: false,
|
||||
}));
|
||||
}
|
||||
} catch (error) {
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'error',
|
||||
message: error instanceof Error ? error.message : 'Merge failed',
|
||||
}));
|
||||
}
|
||||
};
|
||||
|
||||
const handleConflictResolve = async (resolutions: ConflictResolution[]) => {
|
||||
if (!state.currentWorkspaceId) return;
|
||||
|
||||
setState(prev => ({ ...prev, status: 'loading', message: 'Applying resolutions...' }));
|
||||
|
||||
try {
|
||||
const repoPath = workspaceManager.getWorkspacePath(state.currentWorkspaceId);
|
||||
const mergeManager = new MergeManager(repoPath);
|
||||
|
||||
const success = await mergeManager.resolveConflicts(resolutions);
|
||||
|
||||
if (success) {
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'connected',
|
||||
message: 'Conflicts resolved successfully',
|
||||
mergeConflicts: [],
|
||||
showConflictResolution: false,
|
||||
}));
|
||||
} else {
|
||||
throw new Error('Failed to apply resolutions');
|
||||
}
|
||||
} catch (error) {
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
status: 'error',
|
||||
message: error instanceof Error ? error.message : 'Resolution failed',
|
||||
}));
|
||||
}
|
||||
};
|
||||
|
||||
const handleConflictCancel = () => {
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
mergeConflicts: [],
|
||||
showConflictResolution: false,
|
||||
}));
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="min-h-screen bg-gray-900 text-white">
|
||||
<header className="bg-gray-800 border-b border-gray-700 px-6 py-4">
|
||||
<h1 className="text-2xl font-bold">HeiCode Swarm</h1>
|
||||
<p className="text-gray-400 text-sm">K8s-Based AI Agent Orchestration</p>
|
||||
</header>
|
||||
|
||||
<main className="container mx-auto px-6 py-8">
|
||||
<div className="bg-gray-800 rounded-lg p-6 shadow-lg mb-8">
|
||||
<div className="flex items-center space-x-4">
|
||||
<div className={`w-3 h-3 rounded-full ${
|
||||
state.status === 'connected' ? 'bg-green-500' :
|
||||
state.status === 'error' ? 'bg-red-500' :
|
||||
state.status === 'loading' ? 'bg-yellow-500 animate-pulse' :
|
||||
'bg-gray-500'
|
||||
}`} />
|
||||
<div>
|
||||
<h2 className="text-lg font-semibold">
|
||||
{state.status === 'connected' ? 'Connected' :
|
||||
state.status === 'error' ? 'Error' :
|
||||
state.status === 'loading' ? 'Loading...' :
|
||||
'Idle'}
|
||||
</h2>
|
||||
<p className="text-gray-400 text-sm">{state.message}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{state.showConflictResolution ? (
|
||||
<ResultViewer
|
||||
conflicts={state.mergeConflicts}
|
||||
mergedBranches={state.mergedBranches}
|
||||
onResolve={handleConflictResolve}
|
||||
onCancel={handleConflictCancel}
|
||||
/>
|
||||
) : (
|
||||
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6">
|
||||
<div className="space-y-6">
|
||||
<TaskSubmissionForm
|
||||
onSubmit={handleTaskSubmit}
|
||||
disabled={state.status !== 'connected'}
|
||||
/>
|
||||
|
||||
{state.currentWorkspaceId && (
|
||||
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
|
||||
<h3 className="text-lg font-semibold mb-4">Workspace Actions</h3>
|
||||
<button
|
||||
className="w-full bg-purple-600 hover:bg-purple-700 text-white font-semibold px-6 py-2 rounded transition-colors"
|
||||
onClick={handleMergeResults}
|
||||
disabled={state.status !== 'connected'}
|
||||
>
|
||||
Merge Agent Results
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<AgentDashboard
|
||||
orchestratorUrl={orchestratorUrl}
|
||||
workspaceId={state.currentWorkspaceId || undefined}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</main>
|
||||
|
||||
<footer className="fixed bottom-0 left-0 right-0 bg-gray-800 border-t border-gray-700 px-6 py-3">
|
||||
<p className="text-gray-400 text-sm text-center">
|
||||
Phase 5: Desktop Client & Orchestration - Full Integration
|
||||
</p>
|
||||
</footer>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
// Initialize React app
|
||||
const container = document.getElementById('root');
|
||||
if (container) {
|
||||
const root = createRoot(container);
|
||||
root.render(<App />);
|
||||
}
|
||||
|
||||
export default App;
|
||||
@@ -0,0 +1,79 @@
|
||||
# AI 模块 —— 任务复杂度分析
|
||||
|
||||
该模块为桌面客户端提供**任务复杂度分析**能力:根据用户需求评估其规模与难度,推荐应投入的 Agent 数量,作为蜂群规模与成本控制的依据。
|
||||
|
||||
## 组成
|
||||
|
||||
### 1. `complexity-analyzer.ts`
|
||||
核心分析器,调用大模型对任务进行复杂度评估并给出推荐 Agent 数量。
|
||||
|
||||
特性:
|
||||
- 结构化 JSON 结果解析,并对非 JSON 响应做兜底解析
|
||||
- Agent 数量校验(1–50)
|
||||
- 性能监控(目标单次 < 3 秒)
|
||||
- 支持批量分析
|
||||
- 单例模式复用客户端
|
||||
|
||||
用法:
|
||||
```typescript
|
||||
import { ComplexityAnalyzer } from './ai';
|
||||
|
||||
const analyzer = new ComplexityAnalyzer({ temperature: 0.3 });
|
||||
const analysis = await analyzer.analyzeTask('为整个项目补充单元测试');
|
||||
console.log(`推荐 Agent 数:${analysis.agentCount}`);
|
||||
console.log(`复杂度:${analysis.complexity}`);
|
||||
console.log(`理由:${analysis.reasoning}`);
|
||||
```
|
||||
|
||||
### 2. `prompt-builder.ts`
|
||||
基于带标注的数据集构造 few-shot 提示词。
|
||||
|
||||
特性:
|
||||
- 从 `test-data/complexity-dataset.json` 加载样例
|
||||
- 按比例选取多样化样例(低 40% / 中 40% / 高 20%)
|
||||
- 生成完整的 few-shot 提示,批量模式下提供简化提示
|
||||
- 数据集不可用时回退到内置默认样例
|
||||
|
||||
### 3. `validator.ts`
|
||||
对照标注数据集校验分析器的准确率。
|
||||
|
||||
特性与验收目标:
|
||||
- 全量与快速(10 条抽样)校验
|
||||
- 准确率(精确匹配、±2 个 Agent 容差)与平均绝对误差(MAE)
|
||||
- 不匹配项报告与 Markdown 报告生成
|
||||
- 目标:准确率 ≥ 70%(±2 容差)、MAE ≤ 2、单任务分析 < 3 秒
|
||||
|
||||
### 4. `validate-analyzer.ts`
|
||||
独立的校验脚本,便于在集成前验证分析器质量:
|
||||
```bash
|
||||
npm run validate-analyzer
|
||||
```
|
||||
|
||||
## 成本控制
|
||||
|
||||
分析结果与 `CostControlDialog` 组件联动,向用户提示资源开销:
|
||||
- **> 10 个 Agent**:提示并给出资源预估
|
||||
- **> 15 个 Agent**:要求用户显式确认
|
||||
- **达到上限**:遵循用户配置的最大值(默认 20)
|
||||
|
||||
## 数据集格式
|
||||
|
||||
标注数据集 `test-data/complexity-dataset.json` 的样例格式:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"task": "修复 main.py 第 42 行的语法错误",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "单文件单行修复,无依赖与副作用。"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## 实现要点
|
||||
|
||||
1. **Few-shot 学习**:使用数据集中的多样化样例提升判断稳定性。
|
||||
2. **温度 0.3**:降低随机性以获得更一致的预测。
|
||||
3. **稳健性**:解析失败时优雅兜底。
|
||||
4. **先验证后上线**:集成进自动伸缩与任务提交流程前,应先通过校验。
|
||||
@@ -0,0 +1,120 @@
|
||||
/**
|
||||
* Unit tests for complexity analyzer
|
||||
* Run with: npm test
|
||||
*/
|
||||
|
||||
import { ComplexityAnalyzer } from './complexity-analyzer';
|
||||
import { loadExamples, selectFewShotExamples, buildComplexityPrompt } from './prompt-builder';
|
||||
|
||||
describe('ComplexityAnalyzer', () => {
|
||||
let analyzer: ComplexityAnalyzer;
|
||||
|
||||
beforeEach(() => {
|
||||
analyzer = new ComplexityAnalyzer({
|
||||
apiKey: process.env.ANTHROPIC_API_KEY || 'test-key',
|
||||
model: 'claude-opus-4-7',
|
||||
});
|
||||
});
|
||||
|
||||
describe('validateAgentCount', () => {
|
||||
it('should clamp negative values to 1', () => {
|
||||
const result = (analyzer as any).validateAgentCount(-5);
|
||||
expect(result).toBe(1);
|
||||
});
|
||||
|
||||
it('should clamp values above 50', () => {
|
||||
const result = (analyzer as any).validateAgentCount(100);
|
||||
expect(result).toBe(50);
|
||||
});
|
||||
|
||||
it('should round decimal values', () => {
|
||||
const result = (analyzer as any).validateAgentCount(3.7);
|
||||
expect(result).toBe(4);
|
||||
});
|
||||
|
||||
it('should accept valid values', () => {
|
||||
const result = (analyzer as any).validateAgentCount(10);
|
||||
expect(result).toBe(10);
|
||||
});
|
||||
});
|
||||
|
||||
describe('validateComplexity', () => {
|
||||
it('should accept valid complexity levels', () => {
|
||||
expect((analyzer as any).validateComplexity('low')).toBe('low');
|
||||
expect((analyzer as any).validateComplexity('medium')).toBe('medium');
|
||||
expect((analyzer as any).validateComplexity('high')).toBe('high');
|
||||
});
|
||||
|
||||
it('should default to medium for invalid values', () => {
|
||||
expect((analyzer as any).validateComplexity('invalid')).toBe('medium');
|
||||
});
|
||||
});
|
||||
|
||||
describe('inferComplexity', () => {
|
||||
it('should infer low complexity for 1-2 agents', () => {
|
||||
expect((analyzer as any).inferComplexity(1)).toBe('low');
|
||||
expect((analyzer as any).inferComplexity(2)).toBe('low');
|
||||
});
|
||||
|
||||
it('should infer medium complexity for 3-7 agents', () => {
|
||||
expect((analyzer as any).inferComplexity(3)).toBe('medium');
|
||||
expect((analyzer as any).inferComplexity(7)).toBe('medium');
|
||||
});
|
||||
|
||||
it('should infer high complexity for 8+ agents', () => {
|
||||
expect((analyzer as any).inferComplexity(8)).toBe('high');
|
||||
expect((analyzer as any).inferComplexity(20)).toBe('high');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('PromptBuilder', () => {
|
||||
describe('loadExamples', () => {
|
||||
it('should load examples from dataset', () => {
|
||||
const examples = loadExamples();
|
||||
expect(examples.length).toBeGreaterThan(0);
|
||||
expect(examples[0]).toHaveProperty('task');
|
||||
expect(examples[0]).toHaveProperty('expectedAgents');
|
||||
expect(examples[0]).toHaveProperty('complexity');
|
||||
expect(examples[0]).toHaveProperty('reasoning');
|
||||
});
|
||||
});
|
||||
|
||||
describe('selectFewShotExamples', () => {
|
||||
it('should select requested number of examples', () => {
|
||||
const examples = loadExamples();
|
||||
const selected = selectFewShotExamples(examples, 12);
|
||||
expect(selected.length).toBeLessThanOrEqual(12);
|
||||
});
|
||||
|
||||
it('should include diverse complexity levels', () => {
|
||||
const examples = loadExamples();
|
||||
const selected = selectFewShotExamples(examples, 12);
|
||||
const complexities = selected.map((e) => e.complexity);
|
||||
expect(complexities).toContain('low');
|
||||
expect(complexities).toContain('medium');
|
||||
expect(complexities).toContain('high');
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildComplexityPrompt', () => {
|
||||
it('should build prompt with task description', () => {
|
||||
const prompt = buildComplexityPrompt('Fix syntax error');
|
||||
expect(prompt).toContain('Fix syntax error');
|
||||
});
|
||||
|
||||
it('should include few-shot examples', () => {
|
||||
const prompt = buildComplexityPrompt('Test task');
|
||||
expect(prompt).toContain('Task:');
|
||||
expect(prompt).toContain('Analysis:');
|
||||
expect(prompt).toContain('agentCount');
|
||||
});
|
||||
|
||||
it('should include guidelines', () => {
|
||||
const prompt = buildComplexityPrompt('Test task');
|
||||
expect(prompt).toContain('Low complexity');
|
||||
expect(prompt).toContain('Medium complexity');
|
||||
expect(prompt).toContain('High complexity');
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,191 @@
|
||||
import Anthropic from '@anthropic-ai/sdk';
|
||||
import { buildComplexityPrompt } from './prompt-builder';
|
||||
|
||||
export interface ComplexityAnalysis {
|
||||
agentCount: number;
|
||||
reasoning: string;
|
||||
complexity: 'low' | 'medium' | 'high';
|
||||
confidence: number;
|
||||
}
|
||||
|
||||
export interface AnalyzerConfig {
|
||||
apiKey: string;
|
||||
model?: string;
|
||||
maxTokens?: number;
|
||||
temperature?: number;
|
||||
}
|
||||
|
||||
export class ComplexityAnalyzer {
|
||||
private client: Anthropic;
|
||||
private model: string;
|
||||
private maxTokens: number;
|
||||
private temperature: number;
|
||||
|
||||
constructor(config: AnalyzerConfig) {
|
||||
this.client = new Anthropic({
|
||||
apiKey: config.apiKey,
|
||||
});
|
||||
this.model = config.model || 'claude-opus-4-7';
|
||||
this.maxTokens = config.maxTokens || 1024;
|
||||
this.temperature = config.temperature || 0.3;
|
||||
}
|
||||
|
||||
/**
|
||||
* Analyze task complexity and determine required agent count
|
||||
* @param taskDescription - The task to analyze
|
||||
* @returns ComplexityAnalysis with agent count recommendation
|
||||
*/
|
||||
async analyzeTask(taskDescription: string): Promise<ComplexityAnalysis> {
|
||||
const startTime = Date.now();
|
||||
|
||||
try {
|
||||
const prompt = buildComplexityPrompt(taskDescription);
|
||||
|
||||
const response = await this.client.messages.create({
|
||||
model: this.model,
|
||||
max_tokens: this.maxTokens,
|
||||
temperature: this.temperature,
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: prompt,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const elapsedTime = Date.now() - startTime;
|
||||
console.log(`Complexity analysis completed in ${elapsedTime}ms`);
|
||||
|
||||
if (elapsedTime > 3000) {
|
||||
console.warn(`Analysis took ${elapsedTime}ms, exceeding 3s target`);
|
||||
}
|
||||
|
||||
const content = response.content[0];
|
||||
if (content.type !== 'text') {
|
||||
throw new Error('Unexpected response type from Claude API');
|
||||
}
|
||||
|
||||
const analysis = this.parseResponse(content.text);
|
||||
return analysis;
|
||||
} catch (error) {
|
||||
console.error('Error analyzing task complexity:', error);
|
||||
throw new Error(`Failed to analyze task complexity: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse Claude's response into structured ComplexityAnalysis
|
||||
*/
|
||||
private parseResponse(responseText: string): ComplexityAnalysis {
|
||||
try {
|
||||
// Try to extract JSON from response
|
||||
const jsonMatch = responseText.match(/\{[\s\S]*\}/);
|
||||
if (jsonMatch) {
|
||||
const parsed = JSON.parse(jsonMatch[0]);
|
||||
return {
|
||||
agentCount: this.validateAgentCount(parsed.agentCount),
|
||||
reasoning: parsed.reasoning || 'No reasoning provided',
|
||||
complexity: this.validateComplexity(parsed.complexity),
|
||||
confidence: parsed.confidence || 0.8,
|
||||
};
|
||||
}
|
||||
|
||||
// Fallback: try to extract agent count from text
|
||||
const agentMatch = responseText.match(/(\d+)\s*agent/i);
|
||||
if (agentMatch) {
|
||||
const agentCount = parseInt(agentMatch[1], 10);
|
||||
return {
|
||||
agentCount: this.validateAgentCount(agentCount),
|
||||
reasoning: responseText,
|
||||
complexity: this.inferComplexity(agentCount),
|
||||
confidence: 0.6,
|
||||
};
|
||||
}
|
||||
|
||||
throw new Error('Could not parse agent count from response');
|
||||
} catch (error) {
|
||||
console.error('Error parsing response:', responseText);
|
||||
throw new Error(`Failed to parse complexity analysis: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and clamp agent count to reasonable range
|
||||
*/
|
||||
private validateAgentCount(count: number): number {
|
||||
if (isNaN(count) || count < 1) {
|
||||
console.warn(`Invalid agent count ${count}, defaulting to 1`);
|
||||
return 1;
|
||||
}
|
||||
if (count > 50) {
|
||||
console.warn(`Agent count ${count} exceeds maximum, capping at 50`);
|
||||
return 50;
|
||||
}
|
||||
return Math.round(count);
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate complexity level
|
||||
*/
|
||||
private validateComplexity(complexity: string): 'low' | 'medium' | 'high' {
|
||||
const normalized = complexity.toLowerCase();
|
||||
if (normalized === 'low' || normalized === 'medium' || normalized === 'high') {
|
||||
return normalized as 'low' | 'medium' | 'high';
|
||||
}
|
||||
console.warn(`Invalid complexity level ${complexity}, defaulting to medium`);
|
||||
return 'medium';
|
||||
}
|
||||
|
||||
/**
|
||||
* Infer complexity from agent count
|
||||
*/
|
||||
private inferComplexity(agentCount: number): 'low' | 'medium' | 'high' {
|
||||
if (agentCount <= 2) return 'low';
|
||||
if (agentCount <= 7) return 'medium';
|
||||
return 'high';
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch analyze multiple tasks
|
||||
*/
|
||||
async analyzeBatch(tasks: string[]): Promise<ComplexityAnalysis[]> {
|
||||
const results: ComplexityAnalysis[] = [];
|
||||
|
||||
for (const task of tasks) {
|
||||
try {
|
||||
const analysis = await this.analyzeTask(task);
|
||||
results.push(analysis);
|
||||
} catch (error) {
|
||||
console.error(`Failed to analyze task: ${task}`, error);
|
||||
// Push a default analysis for failed tasks
|
||||
results.push({
|
||||
agentCount: 1,
|
||||
reasoning: `Analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
|
||||
complexity: 'low',
|
||||
confidence: 0,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a singleton analyzer instance
|
||||
*/
|
||||
let analyzerInstance: ComplexityAnalyzer | null = null;
|
||||
|
||||
export function getAnalyzer(config?: AnalyzerConfig): ComplexityAnalyzer {
|
||||
if (!analyzerInstance) {
|
||||
if (!config) {
|
||||
throw new Error('Analyzer not initialized. Provide config on first call.');
|
||||
}
|
||||
analyzerInstance = new ComplexityAnalyzer(config);
|
||||
}
|
||||
return analyzerInstance;
|
||||
}
|
||||
|
||||
export function resetAnalyzer(): void {
|
||||
analyzerInstance = null;
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
/**
|
||||
* Example usage of the complexity analyzer
|
||||
* This demonstrates how to use the analyzer in the desktop client
|
||||
*/
|
||||
|
||||
import { ComplexityAnalyzer, validateAnalyzer } from './index';
|
||||
|
||||
async function exampleUsage() {
|
||||
// Initialize analyzer
|
||||
const analyzer = new ComplexityAnalyzer({
|
||||
apiKey: process.env.ANTHROPIC_API_KEY || 'your-api-key',
|
||||
model: 'claude-opus-4-7',
|
||||
temperature: 0.3,
|
||||
});
|
||||
|
||||
// Example 1: Analyze a simple task
|
||||
console.log('=== Example 1: Simple Task ===');
|
||||
const simpleTask = 'Fix syntax error in main.py line 42';
|
||||
const simpleAnalysis = await analyzer.analyzeTask(simpleTask);
|
||||
console.log(`Task: ${simpleTask}`);
|
||||
console.log(`Agents needed: ${simpleAnalysis.agentCount}`);
|
||||
console.log(`Complexity: ${simpleAnalysis.complexity}`);
|
||||
console.log(`Reasoning: ${simpleAnalysis.reasoning}\n`);
|
||||
|
||||
// Example 2: Analyze a medium complexity task
|
||||
console.log('=== Example 2: Medium Complexity Task ===');
|
||||
const mediumTask = 'Add input validation to all API endpoints';
|
||||
const mediumAnalysis = await analyzer.analyzeTask(mediumTask);
|
||||
console.log(`Task: ${mediumTask}`);
|
||||
console.log(`Agents needed: ${mediumAnalysis.agentCount}`);
|
||||
console.log(`Complexity: ${mediumAnalysis.complexity}`);
|
||||
console.log(`Reasoning: ${mediumAnalysis.reasoning}\n`);
|
||||
|
||||
// Example 3: Analyze a complex task
|
||||
console.log('=== Example 3: Complex Task ===');
|
||||
const complexTask = 'Add comprehensive unit tests to the entire project';
|
||||
const complexAnalysis = await analyzer.analyzeTask(complexTask);
|
||||
console.log(`Task: ${complexTask}`);
|
||||
console.log(`Agents needed: ${complexAnalysis.agentCount}`);
|
||||
console.log(`Complexity: ${complexAnalysis.complexity}`);
|
||||
console.log(`Reasoning: ${complexAnalysis.reasoning}\n`);
|
||||
|
||||
// Example 4: Batch analysis
|
||||
console.log('=== Example 4: Batch Analysis ===');
|
||||
const tasks = [
|
||||
'Update README.md with installation instructions',
|
||||
'Implement caching layer using Redis',
|
||||
'Migrate from REST API to GraphQL',
|
||||
];
|
||||
const batchResults = await analyzer.analyzeBatch(tasks);
|
||||
batchResults.forEach((result, i: number) => {
|
||||
console.log(`${i + 1}. ${tasks[i]}: ${result.agentCount} agents (${result.complexity})`);
|
||||
});
|
||||
console.log();
|
||||
|
||||
// Example 5: Run validation (optional)
|
||||
if (process.env.RUN_VALIDATION === 'true') {
|
||||
console.log('=== Example 5: Validation ===');
|
||||
const validationResult = await validateAnalyzer(analyzer, {
|
||||
accuracyThreshold: 0.7,
|
||||
maxMeanAbsoluteError: 2,
|
||||
});
|
||||
console.log(`Validation ${validationResult.passed ? 'PASSED' : 'FAILED'}`);
|
||||
console.log(`Accuracy: ${(validationResult.accuracy * 100).toFixed(1)}%`);
|
||||
console.log(`MAE: ${validationResult.meanAbsoluteError.toFixed(2)}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Run if executed directly
|
||||
if (require.main === module) {
|
||||
exampleUsage().catch((error) => {
|
||||
console.error('Error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
export { exampleUsage };
|
||||
@@ -0,0 +1,10 @@
|
||||
/**
|
||||
* AI Module - Task Complexity Analysis
|
||||
*
|
||||
* This module provides AI-powered task complexity analysis using Claude Opus 4.7.
|
||||
* It determines the optimal number of agents needed to complete a given task.
|
||||
*/
|
||||
|
||||
export { ComplexityAnalyzer, ComplexityAnalysis, AnalyzerConfig, getAnalyzer, resetAnalyzer } from './complexity-analyzer';
|
||||
export { loadExamples, selectFewShotExamples, buildComplexityPrompt, buildSimplifiedPrompt, ComplexityExample } from './prompt-builder';
|
||||
export { validateAnalyzer, quickValidation, generateValidationReport, ValidationResult, ValidationMismatch, ValidationConfig } from './validator';
|
||||
@@ -0,0 +1,172 @@
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
export interface ComplexityExample {
|
||||
task: string;
|
||||
expectedAgents: number;
|
||||
complexity: 'low' | 'medium' | 'high';
|
||||
reasoning: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Load complexity examples from the test dataset
|
||||
*/
|
||||
export function loadExamples(datasetPath?: string): ComplexityExample[] {
|
||||
const defaultPath = path.join(__dirname, '../../../test-data/complexity-dataset.json');
|
||||
const filePath = datasetPath || defaultPath;
|
||||
|
||||
try {
|
||||
const fileContent = fs.readFileSync(filePath, 'utf-8');
|
||||
const examples = JSON.parse(fileContent) as ComplexityExample[];
|
||||
return examples;
|
||||
} catch (error) {
|
||||
console.error(`Failed to load examples from ${filePath}:`, error);
|
||||
return getDefaultExamples();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get default examples if dataset file is not available
|
||||
*/
|
||||
function getDefaultExamples(): ComplexityExample[] {
|
||||
return [
|
||||
{
|
||||
task: 'Fix syntax error in main.py line 42',
|
||||
expectedAgents: 1,
|
||||
complexity: 'low',
|
||||
reasoning: 'Single file, single line fix. No dependencies or side effects.',
|
||||
},
|
||||
{
|
||||
task: 'Add type hints to the calculate_total function',
|
||||
expectedAgents: 1,
|
||||
complexity: 'low',
|
||||
reasoning: 'Single function modification in one file.',
|
||||
},
|
||||
{
|
||||
task: 'Refactor the user authentication module to use JWT tokens',
|
||||
expectedAgents: 3,
|
||||
complexity: 'medium',
|
||||
reasoning: 'Affects multiple functions in auth module, requires updating login/logout flows and adding token validation.',
|
||||
},
|
||||
{
|
||||
task: 'Add input validation to all API endpoints',
|
||||
expectedAgents: 5,
|
||||
complexity: 'medium',
|
||||
reasoning: 'Multiple endpoints across different route files need validation logic.',
|
||||
},
|
||||
{
|
||||
task: 'Add comprehensive unit tests to the entire project',
|
||||
expectedAgents: 12,
|
||||
complexity: 'high',
|
||||
reasoning: 'Testing all modules, functions, edge cases. Requires understanding entire codebase and writing hundreds of test cases.',
|
||||
},
|
||||
{
|
||||
task: 'Refactor entire codebase to follow clean architecture principles',
|
||||
expectedAgents: 15,
|
||||
complexity: 'high',
|
||||
reasoning: 'Major restructuring: separate layers (domain, application, infrastructure), update all imports, maintain functionality.',
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Select diverse examples for few-shot prompting
|
||||
* Ensures representation across low, medium, and high complexity
|
||||
*/
|
||||
export function selectFewShotExamples(
|
||||
examples: ComplexityExample[],
|
||||
count: number = 12
|
||||
): ComplexityExample[] {
|
||||
const lowExamples = examples.filter((e) => e.complexity === 'low');
|
||||
const mediumExamples = examples.filter((e) => e.complexity === 'medium');
|
||||
const highExamples = examples.filter((e) => e.complexity === 'high');
|
||||
|
||||
// Aim for balanced distribution: 40% low, 40% medium, 20% high
|
||||
const lowCount = Math.ceil(count * 0.4);
|
||||
const mediumCount = Math.ceil(count * 0.4);
|
||||
const highCount = count - lowCount - mediumCount;
|
||||
|
||||
const selected: ComplexityExample[] = [];
|
||||
|
||||
// Randomly select from each category
|
||||
selected.push(...selectRandom(lowExamples, lowCount));
|
||||
selected.push(...selectRandom(mediumExamples, mediumCount));
|
||||
selected.push(...selectRandom(highExamples, highCount));
|
||||
|
||||
return selected;
|
||||
}
|
||||
|
||||
/**
|
||||
* Randomly select N items from array
|
||||
*/
|
||||
function selectRandom<T>(array: T[], count: number): T[] {
|
||||
const shuffled = [...array].sort(() => Math.random() - 0.5);
|
||||
return shuffled.slice(0, Math.min(count, array.length));
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the few-shot prompt for complexity analysis
|
||||
*/
|
||||
export function buildComplexityPrompt(taskDescription: string): string {
|
||||
const examples = loadExamples();
|
||||
const fewShotExamples = selectFewShotExamples(examples, 12);
|
||||
|
||||
const examplesText = fewShotExamples
|
||||
.map(
|
||||
(ex) =>
|
||||
`Task: "${ex.task}"
|
||||
Analysis: {
|
||||
"agentCount": ${ex.expectedAgents},
|
||||
"complexity": "${ex.complexity}",
|
||||
"reasoning": "${ex.reasoning}",
|
||||
"confidence": 0.9
|
||||
}`
|
||||
)
|
||||
.join('\n\n');
|
||||
|
||||
return `You are an expert at analyzing software development task complexity and determining how many AI agents are needed to complete the task efficiently.
|
||||
|
||||
Your job is to analyze a task description and return a JSON object with:
|
||||
- agentCount: number of agents needed (1-50)
|
||||
- complexity: "low", "medium", or "high"
|
||||
- reasoning: brief explanation of your decision
|
||||
- confidence: your confidence level (0.0-1.0)
|
||||
|
||||
Guidelines for agent count estimation:
|
||||
- **Low complexity (1-2 agents)**: Single file changes, simple fixes, documentation updates, small additions
|
||||
- **Medium complexity (3-7 agents)**: Multi-file changes, refactoring modules, adding features across several files, implementing new subsystems
|
||||
- **High complexity (8-20 agents)**: Major architectural changes, full test suites, large-scale refactoring, migrating technologies, building new systems
|
||||
|
||||
Consider these factors:
|
||||
1. **Scope**: How many files/modules are affected?
|
||||
2. **Interdependencies**: Do changes require coordination across multiple components?
|
||||
3. **Complexity**: Does it require deep understanding of the codebase?
|
||||
4. **Testing**: How much testing is needed?
|
||||
5. **Risk**: What's the blast radius of potential errors?
|
||||
|
||||
Here are examples of task complexity analysis:
|
||||
|
||||
${examplesText}
|
||||
|
||||
Now analyze this task:
|
||||
|
||||
Task: "${taskDescription}"
|
||||
|
||||
Return ONLY a JSON object with the analysis. No additional text.`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a simplified prompt for faster analysis (used in batch mode)
|
||||
*/
|
||||
export function buildSimplifiedPrompt(taskDescription: string): string {
|
||||
return `Analyze this software development task and estimate how many AI agents (1-50) are needed to complete it.
|
||||
|
||||
Task: "${taskDescription}"
|
||||
|
||||
Consider:
|
||||
- Single file/function changes = 1 agent
|
||||
- Multi-file changes or module refactoring = 3-7 agents
|
||||
- Major architectural changes or full test suites = 8-20 agents
|
||||
|
||||
Return JSON: {"agentCount": N, "complexity": "low|medium|high", "reasoning": "brief explanation"}`;
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Validation script for complexity analyzer
|
||||
* Run with: npm run validate-analyzer
|
||||
*/
|
||||
|
||||
import { ComplexityAnalyzer } from './complexity-analyzer';
|
||||
import { validateAnalyzer, generateValidationReport } from './validator';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
async function main() {
|
||||
console.log('=== Complexity Analyzer Validation ===\n');
|
||||
|
||||
// Check for API key
|
||||
const apiKey = process.env.ANTHROPIC_API_KEY;
|
||||
if (!apiKey) {
|
||||
console.error('Error: ANTHROPIC_API_KEY environment variable not set');
|
||||
console.error('Please set it with: export ANTHROPIC_API_KEY=your_key_here');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Initialize analyzer
|
||||
console.log('Initializing analyzer with Claude Opus 4.7...');
|
||||
const analyzer = new ComplexityAnalyzer({
|
||||
apiKey,
|
||||
model: 'claude-opus-4-7',
|
||||
temperature: 0.3,
|
||||
});
|
||||
|
||||
// Run validation
|
||||
const result = await validateAnalyzer(analyzer, {
|
||||
accuracyThreshold: 0.7,
|
||||
maxMeanAbsoluteError: 2,
|
||||
});
|
||||
|
||||
// Generate report
|
||||
const report = generateValidationReport(result);
|
||||
const reportPath = path.join(__dirname, '../../../validation-report.md');
|
||||
fs.writeFileSync(reportPath, report);
|
||||
console.log(`\nValidation report saved to: ${reportPath}`);
|
||||
|
||||
// Exit with appropriate code
|
||||
if (result.passed) {
|
||||
console.log('\n✓ Validation PASSED - Analyzer meets acceptance criteria');
|
||||
process.exit(0);
|
||||
} else {
|
||||
console.log('\n✗ Validation FAILED - Analyzer does not meet acceptance criteria');
|
||||
console.log('Please review mismatches and tune the prompt.');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
console.error('Validation failed with error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,247 @@
|
||||
import { ComplexityAnalyzer, ComplexityAnalysis } from './complexity-analyzer';
|
||||
import { loadExamples, ComplexityExample } from './prompt-builder';
|
||||
|
||||
export interface ValidationResult {
|
||||
accuracy: number;
|
||||
meanAbsoluteError: number;
|
||||
totalTests: number;
|
||||
correctPredictions: number;
|
||||
withinTwoAgents: number;
|
||||
mismatches: ValidationMismatch[];
|
||||
passed: boolean;
|
||||
}
|
||||
|
||||
export interface ValidationMismatch {
|
||||
task: string;
|
||||
expected: number;
|
||||
predicted: number;
|
||||
error: number;
|
||||
reasoning: string;
|
||||
}
|
||||
|
||||
export interface ValidationConfig {
|
||||
accuracyThreshold?: number; // Default: 0.7 (70%)
|
||||
maxMeanAbsoluteError?: number; // Default: 2
|
||||
datasetPath?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the complexity analyzer against the labeled dataset
|
||||
*/
|
||||
export async function validateAnalyzer(
|
||||
analyzer: ComplexityAnalyzer,
|
||||
config: ValidationConfig = {}
|
||||
): Promise<ValidationResult> {
|
||||
const accuracyThreshold = config.accuracyThreshold || 0.7;
|
||||
const maxMAE = config.maxMeanAbsoluteError || 2;
|
||||
|
||||
console.log('Loading test dataset...');
|
||||
const examples = loadExamples(config.datasetPath);
|
||||
console.log(`Loaded ${examples.length} examples`);
|
||||
|
||||
console.log('Running validation...');
|
||||
const startTime = Date.now();
|
||||
|
||||
const predictions: ComplexityAnalysis[] = [];
|
||||
const mismatches: ValidationMismatch[] = [];
|
||||
let totalError = 0;
|
||||
let correctPredictions = 0;
|
||||
let withinTwoAgents = 0;
|
||||
|
||||
// Analyze each example
|
||||
for (let i = 0; i < examples.length; i++) {
|
||||
const example = examples[i];
|
||||
console.log(`[${i + 1}/${examples.length}] Analyzing: ${example.task.substring(0, 50)}...`);
|
||||
|
||||
try {
|
||||
const prediction = await analyzer.analyzeTask(example.task);
|
||||
predictions.push(prediction);
|
||||
|
||||
const error = Math.abs(prediction.agentCount - example.expectedAgents);
|
||||
totalError += error;
|
||||
|
||||
// Exact match
|
||||
if (prediction.agentCount === example.expectedAgents) {
|
||||
correctPredictions++;
|
||||
withinTwoAgents++;
|
||||
}
|
||||
// Within ±2 agents
|
||||
else if (error <= 2) {
|
||||
withinTwoAgents++;
|
||||
}
|
||||
|
||||
// Record mismatches
|
||||
if (error > 2) {
|
||||
mismatches.push({
|
||||
task: example.task,
|
||||
expected: example.expectedAgents,
|
||||
predicted: prediction.agentCount,
|
||||
error,
|
||||
reasoning: prediction.reasoning,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`Failed to analyze task: ${example.task}`, error);
|
||||
// Count as maximum error
|
||||
totalError += 50;
|
||||
mismatches.push({
|
||||
task: example.task,
|
||||
expected: example.expectedAgents,
|
||||
predicted: 0,
|
||||
error: 50,
|
||||
reasoning: `Analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const elapsedTime = Date.now() - startTime;
|
||||
console.log(`Validation completed in ${elapsedTime}ms`);
|
||||
|
||||
// Calculate metrics
|
||||
const totalTests = examples.length;
|
||||
const accuracy = withinTwoAgents / totalTests;
|
||||
const meanAbsoluteError = totalError / totalTests;
|
||||
|
||||
const passed = accuracy >= accuracyThreshold && meanAbsoluteError <= maxMAE;
|
||||
|
||||
const result: ValidationResult = {
|
||||
accuracy,
|
||||
meanAbsoluteError,
|
||||
totalTests,
|
||||
correctPredictions,
|
||||
withinTwoAgents,
|
||||
mismatches,
|
||||
passed,
|
||||
};
|
||||
|
||||
// Print summary
|
||||
console.log('\n=== Validation Results ===');
|
||||
console.log(`Total tests: ${totalTests}`);
|
||||
console.log(`Exact matches: ${correctPredictions} (${((correctPredictions / totalTests) * 100).toFixed(1)}%)`);
|
||||
console.log(`Within ±2 agents: ${withinTwoAgents} (${(accuracy * 100).toFixed(1)}%)`);
|
||||
console.log(`Mean Absolute Error: ${meanAbsoluteError.toFixed(2)}`);
|
||||
console.log(`Target accuracy: ${(accuracyThreshold * 100).toFixed(0)}%`);
|
||||
console.log(`Target MAE: ≤${maxMAE}`);
|
||||
console.log(`Status: ${passed ? '✓ PASSED' : '✗ FAILED'}`);
|
||||
|
||||
if (mismatches.length > 0) {
|
||||
console.log(`\n=== Mismatches (error >2 agents) ===`);
|
||||
mismatches.slice(0, 10).forEach((m, i) => {
|
||||
console.log(`\n${i + 1}. Task: ${m.task}`);
|
||||
console.log(` Expected: ${m.expected}, Predicted: ${m.predicted}, Error: ${m.error}`);
|
||||
console.log(` Reasoning: ${m.reasoning.substring(0, 100)}...`);
|
||||
});
|
||||
if (mismatches.length > 10) {
|
||||
console.log(`\n... and ${mismatches.length - 10} more mismatches`);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run quick validation on a subset of examples
|
||||
*/
|
||||
export async function quickValidation(
|
||||
analyzer: ComplexityAnalyzer,
|
||||
sampleSize: number = 10
|
||||
): Promise<ValidationResult> {
|
||||
const examples = loadExamples();
|
||||
|
||||
// Select diverse sample
|
||||
const lowExamples = examples.filter((e) => e.complexity === 'low').slice(0, 3);
|
||||
const mediumExamples = examples.filter((e) => e.complexity === 'medium').slice(0, 4);
|
||||
const highExamples = examples.filter((e) => e.complexity === 'high').slice(0, 3);
|
||||
|
||||
const sample = [...lowExamples, ...mediumExamples, ...highExamples];
|
||||
|
||||
console.log(`Running quick validation on ${sample.length} examples...`);
|
||||
|
||||
const predictions: ComplexityAnalysis[] = [];
|
||||
const mismatches: ValidationMismatch[] = [];
|
||||
let totalError = 0;
|
||||
let correctPredictions = 0;
|
||||
let withinTwoAgents = 0;
|
||||
|
||||
for (const example of sample) {
|
||||
try {
|
||||
const prediction = await analyzer.analyzeTask(example.task);
|
||||
predictions.push(prediction);
|
||||
|
||||
const error = Math.abs(prediction.agentCount - example.expectedAgents);
|
||||
totalError += error;
|
||||
|
||||
if (prediction.agentCount === example.expectedAgents) {
|
||||
correctPredictions++;
|
||||
withinTwoAgents++;
|
||||
} else if (error <= 2) {
|
||||
withinTwoAgents++;
|
||||
}
|
||||
|
||||
if (error > 2) {
|
||||
mismatches.push({
|
||||
task: example.task,
|
||||
expected: example.expectedAgents,
|
||||
predicted: prediction.agentCount,
|
||||
error,
|
||||
reasoning: prediction.reasoning,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`Failed to analyze task: ${example.task}`, error);
|
||||
totalError += 50;
|
||||
}
|
||||
}
|
||||
|
||||
const totalTests = sample.length;
|
||||
const accuracy = withinTwoAgents / totalTests;
|
||||
const meanAbsoluteError = totalError / totalTests;
|
||||
|
||||
return {
|
||||
accuracy,
|
||||
meanAbsoluteError,
|
||||
totalTests,
|
||||
correctPredictions,
|
||||
withinTwoAgents,
|
||||
mismatches,
|
||||
passed: accuracy >= 0.7 && meanAbsoluteError <= 2,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a validation report as markdown
|
||||
*/
|
||||
export function generateValidationReport(result: ValidationResult): string {
|
||||
const report = `# Complexity Analyzer Validation Report
|
||||
|
||||
## Summary
|
||||
- **Status**: ${result.passed ? '✓ PASSED' : '✗ FAILED'}
|
||||
- **Total Tests**: ${result.totalTests}
|
||||
- **Accuracy (±2 agents)**: ${(result.accuracy * 100).toFixed(1)}%
|
||||
- **Exact Matches**: ${result.correctPredictions} (${((result.correctPredictions / result.totalTests) * 100).toFixed(1)}%)
|
||||
- **Mean Absolute Error**: ${result.meanAbsoluteError.toFixed(2)}
|
||||
|
||||
## Acceptance Criteria
|
||||
- ✓ Accuracy ≥70%: ${result.accuracy >= 0.7 ? 'PASS' : 'FAIL'}
|
||||
- ✓ MAE ≤2: ${result.meanAbsoluteError <= 2 ? 'PASS' : 'FAIL'}
|
||||
|
||||
## Mismatches (Error >2 agents)
|
||||
${result.mismatches.length === 0 ? 'None' : ''}
|
||||
${result.mismatches
|
||||
.map(
|
||||
(m, i) => `
|
||||
### ${i + 1}. ${m.task}
|
||||
- **Expected**: ${m.expected} agents
|
||||
- **Predicted**: ${m.predicted} agents
|
||||
- **Error**: ${m.error} agents
|
||||
- **Reasoning**: ${m.reasoning}
|
||||
`
|
||||
)
|
||||
.join('\n')}
|
||||
|
||||
---
|
||||
Generated: ${new Date().toISOString()}
|
||||
`;
|
||||
|
||||
return report;
|
||||
}
|
||||
@@ -0,0 +1,342 @@
|
||||
/**
|
||||
* Usage Tracker for Agent Analytics and Cost Control
|
||||
*
|
||||
* Tracks agent usage, compute time, and estimated costs.
|
||||
* Provides budget alerts and usage analytics.
|
||||
*/
|
||||
|
||||
export interface AgentUsageEvent {
|
||||
agentId: string;
|
||||
taskId: string;
|
||||
eventType: 'created' | 'started' | 'completed' | 'failed' | 'terminated';
|
||||
timestamp: number;
|
||||
metadata?: Record<string, any>;
|
||||
}
|
||||
|
||||
export interface AgentSession {
|
||||
agentId: string;
|
||||
taskId: string;
|
||||
startTime: number;
|
||||
endTime?: number;
|
||||
durationSeconds?: number;
|
||||
podName: string;
|
||||
nodeType: string;
|
||||
cpuCores: number;
|
||||
memoryMB: number;
|
||||
status: 'running' | 'completed' | 'failed' | 'terminated';
|
||||
costUSD?: number;
|
||||
}
|
||||
|
||||
export interface UsageMetrics {
|
||||
totalAgents: number;
|
||||
activeAgents: number;
|
||||
completedTasks: number;
|
||||
failedTasks: number;
|
||||
totalComputeHours: number;
|
||||
totalCostUSD: number;
|
||||
averageTaskDuration: number;
|
||||
peakConcurrentAgents: number;
|
||||
}
|
||||
|
||||
export interface CostConfig {
|
||||
cpuCostPerCoreHour: number; // USD per CPU core per hour
|
||||
memoryCostPerGBHour: number; // USD per GB memory per hour
|
||||
budgetLimitUSD?: number;
|
||||
alertThresholds: number[]; // Alert at these percentages of budget
|
||||
}
|
||||
|
||||
export interface BudgetAlert {
|
||||
timestamp: number;
|
||||
currentSpend: number;
|
||||
budgetLimit: number;
|
||||
percentageUsed: number;
|
||||
message: string;
|
||||
severity: 'info' | 'warning' | 'critical';
|
||||
}
|
||||
|
||||
export class UsageTracker {
|
||||
private sessions: Map<string, AgentSession> = new Map();
|
||||
private events: AgentUsageEvent[] = [];
|
||||
private alerts: BudgetAlert[] = [];
|
||||
private costConfig: CostConfig;
|
||||
private alertedThresholds: Set<number> = new Set();
|
||||
|
||||
constructor(costConfig: CostConfig) {
|
||||
this.costConfig = costConfig;
|
||||
}
|
||||
|
||||
/**
|
||||
* Track an agent lifecycle event
|
||||
*/
|
||||
trackEvent(event: AgentUsageEvent): void {
|
||||
this.events.push(event);
|
||||
|
||||
switch (event.eventType) {
|
||||
case 'created':
|
||||
this.handleAgentCreated(event);
|
||||
break;
|
||||
case 'started':
|
||||
this.handleAgentStarted(event);
|
||||
break;
|
||||
case 'completed':
|
||||
case 'failed':
|
||||
case 'terminated':
|
||||
this.handleAgentEnded(event);
|
||||
break;
|
||||
}
|
||||
|
||||
// Check budget after each event
|
||||
this.checkBudgetAlerts();
|
||||
}
|
||||
|
||||
private handleAgentCreated(event: AgentUsageEvent): void {
|
||||
const session: AgentSession = {
|
||||
agentId: event.agentId,
|
||||
taskId: event.taskId,
|
||||
startTime: event.timestamp,
|
||||
podName: event.metadata?.podName || '',
|
||||
nodeType: event.metadata?.nodeType || 'standard',
|
||||
cpuCores: event.metadata?.cpuCores || 2,
|
||||
memoryMB: event.metadata?.memoryMB || 4096,
|
||||
status: 'running'
|
||||
};
|
||||
|
||||
this.sessions.set(event.agentId, session);
|
||||
}
|
||||
|
||||
private handleAgentStarted(event: AgentUsageEvent): void {
|
||||
const session = this.sessions.get(event.agentId);
|
||||
if (session) {
|
||||
session.startTime = event.timestamp;
|
||||
session.status = 'running';
|
||||
}
|
||||
}
|
||||
|
||||
private handleAgentEnded(event: AgentUsageEvent): void {
|
||||
const session = this.sessions.get(event.agentId);
|
||||
if (!session) return;
|
||||
|
||||
session.endTime = event.timestamp;
|
||||
session.durationSeconds = (event.timestamp - session.startTime) / 1000;
|
||||
session.status = event.eventType === 'completed' ? 'completed' :
|
||||
event.eventType === 'failed' ? 'failed' : 'terminated';
|
||||
|
||||
// Calculate cost
|
||||
session.costUSD = this.calculateSessionCost(session);
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate cost for an agent session
|
||||
*/
|
||||
private calculateSessionCost(session: AgentSession): number {
|
||||
if (!session.durationSeconds) return 0;
|
||||
|
||||
const hours = session.durationSeconds / 3600;
|
||||
const cpuCost = session.cpuCores * this.costConfig.cpuCostPerCoreHour * hours;
|
||||
const memoryGB = session.memoryMB / 1024;
|
||||
const memoryCost = memoryGB * this.costConfig.memoryCostPerGBHour * hours;
|
||||
|
||||
return cpuCost + memoryCost;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current usage metrics
|
||||
*/
|
||||
getMetrics(): UsageMetrics {
|
||||
const allSessions = Array.from(this.sessions.values());
|
||||
const activeSessions = allSessions.filter(s => s.status === 'running');
|
||||
const completedSessions = allSessions.filter(s => s.status === 'completed');
|
||||
const failedSessions = allSessions.filter(s => s.status === 'failed');
|
||||
|
||||
const totalComputeSeconds = allSessions.reduce(
|
||||
(sum, s) => sum + (s.durationSeconds || 0),
|
||||
0
|
||||
);
|
||||
|
||||
const totalCost = allSessions.reduce(
|
||||
(sum, s) => sum + (s.costUSD || 0),
|
||||
0
|
||||
);
|
||||
|
||||
const completedDurations = completedSessions
|
||||
.map(s => s.durationSeconds || 0)
|
||||
.filter(d => d > 0);
|
||||
|
||||
const averageDuration = completedDurations.length > 0
|
||||
? completedDurations.reduce((a, b) => a + b, 0) / completedDurations.length
|
||||
: 0;
|
||||
|
||||
// Calculate peak concurrent agents from events
|
||||
const peakConcurrent = this.calculatePeakConcurrency();
|
||||
|
||||
return {
|
||||
totalAgents: allSessions.length,
|
||||
activeAgents: activeSessions.length,
|
||||
completedTasks: completedSessions.length,
|
||||
failedTasks: failedSessions.length,
|
||||
totalComputeHours: totalComputeSeconds / 3600,
|
||||
totalCostUSD: totalCost,
|
||||
averageTaskDuration: averageDuration,
|
||||
peakConcurrentAgents: peakConcurrent
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate peak concurrent agents from event history
|
||||
*/
|
||||
private calculatePeakConcurrency(): number {
|
||||
const timePoints: Array<{ time: number; delta: number }> = [];
|
||||
|
||||
for (const session of this.sessions.values()) {
|
||||
timePoints.push({ time: session.startTime, delta: 1 });
|
||||
if (session.endTime) {
|
||||
timePoints.push({ time: session.endTime, delta: -1 });
|
||||
}
|
||||
}
|
||||
|
||||
timePoints.sort((a, b) => a.time - b.time);
|
||||
|
||||
let current = 0;
|
||||
let peak = 0;
|
||||
|
||||
for (const point of timePoints) {
|
||||
current += point.delta;
|
||||
peak = Math.max(peak, current);
|
||||
}
|
||||
|
||||
return peak;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if budget alerts should be triggered
|
||||
*/
|
||||
private checkBudgetAlerts(): void {
|
||||
if (!this.costConfig.budgetLimitUSD) return;
|
||||
|
||||
const metrics = this.getMetrics();
|
||||
const percentageUsed = (metrics.totalCostUSD / this.costConfig.budgetLimitUSD) * 100;
|
||||
|
||||
for (const threshold of this.costConfig.alertThresholds) {
|
||||
if (percentageUsed >= threshold && !this.alertedThresholds.has(threshold)) {
|
||||
this.alertedThresholds.add(threshold);
|
||||
|
||||
const severity = threshold >= 90 ? 'critical' :
|
||||
threshold >= 75 ? 'warning' : 'info';
|
||||
|
||||
const alert: BudgetAlert = {
|
||||
timestamp: Date.now(),
|
||||
currentSpend: metrics.totalCostUSD,
|
||||
budgetLimit: this.costConfig.budgetLimitUSD,
|
||||
percentageUsed,
|
||||
message: `Budget alert: ${percentageUsed.toFixed(1)}% of budget used ($${metrics.totalCostUSD.toFixed(2)} / $${this.costConfig.budgetLimitUSD})`,
|
||||
severity
|
||||
};
|
||||
|
||||
this.alerts.push(alert);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all budget alerts
|
||||
*/
|
||||
getAlerts(): BudgetAlert[] {
|
||||
return [...this.alerts];
|
||||
}
|
||||
|
||||
/**
|
||||
* Get recent alerts (last N)
|
||||
*/
|
||||
getRecentAlerts(count: number = 10): BudgetAlert[] {
|
||||
return this.alerts.slice(-count);
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all alerts
|
||||
*/
|
||||
clearAlerts(): void {
|
||||
this.alerts = [];
|
||||
this.alertedThresholds.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get usage breakdown by task
|
||||
*/
|
||||
getUsageByTask(): Map<string, { agents: number; cost: number; duration: number }> {
|
||||
const taskMap = new Map<string, { agents: number; cost: number; duration: number }>();
|
||||
|
||||
for (const session of this.sessions.values()) {
|
||||
const existing = taskMap.get(session.taskId) || { agents: 0, cost: 0, duration: 0 };
|
||||
|
||||
taskMap.set(session.taskId, {
|
||||
agents: existing.agents + 1,
|
||||
cost: existing.cost + (session.costUSD || 0),
|
||||
duration: existing.duration + (session.durationSeconds || 0)
|
||||
});
|
||||
}
|
||||
|
||||
return taskMap;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get usage breakdown by time period
|
||||
*/
|
||||
getUsageByPeriod(periodMs: number): Array<{ period: number; agents: number; cost: number }> {
|
||||
const periods = new Map<number, { agents: Set<string>; cost: number }>();
|
||||
|
||||
for (const session of this.sessions.values()) {
|
||||
const periodStart = Math.floor(session.startTime / periodMs) * periodMs;
|
||||
|
||||
const existing = periods.get(periodStart) || { agents: new Set(), cost: 0 };
|
||||
existing.agents.add(session.agentId);
|
||||
existing.cost += session.costUSD || 0;
|
||||
|
||||
periods.set(periodStart, existing);
|
||||
}
|
||||
|
||||
return Array.from(periods.entries())
|
||||
.map(([period, data]) => ({
|
||||
period,
|
||||
agents: data.agents.size,
|
||||
cost: data.cost
|
||||
}))
|
||||
.sort((a, b) => a.period - b.period);
|
||||
}
|
||||
|
||||
/**
|
||||
* Export usage data for analysis
|
||||
*/
|
||||
exportData(): {
|
||||
sessions: AgentSession[];
|
||||
events: AgentUsageEvent[];
|
||||
metrics: UsageMetrics;
|
||||
alerts: BudgetAlert[];
|
||||
} {
|
||||
return {
|
||||
sessions: Array.from(this.sessions.values()),
|
||||
events: [...this.events],
|
||||
metrics: this.getMetrics(),
|
||||
alerts: [...this.alerts]
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset all tracking data
|
||||
*/
|
||||
reset(): void {
|
||||
this.sessions.clear();
|
||||
this.events = [];
|
||||
this.alerts = [];
|
||||
this.alertedThresholds.clear();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Default cost configuration (example AWS pricing)
|
||||
*/
|
||||
export const DEFAULT_COST_CONFIG: CostConfig = {
|
||||
cpuCostPerCoreHour: 0.04, // ~$0.04 per vCPU hour
|
||||
memoryCostPerGBHour: 0.005, // ~$0.005 per GB hour
|
||||
budgetLimitUSD: 100, // $100 default budget
|
||||
alertThresholds: [50, 75, 90, 95] // Alert at 50%, 75%, 90%, 95%
|
||||
};
|
||||
@@ -0,0 +1,215 @@
|
||||
/**
|
||||
* REST API client for the orchestrator service
|
||||
*/
|
||||
|
||||
export interface AgentMetadata {
|
||||
agent_id: string;
|
||||
status: 'idle' | 'busy' | 'handoff' | 'handoff-pending' | 'failed';
|
||||
capabilities: string[];
|
||||
current_task_id?: string;
|
||||
last_heartbeat: number;
|
||||
created_at: number;
|
||||
}
|
||||
|
||||
export interface TaskMetadata {
|
||||
task_id: string;
|
||||
title?: string;
|
||||
description: string;
|
||||
status: 'pending' | 'assigned' | 'in_progress' | 'blocked' | 'completed' | 'failed' | 'cancelled';
|
||||
agent_role?: string;
|
||||
required_capabilities?: string[];
|
||||
depends_on?: string[];
|
||||
parent_task_id?: string;
|
||||
root_task_id?: string;
|
||||
source?: string;
|
||||
assigned_agent_id?: string;
|
||||
context: Record<string, any>;
|
||||
created_at: number;
|
||||
started_at?: number;
|
||||
completed_at?: number;
|
||||
attempt?: number;
|
||||
blocked_reason?: string;
|
||||
max_retries: number;
|
||||
retry_count: number;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export interface HandoffMetadata {
|
||||
handoff_id: string;
|
||||
source_agent_id: string;
|
||||
target_agent_id: string;
|
||||
task_context: Record<string, any>;
|
||||
status: 'pending' | 'completed' | 'failed';
|
||||
created_at: number;
|
||||
completed_at?: number;
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
export interface HealthStatus {
|
||||
status: string;
|
||||
redis: string;
|
||||
active_connections: number;
|
||||
}
|
||||
|
||||
export interface OrchestratorClientConfig {
|
||||
baseUrl: string;
|
||||
timeout?: number;
|
||||
}
|
||||
|
||||
export class OrchestratorClient {
|
||||
private baseUrl: string;
|
||||
private timeout: number;
|
||||
|
||||
constructor(config: OrchestratorClientConfig) {
|
||||
this.baseUrl = config.baseUrl.replace(/\/$/, ''); // Remove trailing slash
|
||||
this.timeout = config.timeout || 10000;
|
||||
}
|
||||
|
||||
/**
|
||||
* Make HTTP request with timeout
|
||||
*/
|
||||
private async request<T>(
|
||||
endpoint: string,
|
||||
options: RequestInit = {}
|
||||
): Promise<T> {
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), this.timeout);
|
||||
|
||||
try {
|
||||
const response = await fetch(`${this.baseUrl}${endpoint}`, {
|
||||
...options,
|
||||
signal: controller.signal,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...options.headers,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.text();
|
||||
throw new Error(`HTTP ${response.status}: ${error}`);
|
||||
}
|
||||
|
||||
return await response.json();
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Health check
|
||||
*/
|
||||
async health(): Promise<HealthStatus> {
|
||||
return this.request<HealthStatus>('/health');
|
||||
}
|
||||
|
||||
/**
|
||||
* List all agents
|
||||
*/
|
||||
async listAgents(): Promise<AgentMetadata[]> {
|
||||
const response = await this.request<{ agents: AgentMetadata[] }>('/agents');
|
||||
return response.agents;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get specific agent
|
||||
*/
|
||||
async getAgent(agentId: string): Promise<AgentMetadata> {
|
||||
return this.request<AgentMetadata>(`/agents/${agentId}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* List idle agents
|
||||
*/
|
||||
async listIdleAgents(): Promise<AgentMetadata[]> {
|
||||
const response = await this.request<{ agents: AgentMetadata[] }>('/agents/idle');
|
||||
return response.agents;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a new task
|
||||
*/
|
||||
async createTask(
|
||||
description: string,
|
||||
context: Record<string, any> = {},
|
||||
maxRetries: number = 3
|
||||
): Promise<TaskMetadata> {
|
||||
return this.request<TaskMetadata>('/tasks', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({
|
||||
description,
|
||||
context,
|
||||
max_retries: maxRetries,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Assign task to agent
|
||||
*/
|
||||
async assignTask(taskId: string, agentId: string): Promise<void> {
|
||||
await this.request('/tasks/assign', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({
|
||||
task_id: taskId,
|
||||
agent_id: agentId,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* List all tasks
|
||||
*/
|
||||
async listTasks(status?: string): Promise<TaskMetadata[]> {
|
||||
const query = status ? `?status=${status}` : '';
|
||||
const response = await this.request<{ tasks: TaskMetadata[] }>(`/tasks${query}`);
|
||||
return response.tasks;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get specific task
|
||||
*/
|
||||
async getTask(taskId: string): Promise<TaskMetadata> {
|
||||
return this.request<TaskMetadata>(`/tasks/${taskId}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* List handoff history
|
||||
*/
|
||||
async listHandoffs(agentId?: string, limit: number = 100): Promise<HandoffMetadata[]> {
|
||||
const params = new URLSearchParams();
|
||||
if (agentId) params.append('agent_id', agentId);
|
||||
params.append('limit', limit.toString());
|
||||
|
||||
const query = params.toString() ? `?${params.toString()}` : '';
|
||||
const response = await this.request<{ handoffs: HandoffMetadata[] }>(`/handoffs${query}`);
|
||||
return response.handoffs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create WebSocket connection for real-time updates
|
||||
*/
|
||||
createWebSocket(agentId: string): WebSocket {
|
||||
const wsUrl = this.baseUrl.replace(/^http/, 'ws');
|
||||
return new WebSocket(`${wsUrl}/ws/${agentId}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Singleton instance
|
||||
*/
|
||||
let clientInstance: OrchestratorClient | null = null;
|
||||
|
||||
export function getOrchestratorClient(config?: OrchestratorClientConfig): OrchestratorClient {
|
||||
if (!clientInstance) {
|
||||
if (!config) {
|
||||
throw new Error('OrchestratorClient not initialized. Provide config on first call.');
|
||||
}
|
||||
clientInstance = new OrchestratorClient(config);
|
||||
}
|
||||
return clientInstance;
|
||||
}
|
||||
|
||||
export function resetOrchestratorClient(): void {
|
||||
clientInstance = null;
|
||||
}
|
||||
@@ -0,0 +1,225 @@
|
||||
import React, { useEffect, useState } from 'react';
|
||||
import { AgentMetadata, getOrchestratorClient } from '../api/orchestrator-client';
|
||||
|
||||
export interface AgentDashboardProps {
|
||||
orchestratorUrl: string;
|
||||
workspaceId?: string;
|
||||
}
|
||||
|
||||
export const AgentDashboard: React.FC<AgentDashboardProps> = ({
|
||||
orchestratorUrl,
|
||||
workspaceId,
|
||||
}) => {
|
||||
const [agents, setAgents] = useState<AgentMetadata[]>([]);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [connected, setConnected] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
const client = getOrchestratorClient({ baseUrl: orchestratorUrl });
|
||||
|
||||
// Initial fetch
|
||||
const fetchAgents = async () => {
|
||||
try {
|
||||
const agentList = await client.listAgents();
|
||||
setAgents(agentList);
|
||||
setConnected(true);
|
||||
setError(null);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Failed to fetch agents');
|
||||
setConnected(false);
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
fetchAgents();
|
||||
|
||||
// Poll for updates every 2 seconds
|
||||
const interval = setInterval(fetchAgents, 2000);
|
||||
|
||||
return () => clearInterval(interval);
|
||||
}, [orchestratorUrl]);
|
||||
|
||||
const getStatusColor = (status: string) => {
|
||||
switch (status) {
|
||||
case 'idle':
|
||||
return 'bg-gray-500';
|
||||
case 'busy':
|
||||
return 'bg-blue-500 animate-pulse';
|
||||
case 'handoff':
|
||||
case 'handoff-pending':
|
||||
return 'bg-yellow-500 animate-pulse';
|
||||
case 'failed':
|
||||
return 'bg-red-500';
|
||||
default:
|
||||
return 'bg-gray-500';
|
||||
}
|
||||
};
|
||||
|
||||
const getStatusText = (status: string) => {
|
||||
switch (status) {
|
||||
case 'idle':
|
||||
return 'Idle';
|
||||
case 'busy':
|
||||
return 'Working';
|
||||
case 'handoff':
|
||||
case 'handoff-pending':
|
||||
return 'Handoff';
|
||||
case 'failed':
|
||||
return 'Failed';
|
||||
default:
|
||||
return 'Unknown';
|
||||
}
|
||||
};
|
||||
|
||||
const formatTimestamp = (timestamp: number) => {
|
||||
const date = new Date(timestamp * 1000);
|
||||
const now = Date.now();
|
||||
const diff = now - date.getTime();
|
||||
|
||||
if (diff < 60000) {
|
||||
return 'Just now';
|
||||
} else if (diff < 3600000) {
|
||||
return `${Math.floor(diff / 60000)}m ago`;
|
||||
} else if (diff < 86400000) {
|
||||
return `${Math.floor(diff / 3600000)}h ago`;
|
||||
} else {
|
||||
return date.toLocaleDateString();
|
||||
}
|
||||
};
|
||||
|
||||
const filteredAgents = workspaceId
|
||||
? agents.filter(agent => agent.agent_id.includes(workspaceId))
|
||||
: agents;
|
||||
|
||||
if (loading) {
|
||||
return (
|
||||
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
|
||||
<h3 className="text-lg font-semibold mb-4">Agent Dashboard</h3>
|
||||
<div className="flex items-center justify-center py-8">
|
||||
<svg className="animate-spin h-8 w-8 text-blue-500" viewBox="0 0 24 24">
|
||||
<circle
|
||||
className="opacity-25"
|
||||
cx="12"
|
||||
cy="12"
|
||||
r="10"
|
||||
stroke="currentColor"
|
||||
strokeWidth="4"
|
||||
fill="none"
|
||||
/>
|
||||
<path
|
||||
className="opacity-75"
|
||||
fill="currentColor"
|
||||
d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"
|
||||
/>
|
||||
</svg>
|
||||
<span className="ml-3 text-gray-400">Loading agents...</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
|
||||
<div className="flex items-center justify-between mb-4">
|
||||
<h3 className="text-lg font-semibold">Agent Dashboard</h3>
|
||||
<div className="flex items-center space-x-2">
|
||||
<div className={`w-2 h-2 rounded-full ${connected ? 'bg-green-500' : 'bg-red-500'}`} />
|
||||
<span className="text-sm text-gray-400">
|
||||
{connected ? 'Connected' : 'Disconnected'}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="bg-red-900/50 border border-red-500 rounded px-4 py-2 text-red-200 text-sm mb-4">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="mb-4 grid grid-cols-3 gap-4">
|
||||
<div className="bg-gray-700 rounded p-3">
|
||||
<p className="text-xs text-gray-400 mb-1">Total Agents</p>
|
||||
<p className="text-2xl font-bold text-white">{filteredAgents.length}</p>
|
||||
</div>
|
||||
<div className="bg-gray-700 rounded p-3">
|
||||
<p className="text-xs text-gray-400 mb-1">Active</p>
|
||||
<p className="text-2xl font-bold text-blue-400">
|
||||
{filteredAgents.filter(a => a.status === 'busy').length}
|
||||
</p>
|
||||
</div>
|
||||
<div className="bg-gray-700 rounded p-3">
|
||||
<p className="text-xs text-gray-400 mb-1">Idle</p>
|
||||
<p className="text-2xl font-bold text-gray-400">
|
||||
{filteredAgents.filter(a => a.status === 'idle').length}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{filteredAgents.length === 0 ? (
|
||||
<div className="text-center py-8 text-gray-500">
|
||||
<p>No agents running</p>
|
||||
<p className="text-sm mt-2">Submit a task to create agents</p>
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-3 max-h-96 overflow-y-auto">
|
||||
{filteredAgents.map((agent) => (
|
||||
<div
|
||||
key={agent.agent_id}
|
||||
className="bg-gray-700 rounded p-4 hover:bg-gray-650 transition-colors"
|
||||
>
|
||||
<div className="flex items-start justify-between">
|
||||
<div className="flex-1">
|
||||
<div className="flex items-center space-x-2 mb-2">
|
||||
<div className={`w-3 h-3 rounded-full ${getStatusColor(agent.status)}`} />
|
||||
<span className="font-medium text-white truncate">
|
||||
{agent.agent_id}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1 text-sm">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-gray-400">Status:</span>
|
||||
<span className="text-gray-300">{getStatusText(agent.status)}</span>
|
||||
</div>
|
||||
|
||||
{agent.current_task_id && (
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-gray-400">Task:</span>
|
||||
<span className="text-blue-400 text-xs truncate max-w-xs">
|
||||
{agent.current_task_id}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-gray-400">Last Heartbeat:</span>
|
||||
<span className="text-gray-300">{formatTimestamp(agent.last_heartbeat)}</span>
|
||||
</div>
|
||||
|
||||
{agent.capabilities.length > 0 && (
|
||||
<div className="mt-2">
|
||||
<span className="text-gray-400 text-xs">Capabilities:</span>
|
||||
<div className="flex flex-wrap gap-1 mt-1">
|
||||
{agent.capabilities.map((cap, idx) => (
|
||||
<span
|
||||
key={idx}
|
||||
className="bg-gray-600 text-gray-300 text-xs px-2 py-0.5 rounded"
|
||||
>
|
||||
{cap}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,260 @@
|
||||
import React, { useState } from 'react';
|
||||
import { ComplexityAnalysis } from '../ai/complexity-analyzer';
|
||||
|
||||
export interface CostControlDialogProps {
|
||||
analysis: ComplexityAnalysis;
|
||||
maxAgents: number;
|
||||
onConfirm: () => void;
|
||||
onCancel: () => void;
|
||||
onAdjust: (newCount: number) => void;
|
||||
}
|
||||
|
||||
export const CostControlDialog: React.FC<CostControlDialogProps> = ({
|
||||
analysis,
|
||||
maxAgents,
|
||||
onConfirm,
|
||||
onCancel,
|
||||
onAdjust,
|
||||
}) => {
|
||||
const [adjustedCount, setAdjustedCount] = useState(analysis.agentCount);
|
||||
const agentCount = analysis.agentCount;
|
||||
|
||||
// Determine warning level
|
||||
const isWarning = agentCount > 10;
|
||||
const requiresConfirmation = agentCount > 15;
|
||||
const exceedsMax = agentCount > maxAgents;
|
||||
|
||||
// Calculate estimated resource cost
|
||||
const estimatedCpuHours = (agentCount * 0.5).toFixed(1); // Assume 30min avg per agent
|
||||
const estimatedCost = (agentCount * 0.05).toFixed(2); // Rough cost estimate
|
||||
|
||||
if (!isWarning && !exceedsMax) {
|
||||
// No warning needed, auto-confirm
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="fixed inset-0 bg-black bg-opacity-50 flex items-center justify-center z-50">
|
||||
<div className="bg-white rounded-lg shadow-xl max-w-2xl w-full mx-4 p-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-start mb-4">
|
||||
<div
|
||||
className={`flex-shrink-0 w-12 h-12 rounded-full flex items-center justify-center ${
|
||||
exceedsMax
|
||||
? 'bg-red-100'
|
||||
: requiresConfirmation
|
||||
? 'bg-orange-100'
|
||||
: 'bg-yellow-100'
|
||||
}`}
|
||||
>
|
||||
<svg
|
||||
className={`w-6 h-6 ${
|
||||
exceedsMax
|
||||
? 'text-red-600'
|
||||
: requiresConfirmation
|
||||
? 'text-orange-600'
|
||||
: 'text-yellow-600'
|
||||
}`}
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
viewBox="0 0 24 24"
|
||||
>
|
||||
<path
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
strokeWidth={2}
|
||||
d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z"
|
||||
/>
|
||||
</svg>
|
||||
</div>
|
||||
<div className="ml-4 flex-1">
|
||||
<h3 className="text-lg font-semibold text-gray-900">
|
||||
{exceedsMax
|
||||
? 'Agent Limit Exceeded'
|
||||
: requiresConfirmation
|
||||
? 'High Agent Count Confirmation Required'
|
||||
: 'High Agent Count Warning'}
|
||||
</h3>
|
||||
<p className="mt-1 text-sm text-gray-600">
|
||||
{exceedsMax
|
||||
? `The task requires ${agentCount} agents, which exceeds your configured maximum of ${maxAgents}.`
|
||||
: requiresConfirmation
|
||||
? `This task will create ${agentCount} agents. Please confirm this is intentional.`
|
||||
: `This task will create ${agentCount} agents, which may consume significant resources.`}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Analysis Details */}
|
||||
<div className="bg-gray-50 rounded-lg p-4 mb-4">
|
||||
<h4 className="text-sm font-medium text-gray-900 mb-2">Task Analysis</h4>
|
||||
<div className="space-y-2 text-sm">
|
||||
<div className="flex justify-between">
|
||||
<span className="text-gray-600">Complexity:</span>
|
||||
<span className="font-medium capitalize">{analysis.complexity}</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span className="text-gray-600">Recommended Agents:</span>
|
||||
<span className="font-medium">{agentCount}</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span className="text-gray-600">Confidence:</span>
|
||||
<span className="font-medium">{(analysis.confidence * 100).toFixed(0)}%</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="mt-3 pt-3 border-t border-gray-200">
|
||||
<p className="text-sm text-gray-700">
|
||||
<span className="font-medium">Reasoning:</span> {analysis.reasoning}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Resource Estimates */}
|
||||
<div className="bg-blue-50 rounded-lg p-4 mb-4">
|
||||
<h4 className="text-sm font-medium text-blue-900 mb-2">Estimated Resource Usage</h4>
|
||||
<div className="grid grid-cols-2 gap-4 text-sm">
|
||||
<div>
|
||||
<span className="text-blue-700">CPU Hours:</span>
|
||||
<span className="ml-2 font-medium text-blue-900">~{estimatedCpuHours}h</span>
|
||||
</div>
|
||||
<div>
|
||||
<span className="text-blue-700">Estimated Cost:</span>
|
||||
<span className="ml-2 font-medium text-blue-900">${estimatedCost}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Agent Count Adjustment */}
|
||||
{!exceedsMax && (
|
||||
<div className="mb-4">
|
||||
<label className="block text-sm font-medium text-gray-700 mb-2">
|
||||
Adjust Agent Count (Optional)
|
||||
</label>
|
||||
<div className="flex items-center space-x-4">
|
||||
<input
|
||||
type="range"
|
||||
min="1"
|
||||
max={maxAgents}
|
||||
value={adjustedCount}
|
||||
onChange={(e) => setAdjustedCount(parseInt(e.target.value, 10))}
|
||||
className="flex-1"
|
||||
/>
|
||||
<input
|
||||
type="number"
|
||||
min="1"
|
||||
max={maxAgents}
|
||||
value={adjustedCount}
|
||||
onChange={(e) => setAdjustedCount(parseInt(e.target.value, 10))}
|
||||
className="w-20 px-3 py-2 border border-gray-300 rounded-md text-sm"
|
||||
/>
|
||||
</div>
|
||||
{adjustedCount !== agentCount && (
|
||||
<p className="mt-2 text-sm text-gray-600">
|
||||
Adjusting from {agentCount} to {adjustedCount} agents may affect task completion
|
||||
quality.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Actions */}
|
||||
<div className="flex justify-end space-x-3">
|
||||
<button
|
||||
onClick={onCancel}
|
||||
className="px-4 py-2 text-sm font-medium text-gray-700 bg-white border border-gray-300 rounded-md hover:bg-gray-50 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500"
|
||||
>
|
||||
Cancel
|
||||
</button>
|
||||
{exceedsMax ? (
|
||||
<button
|
||||
onClick={() => {
|
||||
setAdjustedCount(maxAgents);
|
||||
onAdjust(maxAgents);
|
||||
}}
|
||||
className="px-4 py-2 text-sm font-medium text-white bg-blue-600 rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500"
|
||||
>
|
||||
Use Maximum ({maxAgents} agents)
|
||||
</button>
|
||||
) : (
|
||||
<>
|
||||
{adjustedCount !== agentCount && (
|
||||
<button
|
||||
onClick={() => onAdjust(adjustedCount)}
|
||||
className="px-4 py-2 text-sm font-medium text-white bg-blue-600 rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500"
|
||||
>
|
||||
Use {adjustedCount} Agents
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
onClick={onConfirm}
|
||||
className={`px-4 py-2 text-sm font-medium text-white rounded-md focus:outline-none focus:ring-2 focus:ring-offset-2 ${
|
||||
requiresConfirmation
|
||||
? 'bg-orange-600 hover:bg-orange-700 focus:ring-orange-500'
|
||||
: 'bg-yellow-600 hover:bg-yellow-700 focus:ring-yellow-500'
|
||||
}`}
|
||||
>
|
||||
{requiresConfirmation ? 'Confirm' : 'Proceed'} with {agentCount} Agents
|
||||
</button>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Hook to manage cost control dialog state
|
||||
*/
|
||||
export function useCostControl(maxAgents: number = 20) {
|
||||
const [showDialog, setShowDialog] = useState(false);
|
||||
const [currentAnalysis, setCurrentAnalysis] = useState<ComplexityAnalysis | null>(null);
|
||||
const [onConfirmCallback, setOnConfirmCallback] = useState<(() => void) | null>(null);
|
||||
|
||||
const checkAndConfirm = (
|
||||
analysis: ComplexityAnalysis,
|
||||
onConfirm: () => void
|
||||
): Promise<boolean> => {
|
||||
return new Promise((resolve) => {
|
||||
const agentCount = analysis.agentCount;
|
||||
|
||||
// Auto-approve if within safe limits
|
||||
if (agentCount <= 10 && agentCount <= maxAgents) {
|
||||
onConfirm();
|
||||
resolve(true);
|
||||
return;
|
||||
}
|
||||
|
||||
// Show dialog for high counts or exceeding max
|
||||
setCurrentAnalysis(analysis);
|
||||
setOnConfirmCallback(() => () => {
|
||||
onConfirm();
|
||||
setShowDialog(false);
|
||||
resolve(true);
|
||||
});
|
||||
setShowDialog(true);
|
||||
});
|
||||
};
|
||||
|
||||
const handleCancel = () => {
|
||||
setShowDialog(false);
|
||||
setCurrentAnalysis(null);
|
||||
setOnConfirmCallback(null);
|
||||
};
|
||||
|
||||
const handleAdjust = (newCount: number) => {
|
||||
if (currentAnalysis) {
|
||||
const adjustedAnalysis = { ...currentAnalysis, agentCount: newCount };
|
||||
setCurrentAnalysis(adjustedAnalysis);
|
||||
}
|
||||
};
|
||||
|
||||
return {
|
||||
showDialog,
|
||||
currentAnalysis,
|
||||
checkAndConfirm,
|
||||
handleCancel,
|
||||
handleConfirm: onConfirmCallback,
|
||||
handleAdjust,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,238 @@
|
||||
import React, { useState } from 'react';
|
||||
import { MergeConflict, ConflictResolution } from '../git/merge-manager';
|
||||
|
||||
export interface ResultViewerProps {
|
||||
conflicts: MergeConflict[];
|
||||
mergedBranches: string[];
|
||||
onResolve: (resolutions: ConflictResolution[]) => void;
|
||||
onCancel: () => void;
|
||||
}
|
||||
|
||||
export const ResultViewer: React.FC<ResultViewerProps> = ({
|
||||
conflicts,
|
||||
mergedBranches,
|
||||
onResolve,
|
||||
onCancel,
|
||||
}) => {
|
||||
const [resolutions, setResolutions] = useState<Map<string, ConflictResolution>>(new Map());
|
||||
const [selectedFile, setSelectedFile] = useState<string | null>(
|
||||
conflicts.length > 0 ? conflicts[0].filepath : null
|
||||
);
|
||||
|
||||
const handleResolutionChange = (
|
||||
filepath: string,
|
||||
resolution: 'ours' | 'theirs' | 'manual',
|
||||
manualContent?: string
|
||||
) => {
|
||||
const newResolutions = new Map(resolutions);
|
||||
newResolutions.set(filepath, {
|
||||
filepath,
|
||||
resolution,
|
||||
manualContent,
|
||||
});
|
||||
setResolutions(newResolutions);
|
||||
};
|
||||
|
||||
const handleResolveAll = () => {
|
||||
const allResolutions = Array.from(resolutions.values());
|
||||
|
||||
// Ensure all conflicts have resolutions
|
||||
const unresolvedConflicts = conflicts.filter(
|
||||
conflict => !resolutions.has(conflict.filepath)
|
||||
);
|
||||
|
||||
if (unresolvedConflicts.length > 0) {
|
||||
alert(`Please resolve all conflicts. ${unresolvedConflicts.length} remaining.`);
|
||||
return;
|
||||
}
|
||||
|
||||
onResolve(allResolutions);
|
||||
};
|
||||
|
||||
const selectedConflict = conflicts.find(c => c.filepath === selectedFile);
|
||||
const currentResolution = selectedFile ? resolutions.get(selectedFile) : null;
|
||||
|
||||
if (conflicts.length === 0) {
|
||||
return (
|
||||
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
|
||||
<h3 className="text-lg font-semibold mb-4">Merge Results</h3>
|
||||
|
||||
<div className="bg-green-900/50 border border-green-500 rounded p-4 mb-4">
|
||||
<div className="flex items-center space-x-2">
|
||||
<svg className="w-6 h-6 text-green-400" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
|
||||
</svg>
|
||||
<span className="text-green-200 font-medium">Merge completed successfully!</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<p className="text-sm text-gray-400">Merged branches:</p>
|
||||
<div className="bg-gray-700 rounded p-3 space-y-1">
|
||||
{mergedBranches.map((branch, idx) => (
|
||||
<div key={idx} className="flex items-center space-x-2">
|
||||
<svg className="w-4 h-4 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M7.707 3.293a1 1 0 010 1.414L5.414 7H11a7 7 0 017 7v2a1 1 0 11-2 0v-2a5 5 0 00-5-5H5.414l2.293 2.293a1 1 0 11-1.414 1.414l-4-4a1 1 0 010-1.414l4-4a1 1 0 011.414 0z" clipRule="evenodd" />
|
||||
</svg>
|
||||
<span className="text-sm text-gray-300 font-mono">{branch}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
|
||||
<h3 className="text-lg font-semibold mb-4">Resolve Merge Conflicts</h3>
|
||||
|
||||
<div className="bg-yellow-900/50 border border-yellow-500 rounded p-4 mb-4">
|
||||
<div className="flex items-center space-x-2">
|
||||
<svg className="w-6 h-6 text-yellow-400" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
||||
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z" />
|
||||
</svg>
|
||||
<span className="text-yellow-200 font-medium">
|
||||
{conflicts.length} conflict{conflicts.length > 1 ? 's' : ''} detected
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-3 gap-4">
|
||||
{/* File list */}
|
||||
<div className="col-span-1 bg-gray-700 rounded p-3 max-h-96 overflow-y-auto">
|
||||
<p className="text-xs text-gray-400 mb-2 font-medium">Conflicted Files</p>
|
||||
<div className="space-y-1">
|
||||
{conflicts.map((conflict) => {
|
||||
const resolved = resolutions.has(conflict.filepath);
|
||||
return (
|
||||
<button
|
||||
key={conflict.filepath}
|
||||
className={`w-full text-left px-3 py-2 rounded text-sm transition-colors ${
|
||||
selectedFile === conflict.filepath
|
||||
? 'bg-blue-600 text-white'
|
||||
: 'bg-gray-600 text-gray-300 hover:bg-gray-550'
|
||||
}`}
|
||||
onClick={() => setSelectedFile(conflict.filepath)}
|
||||
>
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="truncate font-mono text-xs">{conflict.filepath}</span>
|
||||
{resolved && (
|
||||
<svg className="w-4 h-4 text-green-400 flex-shrink-0 ml-2" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
|
||||
</svg>
|
||||
)}
|
||||
</div>
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Conflict viewer and resolution */}
|
||||
<div className="col-span-2 space-y-4">
|
||||
{selectedConflict && (
|
||||
<>
|
||||
<div className="bg-gray-700 rounded p-4">
|
||||
<p className="text-sm font-medium text-gray-300 mb-3">
|
||||
{selectedConflict.filepath}
|
||||
</p>
|
||||
|
||||
<div className="space-y-3">
|
||||
<button
|
||||
className={`w-full text-left p-3 rounded border-2 transition-colors ${
|
||||
currentResolution?.resolution === 'ours'
|
||||
? 'border-blue-500 bg-blue-900/30'
|
||||
: 'border-gray-600 hover:border-gray-500'
|
||||
}`}
|
||||
onClick={() => handleResolutionChange(selectedConflict.filepath, 'ours')}
|
||||
>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<span className="text-sm font-medium text-gray-300">Use Current Version</span>
|
||||
{currentResolution?.resolution === 'ours' && (
|
||||
<svg className="w-5 h-5 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
|
||||
</svg>
|
||||
)}
|
||||
</div>
|
||||
<pre className="text-xs text-gray-400 bg-gray-800 p-2 rounded overflow-x-auto max-h-32">
|
||||
{selectedConflict.ours}
|
||||
</pre>
|
||||
</button>
|
||||
|
||||
<button
|
||||
className={`w-full text-left p-3 rounded border-2 transition-colors ${
|
||||
currentResolution?.resolution === 'theirs'
|
||||
? 'border-blue-500 bg-blue-900/30'
|
||||
: 'border-gray-600 hover:border-gray-500'
|
||||
}`}
|
||||
onClick={() => handleResolutionChange(selectedConflict.filepath, 'theirs')}
|
||||
>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<span className="text-sm font-medium text-gray-300">Use Agent Version</span>
|
||||
{currentResolution?.resolution === 'theirs' && (
|
||||
<svg className="w-5 h-5 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
|
||||
</svg>
|
||||
)}
|
||||
</div>
|
||||
<pre className="text-xs text-gray-400 bg-gray-800 p-2 rounded overflow-x-auto max-h-32">
|
||||
{selectedConflict.theirs}
|
||||
</pre>
|
||||
</button>
|
||||
|
||||
<div
|
||||
className={`p-3 rounded border-2 ${
|
||||
currentResolution?.resolution === 'manual'
|
||||
? 'border-blue-500 bg-blue-900/30'
|
||||
: 'border-gray-600'
|
||||
}`}
|
||||
>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<span className="text-sm font-medium text-gray-300">Manual Resolution</span>
|
||||
{currentResolution?.resolution === 'manual' && (
|
||||
<svg className="w-5 h-5 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
|
||||
</svg>
|
||||
)}
|
||||
</div>
|
||||
<textarea
|
||||
className="w-full bg-gray-800 text-gray-300 text-xs font-mono p-2 rounded focus:outline-none focus:ring-2 focus:ring-blue-500"
|
||||
rows={8}
|
||||
placeholder="Enter your manual resolution..."
|
||||
value={currentResolution?.resolution === 'manual' ? currentResolution.manualContent : ''}
|
||||
onChange={(e) =>
|
||||
handleResolutionChange(selectedConflict.filepath, 'manual', e.target.value)
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="mt-6 flex justify-between items-center">
|
||||
<div className="text-sm text-gray-400">
|
||||
Resolved: {resolutions.size} / {conflicts.length}
|
||||
</div>
|
||||
<div className="flex space-x-3">
|
||||
<button
|
||||
className="bg-gray-600 hover:bg-gray-700 text-white font-semibold px-6 py-2 rounded transition-colors"
|
||||
onClick={onCancel}
|
||||
>
|
||||
Cancel
|
||||
</button>
|
||||
<button
|
||||
className="bg-green-600 hover:bg-green-700 text-white font-semibold px-6 py-2 rounded disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
|
||||
onClick={handleResolveAll}
|
||||
disabled={resolutions.size !== conflicts.length}
|
||||
>
|
||||
Apply Resolutions
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,165 @@
|
||||
import React, { useState } from 'react';
|
||||
import { ComplexityAnalysis, getAnalyzer } from '../ai/complexity-analyzer';
|
||||
|
||||
export interface TaskSubmissionFormProps {
|
||||
onSubmit: (task: string, complexity: ComplexityAnalysis) => void;
|
||||
disabled?: boolean;
|
||||
}
|
||||
|
||||
export const TaskSubmissionForm: React.FC<TaskSubmissionFormProps> = ({
|
||||
onSubmit,
|
||||
disabled = false,
|
||||
}) => {
|
||||
const [taskDescription, setTaskDescription] = useState('');
|
||||
const [analyzing, setAnalyzing] = useState(false);
|
||||
const [complexity, setComplexity] = useState<ComplexityAnalysis | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
const handleAnalyze = async () => {
|
||||
if (!taskDescription.trim()) {
|
||||
setError('Please enter a task description');
|
||||
return;
|
||||
}
|
||||
|
||||
setAnalyzing(true);
|
||||
setError(null);
|
||||
|
||||
try {
|
||||
const analyzer = getAnalyzer();
|
||||
const result = await analyzer.analyzeTask(taskDescription);
|
||||
setComplexity(result);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Failed to analyze task');
|
||||
setComplexity(null);
|
||||
} finally {
|
||||
setAnalyzing(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleSubmit = () => {
|
||||
if (!complexity) {
|
||||
setError('Please analyze the task first');
|
||||
return;
|
||||
}
|
||||
|
||||
onSubmit(taskDescription, complexity);
|
||||
|
||||
// Reset form
|
||||
setTaskDescription('');
|
||||
setComplexity(null);
|
||||
setError(null);
|
||||
};
|
||||
|
||||
const getComplexityColor = (level: string) => {
|
||||
switch (level) {
|
||||
case 'low':
|
||||
return 'text-green-400';
|
||||
case 'medium':
|
||||
return 'text-yellow-400';
|
||||
case 'high':
|
||||
return 'text-red-400';
|
||||
default:
|
||||
return 'text-gray-400';
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
|
||||
<h3 className="text-lg font-semibold mb-4">Submit Task</h3>
|
||||
|
||||
<div className="space-y-4">
|
||||
<div>
|
||||
<label className="block text-sm font-medium text-gray-300 mb-2">
|
||||
Task Description
|
||||
</label>
|
||||
<textarea
|
||||
className="w-full bg-gray-700 text-white rounded px-4 py-2 focus:outline-none focus:ring-2 focus:ring-blue-500 disabled:opacity-50 disabled:cursor-not-allowed"
|
||||
rows={6}
|
||||
placeholder="Describe your programming task in detail..."
|
||||
value={taskDescription}
|
||||
onChange={(e) => setTaskDescription(e.target.value)}
|
||||
disabled={disabled || analyzing}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="bg-red-900/50 border border-red-500 rounded px-4 py-2 text-red-200 text-sm">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{complexity && (
|
||||
<div className="bg-gray-700 rounded p-4 space-y-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm font-medium text-gray-300">Complexity:</span>
|
||||
<span className={`text-sm font-bold uppercase ${getComplexityColor(complexity.complexity)}`}>
|
||||
{complexity.complexity}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm font-medium text-gray-300">Recommended Agents:</span>
|
||||
<span className="text-sm font-bold text-blue-400">{complexity.agentCount}</span>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm font-medium text-gray-300">Confidence:</span>
|
||||
<span className="text-sm font-bold text-gray-300">
|
||||
{Math.round(complexity.confidence * 100)}%
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="pt-2 border-t border-gray-600">
|
||||
<p className="text-xs text-gray-400 mb-1">Analysis:</p>
|
||||
<p className="text-sm text-gray-300">{complexity.reasoning}</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex space-x-3">
|
||||
<button
|
||||
className="flex-1 bg-blue-600 hover:bg-blue-700 text-white font-semibold px-6 py-2 rounded disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
|
||||
onClick={handleAnalyze}
|
||||
disabled={disabled || analyzing || !taskDescription.trim()}
|
||||
>
|
||||
{analyzing ? (
|
||||
<span className="flex items-center justify-center">
|
||||
<svg className="animate-spin h-5 w-5 mr-2" viewBox="0 0 24 24">
|
||||
<circle
|
||||
className="opacity-25"
|
||||
cx="12"
|
||||
cy="12"
|
||||
r="10"
|
||||
stroke="currentColor"
|
||||
strokeWidth="4"
|
||||
fill="none"
|
||||
/>
|
||||
<path
|
||||
className="opacity-75"
|
||||
fill="currentColor"
|
||||
d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"
|
||||
/>
|
||||
</svg>
|
||||
Analyzing...
|
||||
</span>
|
||||
) : (
|
||||
'Analyze Complexity'
|
||||
)}
|
||||
</button>
|
||||
|
||||
<button
|
||||
className="flex-1 bg-green-600 hover:bg-green-700 text-white font-semibold px-6 py-2 rounded disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
|
||||
onClick={handleSubmit}
|
||||
disabled={disabled || analyzing || !complexity}
|
||||
>
|
||||
Submit Task
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<p className="text-xs text-gray-500 text-center">
|
||||
Analyze your task to see complexity and agent requirements before submission
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,333 @@
|
||||
/**
|
||||
* Git merge manager for aggregating agent results with conflict resolution
|
||||
*/
|
||||
import * as git from 'isomorphic-git';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import http from 'isomorphic-git/http/node';
|
||||
|
||||
export interface MergeConflict {
|
||||
filepath: string;
|
||||
ours: string;
|
||||
theirs: string;
|
||||
base?: string;
|
||||
}
|
||||
|
||||
export interface MergeResult {
|
||||
success: boolean;
|
||||
conflicts: MergeConflict[];
|
||||
mergedBranches: string[];
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export interface ConflictResolution {
|
||||
filepath: string;
|
||||
resolution: 'ours' | 'theirs' | 'manual';
|
||||
manualContent?: string;
|
||||
}
|
||||
|
||||
export class MergeManager {
|
||||
private repoDir: string;
|
||||
|
||||
constructor(repoDir: string) {
|
||||
this.repoDir = repoDir;
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge multiple agent result branches into a target branch
|
||||
*/
|
||||
async mergeAgentResults(
|
||||
targetBranch: string,
|
||||
agentBranches: string[]
|
||||
): Promise<MergeResult> {
|
||||
const result: MergeResult = {
|
||||
success: true,
|
||||
conflicts: [],
|
||||
mergedBranches: [],
|
||||
};
|
||||
|
||||
try {
|
||||
// Checkout target branch
|
||||
await git.checkout({
|
||||
fs,
|
||||
dir: this.repoDir,
|
||||
ref: targetBranch,
|
||||
});
|
||||
|
||||
// Merge each agent branch sequentially
|
||||
for (const agentBranch of agentBranches) {
|
||||
try {
|
||||
await git.merge({
|
||||
fs,
|
||||
dir: this.repoDir,
|
||||
ours: targetBranch,
|
||||
theirs: agentBranch,
|
||||
author: {
|
||||
name: 'HeiCode Swarm',
|
||||
email: 'swarm@heicode.local',
|
||||
},
|
||||
});
|
||||
|
||||
result.mergedBranches.push(agentBranch);
|
||||
} catch (error) {
|
||||
// Merge conflict detected
|
||||
const conflicts = await this.detectConflicts(agentBranch);
|
||||
result.conflicts.push(...conflicts);
|
||||
result.success = false;
|
||||
|
||||
// Abort the merge to allow manual resolution
|
||||
await this.abortMerge();
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
result.success = false;
|
||||
result.error = error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect merge conflicts between current branch and another branch
|
||||
*/
|
||||
async detectConflicts(branchName: string): Promise<MergeConflict[]> {
|
||||
const conflicts: MergeConflict[] = [];
|
||||
|
||||
try {
|
||||
// Get the status matrix to find conflicted files
|
||||
const status = await git.statusMatrix({ fs, dir: this.repoDir });
|
||||
|
||||
for (const [filepath, HEADStatus, workdirStatus, stageStatus] of status) {
|
||||
// Check if file has conflicts (different in HEAD, workdir, and stage)
|
||||
if (HEADStatus !== workdirStatus || workdirStatus !== stageStatus) {
|
||||
const conflict = await this.getConflictContent(filepath, branchName);
|
||||
if (conflict) {
|
||||
conflicts.push(conflict);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error detecting conflicts:', error);
|
||||
}
|
||||
|
||||
return conflicts;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get conflict content for a specific file
|
||||
*/
|
||||
private async getConflictContent(
|
||||
filepath: string,
|
||||
theirBranch: string
|
||||
): Promise<MergeConflict | null> {
|
||||
try {
|
||||
const fullPath = path.join(this.repoDir, filepath);
|
||||
|
||||
// Read current (ours) version
|
||||
const oursContent = fs.existsSync(fullPath)
|
||||
? fs.readFileSync(fullPath, 'utf-8')
|
||||
: '';
|
||||
|
||||
// Read their version
|
||||
await git.checkout({
|
||||
fs,
|
||||
dir: this.repoDir,
|
||||
ref: theirBranch,
|
||||
filepaths: [filepath],
|
||||
force: true,
|
||||
});
|
||||
|
||||
const theirsContent = fs.existsSync(fullPath)
|
||||
? fs.readFileSync(fullPath, 'utf-8')
|
||||
: '';
|
||||
|
||||
// Restore our version
|
||||
await git.checkout({
|
||||
fs,
|
||||
dir: this.repoDir,
|
||||
ref: 'HEAD',
|
||||
filepaths: [filepath],
|
||||
force: true,
|
||||
});
|
||||
|
||||
return {
|
||||
filepath,
|
||||
ours: oursContent,
|
||||
theirs: theirsContent,
|
||||
};
|
||||
} catch (error) {
|
||||
console.error(`Error getting conflict content for ${filepath}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve conflicts with provided resolutions
|
||||
*/
|
||||
async resolveConflicts(resolutions: ConflictResolution[]): Promise<boolean> {
|
||||
try {
|
||||
for (const resolution of resolutions) {
|
||||
const fullPath = path.join(this.repoDir, resolution.filepath);
|
||||
|
||||
if (resolution.resolution === 'manual' && resolution.manualContent) {
|
||||
// Use manual resolution
|
||||
fs.writeFileSync(fullPath, resolution.manualContent, 'utf-8');
|
||||
} else if (resolution.resolution === 'ours') {
|
||||
// Keep our version (already in place, just stage it)
|
||||
await git.add({ fs, dir: this.repoDir, filepath: resolution.filepath });
|
||||
} else if (resolution.resolution === 'theirs') {
|
||||
// Use their version (need to checkout from merge branch)
|
||||
// This is handled during the merge process
|
||||
}
|
||||
|
||||
// Stage the resolved file
|
||||
await git.add({ fs, dir: this.repoDir, filepath: resolution.filepath });
|
||||
}
|
||||
|
||||
// Commit the merge
|
||||
await git.commit({
|
||||
fs,
|
||||
dir: this.repoDir,
|
||||
message: 'Merge agent results with conflict resolution',
|
||||
author: {
|
||||
name: 'HeiCode Swarm',
|
||||
email: 'swarm@heicode.local',
|
||||
},
|
||||
});
|
||||
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error('Error resolving conflicts:', error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Abort current merge
|
||||
*/
|
||||
async abortMerge(): Promise<void> {
|
||||
try {
|
||||
// Reset to HEAD to abort merge
|
||||
const commits = await git.log({ fs, dir: this.repoDir, depth: 1 });
|
||||
if (commits.length > 0) {
|
||||
const headCommit = commits[0].oid;
|
||||
|
||||
// Get all files
|
||||
const status = await git.statusMatrix({ fs, dir: this.repoDir });
|
||||
|
||||
for (const [filepath] of status) {
|
||||
try {
|
||||
// Restore file to HEAD state
|
||||
await git.checkout({
|
||||
fs,
|
||||
dir: this.repoDir,
|
||||
ref: headCommit,
|
||||
filepaths: [filepath],
|
||||
force: true,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(`Error restoring ${filepath}:`, error);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error aborting merge:', error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get diff between two branches
|
||||
*/
|
||||
async getDiff(branch1: string, branch2: string): Promise<string[]> {
|
||||
const changedFiles: string[] = [];
|
||||
|
||||
try {
|
||||
// Get commits for both branches
|
||||
const commits1 = await git.log({ fs, dir: this.repoDir, ref: branch1, depth: 1 });
|
||||
const commits2 = await git.log({ fs, dir: this.repoDir, ref: branch2, depth: 1 });
|
||||
|
||||
if (commits1.length === 0 || commits2.length === 0) {
|
||||
return changedFiles;
|
||||
}
|
||||
|
||||
const oid1 = commits1[0].oid;
|
||||
const oid2 = commits2[0].oid;
|
||||
|
||||
// Compare trees
|
||||
const tree1 = await git.readTree({ fs, dir: this.repoDir, oid: oid1 });
|
||||
const tree2 = await git.readTree({ fs, dir: this.repoDir, oid: oid2 });
|
||||
|
||||
// Find differences
|
||||
const files1 = new Set(tree1.tree.map(entry => entry.path));
|
||||
const files2 = new Set(tree2.tree.map(entry => entry.path));
|
||||
|
||||
// Files in branch1 but not in branch2
|
||||
for (const file of files1) {
|
||||
if (!files2.has(file)) {
|
||||
changedFiles.push(file);
|
||||
}
|
||||
}
|
||||
|
||||
// Files in branch2 but not in branch1
|
||||
for (const file of files2) {
|
||||
if (!files1.has(file)) {
|
||||
changedFiles.push(file);
|
||||
}
|
||||
}
|
||||
|
||||
// Files in both but with different content
|
||||
for (const file of files1) {
|
||||
if (files2.has(file)) {
|
||||
const entry1 = tree1.tree.find(e => e.path === file);
|
||||
const entry2 = tree2.tree.find(e => e.path === file);
|
||||
|
||||
if (entry1 && entry2 && entry1.oid !== entry2.oid) {
|
||||
changedFiles.push(file);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error getting diff:', error);
|
||||
}
|
||||
|
||||
return changedFiles;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a merge summary
|
||||
*/
|
||||
async createMergeSummary(
|
||||
targetBranch: string,
|
||||
agentBranches: string[]
|
||||
): Promise<{
|
||||
totalChanges: number;
|
||||
fileChanges: Map<string, number>;
|
||||
}> {
|
||||
const fileChanges = new Map<string, number>();
|
||||
let totalChanges = 0;
|
||||
|
||||
for (const agentBranch of agentBranches) {
|
||||
const diff = await this.getDiff(targetBranch, agentBranch);
|
||||
totalChanges += diff.length;
|
||||
|
||||
for (const file of diff) {
|
||||
fileChanges.set(file, (fileChanges.get(file) || 0) + 1);
|
||||
}
|
||||
}
|
||||
|
||||
return { totalChanges, fileChanges };
|
||||
}
|
||||
|
||||
/**
|
||||
* List all branches matching a pattern
|
||||
*/
|
||||
async listAgentBranches(workspaceId: string): Promise<string[]> {
|
||||
try {
|
||||
const branches = await git.listBranches({ fs, dir: this.repoDir });
|
||||
return branches.filter(branch => branch.startsWith(`agent-${workspaceId}-`));
|
||||
} catch (error) {
|
||||
console.error('Error listing branches:', error);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,316 @@
|
||||
import * as git from 'isomorphic-git';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import http from 'isomorphic-git/http/node';
|
||||
|
||||
export interface WorkspaceConfig {
|
||||
workspaceId: string;
|
||||
repoPath: string;
|
||||
gitUrl: string;
|
||||
}
|
||||
|
||||
export interface GitOperationResult {
|
||||
success: boolean;
|
||||
error?: string;
|
||||
data?: any;
|
||||
}
|
||||
|
||||
export class WorkspaceManager {
|
||||
private workspacesRoot: string;
|
||||
|
||||
constructor(workspacesRoot?: string) {
|
||||
this.workspacesRoot = workspacesRoot || path.join(os.homedir(), '.heicode-swarm', 'workspaces');
|
||||
this.ensureWorkspacesRoot();
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize a new bare Git repository for a workspace
|
||||
*/
|
||||
public async initWorkspace(workspaceId: string): Promise<GitOperationResult> {
|
||||
try {
|
||||
const repoPath = path.join(this.workspacesRoot, workspaceId);
|
||||
|
||||
// Create workspace directory
|
||||
if (!fs.existsSync(repoPath)) {
|
||||
fs.mkdirSync(repoPath, { recursive: true });
|
||||
}
|
||||
|
||||
// Initialize bare repository
|
||||
await git.init({
|
||||
fs,
|
||||
dir: repoPath,
|
||||
bare: true,
|
||||
defaultBranch: 'main'
|
||||
});
|
||||
|
||||
const gitUrl = `file://${repoPath}`;
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: {
|
||||
workspaceId,
|
||||
repoPath,
|
||||
gitUrl
|
||||
}
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clone a workspace repository to a local directory
|
||||
*/
|
||||
public async cloneWorkspace(
|
||||
gitUrl: string,
|
||||
targetDir: string,
|
||||
branch: string = 'main'
|
||||
): Promise<GitOperationResult> {
|
||||
try {
|
||||
// Ensure target directory exists
|
||||
if (!fs.existsSync(targetDir)) {
|
||||
fs.mkdirSync(targetDir, { recursive: true });
|
||||
}
|
||||
|
||||
await git.clone({
|
||||
fs,
|
||||
http,
|
||||
dir: targetDir,
|
||||
url: gitUrl,
|
||||
ref: branch,
|
||||
singleBranch: true,
|
||||
depth: 1
|
||||
});
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a new branch in a repository
|
||||
*/
|
||||
public async createBranch(
|
||||
repoDir: string,
|
||||
branchName: string,
|
||||
checkout: boolean = true
|
||||
): Promise<GitOperationResult> {
|
||||
try {
|
||||
// Create branch
|
||||
await git.branch({
|
||||
fs,
|
||||
dir: repoDir,
|
||||
ref: branchName
|
||||
});
|
||||
|
||||
// Checkout if requested
|
||||
if (checkout) {
|
||||
await git.checkout({
|
||||
fs,
|
||||
dir: repoDir,
|
||||
ref: branchName
|
||||
});
|
||||
}
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Commit changes to a repository
|
||||
*/
|
||||
public async commit(
|
||||
repoDir: string,
|
||||
message: string,
|
||||
author: { name: string; email: string }
|
||||
): Promise<GitOperationResult> {
|
||||
try {
|
||||
// Stage all changes
|
||||
const status = await git.statusMatrix({ fs, dir: repoDir });
|
||||
|
||||
for (const [filepath, , worktreeStatus] of status) {
|
||||
if (worktreeStatus !== 1) {
|
||||
// File is modified or new
|
||||
await git.add({ fs, dir: repoDir, filepath });
|
||||
} else if (worktreeStatus === 0) {
|
||||
// File is deleted
|
||||
await git.remove({ fs, dir: repoDir, filepath });
|
||||
}
|
||||
}
|
||||
|
||||
// Commit
|
||||
const sha = await git.commit({
|
||||
fs,
|
||||
dir: repoDir,
|
||||
message,
|
||||
author
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { sha }
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Push changes to remote repository
|
||||
*/
|
||||
public async push(
|
||||
repoDir: string,
|
||||
remote: string = 'origin',
|
||||
branch?: string
|
||||
): Promise<GitOperationResult> {
|
||||
try {
|
||||
await git.push({
|
||||
fs,
|
||||
http,
|
||||
dir: repoDir,
|
||||
remote,
|
||||
ref: branch
|
||||
});
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge branches
|
||||
*/
|
||||
public async merge(
|
||||
repoDir: string,
|
||||
sourceBranch: string,
|
||||
targetBranch: string
|
||||
): Promise<GitOperationResult> {
|
||||
try {
|
||||
// Checkout target branch
|
||||
await git.checkout({
|
||||
fs,
|
||||
dir: repoDir,
|
||||
ref: targetBranch
|
||||
});
|
||||
|
||||
// Merge source branch
|
||||
await git.merge({
|
||||
fs,
|
||||
dir: repoDir,
|
||||
ours: targetBranch,
|
||||
theirs: sourceBranch,
|
||||
author: {
|
||||
name: 'HeiCode Swarm',
|
||||
email: 'swarm@heicode.local'
|
||||
}
|
||||
});
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* List all branches in a repository
|
||||
*/
|
||||
public async listBranches(repoDir: string): Promise<GitOperationResult> {
|
||||
try {
|
||||
const branches = await git.listBranches({
|
||||
fs,
|
||||
dir: repoDir
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { branches }
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current branch name
|
||||
*/
|
||||
public async getCurrentBranch(repoDir: string): Promise<GitOperationResult> {
|
||||
try {
|
||||
const branch = await git.currentBranch({
|
||||
fs,
|
||||
dir: repoDir
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { branch }
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a workspace
|
||||
*/
|
||||
public async deleteWorkspace(workspaceId: string): Promise<GitOperationResult> {
|
||||
try {
|
||||
const repoPath = path.join(this.workspacesRoot, workspaceId);
|
||||
|
||||
if (fs.existsSync(repoPath)) {
|
||||
fs.rmSync(repoPath, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get workspace path
|
||||
*/
|
||||
public getWorkspacePath(workspaceId: string): string {
|
||||
return path.join(this.workspacesRoot, workspaceId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure workspaces root directory exists
|
||||
*/
|
||||
private ensureWorkspacesRoot(): void {
|
||||
if (!fs.existsSync(this.workspacesRoot)) {
|
||||
fs.mkdirSync(this.workspacesRoot, { recursive: true });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>HeiCode Swarm</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
|
||||
'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
|
||||
sans-serif;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script src="../dist/App.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,316 @@
|
||||
/**
|
||||
* K8s auto-scaler for creating and deleting agent pods based on complexity analysis
|
||||
*/
|
||||
import * as k8s from '@kubernetes/client-node';
|
||||
import { ComplexityAnalysis } from '../ai/complexity-analyzer';
|
||||
|
||||
export interface AgentPodConfig {
|
||||
namespace: string;
|
||||
image: string;
|
||||
orchestratorUrl: string;
|
||||
gitUrl: string;
|
||||
workspaceId: string;
|
||||
agentId: string;
|
||||
cpuRequest?: string;
|
||||
memoryRequest?: string;
|
||||
cpuLimit?: string;
|
||||
memoryLimit?: string;
|
||||
}
|
||||
|
||||
export interface ScalingResult {
|
||||
success: boolean;
|
||||
createdPods: string[];
|
||||
deletedPods: string[];
|
||||
errors: string[];
|
||||
}
|
||||
|
||||
export class AutoScaler {
|
||||
private kc: k8s.KubeConfig;
|
||||
private coreApi: k8s.CoreV1Api;
|
||||
private namespace: string;
|
||||
|
||||
constructor(kubeConfig: k8s.KubeConfig, namespace: string = 'default') {
|
||||
this.kc = kubeConfig;
|
||||
this.coreApi = this.kc.makeApiClient(k8s.CoreV1Api);
|
||||
this.namespace = namespace;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scale up agents based on complexity analysis
|
||||
*/
|
||||
async scaleUp(
|
||||
complexity: ComplexityAnalysis,
|
||||
baseConfig: Omit<AgentPodConfig, 'agentId'>
|
||||
): Promise<ScalingResult> {
|
||||
const result: ScalingResult = {
|
||||
success: true,
|
||||
createdPods: [],
|
||||
deletedPods: [],
|
||||
errors: [],
|
||||
};
|
||||
|
||||
const agentCount = complexity.agentCount;
|
||||
|
||||
for (let i = 0; i < agentCount; i++) {
|
||||
const agentId = `agent-${baseConfig.workspaceId}-${i}-${Date.now()}`;
|
||||
const podConfig: AgentPodConfig = {
|
||||
...baseConfig,
|
||||
agentId,
|
||||
};
|
||||
|
||||
try {
|
||||
const podName = await this.createAgentPod(podConfig);
|
||||
result.createdPods.push(podName);
|
||||
} catch (error) {
|
||||
const errorMsg = `Failed to create agent ${agentId}: ${error instanceof Error ? error.message : String(error)}`;
|
||||
result.errors.push(errorMsg);
|
||||
result.success = false;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a single agent pod
|
||||
*/
|
||||
async createAgentPod(config: AgentPodConfig): Promise<string> {
|
||||
const podName = config.agentId;
|
||||
|
||||
const podManifest: k8s.V1Pod = {
|
||||
apiVersion: 'v1',
|
||||
kind: 'Pod',
|
||||
metadata: {
|
||||
name: podName,
|
||||
namespace: config.namespace,
|
||||
labels: {
|
||||
app: 'heicode-swarm-agent',
|
||||
'workspace-id': config.workspaceId,
|
||||
'agent-id': config.agentId,
|
||||
},
|
||||
},
|
||||
spec: {
|
||||
serviceAccountName: 'swarm-agent',
|
||||
restartPolicy: 'Never',
|
||||
containers: [
|
||||
{
|
||||
name: 'agent',
|
||||
image: config.image,
|
||||
env: [
|
||||
{
|
||||
name: 'ORCHESTRATOR_URL',
|
||||
value: config.orchestratorUrl,
|
||||
},
|
||||
{
|
||||
name: 'AGENT_ID',
|
||||
value: config.agentId,
|
||||
},
|
||||
{
|
||||
name: 'GIT_URL',
|
||||
value: config.gitUrl,
|
||||
},
|
||||
{
|
||||
name: 'WORKSPACE_ID',
|
||||
value: config.workspaceId,
|
||||
},
|
||||
{
|
||||
name: 'ANTHROPIC_API_KEY',
|
||||
valueFrom: {
|
||||
secretKeyRef: {
|
||||
name: 'anthropic-api-key',
|
||||
key: 'api-key',
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
resources: {
|
||||
requests: {
|
||||
cpu: config.cpuRequest || '500m',
|
||||
memory: config.memoryRequest || '512Mi',
|
||||
},
|
||||
limits: {
|
||||
cpu: config.cpuLimit || '1000m',
|
||||
memory: config.memoryLimit || '1Gi',
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
try {
|
||||
await this.coreApi.createNamespacedPod(config.namespace, podManifest);
|
||||
return podName;
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to create pod ${podName}: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete agent pods by workspace ID
|
||||
*/
|
||||
async deleteAgentPods(workspaceId: string): Promise<ScalingResult> {
|
||||
const result: ScalingResult = {
|
||||
success: true,
|
||||
createdPods: [],
|
||||
deletedPods: [],
|
||||
errors: [],
|
||||
};
|
||||
|
||||
try {
|
||||
// List pods with workspace-id label
|
||||
const response = await this.coreApi.listNamespacedPod(
|
||||
this.namespace,
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
`workspace-id=${workspaceId}`
|
||||
);
|
||||
|
||||
const pods = response.body.items;
|
||||
|
||||
for (const pod of pods) {
|
||||
const podName = pod.metadata?.name;
|
||||
if (!podName) continue;
|
||||
|
||||
try {
|
||||
await this.coreApi.deleteNamespacedPod(podName, this.namespace);
|
||||
result.deletedPods.push(podName);
|
||||
} catch (error) {
|
||||
const errorMsg = `Failed to delete pod ${podName}: ${error instanceof Error ? error.message : String(error)}`;
|
||||
result.errors.push(errorMsg);
|
||||
result.success = false;
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
result.success = false;
|
||||
result.errors.push(`Failed to list pods: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a specific agent pod
|
||||
*/
|
||||
async deleteAgentPod(agentId: string): Promise<void> {
|
||||
try {
|
||||
await this.coreApi.deleteNamespacedPod(agentId, this.namespace);
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to delete pod ${agentId}: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* List all agent pods for a workspace
|
||||
*/
|
||||
async listAgentPods(workspaceId: string): Promise<k8s.V1Pod[]> {
|
||||
try {
|
||||
const response = await this.coreApi.listNamespacedPod(
|
||||
this.namespace,
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
undefined,
|
||||
`workspace-id=${workspaceId}`
|
||||
);
|
||||
|
||||
return response.body.items;
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to list pods: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get pod status
|
||||
*/
|
||||
async getPodStatus(podName: string): Promise<string | undefined> {
|
||||
try {
|
||||
const response = await this.coreApi.readNamespacedPod(podName, this.namespace);
|
||||
return response.body.status?.phase;
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to get pod status: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for pods to be ready
|
||||
*/
|
||||
async waitForPodsReady(
|
||||
workspaceId: string,
|
||||
timeoutMs: number = 120000
|
||||
): Promise<{ ready: boolean; pods: string[] }> {
|
||||
const startTime = Date.now();
|
||||
|
||||
while (Date.now() - startTime < timeoutMs) {
|
||||
const pods = await this.listAgentPods(workspaceId);
|
||||
const allReady = pods.every(pod => {
|
||||
const phase = pod.status?.phase;
|
||||
return phase === 'Running' || phase === 'Succeeded';
|
||||
});
|
||||
|
||||
if (allReady) {
|
||||
return {
|
||||
ready: true,
|
||||
pods: pods.map(p => p.metadata?.name || '').filter(Boolean),
|
||||
};
|
||||
}
|
||||
|
||||
// Wait 2 seconds before checking again
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
}
|
||||
|
||||
return {
|
||||
ready: false,
|
||||
pods: [],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Scale down idle agents
|
||||
*/
|
||||
async scaleDownIdle(workspaceId: string, idleThresholdMs: number = 300000): Promise<ScalingResult> {
|
||||
const result: ScalingResult = {
|
||||
success: true,
|
||||
createdPods: [],
|
||||
deletedPods: [],
|
||||
errors: [],
|
||||
};
|
||||
|
||||
try {
|
||||
const pods = await this.listAgentPods(workspaceId);
|
||||
const now = Date.now();
|
||||
|
||||
for (const pod of pods) {
|
||||
const podName = pod.metadata?.name;
|
||||
if (!podName) continue;
|
||||
|
||||
// Check if pod has been idle (check creation time as proxy)
|
||||
const creationTime = pod.metadata?.creationTimestamp;
|
||||
if (!creationTime) continue;
|
||||
|
||||
const createdAt = new Date(creationTime).getTime();
|
||||
const idleTime = now - createdAt;
|
||||
|
||||
// If pod is old enough and in Running state, consider it idle
|
||||
// In production, you'd check actual agent activity via orchestrator
|
||||
if (idleTime > idleThresholdMs && pod.status?.phase === 'Running') {
|
||||
try {
|
||||
await this.deleteAgentPod(podName);
|
||||
result.deletedPods.push(podName);
|
||||
} catch (error) {
|
||||
const errorMsg = `Failed to delete idle pod ${podName}: ${error instanceof Error ? error.message : String(error)}`;
|
||||
result.errors.push(errorMsg);
|
||||
result.success = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
result.success = false;
|
||||
result.errors.push(`Failed to scale down: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
import * as k8s from '@kubernetes/client-node';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
|
||||
export interface KubeConfigValidationResult {
|
||||
valid: boolean;
|
||||
error?: string;
|
||||
context?: string;
|
||||
cluster?: string;
|
||||
}
|
||||
|
||||
export class KubeConfigLoader {
|
||||
private kc: k8s.KubeConfig;
|
||||
|
||||
constructor() {
|
||||
this.kc = new k8s.KubeConfig();
|
||||
}
|
||||
|
||||
/**
|
||||
* Load kubeconfig from default locations or KUBECONFIG env var
|
||||
* Priority: KUBECONFIG env var > ~/.kube/config
|
||||
*/
|
||||
public load(): void {
|
||||
const kubeconfigPath = this.getKubeConfigPath();
|
||||
|
||||
if (!kubeconfigPath) {
|
||||
throw new Error('No kubeconfig found. Set KUBECONFIG env var or create ~/.kube/config');
|
||||
}
|
||||
|
||||
if (!fs.existsSync(kubeconfigPath)) {
|
||||
throw new Error(`Kubeconfig file not found at: ${kubeconfigPath}`);
|
||||
}
|
||||
|
||||
try {
|
||||
this.kc.loadFromFile(kubeconfigPath);
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to load kubeconfig: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the loaded kubeconfig
|
||||
*/
|
||||
public validate(): KubeConfigValidationResult {
|
||||
try {
|
||||
const currentContext = this.kc.getCurrentContext();
|
||||
if (!currentContext) {
|
||||
return {
|
||||
valid: false,
|
||||
error: 'No current context set in kubeconfig'
|
||||
};
|
||||
}
|
||||
|
||||
const cluster = this.kc.getCurrentCluster();
|
||||
if (!cluster) {
|
||||
return {
|
||||
valid: false,
|
||||
error: 'No cluster found for current context'
|
||||
};
|
||||
}
|
||||
|
||||
const user = this.kc.getCurrentUser();
|
||||
if (!user) {
|
||||
return {
|
||||
valid: false,
|
||||
error: 'No user found for current context'
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
valid: true,
|
||||
context: currentContext,
|
||||
cluster: cluster.name
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
valid: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the KubeConfig instance for use with K8s clients
|
||||
*/
|
||||
public getKubeConfig(): k8s.KubeConfig {
|
||||
return this.kc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the current context name
|
||||
*/
|
||||
public getCurrentContext(): string | null {
|
||||
return this.kc.getCurrentContext();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the current cluster name
|
||||
*/
|
||||
public getCurrentCluster(): string | null {
|
||||
const cluster = this.kc.getCurrentCluster();
|
||||
return cluster ? cluster.name : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Test connection to the K8s cluster
|
||||
*/
|
||||
public async testConnection(): Promise<{ success: boolean; error?: string }> {
|
||||
try {
|
||||
const coreApi = this.kc.makeApiClient(k8s.CoreV1Api);
|
||||
await coreApi.listNamespace();
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get kubeconfig path from env var or default location
|
||||
*/
|
||||
private getKubeConfigPath(): string | null {
|
||||
// Check KUBECONFIG env var first
|
||||
const kubeconfigEnv = process.env.KUBECONFIG;
|
||||
if (kubeconfigEnv) {
|
||||
// KUBECONFIG can contain multiple paths separated by colons
|
||||
const paths = kubeconfigEnv.split(path.delimiter);
|
||||
for (const p of paths) {
|
||||
if (fs.existsSync(p)) {
|
||||
return p;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Fall back to default location
|
||||
const defaultPath = path.join(os.homedir(), '.kube', 'config');
|
||||
if (fs.existsSync(defaultPath)) {
|
||||
return defaultPath;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convenience function to load and validate kubeconfig
|
||||
*/
|
||||
export async function loadAndValidateKubeConfig(): Promise<{
|
||||
loader: KubeConfigLoader;
|
||||
validation: KubeConfigValidationResult;
|
||||
}> {
|
||||
const loader = new KubeConfigLoader();
|
||||
|
||||
try {
|
||||
loader.load();
|
||||
} catch (error) {
|
||||
return {
|
||||
loader,
|
||||
validation: {
|
||||
valid: false,
|
||||
error: error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
const validation = loader.validate();
|
||||
return { loader, validation };
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
import { app, BrowserWindow } from 'electron';
|
||||
import * as path from 'path';
|
||||
import { KubeConfigLoader } from './k8s/kubeconfig-loader';
|
||||
|
||||
let mainWindow: BrowserWindow | null = null;
|
||||
|
||||
function createWindow(): void {
|
||||
mainWindow = new BrowserWindow({
|
||||
width: 1200,
|
||||
height: 800,
|
||||
webPreferences: {
|
||||
nodeIntegration: true,
|
||||
contextIsolation: false,
|
||||
preload: path.join(__dirname, 'preload.js')
|
||||
},
|
||||
title: 'HeiCode Swarm',
|
||||
backgroundColor: '#1e1e1e'
|
||||
});
|
||||
|
||||
// In development, load from webpack dev server
|
||||
// In production, load from built files
|
||||
if (process.env.NODE_ENV === 'development') {
|
||||
mainWindow.loadURL('http://localhost:3000');
|
||||
mainWindow.webContents.openDevTools();
|
||||
} else {
|
||||
mainWindow.loadFile(path.join(__dirname, '../renderer/index.html'));
|
||||
}
|
||||
|
||||
mainWindow.on('closed', () => {
|
||||
mainWindow = null;
|
||||
});
|
||||
}
|
||||
|
||||
async function initializeApp(): Promise<void> {
|
||||
try {
|
||||
// Load and validate kubeconfig on startup
|
||||
const kubeLoader = new KubeConfigLoader();
|
||||
kubeLoader.load();
|
||||
|
||||
const validation = kubeLoader.validate();
|
||||
if (!validation.valid) {
|
||||
console.error('Kubeconfig validation failed:', validation.error);
|
||||
// Show error dialog to user
|
||||
const { dialog } = require('electron');
|
||||
await dialog.showErrorBox(
|
||||
'Kubernetes Configuration Error',
|
||||
`Failed to load kubeconfig: ${validation.error}\n\nPlease ensure you have a valid kubeconfig at ~/.kube/config or set the KUBECONFIG environment variable.`
|
||||
);
|
||||
app.quit();
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('Kubeconfig loaded successfully');
|
||||
console.log('Current context:', validation.context);
|
||||
console.log('Current cluster:', validation.cluster);
|
||||
|
||||
// Test connection
|
||||
const connectionTest = await kubeLoader.testConnection();
|
||||
if (!connectionTest.success) {
|
||||
console.warn('Failed to connect to Kubernetes cluster:', connectionTest.error);
|
||||
// Don't quit, but warn the user
|
||||
const { dialog } = require('electron');
|
||||
await dialog.showMessageBox({
|
||||
type: 'warning',
|
||||
title: 'Kubernetes Connection Warning',
|
||||
message: 'Could not connect to Kubernetes cluster',
|
||||
detail: `Error: ${connectionTest.error}\n\nThe application will start, but you may not be able to create agent pods.`
|
||||
});
|
||||
} else {
|
||||
console.log('Successfully connected to Kubernetes cluster');
|
||||
}
|
||||
|
||||
createWindow();
|
||||
} catch (error) {
|
||||
console.error('Failed to initialize application:', error);
|
||||
const { dialog } = require('electron');
|
||||
await dialog.showErrorBox(
|
||||
'Initialization Error',
|
||||
`Failed to start application: ${error instanceof Error ? error.message : String(error)}`
|
||||
);
|
||||
app.quit();
|
||||
}
|
||||
}
|
||||
|
||||
// App lifecycle events
|
||||
app.on('ready', initializeApp);
|
||||
|
||||
app.on('window-all-closed', () => {
|
||||
// On macOS, keep app running until user explicitly quits
|
||||
if (process.platform !== 'darwin') {
|
||||
app.quit();
|
||||
}
|
||||
});
|
||||
|
||||
app.on('activate', () => {
|
||||
// On macOS, recreate window when dock icon is clicked
|
||||
if (mainWindow === null) {
|
||||
createWindow();
|
||||
}
|
||||
});
|
||||
|
||||
// Handle uncaught exceptions
|
||||
process.on('uncaughtException', (error) => {
|
||||
console.error('Uncaught exception:', error);
|
||||
});
|
||||
|
||||
process.on('unhandledRejection', (reason, promise) => {
|
||||
console.error('Unhandled rejection at:', promise, 'reason:', reason);
|
||||
});
|
||||
@@ -0,0 +1,10 @@
|
||||
module.exports = {
|
||||
content: [
|
||||
"./src/**/*.{js,jsx,ts,tsx}",
|
||||
"./src/index.html"
|
||||
],
|
||||
theme: {
|
||||
extend: {},
|
||||
},
|
||||
plugins: [],
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2020",
|
||||
"module": "commonjs",
|
||||
"lib": ["ES2020", "DOM"],
|
||||
"jsx": "react",
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"resolveJsonModule": true,
|
||||
"moduleResolution": "node",
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true
|
||||
},
|
||||
"include": ["src/**/*"],
|
||||
"exclude": ["node_modules", "dist", "release"]
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
# 交付说明(Delivery Guide)
|
||||
|
||||
本文件说明 **Agent Swarm(agent_swarm_v5)** 的交付物、构建/部署方式、配置项、验收方法,以及本次交付对各链路的影响与合规要求。系统总体介绍见根目录 [README.md](../README.md)。
|
||||
|
||||
---
|
||||
|
||||
## 一、交付物清单
|
||||
|
||||
| 类别 | 内容 | 位置 |
|
||||
|---|---|---|
|
||||
| 编排器 | FastAPI 编排服务(分解/派发/协作/评审/汇总、Manager 对接) | `orchestrator/` |
|
||||
| 执行单元 | Agent 运行时(WebSocket、并发、OpenAI 执行、Git 提交) | `agent/` |
|
||||
| 桌面客户端 | Electron + React 客户端(提交/分析/看板/聚合) | `desktop-client/` |
|
||||
| 容器镜像定义 | Agent 与编排器的 Dockerfile | `Dockerfile.agent`、`Dockerfile.orchestrator` |
|
||||
| 部署清单 | Redis、编排器、Agent、RBAC、监控等 K8s 清单 | `k8s/` |
|
||||
| 部署脚本 | 一键部署脚本 | `scripts/deploy-orchestrator.sh`、`scripts/deploy-agent.sh` |
|
||||
| 测试 | 契约校验与工作流冒烟测试 | `scripts/test-runtime-contract.py`、`scripts/test-merge-smoke.py` |
|
||||
| 数据集 | 复杂度分析标注数据集 | `test-data/complexity-dataset.json` |
|
||||
| 文档 | 各组件 README 与本交付说明 | `README.md`、各目录 `README.md`、`doc/` |
|
||||
|
||||
> 交付不包含任何密钥、令牌、`.env` 或证书。凭据由部署环境 / `secret_ref` / Key Vault 注入。
|
||||
|
||||
---
|
||||
|
||||
## 二、构建
|
||||
|
||||
### 容器镜像
|
||||
```bash
|
||||
# 编排器镜像
|
||||
docker build -f Dockerfile.orchestrator -t agent-swarm-orchestrator:<tag> .
|
||||
|
||||
# Agent 镜像
|
||||
docker build -f Dockerfile.agent -t agent-swarm-agent:<tag> .
|
||||
```
|
||||
|
||||
### 桌面客户端
|
||||
```bash
|
||||
cd desktop-client
|
||||
npm install
|
||||
npm run build
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、部署(Kubernetes)
|
||||
|
||||
按以下顺序部署(具体清单见 `k8s/`):
|
||||
|
||||
```bash
|
||||
# 1) 命名空间与 RBAC
|
||||
kubectl apply -f k8s/test-namespace.yaml
|
||||
kubectl apply -f k8s/rbac/
|
||||
|
||||
# 2) Redis(权威状态存储)
|
||||
kubectl apply -f k8s/redis-statefulset.yaml
|
||||
|
||||
# 3) 编排器
|
||||
kubectl apply -f k8s/orchestrator-deployment.yaml
|
||||
|
||||
# 4) Agent(按需扩缩)
|
||||
kubectl apply -f k8s/agent-deployment-v2.yaml
|
||||
```
|
||||
|
||||
也可使用脚本:`scripts/deploy-orchestrator.sh`、`scripts/deploy-agent.sh`。
|
||||
|
||||
**凭据(必须经 secret_ref / K8s Secret 注入,禁止明文)**:
|
||||
- 模型访问:`OPENAI_API_KEY`(OpenAI 兼容;如使用自定义端点另配 `OPENAI_API_BASE`)。
|
||||
- Git 推送:`GIT_USERNAME` / `GIT_PASSWORD`(或 `GIT_TOKEN`)。
|
||||
- 服务间鉴权与回调签名:`AGENT_RUNTIME_SERVICE_TOKEN`、`AGENT_CALLBACK_SERVICE_TOKEN`、`AGENT_CALLBACK_SIGNING_SECRET`。
|
||||
|
||||
> 注意:模型后端为 **OpenAI 兼容**。部署时请确保以 `OPENAI_*` 凭据注入 Agent 与编排器,并据此核对/更新 Secret 与清单中的环境变量。
|
||||
|
||||
---
|
||||
|
||||
## 四、配置(环境变量与开关)
|
||||
|
||||
| 变量 | 作用 | 备注 |
|
||||
|---|---|---|
|
||||
| `REDIS_HOST` / `REDIS_PORT` / `REDIS_DB` | Redis 连接 | 生产必需 |
|
||||
| `REDIS_FAKE` / `ALLOW_MEMORY_STORE` | 内存回退 | **仅开发/CI**,生产禁用 |
|
||||
| `ENABLE_PLANNER_FALLBACK` | 无 Manager 分工时启用规划回退 | 默认关闭 |
|
||||
| `ENABLE_REVIEW_LOOP` / `MAX_REVIEW_CYCLES` | 主控评审/重做循环 + 汇总 | 默认关闭 / 默认 2 |
|
||||
| `ENABLE_SUBTASK_HANDOFF` | 动态子任务移交 | 默认关闭 |
|
||||
| `OPENAI_API_KEY` / `OPENAI_API_BASE` / `OPENAI_MODEL` | 模型访问 | Agent 与编排器规划/评审共用 |
|
||||
| `AGENT_RUNTIME_SERVICE_TOKEN` 等 | 鉴权与回调签名 | 与 Manager 契约相关 |
|
||||
| `MAX_CONCURRENT_TASKS` / `TASK_TIMEOUT_SECONDS` | Agent 并发与超时 | Agent 侧 |
|
||||
| `OTEL_EXPORTER_OTLP_ENDPOINT` 等 | 链路追踪 | 可选 |
|
||||
|
||||
各组件完整变量见对应 README:[orchestrator](../orchestrator/README.md)、[agent](../agent/README.md)。
|
||||
|
||||
---
|
||||
|
||||
## 五、验收与验证
|
||||
|
||||
### 1) 自动化测试(无需 Redis 服务器,也无需模型密钥)
|
||||
```bash
|
||||
# 在 agent_swarm_v5 目录下
|
||||
set "REDIS_FAKE=1"
|
||||
python scripts/test-runtime-contract.py # Manager 契约校验
|
||||
python scripts/test-merge-smoke.py # 机制级冒烟测试(评审 / 协作 / 汇总等,32 项)
|
||||
```
|
||||
验收标准:两者均通过(contract checks passed / all merge smoke checks passed)。
|
||||
|
||||
### 2) 端到端工作流测试(自动、确定性、无需密钥)
|
||||
```bash
|
||||
python scripts/test-workflow-e2e.py
|
||||
```
|
||||
该脚本在进程内启动**真实编排器**(uvicorn + 内存存储),通过真实 WebSocket 接入一个**无需密钥的桩 Agent**(`scripts/stub_agent.py`),提交一个目标,并逐项断言完整工作流:
|
||||
|
||||
| 断言 | 对应工作流步骤 |
|
||||
|---|---|
|
||||
| 生成 3 个专家任务且角色为 实现/测试/文档 | 主控**分解**为多专家任务 |
|
||||
| 每个任务均完成 | 子 Agent**执行** |
|
||||
| 至少发生一次主控评审循环 | 主控**评审并退回重做** |
|
||||
| 运行最终 completed | **循环直至达标** |
|
||||
| 存在统一的最终汇总 | 专家产出**汇总**为最终回答 |
|
||||
|
||||
确定性来源:规划/评审/汇总被强制离线(静态分解 + 启发式评审 + 拼接汇总),桩 Agent 在首次「测试」任务返回冲突框架(unittest)、重做时返回对齐框架(pytest),从而稳定地触发**一次**评审重做后通过。
|
||||
|
||||
验收标准:输出 `workflow follows the expected sequence: ...`,退出码 0。
|
||||
|
||||
### 3) 端到端工作流演示(真实 Agent + 模型)
|
||||
启动编排器(开启 `REDIS_FAKE`、`ENABLE_PLANNER_FALLBACK`、`ENABLE_REVIEW_LOOP`)与至少一个真实 Agent(`.env` 提供 `OPENAI_API_KEY`),提交需求,按 `GET /api/swarms/{id}/tasks | /logs | /workflow` 确认:分解 → 派发执行 →(评审/重做)→ 汇总交付。详细步骤见根目录 [README.md](../README.md) 的「快速开始」。
|
||||
|
||||
> 辅助工具:`scripts/stub_agent.py` 是无需密钥的桩 Agent,可单独运行以对手动启动的编排器做 Tier-3 联调(设置 `ORCHESTRATOR_URL` / `AGENT_ID` / `AGENT_CAPABILITIES` 后 `python scripts/stub_agent.py`)。
|
||||
|
||||
---
|
||||
|
||||
## 六、交付影响说明
|
||||
|
||||
依据组织规范,本次交付影响范围如下(PR 中需复述):
|
||||
|
||||
| 链路 | 是否影响 | 说明 |
|
||||
|---|---|---|
|
||||
| Client(桌面客户端) | 是 | 提交/分析/看板/聚合 |
|
||||
| Manager | 否(契约保持) | 面向 Manager 的接口、回调、审批链均保留 |
|
||||
| Swarm(编排器 + Agent) | 是 | 新增分解/协作/评审/汇总能力,均默认关闭 |
|
||||
| Agent | 是 | OpenAI 执行、并发、重连、超时、取消、协作 |
|
||||
| CodeGW | 否 | 无改动 |
|
||||
| 计费 | 是(字段保留) | 模型用量归属(`usage` 与归属头)保留;模型后端改为 OpenAI 兼容 |
|
||||
| 密钥 | 是 | 需以 `secret_ref` 注入 `OPENAI_*` 等凭据 |
|
||||
| 审计 | 否(契约保持) | 事件与签名回调结构保留 |
|
||||
| 发布链路 | 否 | 不涉及非 dev release 仓库的业务改动 |
|
||||
|
||||
---
|
||||
|
||||
## 七、安全与合规
|
||||
|
||||
- **禁止**将密钥、令牌、云凭据、`.env`、证书写入代码、日志、Markdown 或提交记录。
|
||||
- 凭据仅经部署环境 / `secret_ref` / Key Vault 注入;`.env` 已被 `.gitignore` 忽略,仅供本地开发。
|
||||
- 涉及鉴权、审批链、计费、审计的改动须遵循 Manager 安全规则与组织标准。
|
||||
- 生产环境必须使用真实 Redis;严禁开启内存回退(`REDIS_FAKE` / `ALLOW_MEMORY_STORE`)。
|
||||
|
||||
---
|
||||
|
||||
## 八、回退与运维
|
||||
|
||||
- **回退**:所有新增工作流能力(规划回退、评审循环、子任务移交)均由环境开关控制,关闭后行为回到基础的「Manager 编排 + 单任务执行」语义。
|
||||
- **可观测性**:编排器与 Agent 暴露 Prometheus 指标(编排器 `GET /metrics`),可接入 `k8s/prometheus-config.yaml` 与 `k8s/grafana-dashboard.json`。
|
||||
- **健康检查**:`GET /` 与 `GET /health`(含 Redis 连接状态与活跃连接数)。
|
||||
|
||||
---
|
||||
|
||||
## 九、已知事项与遗留项
|
||||
|
||||
以下为已知的陈旧/遗留资产,PR 中应一并说明,并按需清理:
|
||||
|
||||
1. **`k8s/orchestrator-source-configmap.yaml` 为陈旧快照(请勿用于当前部署)。**
|
||||
该 ConfigMap 内嵌的是旧版编排器源码(无规划/评审/汇总)。正式部署走**镜像**方式(`Dockerfile.orchestrator` → `k8s/orchestrator-deployment.yaml`,`swarm-orchestrator:latest`)。如确需源码挂载部署,请从 `orchestrator/` 重新生成:
|
||||
```bash
|
||||
kubectl create configmap orchestrator-source --from-file=orchestrator/ --dry-run=client -o yaml
|
||||
```
|
||||
否则建议删除该文件。文件顶部已加显著警告。
|
||||
|
||||
2. **`scripts/test-agent.py` 为遗留测试脚本。**
|
||||
它针对旧版 Anthropic 初始化流程(使用占位密钥 `sk-ant-test-...`),与当前 OpenAI 运行时不一致,且**不属于**当前测试套件(`scripts/test-runtime-contract.py`、`scripts/test-merge-smoke.py`、`scripts/test-workflow-e2e.py`)。后续应更新为 OpenAI 流程或移除。
|
||||
@@ -0,0 +1,94 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: agent-runtime-config
|
||||
namespace: swarm-system
|
||||
data:
|
||||
ORCHESTRATOR_URL: "ws://orchestrator-service:8000"
|
||||
WORKSPACE_DIR: "/workspace"
|
||||
GIT_REPO_URL: "git://git-test-server.swarm-system.svc.cluster.local/swarm-test.git"
|
||||
GIT_BASE_BRANCH: "main"
|
||||
AGENT_CAPABILITIES: "code_generation,git,python,filesystem,file-operations,editing,validation,verification,shell,reporting,repository-inspection"
|
||||
ENABLE_SUBTASK_HANDOFF: "false"
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-full
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: agent-full
|
||||
component: worker
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: agent-full
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: agent-full
|
||||
component: worker
|
||||
spec:
|
||||
serviceAccountName: orchestrator-sa
|
||||
containers:
|
||||
- name: agent
|
||||
image: python:3.11-slim
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- |
|
||||
set -euxo pipefail
|
||||
apt-get update
|
||||
apt-get install -y git
|
||||
pip install --no-cache-dir -r /app/agent/requirements.txt
|
||||
cd /app
|
||||
python -m agent.main
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: agent-runtime-config
|
||||
env:
|
||||
- name: AGENT_ID
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: OPENAI_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: api-key
|
||||
- name: OPENAI_API_BASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: api-base
|
||||
optional: true
|
||||
- name: OPENAI_MODEL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: model
|
||||
optional: true
|
||||
- name: PYTHONUNBUFFERED
|
||||
value: "1"
|
||||
- name: DEBIAN_FRONTEND
|
||||
value: noninteractive
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: 1000m
|
||||
memory: 1Gi
|
||||
volumeMounts:
|
||||
- name: agent-source
|
||||
mountPath: /app/agent
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
volumes:
|
||||
- name: agent-source
|
||||
configMap:
|
||||
name: agent-source
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
restartPolicy: Always
|
||||
@@ -0,0 +1,241 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: agent
|
||||
component: worker
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: agent
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: agent
|
||||
component: worker
|
||||
spec:
|
||||
serviceAccountName: orchestrator-sa
|
||||
containers:
|
||||
- name: agent
|
||||
image: python:3.11-slim
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
echo "Installing dependencies..."
|
||||
pip install --no-cache-dir \
|
||||
websockets \
|
||||
prometheus-client \
|
||||
gitpython \
|
||||
openai
|
||||
|
||||
echo "Setting up agent code..."
|
||||
mkdir -p /tmp/agent
|
||||
cd /tmp/agent
|
||||
|
||||
echo "Starting agent..."
|
||||
python3 -c "
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
from typing import Optional
|
||||
import websockets
|
||||
from websockets.exceptions import ConnectionClosed
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class SimpleAgent:
|
||||
def __init__(self, orchestrator_url: str, agent_id: str):
|
||||
self.orchestrator_url = orchestrator_url
|
||||
self.agent_id = agent_id
|
||||
self.api_key = os.getenv('OPENAI_API_KEY')
|
||||
self.api_base = os.getenv('OPENAI_API_BASE', 'https://api.openai.com/v1')
|
||||
self.model = os.getenv('OPENAI_MODEL', 'gpt-4o-mini')
|
||||
self.ws = None
|
||||
self.running = True
|
||||
|
||||
async def connect(self):
|
||||
logger.info(f'Connecting to orchestrator at {self.orchestrator_url}')
|
||||
self.ws = await websockets.connect(self.orchestrator_url)
|
||||
|
||||
# Register with orchestrator
|
||||
register_msg = {
|
||||
'type': 'register',
|
||||
'agent_id': self.agent_id,
|
||||
'capabilities': ['code_generation', 'code_review'],
|
||||
'status': 'idle'
|
||||
}
|
||||
await self.ws.send(json.dumps(register_msg))
|
||||
logger.info(f'Agent {self.agent_id} registered')
|
||||
|
||||
async def send_heartbeat(self):
|
||||
while self.running:
|
||||
try:
|
||||
if self.ws:
|
||||
heartbeat_msg = {
|
||||
'type': 'heartbeat',
|
||||
'agent_id': self.agent_id,
|
||||
'status': 'idle',
|
||||
'timestamp': time.time()
|
||||
}
|
||||
await self.ws.send(json.dumps(heartbeat_msg))
|
||||
logger.debug(f'Heartbeat sent')
|
||||
except Exception as e:
|
||||
logger.error(f'Heartbeat error: {e}')
|
||||
await asyncio.sleep(15)
|
||||
|
||||
async def execute_task(self, task_data):
|
||||
task_id = task_data.get('task_id')
|
||||
description = task_data.get('description')
|
||||
|
||||
logger.info(f'Executing task {task_id}: {description}')
|
||||
|
||||
try:
|
||||
# Use OpenAI-compatible API
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
api_key=self.api_key,
|
||||
base_url=self.api_base
|
||||
)
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=[
|
||||
{'role': 'system', 'content': 'You are a helpful coding assistant. Generate clean, working code.'},
|
||||
{'role': 'user', 'content': description}
|
||||
],
|
||||
max_tokens=2000
|
||||
)
|
||||
|
||||
result = response.choices[0].message.content
|
||||
|
||||
# Send result back
|
||||
result_msg = {
|
||||
'type': 'task_result',
|
||||
'task_id': task_id,
|
||||
'agent_id': self.agent_id,
|
||||
'status': 'completed',
|
||||
'result': result,
|
||||
'timestamp': time.time()
|
||||
}
|
||||
await self.ws.send(json.dumps(result_msg))
|
||||
logger.info(f'Task {task_id} completed')
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f'Task execution failed: {e}')
|
||||
error_msg = {
|
||||
'type': 'task_result',
|
||||
'task_id': task_id,
|
||||
'agent_id': self.agent_id,
|
||||
'status': 'failed',
|
||||
'error': str(e),
|
||||
'timestamp': time.time()
|
||||
}
|
||||
await self.ws.send(json.dumps(error_msg))
|
||||
|
||||
async def listen(self):
|
||||
while self.running:
|
||||
try:
|
||||
message = await self.ws.recv()
|
||||
data = json.loads(message)
|
||||
msg_type = data.get('type')
|
||||
|
||||
logger.info(f'Received message: {msg_type}')
|
||||
|
||||
if msg_type == 'task_assignment':
|
||||
await self.execute_task(data)
|
||||
elif msg_type == 'ping':
|
||||
pong_msg = {'type': 'pong', 'agent_id': self.agent_id}
|
||||
await self.ws.send(json.dumps(pong_msg))
|
||||
|
||||
except ConnectionClosed:
|
||||
logger.warning('Connection closed, reconnecting...')
|
||||
await asyncio.sleep(5)
|
||||
await self.connect()
|
||||
except Exception as e:
|
||||
logger.error(f'Listen error: {e}')
|
||||
await asyncio.sleep(1)
|
||||
|
||||
async def run(self):
|
||||
await self.connect()
|
||||
|
||||
# Start heartbeat task
|
||||
heartbeat_task = asyncio.create_task(self.send_heartbeat())
|
||||
|
||||
# Start listening
|
||||
await self.listen()
|
||||
|
||||
heartbeat_task.cancel()
|
||||
|
||||
async def main():
|
||||
orchestrator_url = os.getenv('ORCHESTRATOR_URL', 'ws://orchestrator-service:8000/ws')
|
||||
agent_id = os.getenv('AGENT_ID', f'agent-{uuid.uuid4().hex[:8]}')
|
||||
|
||||
logger.info(f'Starting agent {agent_id}')
|
||||
logger.info(f'Orchestrator URL: {orchestrator_url}')
|
||||
|
||||
agent = SimpleAgent(orchestrator_url, agent_id)
|
||||
await agent.run()
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
"
|
||||
env:
|
||||
- name: AGENT_ID
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: ORCHESTRATOR_URL
|
||||
value: "ws://orchestrator-service:8000/ws/$(AGENT_ID)"
|
||||
- name: OPENAI_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: api-key
|
||||
- name: OPENAI_API_BASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: api-base
|
||||
optional: true
|
||||
- name: OPENAI_MODEL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: model
|
||||
optional: true
|
||||
- name: PYTHONUNBUFFERED
|
||||
value: "1"
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: 1000m
|
||||
memory: 1Gi
|
||||
volumeMounts:
|
||||
- name: agent-source
|
||||
mountPath: /app
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
volumes:
|
||||
- name: agent-source
|
||||
configMap:
|
||||
name: agent-source
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
restartPolicy: Always
|
||||
@@ -0,0 +1,225 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: agent
|
||||
component: worker
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: agent
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: agent
|
||||
component: worker
|
||||
spec:
|
||||
serviceAccountName: orchestrator-sa
|
||||
containers:
|
||||
- name: agent
|
||||
image: python:3.11-slim
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
echo "Installing dependencies..."
|
||||
pip install --no-cache-dir \
|
||||
websockets \
|
||||
openai \
|
||||
prometheus-client
|
||||
|
||||
echo "Starting agent..."
|
||||
python3 -c "
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
from typing import Optional
|
||||
import websockets
|
||||
from websockets.exceptions import ConnectionClosed
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class SimpleAgent:
|
||||
def __init__(self, orchestrator_url: str, agent_id: str):
|
||||
self.orchestrator_url = orchestrator_url
|
||||
self.agent_id = agent_id
|
||||
self.api_key = os.getenv('OPENAI_API_KEY')
|
||||
self.api_base = os.getenv('OPENAI_API_BASE', 'https://api.openai.com/v1')
|
||||
self.model = os.getenv('OPENAI_MODEL', 'gpt-4o-mini')
|
||||
self.ws = None
|
||||
self.running = True
|
||||
|
||||
async def connect(self):
|
||||
logger.info(f'Connecting to orchestrator at {self.orchestrator_url}')
|
||||
self.ws = await websockets.connect(self.orchestrator_url)
|
||||
|
||||
# Register with orchestrator
|
||||
register_msg = {
|
||||
'type': 'register',
|
||||
'agent_id': self.agent_id,
|
||||
'capabilities': ['code_generation', 'code_review'],
|
||||
'status': 'idle'
|
||||
}
|
||||
await self.ws.send(json.dumps(register_msg))
|
||||
logger.info(f'Agent {self.agent_id} registered')
|
||||
|
||||
async def send_heartbeat(self):
|
||||
while self.running:
|
||||
try:
|
||||
if self.ws:
|
||||
heartbeat_msg = {
|
||||
'type': 'heartbeat',
|
||||
'agent_id': self.agent_id,
|
||||
'status': 'idle',
|
||||
'timestamp': time.time()
|
||||
}
|
||||
await self.ws.send(json.dumps(heartbeat_msg))
|
||||
logger.debug(f'Heartbeat sent')
|
||||
except Exception as e:
|
||||
logger.error(f'Heartbeat error: {e}')
|
||||
await asyncio.sleep(15)
|
||||
|
||||
async def execute_task(self, task_data):
|
||||
task_id = task_data.get('task_id')
|
||||
description = task_data.get('description')
|
||||
|
||||
logger.info(f'Executing task {task_id}: {description}')
|
||||
|
||||
try:
|
||||
# Use OpenAI-compatible API
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
api_key=self.api_key,
|
||||
base_url=self.api_base
|
||||
)
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=[
|
||||
{'role': 'system', 'content': 'You are a helpful coding assistant. Generate clean, working code.'},
|
||||
{'role': 'user', 'content': description}
|
||||
],
|
||||
max_tokens=2000
|
||||
)
|
||||
|
||||
result = response.choices[0].message.content
|
||||
|
||||
# Send result back
|
||||
result_msg = {
|
||||
'type': 'task_result',
|
||||
'task_id': task_id,
|
||||
'agent_id': self.agent_id,
|
||||
'status': 'completed',
|
||||
'result': result,
|
||||
'timestamp': time.time()
|
||||
}
|
||||
await self.ws.send(json.dumps(result_msg))
|
||||
logger.info(f'Task {task_id} completed')
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f'Task execution failed: {e}')
|
||||
error_msg = {
|
||||
'type': 'task_result',
|
||||
'task_id': task_id,
|
||||
'agent_id': self.agent_id,
|
||||
'status': 'failed',
|
||||
'error': str(e),
|
||||
'timestamp': time.time()
|
||||
}
|
||||
await self.ws.send(json.dumps(error_msg))
|
||||
|
||||
async def listen(self):
|
||||
while self.running:
|
||||
try:
|
||||
message = await self.ws.recv()
|
||||
data = json.loads(message)
|
||||
msg_type = data.get('type')
|
||||
|
||||
logger.info(f'Received message: {msg_type}')
|
||||
|
||||
if msg_type == 'task_assignment':
|
||||
await self.execute_task(data)
|
||||
elif msg_type == 'ping':
|
||||
pong_msg = {'type': 'pong', 'agent_id': self.agent_id}
|
||||
await self.ws.send(json.dumps(pong_msg))
|
||||
|
||||
except ConnectionClosed:
|
||||
logger.warning('Connection closed, reconnecting...')
|
||||
await asyncio.sleep(5)
|
||||
await self.connect()
|
||||
except Exception as e:
|
||||
logger.error(f'Listen error: {e}')
|
||||
await asyncio.sleep(1)
|
||||
|
||||
async def run(self):
|
||||
await self.connect()
|
||||
|
||||
# Start heartbeat task
|
||||
heartbeat_task = asyncio.create_task(self.send_heartbeat())
|
||||
|
||||
# Start listening
|
||||
await self.listen()
|
||||
|
||||
heartbeat_task.cancel()
|
||||
|
||||
async def main():
|
||||
orchestrator_url = os.getenv('ORCHESTRATOR_URL', 'ws://orchestrator-service:8000/ws')
|
||||
agent_id = os.getenv('AGENT_ID', f'agent-{uuid.uuid4().hex[:8]}')
|
||||
|
||||
logger.info(f'Starting agent {agent_id}')
|
||||
logger.info(f'Orchestrator URL: {orchestrator_url}')
|
||||
|
||||
agent = SimpleAgent(orchestrator_url, agent_id)
|
||||
await agent.run()
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
"
|
||||
env:
|
||||
- name: AGENT_ID
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: ORCHESTRATOR_URL
|
||||
value: "ws://orchestrator-service:8000/ws/$(AGENT_ID)"
|
||||
- name: OPENAI_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: api-key
|
||||
- name: OPENAI_API_BASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: api-base
|
||||
optional: true
|
||||
- name: OPENAI_MODEL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-api-key
|
||||
key: model
|
||||
optional: true
|
||||
- name: PYTHONUNBUFFERED
|
||||
value: "1"
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: 1000m
|
||||
memory: 1Gi
|
||||
restartPolicy: Always
|
||||
@@ -0,0 +1,158 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: agent-${AGENT_ID}
|
||||
labels:
|
||||
app: swarm-agent
|
||||
agent-id: ${AGENT_ID}
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
|
||||
# Init container to clone workspace
|
||||
initContainers:
|
||||
- name: git-clone
|
||||
image: alpine/git:latest
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
if [ -n "$GIT_REPO_URL" ]; then
|
||||
echo "Cloning repository from $GIT_REPO_URL"
|
||||
git clone $GIT_REPO_URL /workspace
|
||||
else
|
||||
echo "No GIT_REPO_URL provided, skipping clone"
|
||||
mkdir -p /workspace
|
||||
fi
|
||||
env:
|
||||
- name: GIT_REPO_URL
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: swarm-config
|
||||
key: git-repo-url
|
||||
optional: true
|
||||
volumeMounts:
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
|
||||
containers:
|
||||
- name: agent
|
||||
image: swarm-agent:latest
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
||||
env:
|
||||
# Agent configuration
|
||||
- name: AGENT_ID
|
||||
value: ${AGENT_ID}
|
||||
|
||||
- name: AGENT_CAPABILITIES
|
||||
value: ${AGENT_CAPABILITIES}
|
||||
|
||||
# Orchestrator connection
|
||||
- name: ORCHESTRATOR_URL
|
||||
value: "ws://orchestrator-service:8000"
|
||||
|
||||
# Workspace configuration
|
||||
- name: WORKSPACE_DIR
|
||||
value: "/workspace"
|
||||
|
||||
- name: GIT_REPO_URL
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: swarm-config
|
||||
key: git-repo-url
|
||||
optional: true
|
||||
|
||||
# Model API key (OpenAI-compatible) — inject via secret_ref, never commit
|
||||
- name: OPENAI_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openai-secret
|
||||
key: api-key
|
||||
|
||||
# Model configuration (OpenAI-compatible)
|
||||
- name: OPENAI_API_BASE
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: swarm-config
|
||||
key: openai-api-base
|
||||
optional: true
|
||||
|
||||
- name: OPENAI_MODEL
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: swarm-config
|
||||
key: openai-model
|
||||
optional: true
|
||||
|
||||
# Git credentials for push access
|
||||
- name: GIT_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: git-credentials
|
||||
key: username
|
||||
optional: true
|
||||
|
||||
- name: GIT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: git-credentials
|
||||
key: password
|
||||
optional: true
|
||||
|
||||
volumeMounts:
|
||||
- name: workspace
|
||||
mountPath: /workspace
|
||||
|
||||
# Resource limits
|
||||
resources:
|
||||
requests:
|
||||
memory: "512Mi"
|
||||
cpu: "250m"
|
||||
limits:
|
||||
memory: "2Gi"
|
||||
cpu: "1000m"
|
||||
|
||||
# Health check
|
||||
livenessProbe:
|
||||
exec:
|
||||
command:
|
||||
- python
|
||||
- -c
|
||||
- "import sys; sys.exit(0)"
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 30
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
|
||||
volumes:
|
||||
- name: workspace
|
||||
emptyDir: {}
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: swarm-config
|
||||
data:
|
||||
git-repo-url: "" # Set this to your Git repository URL
|
||||
openai-model: "gpt-4o-mini"
|
||||
openai-api-base: "https://api.openai.com/v1" # Override for an OpenAI-compatible endpoint
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openai-secret
|
||||
type: Opaque
|
||||
stringData:
|
||||
api-key: "" # Set via secret_ref / kubectl — do NOT commit a real key
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: git-credentials
|
||||
type: Opaque
|
||||
stringData:
|
||||
username: "" # Set this to your Git username
|
||||
password: "" # Set this to your Git password or token
|
||||
@@ -0,0 +1,90 @@
|
||||
apiVersion: apps/v1
|
||||
kind: DaemonSet
|
||||
metadata:
|
||||
name: agent-image-puller
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: agent-image-puller
|
||||
component: optimization
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: agent-image-puller
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: agent-image-puller
|
||||
spec:
|
||||
# Run on all nodes including control plane
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
- key: node-role.kubernetes.io/master
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
|
||||
# Use host network to avoid CNI overhead
|
||||
hostNetwork: true
|
||||
|
||||
initContainers:
|
||||
# Pre-pull the agent image
|
||||
- name: pull-agent-image
|
||||
image: ghcr.io/heicode/swarm-agent:latest
|
||||
command: ['sh', '-c', 'echo "Agent image pulled successfully"']
|
||||
resources:
|
||||
limits:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
|
||||
# Pre-pull common base images
|
||||
- name: pull-python-base
|
||||
image: python:3.11-slim
|
||||
command: ['sh', '-c', 'echo "Python base image pulled"']
|
||||
resources:
|
||||
limits:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
|
||||
containers:
|
||||
# Keep-alive container (does nothing but keeps DaemonSet running)
|
||||
- name: pause
|
||||
image: gcr.io/google_containers/pause:3.9
|
||||
resources:
|
||||
limits:
|
||||
memory: "32Mi"
|
||||
cpu: "10m"
|
||||
requests:
|
||||
memory: "16Mi"
|
||||
cpu: "5m"
|
||||
|
||||
# Optional: Periodic re-pull to get latest images
|
||||
- name: periodic-puller
|
||||
image: docker:24-cli
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
while true; do
|
||||
echo "Checking for image updates..."
|
||||
# This would require docker socket mount in production
|
||||
# For now, just sleep
|
||||
sleep 3600 # Re-check every hour
|
||||
done
|
||||
resources:
|
||||
limits:
|
||||
memory: "64Mi"
|
||||
cpu: "50m"
|
||||
volumeMounts:
|
||||
- name: docker-socket
|
||||
mountPath: /var/run/docker.sock
|
||||
readOnly: true
|
||||
|
||||
volumes:
|
||||
- name: docker-socket
|
||||
hostPath:
|
||||
path: /var/run/docker.sock
|
||||
type: Socket
|
||||
|
||||
# Ensure DaemonSet runs before agent pods
|
||||
priorityClassName: system-node-critical
|
||||
@@ -0,0 +1,91 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: git-test-server
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: git-test-server
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: git-test-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: git-test-server
|
||||
spec:
|
||||
initContainers:
|
||||
- name: init-repo
|
||||
image: alpine:3.20
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
apk add --no-cache git >/dev/null
|
||||
rm -rf /git/swarm-test.git /tmp/swarm-test-src
|
||||
|
||||
mkdir -p /tmp/swarm-test-src
|
||||
cd /tmp/swarm-test-src
|
||||
git init -b main
|
||||
git config user.name "Swarm Test"
|
||||
git config user.email "swarm-test@example.com"
|
||||
|
||||
cat > hello.py <<'PY'
|
||||
def hello_world():
|
||||
return "hello"
|
||||
PY
|
||||
|
||||
git add hello.py
|
||||
git commit -m "Initial test repository"
|
||||
|
||||
mkdir -p /git
|
||||
git clone --bare /tmp/swarm-test-src /git/swarm-test.git
|
||||
git -C /git/swarm-test.git config daemon.receivepack true
|
||||
touch /git/swarm-test.git/git-daemon-export-ok
|
||||
volumeMounts:
|
||||
- name: git-data
|
||||
mountPath: /git
|
||||
containers:
|
||||
- name: git
|
||||
image: alpine:3.20
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
apk add --no-cache git git-daemon >/dev/null
|
||||
git daemon \
|
||||
--verbose \
|
||||
--reuseaddr \
|
||||
--base-path=/git \
|
||||
--export-all \
|
||||
--enable=receive-pack \
|
||||
/git
|
||||
ports:
|
||||
- containerPort: 9418
|
||||
name: git
|
||||
volumeMounts:
|
||||
- name: git-data
|
||||
mountPath: /git
|
||||
volumes:
|
||||
- name: git-data
|
||||
emptyDir: {}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: git-test-server
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: git-test-server
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 9418
|
||||
targetPort: git
|
||||
protocol: TCP
|
||||
name: git
|
||||
selector:
|
||||
app: git-test-server
|
||||
@@ -0,0 +1,488 @@
|
||||
{
|
||||
"dashboard": {
|
||||
"title": "HeiCode Swarm System Health",
|
||||
"tags": ["swarm", "k8s", "agents"],
|
||||
"timezone": "browser",
|
||||
"schemaVersion": 16,
|
||||
"version": 1,
|
||||
"refresh": "10s",
|
||||
"time": {
|
||||
"from": "now-1h",
|
||||
"to": "now"
|
||||
},
|
||||
"panels": [
|
||||
{
|
||||
"id": 1,
|
||||
"title": "Active Agents",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 0, "y": 0, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "swarm:agents:active:total",
|
||||
"legendFormat": "Active Agents"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {"mode": "thresholds"},
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"value": 0, "color": "green"},
|
||||
{"value": 10, "color": "yellow"},
|
||||
{"value": 20, "color": "red"}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"title": "Task Completion Rate",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 6, "y": 0, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "swarm:tasks:completion_rate:5m",
|
||||
"legendFormat": "Tasks/sec"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "ops",
|
||||
"decimals": 2
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"title": "Average Task Duration",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 12, "y": 0, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "swarm:tasks:duration:avg",
|
||||
"legendFormat": "Avg Duration"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "s",
|
||||
"decimals": 1
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"title": "Pod Startup Time (P95)",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 18, "y": 0, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "swarm:pod:startup:p95",
|
||||
"legendFormat": "P95 Startup"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "s",
|
||||
"decimals": 1,
|
||||
"color": {"mode": "thresholds"},
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"value": 0, "color": "green"},
|
||||
{"value": 30, "color": "yellow"},
|
||||
{"value": 60, "color": "red"}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"title": "Agent Count Over Time",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 0, "y": 4, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "swarm:agents:count:by_status",
|
||||
"legendFormat": "{{status}}"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "short", "label": "Agents"},
|
||||
{"format": "short"}
|
||||
],
|
||||
"legend": {"show": true, "alignAsTable": true, "rightSide": false}
|
||||
},
|
||||
{
|
||||
"id": 6,
|
||||
"title": "Task Metrics",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 12, "y": 4, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "rate(swarm_tasks_completed_total[5m])",
|
||||
"legendFormat": "Completed"
|
||||
},
|
||||
{
|
||||
"expr": "rate(swarm_tasks_failed_total[5m])",
|
||||
"legendFormat": "Failed"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "ops", "label": "Tasks/sec"},
|
||||
{"format": "short"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 7,
|
||||
"title": "Handoff Latency Distribution",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 0, "y": 12, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "histogram_quantile(0.50, rate(swarm_handoff_duration_seconds_bucket[5m]))",
|
||||
"legendFormat": "P50"
|
||||
},
|
||||
{
|
||||
"expr": "histogram_quantile(0.95, rate(swarm_handoff_duration_seconds_bucket[5m]))",
|
||||
"legendFormat": "P95"
|
||||
},
|
||||
{
|
||||
"expr": "histogram_quantile(0.99, rate(swarm_handoff_duration_seconds_bucket[5m]))",
|
||||
"legendFormat": "P99"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "s", "label": "Latency"},
|
||||
{"format": "short"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 8,
|
||||
"title": "Agent Creation Rate",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 12, "y": 12, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "swarm:agents:creation_rate:5m",
|
||||
"legendFormat": "Creation Rate"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "ops", "label": "Agents/sec"},
|
||||
{"format": "short"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 9,
|
||||
"title": "System Resource Usage",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 0, "y": 20, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(rate(container_cpu_usage_seconds_total{namespace=\"swarm-system\"}[5m]))",
|
||||
"legendFormat": "CPU Usage"
|
||||
},
|
||||
{
|
||||
"expr": "sum(container_memory_working_set_bytes{namespace=\"swarm-system\"}) / 1024 / 1024 / 1024",
|
||||
"legendFormat": "Memory Usage (GB)"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "short", "label": "Resources"},
|
||||
{"format": "short"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 10,
|
||||
"title": "Redis Operations",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 12, "y": 20, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "rate(redis_commands_processed_total[5m])",
|
||||
"legendFormat": "Commands/sec"
|
||||
},
|
||||
{
|
||||
"expr": "redis_connected_clients",
|
||||
"legendFormat": "Connected Clients"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "ops", "label": "Operations"},
|
||||
{"format": "short"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"title": "Pod Status by Phase",
|
||||
"type": "piechart",
|
||||
"gridPos": {"x": 0, "y": 28, "w": 8, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "count by (phase) (kube_pod_status_phase{namespace=\"swarm-system\"})",
|
||||
"legendFormat": "{{phase}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"title": "Error Rate",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 8, "y": 28, "w": 8, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "rate(swarm_errors_total[5m])",
|
||||
"legendFormat": "{{error_type}}"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "ops", "label": "Errors/sec"},
|
||||
{"format": "short"}
|
||||
],
|
||||
"alert": {
|
||||
"conditions": [
|
||||
{
|
||||
"evaluator": {"params": [0.1], "type": "gt"},
|
||||
"operator": {"type": "and"},
|
||||
"query": {"params": ["A", "5m", "now"]},
|
||||
"reducer": {"params": [], "type": "avg"},
|
||||
"type": "query"
|
||||
}
|
||||
],
|
||||
"executionErrorState": "alerting",
|
||||
"frequency": "60s",
|
||||
"handler": 1,
|
||||
"name": "High Error Rate",
|
||||
"noDataState": "no_data",
|
||||
"notifications": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"title": "Network I/O",
|
||||
"type": "graph",
|
||||
"gridPos": {"x": 16, "y": 28, "w": 8, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "rate(container_network_receive_bytes_total{namespace=\"swarm-system\"}[5m])",
|
||||
"legendFormat": "RX {{pod}}"
|
||||
},
|
||||
{
|
||||
"expr": "rate(container_network_transmit_bytes_total{namespace=\"swarm-system\"}[5m])",
|
||||
"legendFormat": "TX {{pod}}"
|
||||
}
|
||||
],
|
||||
"yaxes": [
|
||||
{"format": "Bps", "label": "Bytes/sec"},
|
||||
{"format": "short"}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 14,
|
||||
"title": "Top Agents by Duration",
|
||||
"type": "table",
|
||||
"gridPos": {"x": 0, "y": 36, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "topk(10, swarm_agent_duration_seconds)",
|
||||
"format": "table",
|
||||
"instant": true
|
||||
}
|
||||
],
|
||||
"transformations": [
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {"Time": true},
|
||||
"indexByName": {},
|
||||
"renameByName": {
|
||||
"agent_id": "Agent ID",
|
||||
"task_id": "Task ID",
|
||||
"Value": "Duration (s)"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 15,
|
||||
"title": "Recent Failures",
|
||||
"type": "table",
|
||||
"gridPos": {"x": 12, "y": 36, "w": 12, "h": 8},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "topk(10, swarm_agent_failures)",
|
||||
"format": "table",
|
||||
"instant": true
|
||||
}
|
||||
],
|
||||
"transformations": [
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {"Time": true},
|
||||
"indexByName": {},
|
||||
"renameByName": {
|
||||
"agent_id": "Agent ID",
|
||||
"task_id": "Task ID",
|
||||
"error_type": "Error Type",
|
||||
"Value": "Count"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 16,
|
||||
"title": "Orchestrator Health",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 0, "y": 44, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "up{job=\"orchestrator\"}",
|
||||
"legendFormat": "Status"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [
|
||||
{"type": "value", "value": "0", "text": "DOWN"},
|
||||
{"type": "value", "value": "1", "text": "UP"}
|
||||
],
|
||||
"color": {"mode": "thresholds"},
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"value": 0, "color": "red"},
|
||||
{"value": 1, "color": "green"}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"title": "Redis Health",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 6, "y": 44, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "up{job=\"redis\"}",
|
||||
"legendFormat": "Status"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [
|
||||
{"type": "value", "value": "0", "text": "DOWN"},
|
||||
{"type": "value", "value": "1", "text": "UP"}
|
||||
],
|
||||
"color": {"mode": "thresholds"},
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"value": 0, "color": "red"},
|
||||
{"value": 1, "color": "green"}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"title": "Kubernetes Cluster Health",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 12, "y": 44, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "up{job=\"kubernetes-apiservers\"}",
|
||||
"legendFormat": "API Server"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [
|
||||
{"type": "value", "value": "0", "text": "DOWN"},
|
||||
{"type": "value", "value": "1", "text": "UP"}
|
||||
],
|
||||
"color": {"mode": "thresholds"},
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{"value": 0, "color": "red"},
|
||||
{"value": 1, "color": "green"}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 19,
|
||||
"title": "Total System Cost (Estimated)",
|
||||
"type": "stat",
|
||||
"gridPos": {"x": 18, "y": 44, "w": 6, "h": 4},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(swarm_agent_cost_usd)",
|
||||
"legendFormat": "Total Cost"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "currencyUSD",
|
||||
"decimals": 2
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"name": "namespace",
|
||||
"type": "query",
|
||||
"query": "label_values(swarm_agent_status, namespace)",
|
||||
"current": {"text": "swarm-system", "value": "swarm-system"},
|
||||
"hide": 0,
|
||||
"includeAll": false,
|
||||
"multi": false,
|
||||
"options": [],
|
||||
"refresh": 1,
|
||||
"regex": "",
|
||||
"sort": 0
|
||||
},
|
||||
{
|
||||
"name": "task_id",
|
||||
"type": "query",
|
||||
"query": "label_values(swarm_agent_status{namespace=\"$namespace\"}, task_id)",
|
||||
"current": {"text": "All", "value": "$__all"},
|
||||
"hide": 0,
|
||||
"includeAll": true,
|
||||
"multi": true,
|
||||
"options": [],
|
||||
"refresh": 1,
|
||||
"regex": "",
|
||||
"sort": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"annotations": {
|
||||
"list": [
|
||||
{
|
||||
"datasource": "Prometheus",
|
||||
"enable": true,
|
||||
"expr": "ALERTS{alertstate=\"firing\"}",
|
||||
"iconColor": "red",
|
||||
"name": "Alerts",
|
||||
"step": "60s",
|
||||
"tagKeys": "alertname",
|
||||
"textFormat": "{{alertname}}",
|
||||
"titleFormat": "Alert"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: orchestrator-config
|
||||
namespace: swarm-system
|
||||
data:
|
||||
REDIS_HOST: "redis-service"
|
||||
REDIS_PORT: "6379"
|
||||
LOG_LEVEL: "INFO"
|
||||
SWARM_RUNTIME_SOURCE: "heicode-swarm-runtime"
|
||||
SWARM_RUNTIME_PLATFORM: "aks"
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: orchestrator
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: orchestrator
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: orchestrator
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: orchestrator
|
||||
spec:
|
||||
serviceAccountName: orchestrator-sa
|
||||
containers:
|
||||
- name: orchestrator
|
||||
image: python:3.11-slim
|
||||
command: ["/bin/bash", "-c"]
|
||||
args:
|
||||
- |
|
||||
apt-get update && apt-get install -y git
|
||||
pip install --no-cache-dir -r /app/orchestrator/requirements.txt
|
||||
cd /app
|
||||
python -m uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
|
||||
ports:
|
||||
- containerPort: 8000
|
||||
name: http
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: orchestrator-config
|
||||
env:
|
||||
- name: AGNET_RUNTIME_SERVICE_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: agnet-runtime-secrets
|
||||
key: runtime-service-token
|
||||
optional: true
|
||||
- name: AGNET_CALLBACK_SERVICE_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: agnet-runtime-secrets
|
||||
key: callback-service-token
|
||||
optional: true
|
||||
- name: AGNET_CALLBACK_SIGNING_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: agnet-runtime-secrets
|
||||
key: callback-signing-secret
|
||||
optional: true
|
||||
volumeMounts:
|
||||
- name: orchestrator-code
|
||||
mountPath: /app/orchestrator
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: 1000m
|
||||
memory: 1Gi
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8000
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8000
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 5
|
||||
volumes:
|
||||
- name: orchestrator-code
|
||||
configMap:
|
||||
name: orchestrator-source
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: orchestrator-service
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: orchestrator
|
||||
spec:
|
||||
type: LoadBalancer
|
||||
ports:
|
||||
- port: 8000
|
||||
targetPort: 8000
|
||||
protocol: TCP
|
||||
name: http
|
||||
selector:
|
||||
app: orchestrator
|
||||
@@ -0,0 +1,73 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: orchestrator-service
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: orchestrator
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 8000
|
||||
targetPort: 8000
|
||||
name: http
|
||||
selector:
|
||||
app: orchestrator
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: orchestrator
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: orchestrator
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: orchestrator
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: orchestrator
|
||||
spec:
|
||||
serviceAccountName: swarm-orchestrator
|
||||
containers:
|
||||
- name: orchestrator
|
||||
image: swarm-orchestrator:latest
|
||||
imagePullPolicy: IfNotPresent
|
||||
ports:
|
||||
- containerPort: 8000
|
||||
name: http
|
||||
env:
|
||||
- name: REDIS_HOST
|
||||
value: "redis-service"
|
||||
- name: REDIS_PORT
|
||||
value: "6379"
|
||||
- name: REDIS_DB
|
||||
value: "0"
|
||||
- name: LOG_LEVEL
|
||||
value: "INFO"
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "500m"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8000
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8000
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 3
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,348 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: prometheus-config
|
||||
namespace: swarm-system
|
||||
data:
|
||||
prometheus.yml: |
|
||||
global:
|
||||
scrape_interval: 15s
|
||||
evaluation_interval: 15s
|
||||
external_labels:
|
||||
cluster: 'swarm-k8s'
|
||||
environment: 'production'
|
||||
|
||||
# Alerting configuration
|
||||
alerting:
|
||||
alertmanagers:
|
||||
- static_configs:
|
||||
- targets:
|
||||
- alertmanager:9093
|
||||
|
||||
# Scrape configurations
|
||||
scrape_configs:
|
||||
# Prometheus itself
|
||||
- job_name: 'prometheus'
|
||||
static_configs:
|
||||
- targets: ['localhost:9090']
|
||||
|
||||
# Orchestrator metrics
|
||||
- job_name: 'orchestrator'
|
||||
kubernetes_sd_configs:
|
||||
- role: pod
|
||||
namespaces:
|
||||
names:
|
||||
- swarm-system
|
||||
relabel_configs:
|
||||
- source_labels: [__meta_kubernetes_pod_label_app]
|
||||
action: keep
|
||||
regex: orchestrator
|
||||
- source_labels: [__meta_kubernetes_pod_name]
|
||||
target_label: pod
|
||||
- source_labels: [__meta_kubernetes_namespace]
|
||||
target_label: namespace
|
||||
- source_labels: [__address__]
|
||||
target_label: __address__
|
||||
regex: ([^:]+)(?::\d+)?
|
||||
replacement: $1:8000
|
||||
|
||||
# Agent pods metrics
|
||||
- job_name: 'agents'
|
||||
kubernetes_sd_configs:
|
||||
- role: pod
|
||||
namespaces:
|
||||
names:
|
||||
- swarm-system
|
||||
relabel_configs:
|
||||
- source_labels: [__meta_kubernetes_pod_label_app]
|
||||
action: keep
|
||||
regex: swarm-agent
|
||||
- source_labels: [__meta_kubernetes_pod_name]
|
||||
target_label: pod
|
||||
- source_labels: [__meta_kubernetes_pod_label_task_id]
|
||||
target_label: task_id
|
||||
- source_labels: [__meta_kubernetes_pod_label_agent_id]
|
||||
target_label: agent_id
|
||||
- source_labels: [__meta_kubernetes_namespace]
|
||||
target_label: namespace
|
||||
- source_labels: [__address__]
|
||||
target_label: __address__
|
||||
regex: ([^:]+)(?::\d+)?
|
||||
replacement: $1:8080
|
||||
|
||||
# Redis metrics (via redis_exporter)
|
||||
- job_name: 'redis'
|
||||
static_configs:
|
||||
- targets: ['redis-exporter:9121']
|
||||
labels:
|
||||
service: 'redis'
|
||||
|
||||
# Kubernetes API server
|
||||
- job_name: 'kubernetes-apiservers'
|
||||
kubernetes_sd_configs:
|
||||
- role: endpoints
|
||||
scheme: https
|
||||
tls_config:
|
||||
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
|
||||
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
|
||||
relabel_configs:
|
||||
- source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
|
||||
action: keep
|
||||
regex: default;kubernetes;https
|
||||
|
||||
# Kubernetes nodes
|
||||
- job_name: 'kubernetes-nodes'
|
||||
kubernetes_sd_configs:
|
||||
- role: node
|
||||
scheme: https
|
||||
tls_config:
|
||||
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
|
||||
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
|
||||
relabel_configs:
|
||||
- action: labelmap
|
||||
regex: __meta_kubernetes_node_label_(.+)
|
||||
|
||||
# Kubernetes pods (general)
|
||||
- job_name: 'kubernetes-pods'
|
||||
kubernetes_sd_configs:
|
||||
- role: pod
|
||||
relabel_configs:
|
||||
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
|
||||
action: keep
|
||||
regex: true
|
||||
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
|
||||
action: replace
|
||||
target_label: __metrics_path__
|
||||
regex: (.+)
|
||||
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
|
||||
action: replace
|
||||
regex: ([^:]+)(?::\d+)?;(\d+)
|
||||
replacement: $1:$2
|
||||
target_label: __address__
|
||||
- action: labelmap
|
||||
regex: __meta_kubernetes_pod_label_(.+)
|
||||
- source_labels: [__meta_kubernetes_namespace]
|
||||
target_label: kubernetes_namespace
|
||||
- source_labels: [__meta_kubernetes_pod_name]
|
||||
target_label: kubernetes_pod_name
|
||||
|
||||
# Recording rules for aggregations
|
||||
rule_files:
|
||||
- /etc/prometheus/rules/*.yml
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: prometheus-rules
|
||||
namespace: swarm-system
|
||||
data:
|
||||
swarm_rules.yml: |
|
||||
groups:
|
||||
- name: swarm_metrics
|
||||
interval: 30s
|
||||
rules:
|
||||
# Agent count by status
|
||||
- record: swarm:agents:count:by_status
|
||||
expr: count by (status) (swarm_agent_status)
|
||||
|
||||
# Total active agents
|
||||
- record: swarm:agents:active:total
|
||||
expr: count(swarm_agent_status{status="running"})
|
||||
|
||||
# Average task duration
|
||||
- record: swarm:tasks:duration:avg
|
||||
expr: avg(swarm_task_duration_seconds)
|
||||
|
||||
# Task completion rate (last 5m)
|
||||
- record: swarm:tasks:completion_rate:5m
|
||||
expr: rate(swarm_tasks_completed_total[5m])
|
||||
|
||||
# Agent creation rate (last 5m)
|
||||
- record: swarm:agents:creation_rate:5m
|
||||
expr: rate(swarm_agents_created_total[5m])
|
||||
|
||||
# Handoff latency p95
|
||||
- record: swarm:handoff:latency:p95
|
||||
expr: histogram_quantile(0.95, rate(swarm_handoff_duration_seconds_bucket[5m]))
|
||||
|
||||
# Pod startup time p95
|
||||
- record: swarm:pod:startup:p95
|
||||
expr: histogram_quantile(0.95, rate(swarm_pod_startup_seconds_bucket[5m]))
|
||||
|
||||
- name: swarm_alerts
|
||||
interval: 30s
|
||||
rules:
|
||||
# Alert if too many agents are failing
|
||||
- alert: HighAgentFailureRate
|
||||
expr: rate(swarm_agents_failed_total[5m]) > 0.1
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "High agent failure rate detected"
|
||||
description: "Agent failure rate is {{ $value }} failures/sec"
|
||||
|
||||
# Alert if orchestrator is down
|
||||
- alert: OrchestratorDown
|
||||
expr: up{job="orchestrator"} == 0
|
||||
for: 1m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "Orchestrator is down"
|
||||
description: "Orchestrator has been down for more than 1 minute"
|
||||
|
||||
# Alert if Redis is down
|
||||
- alert: RedisDown
|
||||
expr: up{job="redis"} == 0
|
||||
for: 1m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "Redis is down"
|
||||
description: "Redis has been down for more than 1 minute"
|
||||
|
||||
# Alert if pod startup is slow
|
||||
- alert: SlowPodStartup
|
||||
expr: swarm:pod:startup:p95 > 60
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "Pod startup time is slow"
|
||||
description: "P95 pod startup time is {{ $value }}s (target: <30s)"
|
||||
|
||||
# Alert if handoff latency is high
|
||||
- alert: HighHandoffLatency
|
||||
expr: swarm:handoff:latency:p95 > 5
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "High handoff latency detected"
|
||||
description: "P95 handoff latency is {{ $value }}s"
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: prometheus
|
||||
namespace: swarm-system
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: prometheus
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: prometheus
|
||||
spec:
|
||||
serviceAccountName: prometheus
|
||||
containers:
|
||||
- name: prometheus
|
||||
image: prom/prometheus:v2.45.0
|
||||
args:
|
||||
- '--config.file=/etc/prometheus/prometheus.yml'
|
||||
- '--storage.tsdb.path=/prometheus'
|
||||
- '--storage.tsdb.retention.time=30d'
|
||||
- '--web.enable-lifecycle'
|
||||
ports:
|
||||
- containerPort: 9090
|
||||
name: web
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/prometheus
|
||||
- name: rules
|
||||
mountPath: /etc/prometheus/rules
|
||||
- name: storage
|
||||
mountPath: /prometheus
|
||||
resources:
|
||||
requests:
|
||||
memory: "2Gi"
|
||||
cpu: "500m"
|
||||
limits:
|
||||
memory: "4Gi"
|
||||
cpu: "2000m"
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: prometheus-config
|
||||
- name: rules
|
||||
configMap:
|
||||
name: prometheus-rules
|
||||
- name: storage
|
||||
persistentVolumeClaim:
|
||||
claimName: prometheus-storage
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: prometheus
|
||||
namespace: swarm-system
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 9090
|
||||
targetPort: 9090
|
||||
name: web
|
||||
selector:
|
||||
app: prometheus
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: prometheus-storage
|
||||
namespace: swarm-system
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 50Gi
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: prometheus
|
||||
namespace: swarm-system
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: prometheus
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources:
|
||||
- nodes
|
||||
- nodes/proxy
|
||||
- services
|
||||
- endpoints
|
||||
- pods
|
||||
verbs: ["get", "list", "watch"]
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resources:
|
||||
- ingresses
|
||||
verbs: ["get", "list", "watch"]
|
||||
- nonResourceURLs: ["/metrics"]
|
||||
verbs: ["get"]
|
||||
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: prometheus
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: prometheus
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: prometheus
|
||||
namespace: swarm-system
|
||||
@@ -0,0 +1,23 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: orchestrator-role
|
||||
namespace: default
|
||||
labels:
|
||||
app: heicode-swarm
|
||||
component: orchestrator
|
||||
rules:
|
||||
# Pod management permissions
|
||||
- apiGroups: [""]
|
||||
resources: ["pods"]
|
||||
verbs: ["create", "delete", "list", "watch", "get"]
|
||||
|
||||
# Pod log access
|
||||
- apiGroups: [""]
|
||||
resources: ["pods/log"]
|
||||
verbs: ["get"]
|
||||
|
||||
# Pod status monitoring
|
||||
- apiGroups: [""]
|
||||
resources: ["pods/status"]
|
||||
verbs: ["get"]
|
||||
@@ -0,0 +1,16 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: orchestrator-rolebinding
|
||||
namespace: default
|
||||
labels:
|
||||
app: heicode-swarm
|
||||
component: orchestrator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: orchestrator-sa
|
||||
namespace: default
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: orchestrator-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
@@ -0,0 +1,8 @@
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: orchestrator-sa
|
||||
namespace: default
|
||||
labels:
|
||||
app: heicode-swarm
|
||||
component: orchestrator
|
||||
@@ -0,0 +1,85 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: redis-service
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: redis
|
||||
spec:
|
||||
ports:
|
||||
- port: 6379
|
||||
targetPort: 6379
|
||||
name: redis
|
||||
clusterIP: None
|
||||
selector:
|
||||
app: redis
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: redis
|
||||
namespace: swarm-system
|
||||
labels:
|
||||
app: redis
|
||||
spec:
|
||||
serviceName: redis-service
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: redis
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: redis
|
||||
spec:
|
||||
containers:
|
||||
- name: redis
|
||||
image: redis:7-alpine
|
||||
ports:
|
||||
- containerPort: 6379
|
||||
name: redis
|
||||
command:
|
||||
- redis-server
|
||||
- --appendonly
|
||||
- "yes"
|
||||
- --appendfsync
|
||||
- everysec
|
||||
- --maxmemory-policy
|
||||
- allkeys-lru
|
||||
- --maxmemory
|
||||
- 256mb
|
||||
volumeMounts:
|
||||
- name: redis-data
|
||||
mountPath: /data
|
||||
resources:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "500m"
|
||||
livenessProbe:
|
||||
tcpSocket:
|
||||
port: 6379
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
readinessProbe:
|
||||
exec:
|
||||
command:
|
||||
- redis-cli
|
||||
- ping
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 3
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: redis-data
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
@@ -0,0 +1,235 @@
|
||||
# Test namespace and RBAC for HeiCode-Swarm testing
|
||||
# Provides isolation and resource limits for test runs
|
||||
|
||||
---
|
||||
# Test namespace
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: heicode-swarm-test
|
||||
labels:
|
||||
environment: test
|
||||
purpose: agent-testing
|
||||
|
||||
---
|
||||
# Resource quota to prevent runaway tests
|
||||
apiVersion: v1
|
||||
kind: ResourceQuota
|
||||
metadata:
|
||||
name: test-resource-quota
|
||||
namespace: heicode-swarm-test
|
||||
spec:
|
||||
hard:
|
||||
# Limit total pods to prevent cluster overload
|
||||
pods: "25"
|
||||
# CPU limits (enough for 20 agents + orchestrator + redis)
|
||||
requests.cpu: "22"
|
||||
limits.cpu: "25"
|
||||
# Memory limits
|
||||
requests.memory: "44Gi"
|
||||
limits.memory: "50Gi"
|
||||
# Storage limits
|
||||
persistentvolumeclaims: "5"
|
||||
requests.storage: "10Gi"
|
||||
|
||||
---
|
||||
# Limit range for individual pods
|
||||
apiVersion: v1
|
||||
kind: LimitRange
|
||||
metadata:
|
||||
name: test-limit-range
|
||||
namespace: heicode-swarm-test
|
||||
spec:
|
||||
limits:
|
||||
# Default limits for containers
|
||||
- type: Container
|
||||
default:
|
||||
cpu: "1"
|
||||
memory: "2Gi"
|
||||
defaultRequest:
|
||||
cpu: "500m"
|
||||
memory: "1Gi"
|
||||
max:
|
||||
cpu: "2"
|
||||
memory: "4Gi"
|
||||
min:
|
||||
cpu: "100m"
|
||||
memory: "128Mi"
|
||||
# Pod limits
|
||||
- type: Pod
|
||||
max:
|
||||
cpu: "2"
|
||||
memory: "4Gi"
|
||||
|
||||
---
|
||||
# ServiceAccount for test orchestrator
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: test-orchestrator-sa
|
||||
namespace: heicode-swarm-test
|
||||
labels:
|
||||
component: orchestrator
|
||||
environment: test
|
||||
|
||||
---
|
||||
# Role for orchestrator pod management
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: test-orchestrator-role
|
||||
namespace: heicode-swarm-test
|
||||
labels:
|
||||
component: orchestrator
|
||||
environment: test
|
||||
rules:
|
||||
# Pod management permissions
|
||||
- apiGroups: [""]
|
||||
resources: ["pods"]
|
||||
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
|
||||
# Pod logs access
|
||||
- apiGroups: [""]
|
||||
resources: ["pods/log"]
|
||||
verbs: ["get", "list"]
|
||||
# Pod exec for debugging (test only)
|
||||
- apiGroups: [""]
|
||||
resources: ["pods/exec"]
|
||||
verbs: ["create"]
|
||||
# ConfigMaps for agent configuration
|
||||
- apiGroups: [""]
|
||||
resources: ["configmaps"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
# Secrets for API keys
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get", "list"]
|
||||
# Services for orchestrator/redis
|
||||
- apiGroups: [""]
|
||||
resources: ["services"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
|
||||
---
|
||||
# RoleBinding to grant orchestrator permissions
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: test-orchestrator-rolebinding
|
||||
namespace: heicode-swarm-test
|
||||
labels:
|
||||
component: orchestrator
|
||||
environment: test
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: test-orchestrator-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: test-orchestrator-sa
|
||||
namespace: heicode-swarm-test
|
||||
|
||||
---
|
||||
# ServiceAccount for test agents
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: test-agent-sa
|
||||
namespace: heicode-swarm-test
|
||||
labels:
|
||||
component: agent
|
||||
environment: test
|
||||
|
||||
---
|
||||
# Role for agent pods (minimal permissions)
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: test-agent-role
|
||||
namespace: heicode-swarm-test
|
||||
labels:
|
||||
component: agent
|
||||
environment: test
|
||||
rules:
|
||||
# Agents can only read their own pod info
|
||||
- apiGroups: [""]
|
||||
resources: ["pods"]
|
||||
verbs: ["get"]
|
||||
resourceNames: [] # Will be restricted to self via admission controller
|
||||
# Read ConfigMaps for configuration
|
||||
- apiGroups: [""]
|
||||
resources: ["configmaps"]
|
||||
verbs: ["get", "list"]
|
||||
# Read Secrets for API keys
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get"]
|
||||
|
||||
---
|
||||
# RoleBinding for agent permissions
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: test-agent-rolebinding
|
||||
namespace: heicode-swarm-test
|
||||
labels:
|
||||
component: agent
|
||||
environment: test
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: test-agent-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: test-agent-sa
|
||||
namespace: heicode-swarm-test
|
||||
|
||||
---
|
||||
# NetworkPolicy to isolate test namespace
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: test-isolation-policy
|
||||
namespace: heicode-swarm-test
|
||||
spec:
|
||||
podSelector: {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
ingress:
|
||||
# Allow traffic within namespace
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
environment: test
|
||||
# Allow traffic from control plane (for kubectl exec, logs)
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: kube-system
|
||||
egress:
|
||||
# Allow DNS
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: kube-system
|
||||
ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
# Allow traffic within namespace
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
environment: test
|
||||
# Allow external API calls (model API, OpenAI-compatible, over HTTPS)
|
||||
- to:
|
||||
- namespaceSelector: {}
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 443
|
||||
# Allow Git operations
|
||||
- to:
|
||||
- namespaceSelector: {}
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 22
|
||||
- protocol: TCP
|
||||
port: 9418
|
||||
@@ -0,0 +1,70 @@
|
||||
# Kind cluster configuration for HeiCode-Swarm local testing
|
||||
# Optimized for agent pod testing with proper resource isolation
|
||||
|
||||
kind: Cluster
|
||||
apiVersion: kind.x-k8s.io/v1alpha4
|
||||
name: heicode-swarm-test
|
||||
|
||||
# Multi-node setup for realistic pod scheduling
|
||||
nodes:
|
||||
# Control plane node
|
||||
- role: control-plane
|
||||
kubeadmConfigPatches:
|
||||
- |
|
||||
kind: InitConfiguration
|
||||
nodeRegistration:
|
||||
kubeletExtraArgs:
|
||||
node-labels: "node-role=control-plane"
|
||||
extraPortMappings:
|
||||
# Expose orchestrator service for desktop client access
|
||||
- containerPort: 30080
|
||||
hostPort: 8080
|
||||
protocol: TCP
|
||||
# Expose Redis for debugging (optional)
|
||||
- containerPort: 30379
|
||||
hostPort: 6379
|
||||
protocol: TCP
|
||||
|
||||
# Worker nodes for agent pods
|
||||
- role: worker
|
||||
kubeadmConfigPatches:
|
||||
- |
|
||||
kind: JoinConfiguration
|
||||
nodeRegistration:
|
||||
kubeletExtraArgs:
|
||||
node-labels: "node-role=worker,workload=agents"
|
||||
# Resource limits per worker to simulate realistic constraints
|
||||
extraMounts:
|
||||
- hostPath: /tmp/heicode-swarm-cache
|
||||
containerPath: /var/lib/containerd
|
||||
|
||||
- role: worker
|
||||
kubeadmConfigPatches:
|
||||
- |
|
||||
kind: JoinConfiguration
|
||||
nodeRegistration:
|
||||
kubeletExtraArgs:
|
||||
node-labels: "node-role=worker,workload=agents"
|
||||
|
||||
# Networking configuration
|
||||
networking:
|
||||
# Disable default CNI to use kindnet (faster startup)
|
||||
disableDefaultCNI: false
|
||||
# Pod subnet
|
||||
podSubnet: "10.244.0.0/16"
|
||||
# Service subnet
|
||||
serviceSubnet: "10.96.0.0/12"
|
||||
|
||||
# Feature gates for testing
|
||||
featureGates:
|
||||
# Enable ephemeral containers for debugging
|
||||
EphemeralContainers: true
|
||||
|
||||
# Runtime configuration
|
||||
containerdConfigPatches:
|
||||
- |-
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
|
||||
endpoint = ["http://localhost:5000"]
|
||||
[plugins."io.containerd.grpc.v1.cri".containerd]
|
||||
# Increase max concurrent downloads for faster image pulls
|
||||
max_concurrent_downloads = 10
|
||||
@@ -0,0 +1,121 @@
|
||||
# Orchestrator(蜂群编排器)
|
||||
|
||||
基于 FastAPI 的编排器,是蜂群系统的控制核心。它对上对接 Heicode Manager(控制面),对下通过 WebSocket 调度多个 Agent 执行单元,负责任务分解、派发、专家协作、质量评审与结果汇总。
|
||||
|
||||
## 能力概览
|
||||
|
||||
- **Manager 对接**:提供面向 Manager 的部署/任务/日志/指标/审批等 REST 接口,事件通过带签名的回调上报。
|
||||
- **WebSocket 调度**:与 Agent 实时双向通信,按能力与容量派发任务。
|
||||
- **任务图与依赖**:支持任务间依赖(`depends_on`),依赖完成后才派发后继任务。
|
||||
- **任务规划(可选)**:在 Manager 未给出明确分工时,用大模型把目标拆解为「实现 → 测试 → 文档」等专家子任务。
|
||||
- **主控评审循环(可选)**:所有任务完成后由「评审者」判断结果是否达标,不达标则把相关任务退回重做,直至通过或达到上限,并对结果做汇总。
|
||||
- **专家协作**:派发时注入依赖产物与同伴信息,支持 Agent 间消息路由。
|
||||
- **持久化**:以 Redis 为权威存储;提供仅限开发/CI 的内存回退。
|
||||
- **故障恢复**:心跳超时检测、任务重派与重试。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
orchestrator/
|
||||
├── main.py # FastAPI 应用:REST 接口、WebSocket、调度循环、评审与汇总
|
||||
├── swarm_runtime.py # Manager 桥接:运行持久化、事件/回调、审批、任务图构建
|
||||
├── planner.py # 任务规划 + 主控评审(critic)+ 结果汇总
|
||||
├── task_queue.py # 任务队列:创建、派发、依赖、重试、重开
|
||||
├── agent_registry.py # Agent 注册表:注册、心跳、状态
|
||||
├── handoff_manager.py # 移交协调与历史
|
||||
├── redis_client.py # 异步 Redis 封装(含受控的内存回退)
|
||||
├── checkpoint_manager.py / model_tuner.py / tracing.py # 检查点、模型调优、链路追踪
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
## 工作流程
|
||||
|
||||
```
|
||||
分解(Plan) → 派发(Dispatch) → 专家执行(Execute) → 协作/移交(Handoff)
|
||||
→ 评审(Review,可选循环重做) → 汇总交付(Deliver)
|
||||
```
|
||||
|
||||
- **分解**:优先采用 Manager 提供的编排方案;若未提供且开启了规划回退,则由 `planner` 生成专家子任务图。
|
||||
- **派发**:调度循环将就绪任务按「能力匹配 + 剩余容量」派发给已连接的空闲 Agent;派发时把已完成依赖的产物与同伴 Agent 信息注入任务上下文。
|
||||
- **评审循环**:任务全部完成后,`planner.review` 判定是否达标;不达标则 `reopen` 指定任务重做,受 `MAX_REVIEW_CYCLES` 约束;通过后由 `planner.synthesize` 生成统一的最终回答。
|
||||
|
||||
## WebSocket 协议(`/ws/{agent_id}`)
|
||||
|
||||
Agent → 编排器:`register`(含 `available_slots`)、`heartbeat`、`task_start`、`task_complete`、`task_failed`、`task_accepted`、`task_rejected`、`blocked_on_handoff`、`handoff_request`、`peer_message`、`status_update`。
|
||||
|
||||
编排器 → Agent:`task_assignment`、`cancel_task`、`peer_message`(路由转发)、各类确认(`registered`、`heartbeat_ack` 等)。
|
||||
|
||||
## REST 接口
|
||||
|
||||
**健康与指标**
|
||||
- `GET /`、`GET /health`、`GET /metrics`(Prometheus)
|
||||
|
||||
**Manager 面(含 `/api/swarms`、`/api/agent/swarm/deployments`、`/api/agnet/deployments` 等别名)**
|
||||
- `POST .../`:创建蜂群部署
|
||||
- `GET .../{id}`:部署概要
|
||||
- `GET .../{id}/tasks | /logs | /events | /metrics | /workflow | /diagnostics`
|
||||
- `POST .../{id}/stop`:停止部署(会向相关 Agent 发送 `cancel_task`)
|
||||
- `POST .../{id}/approvals/{approval_id}`:接收 Manager 审批决定
|
||||
|
||||
**内部/调试**
|
||||
- `GET /agents`、`/agents/idle`、`/agents/{id}`
|
||||
- `POST /tasks`、`POST /tasks/assign`、`GET /tasks`、`GET /tasks/{id}`、`GET /handoffs`
|
||||
|
||||
## 配置(环境变量)
|
||||
|
||||
```bash
|
||||
# 存储
|
||||
REDIS_HOST=redis-service
|
||||
REDIS_PORT=6379
|
||||
REDIS_DB=0
|
||||
REDIS_FAKE=1 # 仅开发/CI:使用内存版 fakeredis
|
||||
ALLOW_MEMORY_STORE=1 # 仅开发/CI:Redis 不可用时回退到内存(生产请勿开启)
|
||||
|
||||
# 工作流开关
|
||||
ENABLE_PLANNER_FALLBACK=1 # 无 Manager 分工时启用规划回退
|
||||
ENABLE_REVIEW_LOOP=1 # 启用主控评审/重做循环 + 结果汇总
|
||||
MAX_REVIEW_CYCLES=2 # 评审最大重做轮数
|
||||
ENABLE_SUBTASK_HANDOFF=false
|
||||
|
||||
# 规划/评审用模型(OpenAI 兼容;未配置时退化为静态分解 + 启发式评审)
|
||||
OPENAI_API_KEY / OPENAI_API_BASE / OPENAI_MODEL
|
||||
MASTER_REVIEW_MODEL / MAX_SUBTASKS / PLANNER_TIMEOUT_SECONDS
|
||||
|
||||
# 安全与回调(与 Manager 契约相关)
|
||||
AGENT_RUNTIME_SERVICE_TOKEN / AGNET_RUNTIME_SERVICE_TOKEN # 服务间鉴权
|
||||
AGENT_CALLBACK_SERVICE_TOKEN / AGENT_CALLBACK_SIGNING_SECRET # 回调令牌与 HMAC 签名
|
||||
|
||||
# 可观测性
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT / SWARM_RUNTIME_SOURCE / SWARM_RUNTIME_PLATFORM
|
||||
```
|
||||
|
||||
> 说明:未配置运行时服务令牌时,Manager 面接口进入「非安全开发模式」(不校验鉴权),仅供本地调试。
|
||||
|
||||
## 持久化与回退
|
||||
|
||||
Redis 为**权威存储**。仅当显式设置 `REDIS_FAKE=1` 或 `ALLOW_MEMORY_STORE=1` 时,才会使用进程内的 `fakeredis` 作为开发/CI 回退;生产环境在 Redis 不可用时会**快速失败**,避免静默丢失持久化与 Manager 状态。
|
||||
|
||||
## 故障恢复
|
||||
|
||||
- Agent 需每 15 秒心跳;超过 30 秒无心跳判定为失败。
|
||||
- 失败 Agent 的任务自动退回队列重派;任务失败按 `max_retries` 重试。
|
||||
- Agent 断连后其在执行的任务会被回收为可重派。
|
||||
|
||||
## 本地运行
|
||||
|
||||
```bash
|
||||
pip install -r orchestrator/requirements.txt
|
||||
|
||||
# 开发模式(内存回退 + 工作流开关)
|
||||
set "REDIS_FAKE=1"
|
||||
set "ENABLE_PLANNER_FALLBACK=1"
|
||||
set "ENABLE_REVIEW_LOOP=1"
|
||||
python -m uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
python scripts/test-runtime-contract.py # Manager 契约校验
|
||||
python scripts/test-merge-smoke.py # 工作流冒烟测试(评审/协作/汇总等)
|
||||
```
|
||||
@@ -0,0 +1,2 @@
|
||||
"""Swarm orchestrator package."""
|
||||
__version__ = "1.0.0"
|
||||
@@ -0,0 +1,149 @@
|
||||
"""Agent registry with Redis-backed state management."""
|
||||
import json
|
||||
import time
|
||||
import logging
|
||||
from typing import Dict, List, Optional
|
||||
from enum import Enum
|
||||
from pydantic import BaseModel
|
||||
from .redis_client import redis_client
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AgentStatus(str, Enum):
|
||||
"""Agent status enumeration."""
|
||||
IDLE = "idle"
|
||||
BUSY = "busy"
|
||||
HANDOFF_PENDING = "handoff-pending"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
class AgentMetadata(BaseModel):
|
||||
"""Agent metadata model."""
|
||||
agent_id: str
|
||||
status: AgentStatus
|
||||
last_heartbeat: float
|
||||
capabilities: List[str]
|
||||
current_task_id: Optional[str] = None
|
||||
|
||||
|
||||
class AgentRegistry:
|
||||
"""Manages agent registration and heartbeat tracking."""
|
||||
|
||||
HEARTBEAT_TIMEOUT = 30 # seconds
|
||||
AGENT_KEY_PREFIX = "agent:"
|
||||
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
async def register_agent(
|
||||
self, agent_id: str, capabilities: List[str]
|
||||
) -> AgentMetadata:
|
||||
"""Register a new agent."""
|
||||
metadata = AgentMetadata(
|
||||
agent_id=agent_id,
|
||||
status=AgentStatus.IDLE,
|
||||
last_heartbeat=time.time(),
|
||||
capabilities=capabilities,
|
||||
)
|
||||
|
||||
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
|
||||
await redis_client.set(key, metadata.model_dump_json())
|
||||
|
||||
logger.info(f"Registered agent {agent_id} with capabilities: {capabilities}")
|
||||
return metadata
|
||||
|
||||
async def deregister_agent(self, agent_id: str):
|
||||
"""Deregister an agent."""
|
||||
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
|
||||
await redis_client.delete(key)
|
||||
logger.info(f"Deregistered agent {agent_id}")
|
||||
|
||||
async def update_heartbeat(self, agent_id: str) -> bool:
|
||||
"""Update agent heartbeat timestamp."""
|
||||
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
|
||||
data = await redis_client.get(key)
|
||||
|
||||
if not data:
|
||||
logger.warning(f"Agent {agent_id} not found for heartbeat update")
|
||||
return False
|
||||
|
||||
metadata = AgentMetadata.model_validate_json(data)
|
||||
metadata.last_heartbeat = time.time()
|
||||
|
||||
await redis_client.set(key, metadata.model_dump_json())
|
||||
return True
|
||||
|
||||
async def update_status(
|
||||
self, agent_id: str, status: AgentStatus, task_id: Optional[str] = None
|
||||
) -> bool:
|
||||
"""Update agent status."""
|
||||
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
|
||||
data = await redis_client.get(key)
|
||||
|
||||
if not data:
|
||||
logger.warning(f"Agent {agent_id} not found for status update")
|
||||
return False
|
||||
|
||||
metadata = AgentMetadata.model_validate_json(data)
|
||||
metadata.status = status
|
||||
metadata.current_task_id = task_id
|
||||
|
||||
await redis_client.set(key, metadata.model_dump_json())
|
||||
logger.info(f"Updated agent {agent_id} status to {status}")
|
||||
return True
|
||||
|
||||
async def get_agent(self, agent_id: str) -> Optional[AgentMetadata]:
|
||||
"""Get agent metadata."""
|
||||
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
|
||||
data = await redis_client.get(key)
|
||||
|
||||
if not data:
|
||||
return None
|
||||
|
||||
return AgentMetadata.model_validate_json(data)
|
||||
|
||||
async def get_all_agents(self) -> List[AgentMetadata]:
|
||||
"""Get all registered agents."""
|
||||
pattern = f"{self.AGENT_KEY_PREFIX}*"
|
||||
keys = await redis_client.keys(pattern)
|
||||
|
||||
agents = []
|
||||
for key in keys:
|
||||
data = await redis_client.get(key)
|
||||
if data:
|
||||
agents.append(AgentMetadata.model_validate_json(data))
|
||||
|
||||
return agents
|
||||
|
||||
async def get_idle_agents(self) -> List[AgentMetadata]:
|
||||
"""Get all idle agents."""
|
||||
all_agents = await self.get_all_agents()
|
||||
return [agent for agent in all_agents if agent.status == AgentStatus.IDLE]
|
||||
|
||||
async def check_failed_agents(self) -> List[str]:
|
||||
"""Check for agents with expired heartbeats and mark as failed."""
|
||||
current_time = time.time()
|
||||
failed_agents = []
|
||||
|
||||
all_agents = await self.get_all_agents()
|
||||
|
||||
for agent in all_agents:
|
||||
if agent.status == AgentStatus.FAILED:
|
||||
continue
|
||||
|
||||
time_since_heartbeat = current_time - agent.last_heartbeat
|
||||
|
||||
if time_since_heartbeat > self.HEARTBEAT_TIMEOUT:
|
||||
await self.update_status(agent.agent_id, AgentStatus.FAILED)
|
||||
failed_agents.append(agent.agent_id)
|
||||
logger.warning(
|
||||
f"Agent {agent.agent_id} marked as failed "
|
||||
f"(no heartbeat for {time_since_heartbeat:.1f}s)"
|
||||
)
|
||||
|
||||
return failed_agents
|
||||
|
||||
|
||||
# Global agent registry instance
|
||||
agent_registry = AgentRegistry()
|
||||
@@ -0,0 +1,240 @@
|
||||
"""
|
||||
Checkpoint Manager for Task Recovery
|
||||
|
||||
Provides partial checkpointing and retry strategies for agent tasks.
|
||||
Enables recovery from failures without restarting entire workflows.
|
||||
"""
|
||||
|
||||
import json
|
||||
import time
|
||||
from typing import Dict, List, Optional, Any
|
||||
from dataclasses import dataclass, asdict
|
||||
from datetime import datetime
|
||||
import redis
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Checkpoint:
|
||||
"""Represents a task checkpoint"""
|
||||
task_id: str
|
||||
agent_id: str
|
||||
checkpoint_id: str
|
||||
timestamp: float
|
||||
phase: str # e.g., "analysis", "implementation", "testing"
|
||||
state: Dict[str, Any] # Serializable state data
|
||||
files_modified: List[str]
|
||||
git_commit: Optional[str] = None
|
||||
metadata: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class RetryPolicy:
|
||||
"""Retry strategy configuration"""
|
||||
max_retries: int = 3
|
||||
backoff_multiplier: float = 2.0
|
||||
initial_delay_seconds: float = 1.0
|
||||
max_delay_seconds: float = 60.0
|
||||
retry_on_errors: List[str] = None # Error types to retry
|
||||
|
||||
|
||||
class CheckpointManager:
|
||||
"""Manages task checkpoints and recovery"""
|
||||
|
||||
def __init__(self, redis_client: redis.Redis):
|
||||
self.redis = redis_client
|
||||
self.checkpoint_ttl = 86400 * 7 # 7 days
|
||||
|
||||
def save_checkpoint(self, checkpoint: Checkpoint) -> bool:
|
||||
"""Save a checkpoint to Redis"""
|
||||
try:
|
||||
key = f"checkpoint:{checkpoint.task_id}:{checkpoint.checkpoint_id}"
|
||||
data = json.dumps(asdict(checkpoint))
|
||||
self.redis.setex(key, self.checkpoint_ttl, data)
|
||||
|
||||
# Add to task's checkpoint list
|
||||
list_key = f"checkpoints:{checkpoint.task_id}"
|
||||
self.redis.lpush(list_key, checkpoint.checkpoint_id)
|
||||
self.redis.expire(list_key, self.checkpoint_ttl)
|
||||
|
||||
logger.info(
|
||||
f"Saved checkpoint {checkpoint.checkpoint_id} for task {checkpoint.task_id}"
|
||||
)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to save checkpoint: {e}")
|
||||
return False
|
||||
|
||||
def get_checkpoint(self, task_id: str, checkpoint_id: str) -> Optional[Checkpoint]:
|
||||
"""Retrieve a specific checkpoint"""
|
||||
try:
|
||||
key = f"checkpoint:{task_id}:{checkpoint_id}"
|
||||
data = self.redis.get(key)
|
||||
if not data:
|
||||
return None
|
||||
|
||||
checkpoint_dict = json.loads(data)
|
||||
return Checkpoint(**checkpoint_dict)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to retrieve checkpoint: {e}")
|
||||
return None
|
||||
|
||||
def get_latest_checkpoint(self, task_id: str) -> Optional[Checkpoint]:
|
||||
"""Get the most recent checkpoint for a task"""
|
||||
try:
|
||||
list_key = f"checkpoints:{task_id}"
|
||||
checkpoint_ids = self.redis.lrange(list_key, 0, 0)
|
||||
|
||||
if not checkpoint_ids:
|
||||
return None
|
||||
|
||||
checkpoint_id = checkpoint_ids[0].decode('utf-8')
|
||||
return self.get_checkpoint(task_id, checkpoint_id)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get latest checkpoint: {e}")
|
||||
return None
|
||||
|
||||
def list_checkpoints(self, task_id: str) -> List[str]:
|
||||
"""List all checkpoint IDs for a task"""
|
||||
try:
|
||||
list_key = f"checkpoints:{task_id}"
|
||||
checkpoint_ids = self.redis.lrange(list_key, 0, -1)
|
||||
return [cid.decode('utf-8') for cid in checkpoint_ids]
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to list checkpoints: {e}")
|
||||
return []
|
||||
|
||||
def delete_checkpoint(self, task_id: str, checkpoint_id: str) -> bool:
|
||||
"""Delete a specific checkpoint"""
|
||||
try:
|
||||
key = f"checkpoint:{task_id}:{checkpoint_id}"
|
||||
self.redis.delete(key)
|
||||
|
||||
# Remove from list
|
||||
list_key = f"checkpoints:{task_id}"
|
||||
self.redis.lrem(list_key, 0, checkpoint_id)
|
||||
|
||||
logger.info(f"Deleted checkpoint {checkpoint_id} for task {task_id}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete checkpoint: {e}")
|
||||
return False
|
||||
|
||||
def cleanup_task_checkpoints(self, task_id: str) -> bool:
|
||||
"""Delete all checkpoints for a task"""
|
||||
try:
|
||||
checkpoint_ids = self.list_checkpoints(task_id)
|
||||
for checkpoint_id in checkpoint_ids:
|
||||
self.delete_checkpoint(task_id, checkpoint_id)
|
||||
|
||||
list_key = f"checkpoints:{task_id}"
|
||||
self.redis.delete(list_key)
|
||||
|
||||
logger.info(f"Cleaned up all checkpoints for task {task_id}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to cleanup checkpoints: {e}")
|
||||
return False
|
||||
|
||||
|
||||
class RetryManager:
|
||||
"""Manages retry logic with exponential backoff"""
|
||||
|
||||
def __init__(self, redis_client: redis.Redis):
|
||||
self.redis = redis_client
|
||||
|
||||
def record_attempt(self, task_id: str, agent_id: str, error: Optional[str] = None):
|
||||
"""Record a task attempt"""
|
||||
key = f"retry:{task_id}"
|
||||
attempt_data = {
|
||||
"agent_id": agent_id,
|
||||
"timestamp": time.time(),
|
||||
"error": error
|
||||
}
|
||||
|
||||
self.redis.lpush(key, json.dumps(attempt_data))
|
||||
self.redis.expire(key, 86400) # 24 hour TTL
|
||||
|
||||
def get_attempt_count(self, task_id: str) -> int:
|
||||
"""Get number of attempts for a task"""
|
||||
key = f"retry:{task_id}"
|
||||
return self.redis.llen(key)
|
||||
|
||||
def should_retry(self, task_id: str, policy: RetryPolicy) -> bool:
|
||||
"""Determine if task should be retried"""
|
||||
attempt_count = self.get_attempt_count(task_id)
|
||||
return attempt_count < policy.max_retries
|
||||
|
||||
def get_retry_delay(self, task_id: str, policy: RetryPolicy) -> float:
|
||||
"""Calculate delay before next retry (exponential backoff)"""
|
||||
attempt_count = self.get_attempt_count(task_id)
|
||||
delay = policy.initial_delay_seconds * (policy.backoff_multiplier ** attempt_count)
|
||||
return min(delay, policy.max_delay_seconds)
|
||||
|
||||
def clear_attempts(self, task_id: str):
|
||||
"""Clear retry history for a task"""
|
||||
key = f"retry:{task_id}"
|
||||
self.redis.delete(key)
|
||||
|
||||
|
||||
class RecoveryCoordinator:
|
||||
"""Coordinates task recovery from checkpoints"""
|
||||
|
||||
def __init__(self, checkpoint_manager: CheckpointManager, retry_manager: RetryManager):
|
||||
self.checkpoint_manager = checkpoint_manager
|
||||
self.retry_manager = retry_manager
|
||||
|
||||
def recover_task(
|
||||
self,
|
||||
task_id: str,
|
||||
retry_policy: Optional[RetryPolicy] = None
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Attempt to recover a failed task from its latest checkpoint
|
||||
|
||||
Returns recovery instructions or None if recovery not possible
|
||||
"""
|
||||
if retry_policy is None:
|
||||
retry_policy = RetryPolicy()
|
||||
|
||||
# Check if we should retry
|
||||
if not self.retry_manager.should_retry(task_id, retry_policy):
|
||||
logger.warning(f"Task {task_id} exceeded max retries")
|
||||
return None
|
||||
|
||||
# Get latest checkpoint
|
||||
checkpoint = self.checkpoint_manager.get_latest_checkpoint(task_id)
|
||||
if not checkpoint:
|
||||
logger.warning(f"No checkpoint found for task {task_id}")
|
||||
return None
|
||||
|
||||
# Calculate retry delay
|
||||
delay = self.retry_manager.get_retry_delay(task_id, retry_policy)
|
||||
|
||||
recovery_plan = {
|
||||
"task_id": task_id,
|
||||
"checkpoint_id": checkpoint.checkpoint_id,
|
||||
"resume_phase": checkpoint.phase,
|
||||
"state": checkpoint.state,
|
||||
"files_modified": checkpoint.files_modified,
|
||||
"git_commit": checkpoint.git_commit,
|
||||
"retry_delay_seconds": delay,
|
||||
"attempt_number": self.retry_manager.get_attempt_count(task_id) + 1
|
||||
}
|
||||
|
||||
logger.info(
|
||||
f"Recovery plan created for task {task_id} from checkpoint {checkpoint.checkpoint_id}"
|
||||
)
|
||||
return recovery_plan
|
||||
|
||||
def mark_recovery_success(self, task_id: str):
|
||||
"""Mark a task as successfully recovered"""
|
||||
self.retry_manager.clear_attempts(task_id)
|
||||
logger.info(f"Task {task_id} recovered successfully")
|
||||
|
||||
def mark_recovery_failure(self, task_id: str, agent_id: str, error: str):
|
||||
"""Record a failed recovery attempt"""
|
||||
self.retry_manager.record_attempt(task_id, agent_id, error)
|
||||
logger.warning(f"Recovery attempt failed for task {task_id}: {error}")
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Handoff manager for task coordination between agents."""
|
||||
import json
|
||||
import time
|
||||
import uuid
|
||||
import logging
|
||||
from typing import Dict, Optional
|
||||
from pydantic import BaseModel
|
||||
from .redis_client import redis_client
|
||||
from .agent_registry import agent_registry, AgentStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class HandoffRequest(BaseModel):
|
||||
"""Handoff request model."""
|
||||
type: str = "handoff"
|
||||
source_agent_id: str
|
||||
target_agent_id: str
|
||||
task_context: Dict
|
||||
|
||||
|
||||
class HandoffRecord(BaseModel):
|
||||
"""Handoff history record."""
|
||||
handoff_id: str
|
||||
source_agent_id: str
|
||||
target_agent_id: str
|
||||
timestamp: float
|
||||
task_context: Dict
|
||||
status: str # pending, completed, failed
|
||||
|
||||
|
||||
class HandoffManager:
|
||||
"""Manages task handoffs between agents."""
|
||||
|
||||
HANDOFF_KEY_PREFIX = "handoff:"
|
||||
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
async def initiate_handoff(
|
||||
self, source_agent_id: str, target_agent_id: str, task_context: Dict
|
||||
) -> Optional[str]:
|
||||
"""Initiate a handoff from source to target agent."""
|
||||
|
||||
# Validate source agent exists and is busy
|
||||
source_agent = await agent_registry.get_agent(source_agent_id)
|
||||
if not source_agent:
|
||||
logger.error(f"Source agent {source_agent_id} not found")
|
||||
return None
|
||||
|
||||
# Validate target agent exists and is idle
|
||||
target_agent = await agent_registry.get_agent(target_agent_id)
|
||||
if not target_agent:
|
||||
logger.error(f"Target agent {target_agent_id} not found")
|
||||
return None
|
||||
|
||||
if target_agent.status != AgentStatus.IDLE:
|
||||
logger.error(
|
||||
f"Target agent {target_agent_id} is not idle (status: {target_agent.status})"
|
||||
)
|
||||
return None
|
||||
|
||||
# Create handoff record
|
||||
handoff_id = str(uuid.uuid4())
|
||||
record = HandoffRecord(
|
||||
handoff_id=handoff_id,
|
||||
source_agent_id=source_agent_id,
|
||||
target_agent_id=target_agent_id,
|
||||
timestamp=time.time(),
|
||||
task_context=task_context,
|
||||
status="pending",
|
||||
)
|
||||
|
||||
# Store in Redis
|
||||
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
|
||||
await redis_client.set(key, record.model_dump_json())
|
||||
|
||||
# Update agent statuses
|
||||
await agent_registry.update_status(source_agent_id, AgentStatus.IDLE)
|
||||
await agent_registry.update_status(
|
||||
target_agent_id, AgentStatus.HANDOFF_PENDING, task_context.get("task_id")
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"Handoff {handoff_id} initiated: {source_agent_id} -> {target_agent_id}"
|
||||
)
|
||||
return handoff_id
|
||||
|
||||
async def complete_handoff(self, handoff_id: str) -> bool:
|
||||
"""Mark handoff as completed."""
|
||||
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
|
||||
data = await redis_client.get(key)
|
||||
|
||||
if not data:
|
||||
logger.error(f"Handoff {handoff_id} not found")
|
||||
return False
|
||||
|
||||
record = HandoffRecord.model_validate_json(data)
|
||||
record.status = "completed"
|
||||
|
||||
await redis_client.set(key, record.model_dump_json())
|
||||
|
||||
# Update target agent to busy
|
||||
await agent_registry.update_status(
|
||||
record.target_agent_id, AgentStatus.BUSY, record.task_context.get("task_id")
|
||||
)
|
||||
|
||||
logger.info(f"Handoff {handoff_id} completed")
|
||||
return True
|
||||
|
||||
async def fail_handoff(self, handoff_id: str, reason: str) -> bool:
|
||||
"""Mark handoff as failed."""
|
||||
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
|
||||
data = await redis_client.get(key)
|
||||
|
||||
if not data:
|
||||
logger.error(f"Handoff {handoff_id} not found")
|
||||
return False
|
||||
|
||||
record = HandoffRecord.model_validate_json(data)
|
||||
record.status = "failed"
|
||||
|
||||
await redis_client.set(key, record.model_dump_json())
|
||||
|
||||
# Revert target agent to idle
|
||||
await agent_registry.update_status(record.target_agent_id, AgentStatus.IDLE)
|
||||
|
||||
logger.warning(f"Handoff {handoff_id} failed: {reason}")
|
||||
return True
|
||||
|
||||
async def get_handoff(self, handoff_id: str) -> Optional[HandoffRecord]:
|
||||
"""Get handoff record by ID."""
|
||||
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
|
||||
data = await redis_client.get(key)
|
||||
|
||||
if not data:
|
||||
return None
|
||||
|
||||
return HandoffRecord.model_validate_json(data)
|
||||
|
||||
async def get_handoff_history(
|
||||
self, agent_id: Optional[str] = None, limit: int = 100
|
||||
) -> list[HandoffRecord]:
|
||||
"""Get handoff history, optionally filtered by agent."""
|
||||
pattern = f"{self.HANDOFF_KEY_PREFIX}*"
|
||||
keys = await redis_client.keys(pattern)
|
||||
|
||||
records = []
|
||||
for key in keys[:limit]:
|
||||
data = await redis_client.get(key)
|
||||
if data:
|
||||
record = HandoffRecord.model_validate_json(data)
|
||||
if agent_id is None or (
|
||||
record.source_agent_id == agent_id
|
||||
or record.target_agent_id == agent_id
|
||||
):
|
||||
records.append(record)
|
||||
|
||||
# Sort by timestamp descending
|
||||
records.sort(key=lambda r: r.timestamp, reverse=True)
|
||||
return records[:limit]
|
||||
|
||||
|
||||
# Global handoff manager instance
|
||||
handoff_manager = HandoffManager()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,386 @@
|
||||
"""
|
||||
Complexity Model Tuner
|
||||
|
||||
Implements feedback loop and retraining for the complexity analysis model.
|
||||
Collects actual vs predicted agent counts and adjusts the model over time.
|
||||
"""
|
||||
|
||||
import json
|
||||
import time
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
from dataclasses import dataclass, asdict
|
||||
from datetime import datetime, timedelta
|
||||
import redis
|
||||
import logging
|
||||
from collections import defaultdict
|
||||
import statistics
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ComplexityPrediction:
|
||||
"""Record of a complexity prediction"""
|
||||
task_id: str
|
||||
timestamp: float
|
||||
task_description: str
|
||||
predicted_agents: int
|
||||
predicted_complexity: str # "simple", "moderate", "complex"
|
||||
confidence: float
|
||||
model_version: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class ComplexityActual:
|
||||
"""Actual outcome of a task"""
|
||||
task_id: str
|
||||
timestamp: float
|
||||
actual_agents: int
|
||||
actual_duration_seconds: float
|
||||
success: bool
|
||||
user_feedback: Optional[str] = None # "too_many", "too_few", "just_right"
|
||||
user_rating: Optional[int] = None # 1-5 scale
|
||||
|
||||
|
||||
@dataclass
|
||||
class ModelMetrics:
|
||||
"""Model performance metrics"""
|
||||
model_version: str
|
||||
total_predictions: int
|
||||
mean_absolute_error: float
|
||||
accuracy_within_1: float # % predictions within ±1 agent
|
||||
accuracy_within_2: float # % predictions within ±2 agents
|
||||
user_satisfaction: float # Average user rating
|
||||
last_updated: float
|
||||
|
||||
|
||||
class ComplexityModelTuner:
|
||||
"""Manages complexity model feedback and tuning"""
|
||||
|
||||
def __init__(self, redis_client: redis.Redis):
|
||||
self.redis = redis_client
|
||||
self.current_model_version = "v1.0"
|
||||
self.data_ttl = 86400 * 30 # 30 days
|
||||
|
||||
def record_prediction(self, prediction: ComplexityPrediction) -> bool:
|
||||
"""Record a complexity prediction"""
|
||||
try:
|
||||
key = f"prediction:{prediction.task_id}"
|
||||
data = json.dumps(asdict(prediction))
|
||||
self.redis.setex(key, self.data_ttl, data)
|
||||
|
||||
# Add to predictions list
|
||||
list_key = f"predictions:{prediction.model_version}"
|
||||
self.redis.lpush(list_key, prediction.task_id)
|
||||
self.redis.expire(list_key, self.data_ttl)
|
||||
|
||||
logger.info(
|
||||
f"Recorded prediction for task {prediction.task_id}: "
|
||||
f"{prediction.predicted_agents} agents"
|
||||
)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to record prediction: {e}")
|
||||
return False
|
||||
|
||||
def record_actual(self, actual: ComplexityActual) -> bool:
|
||||
"""Record actual task outcome"""
|
||||
try:
|
||||
key = f"actual:{actual.task_id}"
|
||||
data = json.dumps(asdict(actual))
|
||||
self.redis.setex(key, self.data_ttl, data)
|
||||
|
||||
# Add to actuals list
|
||||
list_key = "actuals:all"
|
||||
self.redis.lpush(list_key, actual.task_id)
|
||||
self.redis.expire(list_key, self.data_ttl)
|
||||
|
||||
logger.info(
|
||||
f"Recorded actual for task {actual.task_id}: "
|
||||
f"{actual.actual_agents} agents, feedback: {actual.user_feedback}"
|
||||
)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to record actual: {e}")
|
||||
return False
|
||||
|
||||
def get_prediction(self, task_id: str) -> Optional[ComplexityPrediction]:
|
||||
"""Retrieve a prediction"""
|
||||
try:
|
||||
key = f"prediction:{task_id}"
|
||||
data = self.redis.get(key)
|
||||
if not data:
|
||||
return None
|
||||
return ComplexityPrediction(**json.loads(data))
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get prediction: {e}")
|
||||
return None
|
||||
|
||||
def get_actual(self, task_id: str) -> Optional[ComplexityActual]:
|
||||
"""Retrieve actual outcome"""
|
||||
try:
|
||||
key = f"actual:{task_id}"
|
||||
data = self.redis.get(key)
|
||||
if not data:
|
||||
return None
|
||||
return ComplexityActual(**json.loads(data))
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get actual: {e}")
|
||||
return None
|
||||
|
||||
def calculate_metrics(
|
||||
self,
|
||||
model_version: Optional[str] = None,
|
||||
limit: int = 1000
|
||||
) -> Optional[ModelMetrics]:
|
||||
"""Calculate model performance metrics"""
|
||||
if model_version is None:
|
||||
model_version = self.current_model_version
|
||||
|
||||
try:
|
||||
# Get predictions for this model version
|
||||
list_key = f"predictions:{model_version}"
|
||||
task_ids = self.redis.lrange(list_key, 0, limit - 1)
|
||||
|
||||
if not task_ids:
|
||||
logger.warning(f"No predictions found for model {model_version}")
|
||||
return None
|
||||
|
||||
errors = []
|
||||
within_1 = 0
|
||||
within_2 = 0
|
||||
ratings = []
|
||||
|
||||
for task_id_bytes in task_ids:
|
||||
task_id = task_id_bytes.decode('utf-8')
|
||||
|
||||
prediction = self.get_prediction(task_id)
|
||||
actual = self.get_actual(task_id)
|
||||
|
||||
if not prediction or not actual:
|
||||
continue
|
||||
|
||||
# Calculate error
|
||||
error = abs(prediction.predicted_agents - actual.actual_agents)
|
||||
errors.append(error)
|
||||
|
||||
# Check accuracy thresholds
|
||||
if error <= 1:
|
||||
within_1 += 1
|
||||
if error <= 2:
|
||||
within_2 += 1
|
||||
|
||||
# Collect user ratings
|
||||
if actual.user_rating:
|
||||
ratings.append(actual.user_rating)
|
||||
|
||||
if not errors:
|
||||
logger.warning(f"No matched predictions/actuals for model {model_version}")
|
||||
return None
|
||||
|
||||
total = len(errors)
|
||||
mae = statistics.mean(errors)
|
||||
acc_1 = (within_1 / total) * 100
|
||||
acc_2 = (within_2 / total) * 100
|
||||
avg_rating = statistics.mean(ratings) if ratings else 0.0
|
||||
|
||||
metrics = ModelMetrics(
|
||||
model_version=model_version,
|
||||
total_predictions=total,
|
||||
mean_absolute_error=mae,
|
||||
accuracy_within_1=acc_1,
|
||||
accuracy_within_2=acc_2,
|
||||
user_satisfaction=avg_rating,
|
||||
last_updated=time.time()
|
||||
)
|
||||
|
||||
# Cache metrics
|
||||
metrics_key = f"metrics:{model_version}"
|
||||
self.redis.setex(metrics_key, 3600, json.dumps(asdict(metrics)))
|
||||
|
||||
logger.info(
|
||||
f"Model {model_version} metrics: MAE={mae:.2f}, "
|
||||
f"Acc±1={acc_1:.1f}%, Acc±2={acc_2:.1f}%, "
|
||||
f"Satisfaction={avg_rating:.2f}/5"
|
||||
)
|
||||
|
||||
return metrics
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to calculate metrics: {e}")
|
||||
return None
|
||||
|
||||
def get_error_patterns(self, limit: int = 100) -> Dict[str, List[Tuple[str, int, int]]]:
|
||||
"""
|
||||
Analyze error patterns to identify systematic biases
|
||||
|
||||
Returns dict with categories:
|
||||
- overestimated: tasks where we predicted too many agents
|
||||
- underestimated: tasks where we predicted too few agents
|
||||
- accurate: tasks where prediction was close
|
||||
"""
|
||||
patterns = {
|
||||
"overestimated": [],
|
||||
"underestimated": [],
|
||||
"accurate": []
|
||||
}
|
||||
|
||||
try:
|
||||
list_key = "actuals:all"
|
||||
task_ids = self.redis.lrange(list_key, 0, limit - 1)
|
||||
|
||||
for task_id_bytes in task_ids:
|
||||
task_id = task_id_bytes.decode('utf-8')
|
||||
|
||||
prediction = self.get_prediction(task_id)
|
||||
actual = self.get_actual(task_id)
|
||||
|
||||
if not prediction or not actual:
|
||||
continue
|
||||
|
||||
error = prediction.predicted_agents - actual.actual_agents
|
||||
|
||||
entry = (
|
||||
task_id,
|
||||
prediction.predicted_agents,
|
||||
actual.actual_agents
|
||||
)
|
||||
|
||||
if error > 1:
|
||||
patterns["overestimated"].append(entry)
|
||||
elif error < -1:
|
||||
patterns["underestimated"].append(entry)
|
||||
else:
|
||||
patterns["accurate"].append(entry)
|
||||
|
||||
return patterns
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to analyze error patterns: {e}")
|
||||
return patterns
|
||||
|
||||
def get_feedback_summary(self, limit: int = 100) -> Dict[str, int]:
|
||||
"""Summarize user feedback"""
|
||||
feedback_counts = defaultdict(int)
|
||||
|
||||
try:
|
||||
list_key = "actuals:all"
|
||||
task_ids = self.redis.lrange(list_key, 0, limit - 1)
|
||||
|
||||
for task_id_bytes in task_ids:
|
||||
task_id = task_id_bytes.decode('utf-8')
|
||||
actual = self.get_actual(task_id)
|
||||
|
||||
if actual and actual.user_feedback:
|
||||
feedback_counts[actual.user_feedback] += 1
|
||||
|
||||
return dict(feedback_counts)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get feedback summary: {e}")
|
||||
return {}
|
||||
|
||||
def generate_tuning_recommendations(self) -> List[str]:
|
||||
"""Generate recommendations for model tuning based on data"""
|
||||
recommendations = []
|
||||
|
||||
try:
|
||||
# Get current metrics
|
||||
metrics = self.calculate_metrics()
|
||||
if not metrics:
|
||||
return ["Insufficient data for recommendations"]
|
||||
|
||||
# Check accuracy
|
||||
if metrics.accuracy_within_1 < 60:
|
||||
recommendations.append(
|
||||
f"Low accuracy ({metrics.accuracy_within_1:.1f}%). "
|
||||
"Consider retraining with more diverse examples."
|
||||
)
|
||||
|
||||
# Check error patterns
|
||||
patterns = self.get_error_patterns()
|
||||
overestimated = len(patterns["overestimated"])
|
||||
underestimated = len(patterns["underestimated"])
|
||||
total = overestimated + underestimated + len(patterns["accurate"])
|
||||
|
||||
if total > 0:
|
||||
over_pct = (overestimated / total) * 100
|
||||
under_pct = (underestimated / total) * 100
|
||||
|
||||
if over_pct > 40:
|
||||
recommendations.append(
|
||||
f"Model overestimates in {over_pct:.1f}% of cases. "
|
||||
"Consider reducing base agent count or adjusting complexity thresholds."
|
||||
)
|
||||
|
||||
if under_pct > 40:
|
||||
recommendations.append(
|
||||
f"Model underestimates in {under_pct:.1f}% of cases. "
|
||||
"Consider increasing base agent count or lowering complexity thresholds."
|
||||
)
|
||||
|
||||
# Check user satisfaction
|
||||
if metrics.user_satisfaction < 3.5:
|
||||
recommendations.append(
|
||||
f"Low user satisfaction ({metrics.user_satisfaction:.1f}/5). "
|
||||
"Review user feedback and adjust model accordingly."
|
||||
)
|
||||
|
||||
# Check feedback
|
||||
feedback = self.get_feedback_summary()
|
||||
if feedback.get("too_many", 0) > feedback.get("too_few", 0) * 2:
|
||||
recommendations.append(
|
||||
"Users frequently report 'too many agents'. "
|
||||
"Consider reducing default agent counts."
|
||||
)
|
||||
elif feedback.get("too_few", 0) > feedback.get("too_many", 0) * 2:
|
||||
recommendations.append(
|
||||
"Users frequently report 'too few agents'. "
|
||||
"Consider increasing default agent counts."
|
||||
)
|
||||
|
||||
if not recommendations:
|
||||
recommendations.append(
|
||||
f"Model performing well (MAE={metrics.mean_absolute_error:.2f}, "
|
||||
f"Acc±1={metrics.accuracy_within_1:.1f}%). Continue monitoring."
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to generate recommendations: {e}")
|
||||
recommendations.append(f"Error generating recommendations: {e}")
|
||||
|
||||
return recommendations
|
||||
|
||||
def export_training_data(self, limit: int = 1000) -> List[Dict]:
|
||||
"""Export prediction/actual pairs for model retraining"""
|
||||
training_data = []
|
||||
|
||||
try:
|
||||
list_key = "actuals:all"
|
||||
task_ids = self.redis.lrange(list_key, 0, limit - 1)
|
||||
|
||||
for task_id_bytes in task_ids:
|
||||
task_id = task_id_bytes.decode('utf-8')
|
||||
|
||||
prediction = self.get_prediction(task_id)
|
||||
actual = self.get_actual(task_id)
|
||||
|
||||
if not prediction or not actual:
|
||||
continue
|
||||
|
||||
training_data.append({
|
||||
"task_description": prediction.task_description,
|
||||
"predicted_agents": prediction.predicted_agents,
|
||||
"actual_agents": actual.actual_agents,
|
||||
"duration_seconds": actual.actual_duration_seconds,
|
||||
"success": actual.success,
|
||||
"user_feedback": actual.user_feedback,
|
||||
"user_rating": actual.user_rating
|
||||
})
|
||||
|
||||
logger.info(f"Exported {len(training_data)} training examples")
|
||||
return training_data
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to export training data: {e}")
|
||||
return []
|
||||
@@ -0,0 +1,261 @@
|
||||
"""LLM task planner (Manager-first fallback).
|
||||
|
||||
Ported from agent_swarm_v4. This planner is used ONLY as a fallback to decompose a
|
||||
swarm objective into specialist subtasks when the Manager's orchestration_plan does
|
||||
not provide an explicit agent breakdown AND the operator opts in via
|
||||
``ENABLE_PLANNER_FALLBACK``. It never overrides a Manager-supplied plan.
|
||||
|
||||
It degrades gracefully: with no API key or on any error it returns a static
|
||||
implementation -> testing -> documentation plan, so the runtime never hard-depends on
|
||||
the model being reachable.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from typing import List, Dict
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
from openai import AsyncOpenAI
|
||||
except Exception: # pragma: no cover - only when openai is absent
|
||||
AsyncOpenAI = None
|
||||
|
||||
|
||||
def _planner_timeout() -> float:
|
||||
try:
|
||||
return float(os.getenv("PLANNER_TIMEOUT_SECONDS", "45") or 45)
|
||||
except ValueError:
|
||||
return 45.0
|
||||
|
||||
|
||||
def _max_subtasks() -> int:
|
||||
try:
|
||||
return int(os.getenv("MAX_SUBTASKS", "6") or 6)
|
||||
except ValueError:
|
||||
return 6
|
||||
|
||||
|
||||
class Planner:
|
||||
"""Decomposes an objective into specialist subtask specs."""
|
||||
|
||||
def __init__(self):
|
||||
api_key = os.getenv("OPENAI_API_KEY") or os.getenv("MODEL_API_KEY")
|
||||
api_base = (
|
||||
os.getenv("OPENAI_API_BASE")
|
||||
or os.getenv("MODEL_API_BASE")
|
||||
or "https://api.openai.com/v1"
|
||||
)
|
||||
model = (
|
||||
os.getenv("OPENAI_MODEL")
|
||||
or os.getenv("MODEL_NAME")
|
||||
or os.getenv("MODEL_ID")
|
||||
or "gpt-4o-mini"
|
||||
)
|
||||
self.model = os.getenv("MASTER_REVIEW_MODEL", model)
|
||||
self.client = (
|
||||
AsyncOpenAI(api_key=api_key, base_url=api_base)
|
||||
if (api_key and AsyncOpenAI is not None)
|
||||
else None
|
||||
)
|
||||
|
||||
def _static_plan(self, run_id: str) -> List[Dict]:
|
||||
return [
|
||||
{
|
||||
"subtask_id": f"{run_id}-implementation",
|
||||
"description": "Implement the core code required by the user request.",
|
||||
"required_capabilities": ["python", "code_generation"],
|
||||
"role": "implementation",
|
||||
"depends_on": [],
|
||||
},
|
||||
{
|
||||
"subtask_id": f"{run_id}-testing",
|
||||
"description": "Write tests for the implemented functionality.",
|
||||
"required_capabilities": ["testing", "pytest"],
|
||||
"role": "testing",
|
||||
"depends_on": [f"{run_id}-implementation"],
|
||||
},
|
||||
{
|
||||
"subtask_id": f"{run_id}-documentation",
|
||||
"description": "Write concise documentation based on the implementation and tests.",
|
||||
"required_capabilities": ["technical-writing", "general"],
|
||||
"role": "documentation",
|
||||
"depends_on": [f"{run_id}-implementation", f"{run_id}-testing"],
|
||||
},
|
||||
]
|
||||
|
||||
async def build_plan(self, run_id: str, objective: str) -> List[Dict]:
|
||||
"""Return specialist subtask specs for an objective, or a static fallback."""
|
||||
fallback = self._static_plan(run_id)
|
||||
if not self.client:
|
||||
return fallback
|
||||
try:
|
||||
response = await self.client.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=[
|
||||
{
|
||||
"role": "system",
|
||||
"content": (
|
||||
"Break the request into at most 6 specialist subtasks. "
|
||||
"Return JSON with key 'subtasks'; each subtask has subtask_id, "
|
||||
"description, role, required_capabilities (list), depends_on (list of subtask_ids)."
|
||||
),
|
||||
},
|
||||
{"role": "user", "content": f"Run id: {run_id}\nObjective: {objective}"},
|
||||
],
|
||||
response_format={"type": "json_object"},
|
||||
timeout=_planner_timeout(),
|
||||
)
|
||||
content = response.choices[0].message.content or "{}"
|
||||
subtasks = (json.loads(content) or {}).get("subtasks")
|
||||
if not subtasks:
|
||||
return fallback
|
||||
return subtasks[: _max_subtasks()]
|
||||
except Exception as e:
|
||||
logger.warning(f"Planner LLM call failed ({e}); using static fallback plan")
|
||||
return fallback
|
||||
|
||||
async def review(self, objective: str, tasks: List[Dict], results: Dict) -> Dict:
|
||||
"""Judge whether the combined specialist results are good enough.
|
||||
|
||||
Returns {accepted: bool, summary: str, retry_tasks: [task_id, ...]}. Falls back to a
|
||||
deterministic consistency heuristic when no model is available or the call fails, so the
|
||||
review gate degrades safely instead of blocking the run.
|
||||
"""
|
||||
artifact_summary = self._summarize_results(results)
|
||||
if self.client:
|
||||
try:
|
||||
response = await self.client.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=[
|
||||
{
|
||||
"role": "system",
|
||||
"content": (
|
||||
"Evaluate whether the specialist results jointly satisfy the objective and are "
|
||||
"semantically aligned. Compare claimed error behavior, function/API names, and usage "
|
||||
"examples across specialists. Reject when behavior claims conflict or the objective is "
|
||||
"unmet. Return JSON with keys: accepted (bool), summary (str), retry_tasks (list of the "
|
||||
"result keys that must be redone)."
|
||||
),
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": json.dumps(
|
||||
{
|
||||
"objective": objective,
|
||||
"tasks": tasks,
|
||||
"results": artifact_summary,
|
||||
"result_keys": list(results.keys()),
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
},
|
||||
],
|
||||
response_format={"type": "json_object"},
|
||||
timeout=_planner_timeout(),
|
||||
)
|
||||
result = json.loads(response.choices[0].message.content or "{}")
|
||||
if result:
|
||||
result.setdefault("accepted", True)
|
||||
result.setdefault("summary", "accepted")
|
||||
result.setdefault("retry_tasks", [])
|
||||
# Only keep retry targets that are real result keys.
|
||||
result["retry_tasks"] = [t for t in result["retry_tasks"] if t in results]
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.warning(f"Review LLM call failed ({e}); using heuristic consistency check")
|
||||
|
||||
consistency = self._heuristic_consistency_check(results)
|
||||
if not consistency["accepted"]:
|
||||
return consistency
|
||||
return {"accepted": True, "summary": "fallback acceptance", "retry_tasks": []}
|
||||
|
||||
async def synthesize(self, objective: str, results: Dict) -> str:
|
||||
"""Compose one coherent answer from the specialist results.
|
||||
|
||||
Uses the model when available; otherwise concatenates specialist summaries so a
|
||||
unified response always exists.
|
||||
"""
|
||||
summary = self._summarize_results(results)
|
||||
deterministic = " | ".join(
|
||||
f"{key}: {value.get('summary') or value.get('changes') or 'no summary'}"
|
||||
for key, value in summary.items()
|
||||
) or objective
|
||||
if not self.client:
|
||||
return deterministic
|
||||
try:
|
||||
response = await self.client.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=[
|
||||
{
|
||||
"role": "system",
|
||||
"content": (
|
||||
"Synthesize the specialist results into one concise, user-facing answer to the "
|
||||
"objective. Resolve overlaps in favor of implementation semantics."
|
||||
),
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": json.dumps(
|
||||
{"objective": objective, "results": summary}, ensure_ascii=False
|
||||
),
|
||||
},
|
||||
],
|
||||
timeout=_planner_timeout(),
|
||||
)
|
||||
return (response.choices[0].message.content or "").strip() or deterministic
|
||||
except Exception as e:
|
||||
logger.warning(f"Synthesis LLM call failed ({e}); using concatenated summary")
|
||||
return deterministic
|
||||
|
||||
def _summarize_results(self, results: Dict) -> Dict:
|
||||
summary = {}
|
||||
for task_id, payload in results.items():
|
||||
result = (payload or {}).get("result", {}) or {}
|
||||
subtasks = result.get("subtasks", []) or []
|
||||
files, combined_changes, combined_summary = [], [], []
|
||||
for item in subtasks:
|
||||
files.extend(item.get("files_modified", []) or [])
|
||||
if item.get("changes"):
|
||||
combined_changes.append(item.get("changes"))
|
||||
if item.get("summary"):
|
||||
combined_summary.append(item.get("summary"))
|
||||
if not combined_summary and result.get("summary"):
|
||||
combined_summary.append(result.get("summary"))
|
||||
summary[task_id] = {
|
||||
"files_modified": files,
|
||||
"summary": " ".join(combined_summary),
|
||||
"changes": " ".join(combined_changes),
|
||||
}
|
||||
return summary
|
||||
|
||||
def _heuristic_consistency_check(self, results: Dict) -> Dict:
|
||||
combined_text, task_ids = [], []
|
||||
for task_id, payload in results.items():
|
||||
task_ids.append(task_id)
|
||||
result = (payload or {}).get("result", {}) or {}
|
||||
for item in result.get("subtasks", []) or []:
|
||||
combined_text.append(item.get("summary", "") or "")
|
||||
combined_text.append(item.get("changes", "") or "")
|
||||
for file_item in item.get("files", []) or []:
|
||||
combined_text.append(file_item.get("content", "") or "")
|
||||
|
||||
joined = "\n".join(combined_text).lower()
|
||||
if ("valueerror" in joined) and ("zerodivisionerror" in joined or "zero division" in joined):
|
||||
retry = [t for t in task_ids if "documentation" in t or "testing" in t] or task_ids
|
||||
return {
|
||||
"accepted": False,
|
||||
"summary": "conflicting error semantics detected between specialists",
|
||||
"retry_tasks": retry,
|
||||
}
|
||||
if ("pytest" in joined) and ("unittest" in joined):
|
||||
retry = [t for t in task_ids if "testing" in t or "documentation" in t] or task_ids
|
||||
return {
|
||||
"accepted": False,
|
||||
"summary": "conflicting test framework expectations detected between specialists",
|
||||
"retry_tasks": retry,
|
||||
}
|
||||
return {"accepted": True, "summary": "heuristic consistency acceptance", "retry_tasks": []}
|
||||
|
||||
|
||||
planner = Planner()
|
||||
@@ -0,0 +1,143 @@
|
||||
"""Redis client for orchestrator state management.
|
||||
|
||||
Redis is the canonical store. An in-memory fallback (backed by the in-process
|
||||
``fakeredis`` emulator, which faithfully implements the Redis list/hash API) is
|
||||
available ONLY for local development and CI, and ONLY when explicitly enabled via
|
||||
``REDIS_FAKE=1`` or ``ALLOW_MEMORY_STORE=1``. In production neither flag is set, so a
|
||||
Redis outage fails fast on startup instead of silently dropping durability/Manager
|
||||
state.
|
||||
"""
|
||||
import os
|
||||
from typing import Optional
|
||||
import logging
|
||||
|
||||
try: # redis-py is required in production; guarded so dev/CI can run on fakeredis only.
|
||||
import redis.asyncio as redis
|
||||
except Exception: # pragma: no cover - exercised only when redis-py is absent
|
||||
redis = None
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _truthy(value: Optional[str]) -> bool:
|
||||
return (value or "").strip().lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
class RedisClient:
|
||||
"""Async Redis client wrapper for orchestrator operations."""
|
||||
|
||||
def __init__(self):
|
||||
self.client = None
|
||||
self.host = os.getenv("REDIS_HOST", "redis-service")
|
||||
self.port = int(os.getenv("REDIS_PORT", "6379"))
|
||||
self.db = int(os.getenv("REDIS_DB", "0"))
|
||||
|
||||
def _fallback_allowed(self) -> bool:
|
||||
return _truthy(os.getenv("REDIS_FAKE")) or _truthy(os.getenv("ALLOW_MEMORY_STORE"))
|
||||
|
||||
def _make_fake_client(self):
|
||||
"""Return an in-process fakeredis client (dev/CI fallback only)."""
|
||||
import fakeredis.aioredis as fakeredis # imported lazily; dev/CI dependency
|
||||
return fakeredis.FakeRedis(decode_responses=True)
|
||||
|
||||
async def connect(self):
|
||||
"""Establish Redis connection, or a gated in-memory fallback for dev/CI."""
|
||||
# Explicit fake mode (used by local runs and tests) short-circuits real Redis.
|
||||
if _truthy(os.getenv("REDIS_FAKE")):
|
||||
self.client = self._make_fake_client()
|
||||
logger.warning("REDIS_FAKE enabled; using in-memory fakeredis (NOT for production)")
|
||||
return
|
||||
|
||||
try:
|
||||
if redis is None:
|
||||
raise RuntimeError("redis-py is not installed")
|
||||
self.client = redis.Redis(
|
||||
host=self.host,
|
||||
port=self.port,
|
||||
db=self.db,
|
||||
decode_responses=True,
|
||||
socket_connect_timeout=5,
|
||||
socket_keepalive=True,
|
||||
)
|
||||
await self.client.ping()
|
||||
logger.info(f"Connected to Redis at {self.host}:{self.port}")
|
||||
except Exception as e:
|
||||
if self._fallback_allowed():
|
||||
logger.warning(
|
||||
f"Redis unavailable ({e}); ALLOW_MEMORY_STORE set, using in-memory fakeredis "
|
||||
"fallback (NOT for production)"
|
||||
)
|
||||
self.client = self._make_fake_client()
|
||||
return
|
||||
logger.error(f"Failed to connect to Redis: {e}")
|
||||
raise
|
||||
|
||||
async def disconnect(self):
|
||||
"""Close Redis connection."""
|
||||
if self.client:
|
||||
await self.client.close()
|
||||
logger.info("Disconnected from Redis")
|
||||
|
||||
async def set(self, key: str, value: str, ex: Optional[int] = None):
|
||||
"""Set key-value pair with optional expiration."""
|
||||
await self.client.set(key, value, ex=ex)
|
||||
|
||||
async def get(self, key: str) -> Optional[str]:
|
||||
"""Get value by key."""
|
||||
return await self.client.get(key)
|
||||
|
||||
async def delete(self, key: str):
|
||||
"""Delete key."""
|
||||
await self.client.delete(key)
|
||||
|
||||
async def exists(self, key: str) -> bool:
|
||||
"""Check if key exists."""
|
||||
return await self.client.exists(key) > 0
|
||||
|
||||
async def hset(self, name: str, key: str, value: str):
|
||||
"""Set hash field."""
|
||||
await self.client.hset(name, key, value)
|
||||
|
||||
async def hget(self, name: str, key: str) -> Optional[str]:
|
||||
"""Get hash field."""
|
||||
return await self.client.hget(name, key)
|
||||
|
||||
async def hgetall(self, name: str) -> dict:
|
||||
"""Get all hash fields."""
|
||||
return await self.client.hgetall(name)
|
||||
|
||||
async def hdel(self, name: str, *keys: str):
|
||||
"""Delete hash fields."""
|
||||
await self.client.hdel(name, *keys)
|
||||
|
||||
async def keys(self, pattern: str) -> list:
|
||||
"""Get keys matching pattern."""
|
||||
return await self.client.keys(pattern)
|
||||
|
||||
async def lpush(self, key: str, *values: str):
|
||||
"""Push values to list head."""
|
||||
await self.client.lpush(key, *values)
|
||||
|
||||
async def rpush(self, key: str, *values: str):
|
||||
"""Push values to list tail."""
|
||||
await self.client.rpush(key, *values)
|
||||
|
||||
async def rpop(self, key: str) -> Optional[str]:
|
||||
"""Pop value from list tail."""
|
||||
return await self.client.rpop(key)
|
||||
|
||||
async def llen(self, key: str) -> int:
|
||||
"""Get list length."""
|
||||
return await self.client.llen(key)
|
||||
|
||||
async def lrange(self, key: str, start: int, end: int) -> list:
|
||||
"""Get list range."""
|
||||
return await self.client.lrange(key, start, end)
|
||||
|
||||
async def lrem(self, key: str, count: int, value: str) -> int:
|
||||
"""Remove values from list."""
|
||||
return await self.client.lrem(key, count, value)
|
||||
|
||||
|
||||
# Global Redis client instance
|
||||
redis_client = RedisClient()
|
||||
@@ -0,0 +1,17 @@
|
||||
fastapi==0.115.0
|
||||
uvicorn[standard]==0.32.0
|
||||
websockets==13.1
|
||||
redis==5.2.0
|
||||
openai==1.55.3
|
||||
pydantic==2.9.2
|
||||
python-dotenv==1.0.1
|
||||
prometheus-client==0.20.0
|
||||
httpx==0.28.1
|
||||
# dev/CI only: in-memory Redis emulator for the gated REDIS_FAKE/ALLOW_MEMORY_STORE fallback
|
||||
fakeredis==2.26.1
|
||||
opentelemetry-api==1.24.0
|
||||
opentelemetry-sdk==1.24.0
|
||||
opentelemetry-exporter-otlp==1.24.0
|
||||
opentelemetry-instrumentation-redis==0.45b0
|
||||
opentelemetry-instrumentation-requests==0.45b0
|
||||
opentelemetry-instrumentation-logging==0.45b0
|
||||
@@ -0,0 +1,781 @@
|
||||
"""Agent Manager compatible swarm runtime bridge."""
|
||||
import asyncio
|
||||
from copy import deepcopy
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
import httpx
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from .redis_client import redis_client
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CallbackConfig(BaseModel):
|
||||
"""Callback settings provided by Agent Manager."""
|
||||
|
||||
url: Optional[str] = None
|
||||
signing_secret_ref: Optional[str] = None
|
||||
subscribed_events: List[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class SwarmRun(BaseModel):
|
||||
"""Runtime-owned representation of a swarm run."""
|
||||
|
||||
deployment_id: str
|
||||
swarm_id: str
|
||||
mode: str = "swarm"
|
||||
status: str
|
||||
objective: str
|
||||
manager_deployment_id: Optional[str] = None
|
||||
correlation_id: Optional[str] = None
|
||||
callback: CallbackConfig = Field(default_factory=CallbackConfig)
|
||||
task_ids: List[str] = Field(default_factory=list)
|
||||
approvals: Dict[str, Dict[str, Any]] = Field(default_factory=dict)
|
||||
metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
request_body: Dict[str, Any] = Field(default_factory=dict)
|
||||
created_at: float = Field(default_factory=time.time)
|
||||
updated_at: float = Field(default_factory=time.time)
|
||||
|
||||
|
||||
class RuntimeValidationError(ValueError):
|
||||
"""Validation error returned in the Agent Manager runtime envelope."""
|
||||
|
||||
def __init__(self, message: str, code: str = "VALIDATION_ERROR"):
|
||||
super().__init__(message)
|
||||
self.message = message
|
||||
self.code = code
|
||||
|
||||
|
||||
class SwarmRuntime:
|
||||
"""Stores swarm runs and emits Agent Manager callback events."""
|
||||
|
||||
RUN_KEY_PREFIX = "swarm:"
|
||||
TASK_RUN_KEY_PREFIX = "swarm_task:"
|
||||
IDEMPOTENCY_KEY_PREFIX = "swarm_idempotency:"
|
||||
EVENT_KEY_PREFIX = "swarm_events:"
|
||||
|
||||
def __init__(self):
|
||||
self.callback_service_token = (
|
||||
os.getenv("AGENT_CALLBACK_SERVICE_TOKEN")
|
||||
or os.getenv("AGNET_CALLBACK_SERVICE_TOKEN")
|
||||
)
|
||||
self.callback_signing_secret = (
|
||||
os.getenv("AGENT_CALLBACK_SIGNING_SECRET")
|
||||
or os.getenv("AGNET_CALLBACK_SIGNING_SECRET")
|
||||
)
|
||||
self.runtime_source = os.getenv("SWARM_RUNTIME_SOURCE", "heicode-swarm-runtime")
|
||||
|
||||
def multi_agent_workflow_enabled(self, body: Optional[Dict[str, Any]] = None) -> bool:
|
||||
"""Return whether the DAG-style runtime workflow is enabled."""
|
||||
if os.getenv("ENABLE_SUBTASK_HANDOFF", "false").lower() not in {"1", "true", "yes"}:
|
||||
return False
|
||||
plan = (body or {}).get("orchestration_plan") or {}
|
||||
agents = plan.get("agents") or (body or {}).get("agents") or []
|
||||
return len(agents) > 0
|
||||
|
||||
async def health(self) -> Dict[str, Any]:
|
||||
"""Return Agent Manager compatible runtime health."""
|
||||
await redis_client.client.ping()
|
||||
return {
|
||||
"success": True,
|
||||
"data": {
|
||||
"status": "healthy",
|
||||
"service": "heicode-swarm-runtime",
|
||||
"version": "1.0.0",
|
||||
"runtime": os.getenv("SWARM_RUNTIME_PLATFORM", "aks"),
|
||||
"time": self._now_iso(),
|
||||
"capabilities": [
|
||||
"swarm.create",
|
||||
"task.flow",
|
||||
"handoff.events",
|
||||
"artifact.events",
|
||||
"approval.pause_resume",
|
||||
"deployment.stop",
|
||||
"runtime.tasks.query",
|
||||
"runtime.logs.query",
|
||||
"runtime.events.query",
|
||||
"runtime.metrics.query",
|
||||
"runtime.workflow.query",
|
||||
"runtime.diagnostics.query",
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
def normalize_create_request(self, body: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Normalize the Manager swarm request into the runtime request shape."""
|
||||
normalized = deepcopy(body or {})
|
||||
metadata = dict(normalized.get("metadata") or {})
|
||||
callback = dict(normalized.get("callback") or {})
|
||||
requirement = dict(normalized.get("requirement") or {})
|
||||
model_selection = dict(normalized.get("model_selection") or {})
|
||||
existing_plan = dict(normalized.get("orchestration_plan") or {})
|
||||
|
||||
uses_new_shape = normalized.get("mode") == "swarm" or bool(requirement)
|
||||
|
||||
if uses_new_shape:
|
||||
primary_model = (
|
||||
model_selection.get("primary_model")
|
||||
or existing_plan.get("model_id")
|
||||
or normalized.get("model_id")
|
||||
)
|
||||
budget = deepcopy(
|
||||
normalized.get("budget")
|
||||
or existing_plan.get("budget")
|
||||
or {}
|
||||
)
|
||||
if not existing_plan.get("objective") and requirement.get("objective"):
|
||||
existing_plan["objective"] = requirement.get("objective")
|
||||
existing_plan.setdefault("sub_mode", "swarm")
|
||||
existing_plan.setdefault("risk_level", normalized.get("risk_level") or "medium")
|
||||
existing_plan.setdefault("budget", budget)
|
||||
existing_plan["model_id"] = primary_model
|
||||
existing_plan["requirement"] = self._redact_sensitive(requirement)
|
||||
existing_plan["attachments"] = self._redact_sensitive(
|
||||
requirement.get("attachments") or []
|
||||
)
|
||||
existing_plan["constraints"] = self._redact_sensitive(
|
||||
requirement.get("constraints") or []
|
||||
)
|
||||
existing_plan["acceptance_criteria"] = self._redact_sensitive(
|
||||
requirement.get("acceptance_criteria") or []
|
||||
)
|
||||
if not existing_plan.get("agents"):
|
||||
existing_plan["agents"] = normalized.get("agents") or []
|
||||
|
||||
normalized["mode"] = "swarm"
|
||||
normalized["conversation_id"] = (
|
||||
normalized.get("conversation_id")
|
||||
or metadata.get("conversation_id")
|
||||
)
|
||||
normalized["requirement"] = requirement
|
||||
normalized["model_selection"] = model_selection
|
||||
normalized["model_id"] = primary_model
|
||||
normalized["orchestration_plan"] = existing_plan
|
||||
if normalized.get("conversation_id"):
|
||||
metadata.setdefault("conversation_id", normalized.get("conversation_id"))
|
||||
if requirement.get("objective"):
|
||||
metadata.setdefault("objective", requirement.get("objective"))
|
||||
if model_selection.get("type"):
|
||||
metadata.setdefault("model_selection_type", model_selection.get("type"))
|
||||
else:
|
||||
normalized["orchestration_plan"] = existing_plan
|
||||
|
||||
normalized["metadata"] = metadata
|
||||
normalized["callback"] = callback
|
||||
return normalized
|
||||
|
||||
def validate_create_request(self, body: Dict[str, Any]):
|
||||
"""Validate the Manager create request before a runtime run is persisted."""
|
||||
normalized = self.normalize_create_request(body)
|
||||
plan = normalized.get("orchestration_plan")
|
||||
metadata = normalized.get("metadata")
|
||||
callback = normalized.get("callback")
|
||||
required_paths = {
|
||||
"orchestration_plan.objective": plan.get("objective") if isinstance(plan, dict) else None,
|
||||
"callback.url": callback.get("url") if isinstance(callback, dict) else None,
|
||||
"metadata.manager_deployment_id": metadata.get("manager_deployment_id") if isinstance(metadata, dict) else None,
|
||||
}
|
||||
missing = [path for path, value in required_paths.items() if value in (None, "", [], {})]
|
||||
if missing:
|
||||
raise RuntimeValidationError(f"Missing required field(s): {', '.join(missing)}")
|
||||
|
||||
if normalized.get("mode") not in (None, "swarm"):
|
||||
raise RuntimeValidationError("mode must be 'swarm'")
|
||||
if normalized.get("model_selection"):
|
||||
selection_type = (normalized.get("model_selection") or {}).get("type")
|
||||
if selection_type not in (None, "primary"):
|
||||
raise RuntimeValidationError("model_selection.type must be 'primary' for swarm")
|
||||
|
||||
billing_secret = (normalized.get("billing_context") or {}).get("secret_ref")
|
||||
if billing_secret and not self._is_azkv_ref(billing_secret):
|
||||
raise RuntimeValidationError("billing_context.secret_ref must use azkv://")
|
||||
|
||||
self._validate_resource_grants(normalized.get("resource_grants") or [], "resource_grants")
|
||||
for index, agent in enumerate(plan.get("agents") or []):
|
||||
self._validate_resource_grants(
|
||||
agent.get("resource_grants") or [],
|
||||
f"orchestration_plan.agents[{index}].resource_grants",
|
||||
)
|
||||
for index, agent in enumerate(normalized.get("agents") or []):
|
||||
self._validate_resource_grants(
|
||||
agent.get("resource_grants") or [],
|
||||
f"agents[{index}].resource_grants",
|
||||
)
|
||||
|
||||
for path in ("metadata", "resource_grants", "callback"):
|
||||
value = normalized.get(path)
|
||||
if value is not None:
|
||||
self._reject_plaintext_secrets(value, path)
|
||||
|
||||
async def get_or_create_run(
|
||||
self,
|
||||
body: Dict[str, Any],
|
||||
idempotency_key: Optional[str],
|
||||
correlation_id: Optional[str],
|
||||
) -> tuple[SwarmRun, bool]:
|
||||
"""Create a swarm run or return the previous run for an idempotency key."""
|
||||
body = self.normalize_create_request(body)
|
||||
if idempotency_key:
|
||||
existing_swarm_id = await redis_client.get(
|
||||
f"{self.IDEMPOTENCY_KEY_PREFIX}{idempotency_key}"
|
||||
)
|
||||
if existing_swarm_id:
|
||||
existing_run = await self.get_run(existing_swarm_id)
|
||||
if existing_run:
|
||||
return existing_run, False
|
||||
|
||||
metadata = body.get("metadata") or {}
|
||||
plan = body.get("orchestration_plan") or {}
|
||||
callback = CallbackConfig.model_validate(body.get("callback") or {})
|
||||
|
||||
swarm_id = f"swarm-{uuid.uuid4().hex[:12]}"
|
||||
deployment_id = f"runtime-dep-{uuid.uuid4().hex[:12]}"
|
||||
manager_deployment_id = (
|
||||
metadata.get("manager_deployment_id")
|
||||
or metadata.get("heicode_deployment_id")
|
||||
)
|
||||
objective = (
|
||||
plan.get("objective")
|
||||
or metadata.get("objective")
|
||||
or "Swarm runtime task"
|
||||
)
|
||||
|
||||
run = SwarmRun(
|
||||
deployment_id=deployment_id,
|
||||
swarm_id=swarm_id,
|
||||
mode="swarm",
|
||||
status="running",
|
||||
objective=objective,
|
||||
manager_deployment_id=manager_deployment_id,
|
||||
correlation_id=correlation_id or metadata.get("correlation_id"),
|
||||
callback=callback,
|
||||
metadata=self._redact_sensitive(metadata),
|
||||
request_body=self._redact_sensitive(body),
|
||||
)
|
||||
|
||||
if self._requires_approval(body):
|
||||
approval = self._build_approval(body, run)
|
||||
run.status = "waiting_approval"
|
||||
run.approvals[approval["approval_id"]] = approval
|
||||
|
||||
await self.save_run(run)
|
||||
|
||||
if idempotency_key:
|
||||
await redis_client.set(
|
||||
f"{self.IDEMPOTENCY_KEY_PREFIX}{idempotency_key}",
|
||||
run.swarm_id,
|
||||
ex=86400,
|
||||
)
|
||||
|
||||
await self.emit_event(
|
||||
run,
|
||||
"deployment.status_changed",
|
||||
payload=self.status_payload(run, phase="Plan"),
|
||||
)
|
||||
|
||||
for approval in run.approvals.values():
|
||||
await self.emit_event(run, "approval.requested", payload=approval)
|
||||
|
||||
return run, True
|
||||
|
||||
async def save_run(self, run: SwarmRun):
|
||||
"""Persist a swarm run."""
|
||||
run.updated_at = time.time()
|
||||
await redis_client.set(
|
||||
f"{self.RUN_KEY_PREFIX}{run.swarm_id}",
|
||||
run.model_dump_json(),
|
||||
)
|
||||
|
||||
async def get_run(self, swarm_id: str) -> Optional[SwarmRun]:
|
||||
"""Get a swarm run by id."""
|
||||
data = await redis_client.get(f"{self.RUN_KEY_PREFIX}{swarm_id}")
|
||||
if not data:
|
||||
return None
|
||||
return SwarmRun.model_validate_json(data)
|
||||
|
||||
async def get_run_by_identifier(self, identifier: str) -> Optional[SwarmRun]:
|
||||
"""Find a run by swarm id, runtime deployment id, or manager deployment id."""
|
||||
direct = await self.get_run(identifier)
|
||||
if direct:
|
||||
return direct
|
||||
|
||||
for key in await redis_client.keys(f"{self.RUN_KEY_PREFIX}*"):
|
||||
data = await redis_client.get(key)
|
||||
if not data:
|
||||
continue
|
||||
run = SwarmRun.model_validate_json(data)
|
||||
if identifier in {run.deployment_id, run.manager_deployment_id, run.swarm_id}:
|
||||
return run
|
||||
return None
|
||||
|
||||
async def get_run_for_task(self, task_id: str) -> Optional[SwarmRun]:
|
||||
"""Find the swarm run that owns a task."""
|
||||
swarm_id = await redis_client.get(f"{self.TASK_RUN_KEY_PREFIX}{task_id}")
|
||||
if not swarm_id:
|
||||
return None
|
||||
return await self.get_run(swarm_id)
|
||||
|
||||
async def attach_task(self, run: SwarmRun, task_id: str):
|
||||
"""Associate a queue task with a swarm run."""
|
||||
if task_id not in run.task_ids:
|
||||
run.task_ids.append(task_id)
|
||||
await redis_client.set(f"{self.TASK_RUN_KEY_PREFIX}{task_id}", run.swarm_id)
|
||||
await self.save_run(run)
|
||||
|
||||
async def stop_run(self, deployment_id: str, reason: str = "") -> Optional[SwarmRun]:
|
||||
"""Stop a run by runtime deployment id or swarm id."""
|
||||
run = await self.find_run_by_deployment_id(deployment_id)
|
||||
if not run:
|
||||
return None
|
||||
|
||||
run.status = "stopped"
|
||||
await self.save_run(run)
|
||||
await self.emit_event(
|
||||
run,
|
||||
"deployment.status_changed",
|
||||
payload=self.status_payload(
|
||||
run,
|
||||
phase="Deliver",
|
||||
reason=reason or "Heicode Manager requested stop",
|
||||
),
|
||||
)
|
||||
return run
|
||||
|
||||
async def find_run_by_deployment_id(self, deployment_id: str) -> Optional[SwarmRun]:
|
||||
"""Find a run by runtime deployment id or swarm id."""
|
||||
return await self.get_run_by_identifier(deployment_id)
|
||||
|
||||
async def record_approval_decision(
|
||||
self,
|
||||
swarm_id: str,
|
||||
approval_id: str,
|
||||
decision: Dict[str, Any],
|
||||
) -> Optional[SwarmRun]:
|
||||
"""Persist an approval decision and update run state."""
|
||||
run = await self.get_run_by_identifier(swarm_id)
|
||||
if not run:
|
||||
return None
|
||||
|
||||
approval = run.approvals.get(approval_id, {"approval_id": approval_id})
|
||||
approval["decision"] = decision.get("decision")
|
||||
credential_lease = decision.get("credential_lease") or {}
|
||||
approval["credential_ref"] = (
|
||||
decision.get("credential_ref") or credential_lease.get("credential_ref")
|
||||
)
|
||||
approval["lease_id"] = decision.get("lease_id") or credential_lease.get("lease_id")
|
||||
approval["lease_expires_at"] = (
|
||||
decision.get("lease_expires_at") or credential_lease.get("expires_at")
|
||||
)
|
||||
approval["decided_at"] = self._now_iso()
|
||||
run.approvals[approval_id] = self._redact_sensitive(approval)
|
||||
|
||||
if decision.get("decision") == "approved":
|
||||
run.status = "running"
|
||||
elif decision.get("decision") == "rejected":
|
||||
run.status = "blocked"
|
||||
|
||||
await self.save_run(run)
|
||||
await self.emit_event(
|
||||
run,
|
||||
"deployment.status_changed",
|
||||
payload=self.status_payload(
|
||||
run,
|
||||
phase="Review",
|
||||
approval_id=approval_id,
|
||||
decision=decision.get("decision"),
|
||||
reason=decision.get("reason"),
|
||||
),
|
||||
)
|
||||
if run.status == "blocked":
|
||||
await self.emit_event(run, "task.blocked", payload={
|
||||
"approval_id": approval_id,
|
||||
"reason": decision.get("reason") or "Approval rejected",
|
||||
"runtime_deployment_id": run.deployment_id,
|
||||
})
|
||||
await self.emit_event(run, "timeline.updated", payload={
|
||||
"summary": "Swarm blocked by approval rejection",
|
||||
"approval_id": approval_id,
|
||||
})
|
||||
return run
|
||||
|
||||
async def emit_event(
|
||||
self,
|
||||
run: SwarmRun,
|
||||
event_type: str,
|
||||
task_id: Optional[str] = None,
|
||||
agent_instance_id: Optional[str] = None,
|
||||
payload: Optional[Dict[str, Any]] = None,
|
||||
artifact: Optional[Dict[str, Any]] = None,
|
||||
):
|
||||
"""Record and optionally emit a Manager callback event."""
|
||||
callback = run.callback
|
||||
redacted_payload = self._redact_sensitive(payload or {})
|
||||
redacted_artifact = self._redact_sensitive(artifact) if artifact else None
|
||||
|
||||
if redacted_artifact and event_type == "artifact.created":
|
||||
redacted_payload = {
|
||||
**redacted_artifact,
|
||||
**redacted_payload,
|
||||
}
|
||||
|
||||
event_id = f"evt_{uuid.uuid4().hex}"
|
||||
body: Dict[str, Any] = {
|
||||
"event_id": event_id,
|
||||
"idempotency_key": event_id,
|
||||
"event_type": event_type,
|
||||
"deployment_id": run.manager_deployment_id or run.deployment_id,
|
||||
"runtime_deployment_id": run.deployment_id,
|
||||
"swarm_id": run.swarm_id,
|
||||
"agent_instance_id": agent_instance_id,
|
||||
"task_id": task_id,
|
||||
"occurred_at": self._now_iso(),
|
||||
"correlation_id": run.correlation_id,
|
||||
"source": self.runtime_source,
|
||||
"payload": redacted_payload,
|
||||
}
|
||||
if redacted_artifact:
|
||||
body["artifact"] = redacted_artifact
|
||||
|
||||
raw_body = json.dumps(body, ensure_ascii=False, separators=(",", ":"))
|
||||
await self._store_event(run.swarm_id, raw_body)
|
||||
|
||||
if not callback.url:
|
||||
return
|
||||
if callback.subscribed_events and event_type not in callback.subscribed_events:
|
||||
return
|
||||
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"X-Agent-Event-Id": event_id,
|
||||
"X-Agnet-Event-Id": event_id,
|
||||
}
|
||||
if run.correlation_id:
|
||||
headers["X-Correlation-ID"] = run.correlation_id
|
||||
if self.callback_service_token:
|
||||
headers["X-Agent-Service-Token"] = self.callback_service_token
|
||||
headers["X-Agnet-Service-Token"] = self.callback_service_token
|
||||
if self.callback_signing_secret:
|
||||
timestamp = str(int(time.time() * 1000))
|
||||
signature_payload = f"{timestamp}.{event_id}.{raw_body}"
|
||||
signature = hmac.new(
|
||||
self.callback_signing_secret.encode("utf-8"),
|
||||
signature_payload.encode("utf-8"),
|
||||
hashlib.sha256,
|
||||
).hexdigest()
|
||||
headers["X-Agent-Timestamp"] = timestamp
|
||||
headers["X-Agent-Signature"] = f"sha256={signature}"
|
||||
headers["X-Agnet-Timestamp"] = timestamp
|
||||
headers["X-Agnet-Signature"] = f"sha256={signature}"
|
||||
|
||||
if callback.url and (not callback.subscribed_events or event_type in callback.subscribed_events):
|
||||
await self._upsert_callback_attempt(
|
||||
run.swarm_id,
|
||||
event_id,
|
||||
{
|
||||
"event_id": event_id,
|
||||
"event_type": event_type,
|
||||
"url": callback.url,
|
||||
"status": "pending",
|
||||
"attempted_at": self._now_iso(),
|
||||
},
|
||||
)
|
||||
asyncio.create_task(
|
||||
self._post_callback(run.swarm_id, callback.url, raw_body, headers, event_type, event_id)
|
||||
)
|
||||
|
||||
async def list_events(
|
||||
self,
|
||||
swarm_id: str,
|
||||
limit: int = 100,
|
||||
cursor: Optional[str] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Return stored runtime events for a swarm."""
|
||||
start = int(cursor or 0)
|
||||
safe_limit = max(1, min(limit, 500))
|
||||
items = await redis_client.lrange(
|
||||
f"{self.EVENT_KEY_PREFIX}{swarm_id}",
|
||||
start,
|
||||
start + safe_limit - 1,
|
||||
)
|
||||
events = [json.loads(item) for item in items]
|
||||
next_cursor = str(start + safe_limit) if len(events) == safe_limit else None
|
||||
return {"events": events, "next_cursor": next_cursor}
|
||||
|
||||
async def _store_event(self, swarm_id: str, raw_body: str):
|
||||
await redis_client.rpush(f"{self.EVENT_KEY_PREFIX}{swarm_id}", raw_body)
|
||||
|
||||
async def _post_callback(
|
||||
self,
|
||||
swarm_id: str,
|
||||
url: str,
|
||||
raw_body: str,
|
||||
headers: Dict[str, str],
|
||||
event_type: str,
|
||||
event_id: str,
|
||||
):
|
||||
"""Send callback without blocking the agent WebSocket loop."""
|
||||
if not url:
|
||||
return
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
response = await client.post(url, content=raw_body, headers=headers)
|
||||
await self._upsert_callback_attempt(
|
||||
swarm_id,
|
||||
event_id,
|
||||
{
|
||||
"event_id": event_id,
|
||||
"event_type": event_type,
|
||||
"url": url,
|
||||
"status": "delivered" if response.status_code < 400 else "failed",
|
||||
"response_status": response.status_code,
|
||||
"attempted_at": self._now_iso(),
|
||||
},
|
||||
)
|
||||
if response.status_code >= 400:
|
||||
logger.warning(
|
||||
"Manager callback %s failed with status %s: %s",
|
||||
event_type,
|
||||
response.status_code,
|
||||
response.text[:500],
|
||||
)
|
||||
except Exception as exc:
|
||||
await self._upsert_callback_attempt(
|
||||
swarm_id,
|
||||
event_id,
|
||||
{
|
||||
"event_id": event_id,
|
||||
"event_type": event_type,
|
||||
"url": url,
|
||||
"status": "failed",
|
||||
"error": str(exc),
|
||||
"attempted_at": self._now_iso(),
|
||||
},
|
||||
)
|
||||
logger.warning("Manager callback %s failed: %s", event_type, exc)
|
||||
|
||||
def build_task_descriptions(self, body: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
"""Create a simple task graph from a Manager orchestration request."""
|
||||
body = self.normalize_create_request(body)
|
||||
plan = body.get("orchestration_plan") or {}
|
||||
agents = plan.get("agents") or body.get("agents") or []
|
||||
requirement = body.get("requirement") or {}
|
||||
objective = (
|
||||
requirement.get("objective")
|
||||
or plan.get("objective")
|
||||
or "Complete swarm objective"
|
||||
)
|
||||
multi_agent_enabled = self.multi_agent_workflow_enabled(body)
|
||||
context = {
|
||||
"orchestration_plan": self._redact_sensitive(plan),
|
||||
"resource_grants": self._redact_sensitive(body.get("resource_grants") or []),
|
||||
"sub_mode": body.get("sub_mode") or plan.get("sub_mode"),
|
||||
"conversation_id": body.get("conversation_id"),
|
||||
"mode": body.get("mode") or "swarm",
|
||||
"requirement": self._redact_sensitive(requirement),
|
||||
}
|
||||
|
||||
if not agents or not multi_agent_enabled:
|
||||
return [{
|
||||
"task_id": "task-1",
|
||||
"title": "Swarm objective",
|
||||
"description": objective,
|
||||
"agent_role": "general",
|
||||
"required_capabilities": ["general"],
|
||||
"depends_on": [],
|
||||
"parent_task_id": None,
|
||||
"root_task_id": "task-1",
|
||||
"source": "runtime_bridge",
|
||||
"workflow_mode": "single_agent",
|
||||
"allow_handoff": False,
|
||||
"context": context,
|
||||
}]
|
||||
|
||||
tasks = []
|
||||
for index, agent in enumerate(agents, start=1):
|
||||
role = agent.get("role") or f"agent-{index}"
|
||||
title = agent.get("title") or f"{role} task"
|
||||
description = agent.get("description") or f"[{role}] {objective}"
|
||||
task_id = agent.get("task_id") or f"task-{index}"
|
||||
required_capabilities = agent.get("required_capabilities") or [role]
|
||||
tasks.append({
|
||||
"task_id": task_id,
|
||||
"title": title,
|
||||
"description": description,
|
||||
"agent_role": role,
|
||||
"required_capabilities": required_capabilities,
|
||||
"depends_on": agent.get("depends_on") or [],
|
||||
"parent_task_id": None,
|
||||
"root_task_id": task_id,
|
||||
"source": "runtime_bridge",
|
||||
"workflow_mode": "multi_agent",
|
||||
"allow_handoff": True,
|
||||
"context": {
|
||||
**context,
|
||||
"agent_role": role,
|
||||
"workflow_mode": "multi_agent",
|
||||
"resource_grants": self._redact_sensitive(
|
||||
agent.get("resource_grants") or []
|
||||
),
|
||||
},
|
||||
})
|
||||
return tasks
|
||||
|
||||
def task_event_payload(self, task: Any, task_spec: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Return the minimum task graph payload required by Manager."""
|
||||
return {
|
||||
"task_id": task.task_id,
|
||||
"title": task.title or task_spec.get("title", task.description[:80]),
|
||||
"description": task.description,
|
||||
"agent_role": task.agent_role or task_spec.get("agent_role", "general"),
|
||||
"status": task.status.value if hasattr(task.status, "value") else task.status,
|
||||
"depends_on": task.depends_on or task_spec.get("depends_on") or [],
|
||||
"parent_task_id": task.parent_task_id or task_spec.get("parent_task_id"),
|
||||
"root_task_id": task.root_task_id or task_spec.get("root_task_id"),
|
||||
"required_capabilities": task.required_capabilities or task_spec.get("required_capabilities") or [],
|
||||
"source": task.source or task_spec.get("source", "runtime_bridge"),
|
||||
"attempt": getattr(task, "retry_count", 0),
|
||||
}
|
||||
|
||||
def _requires_approval(self, body: Dict[str, Any]) -> bool:
|
||||
plan = body.get("orchestration_plan") or {}
|
||||
agile = plan.get("agile_context") or {}
|
||||
risk = str(plan.get("risk_level") or body.get("risk_level") or "").lower()
|
||||
return bool(agile.get("requires_user_approval")) or risk == "high"
|
||||
|
||||
def status_payload(
|
||||
self,
|
||||
run: SwarmRun,
|
||||
phase: str,
|
||||
reason: Optional[str] = None,
|
||||
approval_id: Optional[str] = None,
|
||||
decision: Optional[str] = None,
|
||||
deliverable: Optional[Dict[str, Any]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Build the standard runtime status payload."""
|
||||
payload: Dict[str, Any] = {
|
||||
"status": run.status,
|
||||
"runtime_execution_status": run.status,
|
||||
"runtime_deployment_id": run.deployment_id,
|
||||
"manager_deployment_id": run.manager_deployment_id,
|
||||
"phase": phase,
|
||||
}
|
||||
if reason:
|
||||
payload["reason"] = reason
|
||||
if approval_id:
|
||||
payload["approval_id"] = approval_id
|
||||
if decision:
|
||||
payload["decision"] = decision
|
||||
if deliverable is not None:
|
||||
payload["deliverable"] = deliverable
|
||||
return payload
|
||||
|
||||
async def _upsert_callback_attempt(
|
||||
self,
|
||||
swarm_id: str,
|
||||
event_id: str,
|
||||
attempt: Dict[str, Any],
|
||||
):
|
||||
run = await self.get_run(swarm_id)
|
||||
if not run:
|
||||
return
|
||||
callback_attempts = list(run.metadata.get("callback_attempts") or [])
|
||||
filtered = [item for item in callback_attempts if item.get("event_id") != event_id]
|
||||
filtered.append(self._redact_sensitive(attempt))
|
||||
run.metadata["callback_attempts"] = filtered[-50:]
|
||||
await self.save_run(run)
|
||||
|
||||
def _build_approval(self, body: Dict[str, Any], run: SwarmRun) -> Dict[str, Any]:
|
||||
grants = body.get("resource_grants") or []
|
||||
grant = grants[0] if grants else {}
|
||||
return {
|
||||
"approval_id": f"appr_{uuid.uuid4().hex[:12]}",
|
||||
"operation": "git.write",
|
||||
"resource_id": grant.get("resource_id", "repo-main"),
|
||||
"resource_type": grant.get("resource_type", "git"),
|
||||
"resource_scope": ",".join(grant.get("permission_scope") or []),
|
||||
"target_role": grant.get("target_role", "general"),
|
||||
"risk_level": "high",
|
||||
"requires_credential": True,
|
||||
"secret_ref": grant.get("secret_ref") or grant.get("ref"),
|
||||
"ttl_seconds": 900,
|
||||
"reason": "High-risk swarm run requires Manager approval",
|
||||
"runtime_deployment_id": run.deployment_id,
|
||||
}
|
||||
|
||||
def _validate_resource_grants(self, grants: List[Dict[str, Any]], path: str):
|
||||
for index, grant in enumerate(grants):
|
||||
for key in ("secret_ref", "ref"):
|
||||
value = grant.get(key)
|
||||
if value and not self._is_azkv_ref(value):
|
||||
raise RuntimeValidationError(f"{path}[{index}].{key} must use azkv://")
|
||||
|
||||
def _reject_plaintext_secrets(self, value: Any, path: str):
|
||||
if isinstance(value, dict):
|
||||
for key, item in value.items():
|
||||
child_path = f"{path}.{key}"
|
||||
key_lower = key.lower()
|
||||
is_ref_key = key_lower.endswith("_ref") or key_lower == "ref"
|
||||
if self._looks_sensitive_key(key_lower) and not is_ref_key:
|
||||
raise RuntimeValidationError(
|
||||
f"Plaintext secret-like field is not allowed: {child_path}"
|
||||
)
|
||||
if is_ref_key and isinstance(item, str) and item and not self._is_azkv_ref(item):
|
||||
raise RuntimeValidationError(f"{child_path} must use azkv://")
|
||||
self._reject_plaintext_secrets(item, child_path)
|
||||
elif isinstance(value, list):
|
||||
for index, item in enumerate(value):
|
||||
self._reject_plaintext_secrets(item, f"{path}[{index}]")
|
||||
|
||||
def _looks_sensitive_key(self, key: str) -> bool:
|
||||
return (
|
||||
any(part in key for part in ("token", "password", "passwd", "secret", "private_key", "api_key"))
|
||||
or key.endswith("_key")
|
||||
)
|
||||
|
||||
def _is_azkv_ref(self, value: Any) -> bool:
|
||||
return isinstance(value, str) and value.startswith("azkv://")
|
||||
|
||||
def _redact_sensitive(self, value: Any) -> Any:
|
||||
"""Remove obvious plaintext secrets from callback-safe payloads."""
|
||||
sensitive_keys = {
|
||||
"password",
|
||||
"passwd",
|
||||
"token",
|
||||
"api_token",
|
||||
"access_token",
|
||||
"refresh_token",
|
||||
"private_key",
|
||||
"access_key",
|
||||
"secret",
|
||||
"client_secret",
|
||||
"connection_string",
|
||||
}
|
||||
if isinstance(value, dict):
|
||||
redacted = {}
|
||||
for key, item in value.items():
|
||||
key_lower = key.lower()
|
||||
if key_lower in {"secret_ref", "credential_ref", "signing_secret_ref"}:
|
||||
redacted[key] = item
|
||||
elif key_lower in sensitive_keys or key_lower.endswith(("_token", "_secret", "_password", "_key")):
|
||||
redacted[key] = "[redacted]"
|
||||
else:
|
||||
redacted[key] = self._redact_sensitive(item)
|
||||
return redacted
|
||||
if isinstance(value, list):
|
||||
return [self._redact_sensitive(item) for item in value]
|
||||
return value
|
||||
|
||||
def _now_iso(self) -> str:
|
||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
swarm_runtime = SwarmRuntime()
|
||||
@@ -0,0 +1,511 @@
|
||||
"""Task queue management with dependency-aware failure recovery."""
|
||||
import json
|
||||
import time
|
||||
import uuid
|
||||
import logging
|
||||
from typing import Dict, List, Optional
|
||||
from enum import Enum
|
||||
from pydantic import BaseModel, Field
|
||||
from .redis_client import redis_client
|
||||
from .agent_registry import agent_registry, AgentStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TaskStatus(str, Enum):
|
||||
"""Task status enumeration."""
|
||||
PENDING = "pending"
|
||||
ASSIGNED = "assigned"
|
||||
IN_PROGRESS = "in_progress"
|
||||
BLOCKED = "blocked"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
CANCELLED = "cancelled"
|
||||
|
||||
|
||||
class Task(BaseModel):
|
||||
"""Task model."""
|
||||
task_id: str
|
||||
title: Optional[str] = None
|
||||
description: str
|
||||
status: TaskStatus
|
||||
agent_role: str = "general"
|
||||
required_capabilities: List[str] = Field(default_factory=list)
|
||||
depends_on: List[str] = Field(default_factory=list)
|
||||
parent_task_id: Optional[str] = None
|
||||
root_task_id: Optional[str] = None
|
||||
source: str = "manual"
|
||||
assigned_agent_id: Optional[str] = None
|
||||
created_at: float
|
||||
started_at: Optional[float] = None
|
||||
completed_at: Optional[float] = None
|
||||
result: Optional[str] = None
|
||||
blocked_reason: Optional[str] = None
|
||||
child_task_ids: List[str] = Field(default_factory=list)
|
||||
retry_count: int = 0
|
||||
max_retries: int = 3
|
||||
context: Dict = Field(default_factory=dict)
|
||||
|
||||
|
||||
class TaskQueue:
|
||||
"""Manages task assignment and reassignment with failure recovery."""
|
||||
|
||||
TASK_KEY_PREFIX = "task:"
|
||||
PENDING_QUEUE_KEY = "queue:pending"
|
||||
AGENT_TASK_KEY_PREFIX = "agent_task:"
|
||||
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
async def _save_task(self, task: Task):
|
||||
"""Persist a task snapshot."""
|
||||
key = f"{self.TASK_KEY_PREFIX}{task.task_id}"
|
||||
await redis_client.set(key, task.model_dump_json())
|
||||
|
||||
async def create_task(
|
||||
self,
|
||||
description: str,
|
||||
context: Optional[Dict] = None,
|
||||
max_retries: int = 3,
|
||||
task_id: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
agent_role: str = "general",
|
||||
required_capabilities: Optional[List[str]] = None,
|
||||
depends_on: Optional[List[str]] = None,
|
||||
parent_task_id: Optional[str] = None,
|
||||
root_task_id: Optional[str] = None,
|
||||
source: str = "manual",
|
||||
enqueue: bool = True,
|
||||
) -> Task:
|
||||
"""Create a new task and add to pending queue."""
|
||||
task = Task(
|
||||
task_id=task_id or str(uuid.uuid4()),
|
||||
title=title,
|
||||
description=description,
|
||||
status=TaskStatus.PENDING,
|
||||
agent_role=agent_role,
|
||||
required_capabilities=required_capabilities or [],
|
||||
depends_on=depends_on or [],
|
||||
parent_task_id=parent_task_id,
|
||||
root_task_id=root_task_id,
|
||||
source=source,
|
||||
created_at=time.time(),
|
||||
max_retries=max_retries,
|
||||
context=context or {},
|
||||
)
|
||||
|
||||
await self._save_task(task)
|
||||
|
||||
if enqueue:
|
||||
await redis_client.lpush(self.PENDING_QUEUE_KEY, task.task_id)
|
||||
|
||||
logger.info(f"Created task {task.task_id}: {description}")
|
||||
return task
|
||||
|
||||
async def add_child_task(self, parent_task_id: str, child_task_id: str):
|
||||
"""Register a child task on a parent task."""
|
||||
parent = await self.get_task(parent_task_id)
|
||||
if not parent:
|
||||
return
|
||||
if child_task_id not in parent.child_task_ids:
|
||||
parent.child_task_ids.append(child_task_id)
|
||||
await self._save_task(parent)
|
||||
|
||||
async def get_ready_pending_task(
|
||||
self,
|
||||
agent_capabilities: Optional[List[str]] = None,
|
||||
) -> Optional[Task]:
|
||||
"""Return and dequeue the next dispatchable task for an agent."""
|
||||
pending_ids = await redis_client.lrange(self.PENDING_QUEUE_KEY, 0, -1)
|
||||
capabilities = set(agent_capabilities or [])
|
||||
|
||||
for task_id in pending_ids:
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
await self.remove_pending_task(task_id)
|
||||
continue
|
||||
if task.status != TaskStatus.PENDING:
|
||||
await self.remove_pending_task(task_id)
|
||||
continue
|
||||
if not await self.is_task_ready(task):
|
||||
continue
|
||||
if not self.can_agent_run_task(task, capabilities):
|
||||
continue
|
||||
|
||||
await self.remove_pending_task(task_id)
|
||||
return task
|
||||
|
||||
return None
|
||||
|
||||
async def is_task_ready(self, task: Task) -> bool:
|
||||
"""Return True when all dependencies are terminal and successful."""
|
||||
if task.status != TaskStatus.PENDING:
|
||||
return False
|
||||
|
||||
for dependency_id in task.depends_on:
|
||||
dependency = await self.get_task(dependency_id)
|
||||
if not dependency or dependency.status != TaskStatus.COMPLETED:
|
||||
return False
|
||||
return True
|
||||
|
||||
def can_agent_run_task(self, task: Task, capabilities: set[str]) -> bool:
|
||||
"""Return whether an agent capability set satisfies task requirements."""
|
||||
required = set(task.required_capabilities or [])
|
||||
if not required:
|
||||
return True
|
||||
return required.issubset(capabilities)
|
||||
|
||||
async def assign_task(self, task_id: str, agent_id: str) -> bool:
|
||||
"""Assign a task to an agent."""
|
||||
# Get task
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
logger.error(f"Task {task_id} not found")
|
||||
return False
|
||||
|
||||
if task.status not in [TaskStatus.PENDING, TaskStatus.FAILED]:
|
||||
logger.error(f"Task {task_id} cannot be assigned (status: {task.status})")
|
||||
return False
|
||||
if task.status == TaskStatus.PENDING and not await self.is_task_ready(task):
|
||||
logger.error(f"Task {task_id} cannot be assigned before dependencies complete")
|
||||
return False
|
||||
|
||||
# Verify agent is idle
|
||||
agent = await agent_registry.get_agent(agent_id)
|
||||
if not agent or agent.status != AgentStatus.IDLE:
|
||||
logger.error(f"Agent {agent_id} is not available for task assignment")
|
||||
return False
|
||||
if not self.can_agent_run_task(task, set(agent.capabilities)):
|
||||
logger.error(f"Agent {agent_id} does not satisfy task {task_id} capabilities")
|
||||
return False
|
||||
|
||||
# Update task
|
||||
task.status = TaskStatus.ASSIGNED
|
||||
task.assigned_agent_id = agent_id
|
||||
task.started_at = time.time()
|
||||
task.blocked_reason = None
|
||||
|
||||
await self._save_task(task)
|
||||
|
||||
# Track agent's current task
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{agent_id}"
|
||||
await redis_client.set(agent_task_key, task_id)
|
||||
|
||||
# Update agent status
|
||||
await agent_registry.update_status(agent_id, AgentStatus.BUSY, task_id)
|
||||
|
||||
logger.info(f"Assigned task {task_id} to agent {agent_id}")
|
||||
return True
|
||||
|
||||
async def start_task(self, task_id: str) -> bool:
|
||||
"""Mark task as in progress."""
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
return False
|
||||
|
||||
task.status = TaskStatus.IN_PROGRESS
|
||||
task.blocked_reason = None
|
||||
await self._save_task(task)
|
||||
|
||||
logger.info(f"Task {task_id} started")
|
||||
return True
|
||||
|
||||
async def block_task(
|
||||
self,
|
||||
task_id: str,
|
||||
reason: str = "",
|
||||
release_agent: bool = True,
|
||||
) -> Optional[Task]:
|
||||
"""Mark a task blocked while waiting for delegated child work."""
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
return None
|
||||
if task.status in {TaskStatus.COMPLETED, TaskStatus.FAILED, TaskStatus.CANCELLED}:
|
||||
return task
|
||||
|
||||
previous_agent_id = task.assigned_agent_id
|
||||
task.status = TaskStatus.BLOCKED
|
||||
task.blocked_reason = reason or "Waiting for delegated handoff work"
|
||||
task.assigned_agent_id = None if release_agent else task.assigned_agent_id
|
||||
|
||||
await self.remove_pending_task(task_id)
|
||||
await self._save_task(task)
|
||||
|
||||
if release_agent and previous_agent_id:
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{previous_agent_id}"
|
||||
await redis_client.delete(agent_task_key)
|
||||
await agent_registry.update_status(previous_agent_id, AgentStatus.IDLE)
|
||||
|
||||
logger.info(f"Task {task_id} blocked: {task.blocked_reason}")
|
||||
return task
|
||||
|
||||
async def complete_task(self, task_id: str, result: str = None) -> bool:
|
||||
"""Mark task as completed."""
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
return False
|
||||
if task.status == TaskStatus.CANCELLED:
|
||||
logger.warning(f"Ignoring completion for cancelled task {task_id}")
|
||||
return False
|
||||
|
||||
task.status = TaskStatus.COMPLETED
|
||||
task.completed_at = time.time()
|
||||
task.blocked_reason = None
|
||||
if result:
|
||||
task.result = result
|
||||
|
||||
await self._save_task(task)
|
||||
|
||||
# Clear agent's current task
|
||||
if task.assigned_agent_id:
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
|
||||
await redis_client.delete(agent_task_key)
|
||||
|
||||
# Update agent to idle
|
||||
await agent_registry.update_status(task.assigned_agent_id, AgentStatus.IDLE)
|
||||
|
||||
logger.info(f"Task {task_id} completed")
|
||||
return True
|
||||
|
||||
async def fail_task(self, task_id: str, reason: str = "") -> bool:
|
||||
"""Mark task as failed and handle retry logic."""
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
return False
|
||||
if task.status == TaskStatus.CANCELLED:
|
||||
logger.warning(f"Ignoring failure for cancelled task {task_id}: {reason}")
|
||||
return False
|
||||
|
||||
task.retry_count += 1
|
||||
|
||||
# Clear agent's current task
|
||||
if task.assigned_agent_id:
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
|
||||
await redis_client.delete(agent_task_key)
|
||||
|
||||
# Update agent to idle
|
||||
await agent_registry.update_status(task.assigned_agent_id, AgentStatus.IDLE)
|
||||
|
||||
# Check if we should retry
|
||||
if task.retry_count < task.max_retries:
|
||||
task.status = TaskStatus.PENDING
|
||||
task.assigned_agent_id = None
|
||||
task.started_at = None
|
||||
|
||||
# Re-add to pending queue
|
||||
await redis_client.lpush(self.PENDING_QUEUE_KEY, task_id)
|
||||
|
||||
logger.warning(
|
||||
f"Task {task_id} failed (retry {task.retry_count}/{task.max_retries}): {reason}"
|
||||
)
|
||||
else:
|
||||
task.status = TaskStatus.FAILED
|
||||
task.completed_at = time.time()
|
||||
|
||||
logger.error(
|
||||
f"Task {task_id} permanently failed after {task.retry_count} retries: {reason}"
|
||||
)
|
||||
|
||||
task.blocked_reason = None if task.status == TaskStatus.PENDING else task.blocked_reason
|
||||
await self._save_task(task)
|
||||
|
||||
return True
|
||||
|
||||
async def cancel_task(self, task_id: str, reason: str = "") -> bool:
|
||||
"""Cancel a task without retrying it."""
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
return False
|
||||
if task.status in {TaskStatus.COMPLETED, TaskStatus.FAILED, TaskStatus.CANCELLED}:
|
||||
logger.info(f"Task {task_id} already terminal ({task.status}); skip cancellation")
|
||||
return False
|
||||
|
||||
if task.assigned_agent_id:
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
|
||||
await redis_client.delete(agent_task_key)
|
||||
|
||||
await self.remove_pending_task(task_id)
|
||||
task.status = TaskStatus.CANCELLED
|
||||
task.completed_at = time.time()
|
||||
task.blocked_reason = None
|
||||
if reason:
|
||||
task.result = json.dumps({"cancelled": True, "reason": reason})
|
||||
|
||||
await self._save_task(task)
|
||||
logger.info(f"Task {task_id} cancelled: {reason}")
|
||||
return True
|
||||
|
||||
async def release_task(self, task_id: str, agent_id: Optional[str] = None) -> bool:
|
||||
"""Return an assigned/in-progress task to the pending queue (e.g. agent rejected it).
|
||||
|
||||
Unlike fail_task this does not increment retry_count: a capacity rejection is not
|
||||
a task failure, just a dispatch that needs to find a different agent.
|
||||
"""
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
return False
|
||||
if task.status not in {TaskStatus.ASSIGNED, TaskStatus.IN_PROGRESS}:
|
||||
return False
|
||||
if agent_id and task.assigned_agent_id and task.assigned_agent_id != agent_id:
|
||||
return False
|
||||
|
||||
previous_agent_id = task.assigned_agent_id
|
||||
task.status = TaskStatus.PENDING
|
||||
task.assigned_agent_id = None
|
||||
task.started_at = None
|
||||
task.blocked_reason = None
|
||||
await self._save_task(task)
|
||||
await self.requeue_task(task_id)
|
||||
|
||||
if previous_agent_id:
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{previous_agent_id}"
|
||||
await redis_client.delete(agent_task_key)
|
||||
await agent_registry.update_status(previous_agent_id, AgentStatus.IDLE)
|
||||
|
||||
logger.info(f"Released task {task_id} back to pending queue")
|
||||
return True
|
||||
|
||||
async def reopen_task(self, task_id: str) -> bool:
|
||||
"""Re-open a completed task for another round (review loop rejected its result).
|
||||
|
||||
Resets the task to PENDING and requeues it without touching retry_count (a review
|
||||
rejection is a quality decision, not a failure). The review-cycle budget in the
|
||||
orchestrator bounds how many times this can happen.
|
||||
"""
|
||||
task = await self.get_task(task_id)
|
||||
if not task:
|
||||
return False
|
||||
if task.status not in {TaskStatus.COMPLETED, TaskStatus.FAILED}:
|
||||
return False
|
||||
|
||||
if task.assigned_agent_id:
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
|
||||
await redis_client.delete(agent_task_key)
|
||||
|
||||
task.status = TaskStatus.PENDING
|
||||
task.assigned_agent_id = None
|
||||
task.started_at = None
|
||||
task.completed_at = None
|
||||
task.blocked_reason = None
|
||||
await self._save_task(task)
|
||||
await self.requeue_task(task_id)
|
||||
logger.info(f"Re-opened task {task_id} for another review cycle")
|
||||
return True
|
||||
|
||||
async def reassign_agent_tasks(self, failed_agent_id: str) -> List[str]:
|
||||
"""Reassign all tasks from a failed agent back to pending queue."""
|
||||
# Get agent's current task
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{failed_agent_id}"
|
||||
task_id = await redis_client.get(agent_task_key)
|
||||
|
||||
reassigned_tasks = []
|
||||
|
||||
if task_id:
|
||||
await self.fail_task(task_id, f"Agent {failed_agent_id} failed")
|
||||
reassigned_tasks.append(task_id)
|
||||
|
||||
logger.info(
|
||||
f"Reassigned {len(reassigned_tasks)} tasks from failed agent {failed_agent_id}"
|
||||
)
|
||||
return reassigned_tasks
|
||||
|
||||
async def recover_orphaned_tasks(
|
||||
self,
|
||||
active_agent_ids: set[str],
|
||||
stale_after_seconds: int = 30,
|
||||
) -> List[str]:
|
||||
"""Recover assigned/in-progress tasks whose agents are no longer connected."""
|
||||
current_time = time.time()
|
||||
recoverable_statuses = {TaskStatus.ASSIGNED, TaskStatus.IN_PROGRESS}
|
||||
recovered_tasks = []
|
||||
|
||||
for task in await self.get_all_tasks():
|
||||
if task.status not in recoverable_statuses:
|
||||
continue
|
||||
|
||||
if not task.assigned_agent_id:
|
||||
continue
|
||||
|
||||
if task.assigned_agent_id in active_agent_ids:
|
||||
continue
|
||||
|
||||
if task.started_at and current_time - task.started_at < stale_after_seconds:
|
||||
continue
|
||||
|
||||
old_agent_id = task.assigned_agent_id
|
||||
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{old_agent_id}"
|
||||
await redis_client.delete(agent_task_key)
|
||||
|
||||
task.status = TaskStatus.PENDING
|
||||
task.assigned_agent_id = None
|
||||
task.started_at = None
|
||||
|
||||
if await self.is_task_ready(task):
|
||||
await self._save_task(task)
|
||||
await redis_client.lpush(self.PENDING_QUEUE_KEY, task.task_id)
|
||||
else:
|
||||
await self._save_task(task)
|
||||
|
||||
recovered_tasks.append(task.task_id)
|
||||
logger.warning(
|
||||
f"Recovered orphaned task {task.task_id} from inactive agent {old_agent_id}"
|
||||
)
|
||||
|
||||
return recovered_tasks
|
||||
|
||||
async def get_task(self, task_id: str) -> Optional[Task]:
|
||||
"""Get task by ID."""
|
||||
key = f"{self.TASK_KEY_PREFIX}{task_id}"
|
||||
data = await redis_client.get(key)
|
||||
|
||||
if not data:
|
||||
return None
|
||||
|
||||
return Task.model_validate_json(data)
|
||||
|
||||
async def get_next_pending_task(self) -> Optional[Task]:
|
||||
"""Get next pending task from queue."""
|
||||
task_id = await redis_client.rpop(self.PENDING_QUEUE_KEY)
|
||||
|
||||
if not task_id:
|
||||
return None
|
||||
|
||||
return await self.get_task(task_id)
|
||||
|
||||
async def requeue_task(self, task_id: str):
|
||||
"""Put a task back on the pending queue."""
|
||||
await redis_client.rpush(self.PENDING_QUEUE_KEY, task_id)
|
||||
|
||||
async def remove_pending_task(self, task_id: str):
|
||||
"""Remove a task from the pending queue if present."""
|
||||
await redis_client.lrem(self.PENDING_QUEUE_KEY, 0, task_id)
|
||||
|
||||
async def get_pending_count(self) -> int:
|
||||
"""Get count of pending tasks."""
|
||||
return await redis_client.llen(self.PENDING_QUEUE_KEY)
|
||||
|
||||
async def get_dependents(self, task_id: str) -> List[Task]:
|
||||
"""Return tasks that directly depend on a task."""
|
||||
return [
|
||||
task for task in await self.get_all_tasks()
|
||||
if task_id in task.depends_on
|
||||
]
|
||||
|
||||
async def get_all_tasks(self, status: Optional[TaskStatus] = None) -> List[Task]:
|
||||
"""Get all tasks, optionally filtered by status."""
|
||||
pattern = f"{self.TASK_KEY_PREFIX}*"
|
||||
keys = await redis_client.keys(pattern)
|
||||
|
||||
tasks = []
|
||||
for key in keys:
|
||||
data = await redis_client.get(key)
|
||||
if data:
|
||||
task = Task.model_validate_json(data)
|
||||
if status is None or task.status == status:
|
||||
tasks.append(task)
|
||||
|
||||
return tasks
|
||||
|
||||
|
||||
# Global task queue instance
|
||||
task_queue = TaskQueue()
|
||||
@@ -0,0 +1,415 @@
|
||||
"""Regression tests for the multi-agent DAG workflow."""
|
||||
import fnmatch
|
||||
import os
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from orchestrator import main as orchestrator_main
|
||||
from orchestrator import swarm_runtime as swarm_runtime_module
|
||||
from orchestrator import task_queue as task_queue_module
|
||||
|
||||
|
||||
class FakeRedisClient:
|
||||
"""Tiny in-memory async Redis replacement for task-queue tests."""
|
||||
|
||||
def __init__(self):
|
||||
self.kv = {}
|
||||
self.lists = {}
|
||||
|
||||
async def set(self, key, value, ex=None):
|
||||
self.kv[key] = value
|
||||
|
||||
async def get(self, key):
|
||||
return self.kv.get(key)
|
||||
|
||||
async def delete(self, key):
|
||||
self.kv.pop(key, None)
|
||||
self.lists.pop(key, None)
|
||||
|
||||
async def keys(self, pattern):
|
||||
keys = list(self.kv.keys()) + list(self.lists.keys())
|
||||
return sorted([key for key in keys if fnmatch.fnmatch(key, pattern)])
|
||||
|
||||
async def lpush(self, key, *values):
|
||||
self.lists.setdefault(key, [])
|
||||
for value in values:
|
||||
self.lists[key].insert(0, value)
|
||||
|
||||
async def rpush(self, key, *values):
|
||||
self.lists.setdefault(key, [])
|
||||
self.lists[key].extend(values)
|
||||
|
||||
async def rpop(self, key):
|
||||
items = self.lists.get(key, [])
|
||||
if not items:
|
||||
return None
|
||||
return items.pop()
|
||||
|
||||
async def llen(self, key):
|
||||
return len(self.lists.get(key, []))
|
||||
|
||||
async def lrange(self, key, start, end):
|
||||
items = list(self.lists.get(key, []))
|
||||
if end == -1:
|
||||
end = len(items) - 1
|
||||
return items[start : end + 1]
|
||||
|
||||
async def lrem(self, key, count, value):
|
||||
items = self.lists.get(key, [])
|
||||
removed = 0
|
||||
kept = []
|
||||
for item in items:
|
||||
if item == value and (count == 0 or removed < count):
|
||||
removed += 1
|
||||
continue
|
||||
kept.append(item)
|
||||
self.lists[key] = kept
|
||||
return removed
|
||||
|
||||
|
||||
class MultiAgentWorkflowTests(unittest.IsolatedAsyncioTestCase):
|
||||
"""Covers DAG task creation, dependency dispatch, and parent-child resolution."""
|
||||
|
||||
def setUp(self):
|
||||
self.fake_redis = FakeRedisClient()
|
||||
self.redis_patch = patch.object(task_queue_module, "redis_client", self.fake_redis)
|
||||
self.redis_patch.start()
|
||||
self.addCleanup(self.redis_patch.stop)
|
||||
|
||||
async def test_build_task_descriptions_respects_feature_flag(self):
|
||||
body = {
|
||||
"orchestration_plan": {
|
||||
"objective": "Implement a feature",
|
||||
"sub_mode": "code",
|
||||
"risk_level": "low",
|
||||
"budget": {"duration_seconds": 600, "token_limit": 1000},
|
||||
"agents": [
|
||||
{
|
||||
"task_id": "backend-1",
|
||||
"role": "backend",
|
||||
"title": "Backend task",
|
||||
"description": "Implement backend changes",
|
||||
"depends_on": [],
|
||||
},
|
||||
{
|
||||
"task_id": "tests-1",
|
||||
"role": "testing",
|
||||
"title": "Testing task",
|
||||
"description": "Add tests",
|
||||
"depends_on": ["backend-1"],
|
||||
},
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
runtime = swarm_runtime_module.SwarmRuntime()
|
||||
with patch.dict(os.environ, {"ENABLE_SUBTASK_HANDOFF": "false"}, clear=False):
|
||||
tasks = runtime.build_task_descriptions(body)
|
||||
self.assertEqual(len(tasks), 1)
|
||||
self.assertEqual(tasks[0]["workflow_mode"], "single_agent")
|
||||
|
||||
with patch.dict(os.environ, {"ENABLE_SUBTASK_HANDOFF": "true"}, clear=False):
|
||||
tasks = runtime.build_task_descriptions(body)
|
||||
self.assertEqual(len(tasks), 2)
|
||||
self.assertEqual(tasks[0]["task_id"], "backend-1")
|
||||
self.assertEqual(tasks[1]["depends_on"], ["backend-1"])
|
||||
self.assertEqual(tasks[1]["workflow_mode"], "multi_agent")
|
||||
|
||||
async def test_ready_pending_task_respects_dependencies_and_capabilities(self):
|
||||
parent = await task_queue_module.task_queue.create_task(
|
||||
task_id="parent-1",
|
||||
title="Backend",
|
||||
description="Implement backend",
|
||||
agent_role="backend",
|
||||
required_capabilities=["backend"],
|
||||
source="runtime_bridge",
|
||||
)
|
||||
child = await task_queue_module.task_queue.create_task(
|
||||
task_id="child-1",
|
||||
title="Tests",
|
||||
description="Add tests",
|
||||
agent_role="testing",
|
||||
required_capabilities=["testing"],
|
||||
depends_on=[parent.task_id],
|
||||
root_task_id=parent.task_id,
|
||||
source="runtime_bridge",
|
||||
)
|
||||
|
||||
task = await task_queue_module.task_queue.get_ready_pending_task(["testing"])
|
||||
self.assertIsNone(task, "dependent task should not dispatch before parent completion")
|
||||
|
||||
task = await task_queue_module.task_queue.get_ready_pending_task(["backend"])
|
||||
self.assertIsNotNone(task)
|
||||
self.assertEqual(task.task_id, parent.task_id)
|
||||
|
||||
await task_queue_module.task_queue.complete_task(parent.task_id, "done")
|
||||
task = await task_queue_module.task_queue.get_ready_pending_task(["testing"])
|
||||
self.assertIsNotNone(task)
|
||||
self.assertEqual(task.task_id, child.task_id)
|
||||
|
||||
async def test_finalize_parent_after_child_success(self):
|
||||
event_calls = []
|
||||
|
||||
async def fake_emit_event(*args, **kwargs):
|
||||
event_calls.append((args, kwargs))
|
||||
|
||||
runtime_patch = patch.object(orchestrator_main.swarm_runtime, "emit_event", fake_emit_event)
|
||||
runtime_patch.start()
|
||||
self.addCleanup(runtime_patch.stop)
|
||||
|
||||
parent = await task_queue_module.task_queue.create_task(
|
||||
task_id="parent-1",
|
||||
description="Parent task",
|
||||
title="Parent",
|
||||
agent_role="backend",
|
||||
source="runtime_bridge",
|
||||
)
|
||||
child = await task_queue_module.task_queue.create_task(
|
||||
task_id="child-1",
|
||||
description="Child task",
|
||||
title="Child",
|
||||
agent_role="testing",
|
||||
parent_task_id=parent.task_id,
|
||||
root_task_id=parent.task_id,
|
||||
source="dynamic_handoff",
|
||||
)
|
||||
await task_queue_module.task_queue.add_child_task(parent.task_id, child.task_id)
|
||||
await task_queue_module.task_queue.block_task(parent.task_id, "Waiting on child")
|
||||
await task_queue_module.task_queue.complete_task(child.task_id, "child complete")
|
||||
|
||||
refreshed_child = await task_queue_module.task_queue.get_task(child.task_id)
|
||||
run = SimpleNamespace()
|
||||
await orchestrator_main.finalize_parent_after_child(
|
||||
run,
|
||||
refreshed_child,
|
||||
agent_id="agent-1",
|
||||
success=True,
|
||||
summary="Delegated child task completed",
|
||||
)
|
||||
|
||||
refreshed_parent = await task_queue_module.task_queue.get_task(parent.task_id)
|
||||
self.assertEqual(refreshed_parent.status, task_queue_module.TaskStatus.COMPLETED)
|
||||
self.assertGreaterEqual(len(event_calls), 2)
|
||||
|
||||
async def test_finalize_parent_after_child_failure_is_terminal(self):
|
||||
async def fake_emit_event(*args, **kwargs):
|
||||
return None
|
||||
|
||||
runtime_patch = patch.object(orchestrator_main.swarm_runtime, "emit_event", fake_emit_event)
|
||||
runtime_patch.start()
|
||||
self.addCleanup(runtime_patch.stop)
|
||||
|
||||
parent = await task_queue_module.task_queue.create_task(
|
||||
task_id="parent-2",
|
||||
description="Parent task",
|
||||
title="Parent",
|
||||
agent_role="backend",
|
||||
source="runtime_bridge",
|
||||
max_retries=3,
|
||||
)
|
||||
child = await task_queue_module.task_queue.create_task(
|
||||
task_id="child-2",
|
||||
description="Child task",
|
||||
title="Child",
|
||||
agent_role="testing",
|
||||
parent_task_id=parent.task_id,
|
||||
root_task_id=parent.task_id,
|
||||
source="dynamic_handoff",
|
||||
max_retries=1,
|
||||
)
|
||||
await task_queue_module.task_queue.add_child_task(parent.task_id, child.task_id)
|
||||
await task_queue_module.task_queue.block_task(parent.task_id, "Waiting on child")
|
||||
await task_queue_module.task_queue.fail_task(child.task_id, "child failed")
|
||||
|
||||
refreshed_child = await task_queue_module.task_queue.get_task(child.task_id)
|
||||
run = SimpleNamespace()
|
||||
await orchestrator_main.finalize_parent_after_child(
|
||||
run,
|
||||
refreshed_child,
|
||||
agent_id="agent-1",
|
||||
success=False,
|
||||
summary="child failed",
|
||||
)
|
||||
|
||||
refreshed_parent = await task_queue_module.task_queue.get_task(parent.task_id)
|
||||
self.assertEqual(refreshed_parent.status, task_queue_module.TaskStatus.FAILED)
|
||||
|
||||
async def test_build_result_artifact_emits_document_without_git_branch(self):
|
||||
run = SimpleNamespace(
|
||||
deployment_id="runtime-dep-1",
|
||||
manager_deployment_id="dep-manager-1",
|
||||
swarm_id="swarm-1",
|
||||
)
|
||||
task = SimpleNamespace(
|
||||
task_id="task-1",
|
||||
title="Backend implementation",
|
||||
agent_role="backend",
|
||||
parent_task_id=None,
|
||||
root_task_id="task-1",
|
||||
context={},
|
||||
)
|
||||
result = {
|
||||
"summary": "Implemented backend endpoints and startup notes",
|
||||
"files_modified": ["backend/app.py", "README.md"],
|
||||
"changes": "Added handlers and docs",
|
||||
"git_skipped": "Workspace is not a Git checkout",
|
||||
}
|
||||
|
||||
artifact = orchestrator_main.build_result_artifact(run, task, result)
|
||||
|
||||
self.assertEqual(artifact["artifact_type"], "deployment_manifest")
|
||||
self.assertEqual(artifact["uri"], "runtime://swarm-1/artifacts/task-1")
|
||||
self.assertEqual(
|
||||
artifact["metadata"]["files_modified"],
|
||||
["backend/app.py", "README.md"],
|
||||
)
|
||||
|
||||
async def test_artifact_created_event_copies_artifact_into_payload(self):
|
||||
fake_runtime_redis = FakeRedisClient()
|
||||
runtime_patch = patch.object(
|
||||
swarm_runtime_module,
|
||||
"redis_client",
|
||||
SimpleNamespace(
|
||||
set=fake_runtime_redis.set,
|
||||
get=fake_runtime_redis.get,
|
||||
keys=fake_runtime_redis.keys,
|
||||
rpush=fake_runtime_redis.rpush,
|
||||
lrange=fake_runtime_redis.lrange,
|
||||
),
|
||||
)
|
||||
runtime_patch.start()
|
||||
self.addCleanup(runtime_patch.stop)
|
||||
|
||||
runtime = swarm_runtime_module.SwarmRuntime()
|
||||
run = swarm_runtime_module.SwarmRun(
|
||||
deployment_id="runtime-dep-1",
|
||||
swarm_id="swarm-1",
|
||||
status="running",
|
||||
objective="Test callback payload",
|
||||
manager_deployment_id="dep-manager-1",
|
||||
)
|
||||
|
||||
await runtime.emit_event(
|
||||
run,
|
||||
"artifact.created",
|
||||
task_id="task-1",
|
||||
artifact={
|
||||
"artifact_id": "art_task-1",
|
||||
"artifact_type": "document",
|
||||
"title": "Task result",
|
||||
"summary": "Generated a result summary",
|
||||
"uri": "runtime://swarm-1/artifacts/task-1",
|
||||
},
|
||||
)
|
||||
|
||||
logs = await runtime.list_events(run.swarm_id)
|
||||
event = logs["events"][-1]
|
||||
self.assertEqual(event["payload"]["artifact_id"], "art_task-1")
|
||||
self.assertEqual(event["payload"]["artifact_type"], "document")
|
||||
self.assertEqual(event["artifact"]["uri"], "runtime://swarm-1/artifacts/task-1")
|
||||
|
||||
async def test_validate_create_request_accepts_new_swarm_shape(self):
|
||||
runtime = swarm_runtime_module.SwarmRuntime()
|
||||
runtime.validate_create_request({
|
||||
"mode": "swarm",
|
||||
"conversation_id": "conv-1",
|
||||
"requirement": {
|
||||
"objective": "Build a swarm-delivered feature",
|
||||
"context": ["ticket-123"],
|
||||
"attachments": [],
|
||||
"constraints": ["keep existing API"],
|
||||
"acceptance_criteria": ["tests pass"],
|
||||
},
|
||||
"model_selection": {
|
||||
"type": "primary",
|
||||
"primary_model": "gpt-5.4",
|
||||
},
|
||||
"metadata": {
|
||||
"manager_deployment_id": "dep-manager-1",
|
||||
"correlation_id": "corr-1",
|
||||
},
|
||||
"callback": {
|
||||
"url": "https://manager.example/api/agent/callbacks/runtime-events",
|
||||
},
|
||||
})
|
||||
|
||||
async def test_validate_create_request_rejects_missing_objective(self):
|
||||
runtime = swarm_runtime_module.SwarmRuntime()
|
||||
with self.assertRaises(swarm_runtime_module.RuntimeValidationError):
|
||||
runtime.validate_create_request({
|
||||
"mode": "swarm",
|
||||
"requirement": {},
|
||||
"metadata": {"manager_deployment_id": "dep-manager-1"},
|
||||
"callback": {"url": "https://manager.example/callback"},
|
||||
})
|
||||
|
||||
async def test_get_run_by_identifier_supports_manager_deployment_id(self):
|
||||
fake_runtime_redis = FakeRedisClient()
|
||||
runtime_patch = patch.object(
|
||||
swarm_runtime_module,
|
||||
"redis_client",
|
||||
SimpleNamespace(
|
||||
set=fake_runtime_redis.set,
|
||||
get=fake_runtime_redis.get,
|
||||
keys=fake_runtime_redis.keys,
|
||||
rpush=fake_runtime_redis.rpush,
|
||||
lrange=fake_runtime_redis.lrange,
|
||||
),
|
||||
)
|
||||
runtime_patch.start()
|
||||
self.addCleanup(runtime_patch.stop)
|
||||
|
||||
runtime = swarm_runtime_module.SwarmRuntime()
|
||||
run = swarm_runtime_module.SwarmRun(
|
||||
deployment_id="runtime-dep-1",
|
||||
swarm_id="swarm-1",
|
||||
status="running",
|
||||
objective="Test identifier lookup",
|
||||
manager_deployment_id="dep-manager-1",
|
||||
)
|
||||
await runtime.save_run(run)
|
||||
|
||||
loaded = await runtime.get_run_by_identifier("dep-manager-1")
|
||||
self.assertIsNotNone(loaded)
|
||||
self.assertEqual(loaded.swarm_id, "swarm-1")
|
||||
|
||||
async def test_runtime_routes_include_agent_swarm_paths(self):
|
||||
route_paths = {route.path for route in orchestrator_main.app.routes}
|
||||
self.assertIn("/api/agent/health", route_paths)
|
||||
self.assertIn("/api/agent/swarm/deployments", route_paths)
|
||||
self.assertIn("/api/agent/swarm/deployments/{deployment_id}/events", route_paths)
|
||||
self.assertIn("/api/agent/swarm/deployments/{deployment_id}/workflow", route_paths)
|
||||
self.assertIn("/api/agent/swarm/deployments/{deployment_id}/diagnostics", route_paths)
|
||||
|
||||
async def test_build_deliverable_fact_marks_summary_only_without_diff(self):
|
||||
result = {"summary": "Only a plan document"}
|
||||
deliverable = orchestrator_main.build_deliverable_fact(result)
|
||||
self.assertFalse(deliverable["has_deliverable"])
|
||||
self.assertTrue(deliverable["summary_only"])
|
||||
self.assertFalse(deliverable["has_diff"])
|
||||
|
||||
async def test_build_workflow_phases_returns_fixed_phase_order(self):
|
||||
run = SimpleNamespace(status="running", approvals={})
|
||||
task = SimpleNamespace(
|
||||
task_id="task-1",
|
||||
title="Backend",
|
||||
agent_role="backend",
|
||||
status=SimpleNamespace(value="in_progress"),
|
||||
started_at=10,
|
||||
completed_at=None,
|
||||
result=None,
|
||||
)
|
||||
phases = orchestrator_main.build_workflow_phases(
|
||||
run,
|
||||
[task],
|
||||
[{"event_type": "task.created", "payload": {}}],
|
||||
)
|
||||
self.assertEqual(
|
||||
[phase["name"] for phase in phases],
|
||||
["Plan", "Dispatch", "Execute", "Handoff", "Review", "Deliver"],
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,329 @@
|
||||
"""
|
||||
OpenTelemetry Distributed Tracing Setup
|
||||
|
||||
Provides distributed tracing for agent handoffs and task flows.
|
||||
Enables end-to-end visibility across the swarm system.
|
||||
"""
|
||||
|
||||
import os
|
||||
import logging
|
||||
from typing import Optional, Dict, Any
|
||||
from contextlib import contextmanager
|
||||
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
|
||||
from opentelemetry.sdk.resources import Resource, SERVICE_NAME, SERVICE_VERSION
|
||||
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
|
||||
from opentelemetry.instrumentation.redis import RedisInstrumentor
|
||||
from opentelemetry.instrumentation.requests import RequestsInstrumentor
|
||||
from opentelemetry.instrumentation.logging import LoggingInstrumentor
|
||||
from opentelemetry.trace import Status, StatusCode, SpanKind
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SwarmTracer:
|
||||
"""Manages distributed tracing for the swarm system"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
service_name: str,
|
||||
service_version: str = "1.0.0",
|
||||
otlp_endpoint: Optional[str] = None,
|
||||
enable_console: bool = False
|
||||
):
|
||||
self.service_name = service_name
|
||||
self.service_version = service_version
|
||||
self.otlp_endpoint = otlp_endpoint or os.getenv(
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"http://localhost:4317"
|
||||
)
|
||||
self.enable_console = enable_console
|
||||
|
||||
self._setup_tracing()
|
||||
|
||||
def _setup_tracing(self):
|
||||
"""Initialize OpenTelemetry tracing"""
|
||||
# Create resource with service information
|
||||
resource = Resource.create({
|
||||
SERVICE_NAME: self.service_name,
|
||||
SERVICE_VERSION: self.service_version,
|
||||
"deployment.environment": os.getenv("ENVIRONMENT", "production"),
|
||||
"k8s.namespace": os.getenv("K8S_NAMESPACE", "swarm-system"),
|
||||
"k8s.pod.name": os.getenv("HOSTNAME", "unknown"),
|
||||
})
|
||||
|
||||
# Create tracer provider
|
||||
provider = TracerProvider(resource=resource)
|
||||
|
||||
# Add OTLP exporter
|
||||
try:
|
||||
otlp_exporter = OTLPSpanExporter(endpoint=self.otlp_endpoint)
|
||||
provider.add_span_processor(BatchSpanProcessor(otlp_exporter))
|
||||
logger.info(f"OTLP exporter configured: {self.otlp_endpoint}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to configure OTLP exporter: {e}")
|
||||
|
||||
# Add console exporter for debugging
|
||||
if self.enable_console:
|
||||
console_exporter = ConsoleSpanExporter()
|
||||
provider.add_span_processor(BatchSpanProcessor(console_exporter))
|
||||
|
||||
# Set global tracer provider
|
||||
trace.set_tracer_provider(provider)
|
||||
|
||||
# Auto-instrument libraries
|
||||
self._instrument_libraries()
|
||||
|
||||
self.tracer = trace.get_tracer(__name__)
|
||||
logger.info(f"Tracing initialized for service: {self.service_name}")
|
||||
|
||||
def _instrument_libraries(self):
|
||||
"""Auto-instrument common libraries"""
|
||||
try:
|
||||
RedisInstrumentor().instrument()
|
||||
RequestsInstrumentor().instrument()
|
||||
LoggingInstrumentor().instrument()
|
||||
logger.info("Auto-instrumentation enabled")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to auto-instrument libraries: {e}")
|
||||
|
||||
@contextmanager
|
||||
def trace_operation(
|
||||
self,
|
||||
operation_name: str,
|
||||
attributes: Optional[Dict[str, Any]] = None,
|
||||
kind: SpanKind = SpanKind.INTERNAL
|
||||
):
|
||||
"""
|
||||
Context manager for tracing an operation
|
||||
|
||||
Usage:
|
||||
with tracer.trace_operation("process_task", {"task_id": "123"}):
|
||||
# do work
|
||||
pass
|
||||
"""
|
||||
with self.tracer.start_as_current_span(
|
||||
operation_name,
|
||||
kind=kind,
|
||||
attributes=attributes or {}
|
||||
) as span:
|
||||
try:
|
||||
yield span
|
||||
except Exception as e:
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
span.record_exception(e)
|
||||
raise
|
||||
|
||||
def trace_task_submission(self, task_id: str, task_description: str):
|
||||
"""Trace task submission"""
|
||||
with self.trace_operation(
|
||||
"task.submit",
|
||||
{
|
||||
"task.id": task_id,
|
||||
"task.description": task_description[:100] # Truncate
|
||||
},
|
||||
kind=SpanKind.PRODUCER
|
||||
) as span:
|
||||
span.add_event("task_submitted")
|
||||
return span
|
||||
|
||||
def trace_agent_creation(self, agent_id: str, task_id: str, pod_name: str):
|
||||
"""Trace agent pod creation"""
|
||||
with self.trace_operation(
|
||||
"agent.create",
|
||||
{
|
||||
"agent.id": agent_id,
|
||||
"task.id": task_id,
|
||||
"k8s.pod.name": pod_name
|
||||
}
|
||||
) as span:
|
||||
span.add_event("agent_pod_created")
|
||||
return span
|
||||
|
||||
def trace_agent_execution(self, agent_id: str, task_id: str):
|
||||
"""Trace agent task execution"""
|
||||
with self.trace_operation(
|
||||
"agent.execute",
|
||||
{
|
||||
"agent.id": agent_id,
|
||||
"task.id": task_id
|
||||
}
|
||||
) as span:
|
||||
span.add_event("agent_started")
|
||||
return span
|
||||
|
||||
def trace_handoff(
|
||||
self,
|
||||
from_agent_id: str,
|
||||
to_agent_id: str,
|
||||
task_id: str,
|
||||
handoff_reason: str
|
||||
):
|
||||
"""Trace agent handoff"""
|
||||
with self.trace_operation(
|
||||
"agent.handoff",
|
||||
{
|
||||
"handoff.from_agent": from_agent_id,
|
||||
"handoff.to_agent": to_agent_id,
|
||||
"task.id": task_id,
|
||||
"handoff.reason": handoff_reason
|
||||
},
|
||||
kind=SpanKind.CLIENT
|
||||
) as span:
|
||||
span.add_event("handoff_initiated")
|
||||
return span
|
||||
|
||||
def trace_result_aggregation(self, task_id: str, agent_count: int):
|
||||
"""Trace result aggregation"""
|
||||
with self.trace_operation(
|
||||
"result.aggregate",
|
||||
{
|
||||
"task.id": task_id,
|
||||
"agent.count": agent_count
|
||||
}
|
||||
) as span:
|
||||
span.add_event("aggregation_started")
|
||||
return span
|
||||
|
||||
def add_event(self, name: str, attributes: Optional[Dict[str, Any]] = None):
|
||||
"""Add an event to the current span"""
|
||||
span = trace.get_current_span()
|
||||
if span:
|
||||
span.add_event(name, attributes or {})
|
||||
|
||||
def set_attribute(self, key: str, value: Any):
|
||||
"""Set an attribute on the current span"""
|
||||
span = trace.get_current_span()
|
||||
if span:
|
||||
span.set_attribute(key, value)
|
||||
|
||||
def record_error(self, error: Exception):
|
||||
"""Record an error in the current span"""
|
||||
span = trace.get_current_span()
|
||||
if span:
|
||||
span.set_status(Status(StatusCode.ERROR, str(error)))
|
||||
span.record_exception(error)
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_tracer_instance: Optional[SwarmTracer] = None
|
||||
|
||||
|
||||
def initialize_tracing(
|
||||
service_name: str,
|
||||
service_version: str = "1.0.0",
|
||||
otlp_endpoint: Optional[str] = None,
|
||||
enable_console: bool = False
|
||||
) -> SwarmTracer:
|
||||
"""Initialize global tracing instance"""
|
||||
global _tracer_instance
|
||||
_tracer_instance = SwarmTracer(
|
||||
service_name=service_name,
|
||||
service_version=service_version,
|
||||
otlp_endpoint=otlp_endpoint,
|
||||
enable_console=enable_console
|
||||
)
|
||||
return _tracer_instance
|
||||
|
||||
|
||||
def get_tracer() -> Optional[SwarmTracer]:
|
||||
"""Get the global tracer instance"""
|
||||
return _tracer_instance
|
||||
|
||||
|
||||
# Decorator for tracing functions
|
||||
def traced(operation_name: Optional[str] = None, **span_attributes):
|
||||
"""
|
||||
Decorator to automatically trace a function
|
||||
|
||||
Usage:
|
||||
@traced("my_operation", task_id="123")
|
||||
def my_function():
|
||||
pass
|
||||
"""
|
||||
def decorator(func):
|
||||
def wrapper(*args, **kwargs):
|
||||
tracer = get_tracer()
|
||||
if not tracer:
|
||||
return func(*args, **kwargs)
|
||||
|
||||
op_name = operation_name or f"{func.__module__}.{func.__name__}"
|
||||
with tracer.trace_operation(op_name, span_attributes):
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
return decorator
|
||||
|
||||
|
||||
# Context propagation helpers
|
||||
def inject_trace_context(headers: Dict[str, str]) -> Dict[str, str]:
|
||||
"""
|
||||
Inject trace context into HTTP headers for propagation
|
||||
|
||||
Usage:
|
||||
headers = inject_trace_context({})
|
||||
requests.post(url, headers=headers)
|
||||
"""
|
||||
from opentelemetry.propagate import inject
|
||||
inject(headers)
|
||||
return headers
|
||||
|
||||
|
||||
def extract_trace_context(headers: Dict[str, str]):
|
||||
"""
|
||||
Extract trace context from HTTP headers
|
||||
|
||||
Usage:
|
||||
extract_trace_context(request.headers)
|
||||
"""
|
||||
from opentelemetry.propagate import extract
|
||||
return extract(headers)
|
||||
|
||||
|
||||
# Example usage patterns
|
||||
"""
|
||||
# In orchestrator/main.py:
|
||||
from orchestrator.tracing import initialize_tracing, get_tracer
|
||||
|
||||
tracer = initialize_tracing(
|
||||
service_name="swarm-orchestrator",
|
||||
service_version="1.0.0",
|
||||
otlp_endpoint="http://otel-collector:4317"
|
||||
)
|
||||
|
||||
# Trace task submission
|
||||
with tracer.trace_task_submission(task_id, description):
|
||||
# Submit task logic
|
||||
pass
|
||||
|
||||
# In agent/main.py:
|
||||
from orchestrator.tracing import initialize_tracing, get_tracer
|
||||
|
||||
tracer = initialize_tracing(
|
||||
service_name="swarm-agent",
|
||||
service_version="1.0.0"
|
||||
)
|
||||
|
||||
# Trace agent execution
|
||||
with tracer.trace_agent_execution(agent_id, task_id):
|
||||
# Execute task
|
||||
pass
|
||||
|
||||
# Trace handoff
|
||||
with tracer.trace_handoff(from_agent, to_agent, task_id, reason):
|
||||
# Perform handoff
|
||||
pass
|
||||
|
||||
# Using decorator
|
||||
@traced("process_subtask", task_id="123")
|
||||
def process_subtask():
|
||||
pass
|
||||
|
||||
# Manual span management
|
||||
tracer = get_tracer()
|
||||
with tracer.trace_operation("custom_operation", {"key": "value"}):
|
||||
tracer.add_event("checkpoint_reached")
|
||||
tracer.set_attribute("result_count", 42)
|
||||
"""
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/bin/bash
|
||||
# Deploy agent to Kubernetes cluster
|
||||
|
||||
set -e
|
||||
|
||||
echo "=========================================="
|
||||
echo "Agent Deployment Script"
|
||||
echo "=========================================="
|
||||
|
||||
# Configuration
|
||||
AGENT_IMAGE="swarm-agent:latest"
|
||||
AGENT_ID="${AGENT_ID:-agent-$(date +%s)}"
|
||||
AGENT_CAPABILITIES="${AGENT_CAPABILITIES:-general,python}"
|
||||
|
||||
echo "Agent ID: $AGENT_ID"
|
||||
echo "Agent Capabilities: $AGENT_CAPABILITIES"
|
||||
|
||||
# Check required environment variables
|
||||
if [ -z "$OPENAI_API_KEY" ]; then
|
||||
echo "Error: OPENAI_API_KEY environment variable not set"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Build Docker image
|
||||
echo ""
|
||||
echo "Building Docker image..."
|
||||
docker build -f Dockerfile.agent -t $AGENT_IMAGE .
|
||||
|
||||
# Load image into kind cluster
|
||||
echo ""
|
||||
echo "Loading image into kind cluster..."
|
||||
kind load docker-image $AGENT_IMAGE
|
||||
|
||||
# Create secrets if they don't exist
|
||||
echo ""
|
||||
echo "Creating/updating secrets..."
|
||||
|
||||
# Model API key (OpenAI-compatible)
|
||||
kubectl create secret generic openai-secret \
|
||||
--from-literal=api-key="$OPENAI_API_KEY" \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
# Git credentials (optional)
|
||||
if [ -n "$GIT_USERNAME" ] && [ -n "$GIT_PASSWORD" ]; then
|
||||
kubectl create secret generic git-credentials \
|
||||
--from-literal=username="$GIT_USERNAME" \
|
||||
--from-literal=password="$GIT_PASSWORD" \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
else
|
||||
echo "Note: GIT_USERNAME and GIT_PASSWORD not set, skipping git credentials"
|
||||
fi
|
||||
|
||||
# Create ConfigMap
|
||||
echo ""
|
||||
echo "Creating/updating ConfigMap..."
|
||||
kubectl create configmap swarm-config \
|
||||
--from-literal=git-repo-url="${GIT_REPO_URL:-}" \
|
||||
--from-literal=openai-model="${OPENAI_MODEL:-gpt-4o-mini}" \
|
||||
--from-literal=openai-api-base="${OPENAI_API_BASE:-https://api.openai.com/v1}" \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
|
||||
# Deploy agent pod
|
||||
echo ""
|
||||
echo "Deploying agent pod..."
|
||||
export AGENT_ID
|
||||
export AGENT_CAPABILITIES
|
||||
envsubst < k8s/agent-pod-template.yaml | kubectl apply -f -
|
||||
|
||||
# Wait for pod to be ready
|
||||
echo ""
|
||||
echo "Waiting for agent pod to be ready..."
|
||||
kubectl wait --for=condition=Ready pod/agent-$AGENT_ID --timeout=60s || true
|
||||
|
||||
# Show pod status
|
||||
echo ""
|
||||
echo "Agent pod status:"
|
||||
kubectl get pod agent-$AGENT_ID
|
||||
|
||||
# Show logs
|
||||
echo ""
|
||||
echo "Agent logs (last 20 lines):"
|
||||
kubectl logs agent-$AGENT_ID --tail=20 || echo "Pod not ready yet"
|
||||
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Agent deployment complete!"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo "To view logs:"
|
||||
echo " kubectl logs -f agent-$AGENT_ID"
|
||||
echo ""
|
||||
echo "To delete agent:"
|
||||
echo " kubectl delete pod agent-$AGENT_ID"
|
||||
@@ -0,0 +1,33 @@
|
||||
#!/bin/bash
|
||||
# Build and deploy orchestrator to Kubernetes
|
||||
|
||||
set -e
|
||||
|
||||
echo "Building orchestrator Docker image..."
|
||||
docker build -f Dockerfile.orchestrator -t swarm-orchestrator:latest .
|
||||
|
||||
echo "Loading image into kind cluster..."
|
||||
kind load docker-image swarm-orchestrator:latest --name swarm-cluster
|
||||
|
||||
echo "Deploying Redis StatefulSet..."
|
||||
kubectl apply -f k8s/redis-statefulset.yaml
|
||||
|
||||
echo "Waiting for Redis to be ready..."
|
||||
kubectl wait --for=condition=ready pod -l app=redis -n swarm-system --timeout=120s
|
||||
|
||||
echo "Deploying orchestrator..."
|
||||
kubectl apply -f k8s/orchestrator-deployment.yaml
|
||||
|
||||
echo "Waiting for orchestrator to be ready..."
|
||||
kubectl wait --for=condition=ready pod -l app=orchestrator -n swarm-system --timeout=120s
|
||||
|
||||
echo "Orchestrator deployed successfully!"
|
||||
echo ""
|
||||
echo "Check status:"
|
||||
echo " kubectl get pods -n swarm-system"
|
||||
echo ""
|
||||
echo "View logs:"
|
||||
echo " kubectl logs -f -l app=orchestrator -n swarm-system"
|
||||
echo ""
|
||||
echo "Port forward to access API:"
|
||||
echo " kubectl port-forward -n swarm-system svc/orchestrator-service 8000:8000"
|
||||
@@ -0,0 +1,174 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Render a terminal-style HTML snapshot from a swarm poster summary JSON."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description="Render a terminal-style swarm snapshot HTML.")
|
||||
parser.add_argument("summary_json", help="Path to the swarm poster summary JSON file.")
|
||||
parser.add_argument(
|
||||
"--output-html",
|
||||
help="Optional output HTML path. Defaults next to the summary file.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def fmt_ts(ts: float | None) -> str:
|
||||
if not ts:
|
||||
return "-"
|
||||
return datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
summary_path = Path(args.summary_json).resolve()
|
||||
data = json.loads(summary_path.read_text(encoding="utf-8"))
|
||||
|
||||
swarm = data["swarm"]
|
||||
task = data["task"]
|
||||
metrics = data["metrics"]
|
||||
|
||||
out_path = (
|
||||
Path(args.output_html).resolve()
|
||||
if args.output_html
|
||||
else summary_path.with_name(summary_path.stem + "-terminal.html")
|
||||
)
|
||||
|
||||
budget_ratio = metrics.get("budget", {}).get("duration_ratio")
|
||||
budget_display = f"{budget_ratio * 100:.1f}%" if isinstance(budget_ratio, (int, float)) else "-"
|
||||
|
||||
transcript = [
|
||||
"$ python3 scripts/run_swarm_poster_demo.py",
|
||||
f"[swarm] deployment_id={swarm['deployment_id']}",
|
||||
f"[swarm] swarm_id={swarm['swarm_id']}",
|
||||
f"[task] task_id={task['task_id']}",
|
||||
f"[task] status={task['status']}",
|
||||
f"[task] assigned_agent={task.get('assigned_agent_id') or '-'}",
|
||||
"",
|
||||
"$ curl -H \"Authorization: Bearer ***\" /api/swarms/{swarm_id}/tasks",
|
||||
f"task.started_at = {fmt_ts(data.get('started_at'))}",
|
||||
f"task.halfway_at = {fmt_ts(data.get('halfway_at'))}",
|
||||
f"halfway.observed_at = {fmt_ts(data.get('halfway_observed_at'))}",
|
||||
"",
|
||||
"$ curl -H \"Authorization: Bearer ***\" /api/swarms/{swarm_id}/metrics",
|
||||
f"runtime.status = {metrics.get('status')}",
|
||||
f"runtime.tasks_total = {metrics.get('tasks_total')}",
|
||||
f"runtime.tasks_by_status = {json.dumps(metrics.get('tasks_by_status', {}), ensure_ascii=False)}",
|
||||
f"runtime.agents_connected = {metrics.get('agents_connected')}",
|
||||
f"runtime.budget.duration_seconds = {metrics.get('budget', {}).get('duration_seconds')}",
|
||||
f"runtime.budget.duration_ratio = {budget_display}",
|
||||
"",
|
||||
"# milestone",
|
||||
f"> 任务开始: {fmt_ts(data.get('started_at'))}",
|
||||
f"> 任务过半: {fmt_ts(data.get('halfway_observed_at'))}",
|
||||
]
|
||||
|
||||
lines_html = "\n".join(f"<div class='line'>{line}</div>" for line in transcript)
|
||||
|
||||
html = f"""<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Swarm Terminal Snapshot</title>
|
||||
<style>
|
||||
body {{
|
||||
margin: 0;
|
||||
background:
|
||||
radial-gradient(circle at top right, rgba(74, 222, 128, 0.10), transparent 20%),
|
||||
radial-gradient(circle at top left, rgba(96, 165, 250, 0.10), transparent 24%),
|
||||
#0a0f14;
|
||||
color: #d7e3ee;
|
||||
font-family: Menlo, Monaco, "SFMono-Regular", "JetBrains Mono", monospace;
|
||||
}}
|
||||
.frame {{
|
||||
width: 1600px;
|
||||
min-height: 900px;
|
||||
margin: 0 auto;
|
||||
padding: 48px;
|
||||
box-sizing: border-box;
|
||||
}}
|
||||
.window {{
|
||||
background: #0b1220;
|
||||
border: 1px solid rgba(148, 163, 184, 0.20);
|
||||
border-radius: 18px;
|
||||
overflow: hidden;
|
||||
box-shadow: 0 30px 80px rgba(0, 0, 0, 0.45);
|
||||
}}
|
||||
.topbar {{
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 14px 18px;
|
||||
background: #121a2a;
|
||||
border-bottom: 1px solid rgba(148, 163, 184, 0.12);
|
||||
}}
|
||||
.dots {{
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}}
|
||||
.dot {{
|
||||
width: 12px;
|
||||
height: 12px;
|
||||
border-radius: 50%;
|
||||
}}
|
||||
.dot.red {{ background: #fb7185; }}
|
||||
.dot.yellow {{ background: #fbbf24; }}
|
||||
.dot.green {{ background: #4ade80; }}
|
||||
.title {{
|
||||
color: #93a4b8;
|
||||
font-size: 15px;
|
||||
letter-spacing: 0.02em;
|
||||
}}
|
||||
.terminal {{
|
||||
padding: 28px 32px 36px;
|
||||
font-size: 24px;
|
||||
line-height: 1.7;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}}
|
||||
.line:nth-child(1),
|
||||
.line:nth-child(8),
|
||||
.line:nth-child(13) {{
|
||||
color: #7dd3fc;
|
||||
}}
|
||||
.line:nth-last-child(2),
|
||||
.line:nth-last-child(1) {{
|
||||
color: #bef264;
|
||||
font-weight: 600;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="frame">
|
||||
<div class="window">
|
||||
<div class="topbar">
|
||||
<div class="dots">
|
||||
<span class="dot red"></span>
|
||||
<span class="dot yellow"></span>
|
||||
<span class="dot green"></span>
|
||||
</div>
|
||||
<div class="title">swarm-terminal-snapshot</div>
|
||||
<div class="title">{swarm['swarm_id']}</div>
|
||||
</div>
|
||||
<div class="terminal">
|
||||
{lines_html}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
out_path.write_text(html, encoding="utf-8")
|
||||
print(out_path)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,660 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Create a real swarm task, wait for start and half-progress, and render a poster page."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
from urllib.error import HTTPError, URLError
|
||||
from urllib.parse import urlencode
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
OUTPUT_DIR = ROOT / "artifacts" / "swarm-posters"
|
||||
|
||||
|
||||
@dataclass
|
||||
class ApiConfig:
|
||||
base_url: str
|
||||
token: str
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Create a swarm task, wait for real progress milestones, and build a poster.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--base-url",
|
||||
default=os.getenv("SWARM_BASE_URL", "http://52.139.240.116:8000"),
|
||||
help="Base URL of the running swarm runtime.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--runtime-token",
|
||||
default=os.getenv("AGNET_RUNTIME_SERVICE_TOKEN"),
|
||||
help="Runtime bearer token. If omitted, read from the Kubernetes secret.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--budget-seconds",
|
||||
type=int,
|
||||
default=40,
|
||||
help="Task budget duration used to compute the half-progress milestone.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--poll-interval",
|
||||
type=float,
|
||||
default=3.0,
|
||||
help="Polling interval in seconds.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--timeout-seconds",
|
||||
type=int,
|
||||
default=240,
|
||||
help="Maximum time to wait for the half-progress milestone.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--objective",
|
||||
default=(
|
||||
"Build a blog system MVP in the configured workspace repository. "
|
||||
"Deliver a runnable implementation with a simple backend and frontend for posts: "
|
||||
"list posts, view a post, create, edit, and delete. Keep setup minimal and include short run instructions."
|
||||
),
|
||||
help="High-level objective for the swarm.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--task-description",
|
||||
default=(
|
||||
"Create a runnable blog system MVP in the target repository with post list, "
|
||||
"post detail, and create/edit/delete flows. Keep the implementation pragmatic and "
|
||||
"coherent with the existing repo. Commit and push the result branch when changes are ready."
|
||||
),
|
||||
help="Concrete task description sent to the agent.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--poster-title",
|
||||
default="Swarm Blog MVP Live Demo",
|
||||
help="Poster title rendered into the HTML.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def iso_now() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
|
||||
|
||||
|
||||
def to_human_time(timestamp: Optional[float]) -> str:
|
||||
if not timestamp:
|
||||
return "-"
|
||||
return datetime.fromtimestamp(timestamp, tz=timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
|
||||
|
||||
|
||||
def read_runtime_token(explicit: Optional[str]) -> str:
|
||||
if explicit:
|
||||
return explicit
|
||||
|
||||
cmd = [
|
||||
"kubectl",
|
||||
"get",
|
||||
"secret",
|
||||
"-n",
|
||||
"swarm-system",
|
||||
"agnet-runtime-secrets",
|
||||
"-o",
|
||||
"jsonpath={.data.runtime-service-token}",
|
||||
]
|
||||
encoded = subprocess.check_output(cmd, text=True).strip()
|
||||
if not encoded:
|
||||
raise RuntimeError("Runtime token was not provided and could not be read from Kubernetes.")
|
||||
return base64.b64decode(encoded).decode("utf-8")
|
||||
|
||||
|
||||
def api_request(
|
||||
config: ApiConfig,
|
||||
path: str,
|
||||
method: str = "GET",
|
||||
body: Optional[dict[str, Any]] = None,
|
||||
headers: Optional[dict[str, str]] = None,
|
||||
query: Optional[dict[str, Any]] = None,
|
||||
) -> dict[str, Any]:
|
||||
query_string = f"?{urlencode(query)}" if query else ""
|
||||
url = f"{config.base_url.rstrip('/')}{path}{query_string}"
|
||||
request_headers = {
|
||||
"Authorization": f"Bearer {config.token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
if headers:
|
||||
request_headers.update(headers)
|
||||
|
||||
payload = None
|
||||
if body is not None:
|
||||
payload = json.dumps(body).encode("utf-8")
|
||||
|
||||
req = Request(url, data=payload, headers=request_headers, method=method)
|
||||
try:
|
||||
with urlopen(req, timeout=30) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
except HTTPError as exc:
|
||||
raw = exc.read().decode("utf-8", errors="replace")
|
||||
raise RuntimeError(f"API request failed with {exc.code}: {raw}") from exc
|
||||
except URLError as exc:
|
||||
raise RuntimeError(f"API request failed: {exc}") from exc
|
||||
|
||||
|
||||
def create_swarm(config: ApiConfig, args: argparse.Namespace) -> dict[str, Any]:
|
||||
request_id = iso_now()
|
||||
correlation_id = f"corr_poster_demo_{request_id}"
|
||||
idempotency_key = f"idem_poster_demo_{request_id}"
|
||||
manager_deployment_id = f"dep_poster_demo_{request_id}"
|
||||
|
||||
agents = [
|
||||
{
|
||||
"task_id": f"poster-blog-{request_id.lower()}",
|
||||
"role": "fullstack",
|
||||
"title": "Implement blog system MVP",
|
||||
"description": args.task_description,
|
||||
"depends_on": [],
|
||||
}
|
||||
]
|
||||
|
||||
body = {
|
||||
"orchestration_plan": {
|
||||
"objective": args.objective,
|
||||
"sub_mode": "code",
|
||||
"risk_level": "low",
|
||||
"budget": {
|
||||
"duration_seconds": args.budget_seconds,
|
||||
"token_limit": 120000,
|
||||
},
|
||||
"agents": agents,
|
||||
},
|
||||
"agents": agents,
|
||||
"callback": {
|
||||
"url": f"{config.base_url.rstrip('/')}/health",
|
||||
"subscribed_events": [
|
||||
"deployment.status_changed",
|
||||
"task.created",
|
||||
"task.running",
|
||||
"task.completed",
|
||||
"task.failed",
|
||||
"timeline.updated",
|
||||
"artifact.created",
|
||||
],
|
||||
},
|
||||
"metadata": {
|
||||
"manager_deployment_id": manager_deployment_id,
|
||||
"correlation_id": correlation_id,
|
||||
},
|
||||
}
|
||||
response = api_request(
|
||||
config,
|
||||
"/api/swarms",
|
||||
method="POST",
|
||||
body=body,
|
||||
headers={
|
||||
"X-Correlation-Id": correlation_id,
|
||||
"X-Idempotency-Key": idempotency_key,
|
||||
},
|
||||
)
|
||||
if not response.get("success"):
|
||||
raise RuntimeError(f"Swarm creation failed: {json.dumps(response, ensure_ascii=False)}")
|
||||
return response["data"]
|
||||
|
||||
|
||||
def fetch_tasks(config: ApiConfig, swarm_id: str) -> list[dict[str, Any]]:
|
||||
response = api_request(config, f"/api/swarms/{swarm_id}/tasks")
|
||||
return response["data"]["tasks"]
|
||||
|
||||
|
||||
def fetch_logs(config: ApiConfig, swarm_id: str, limit: int = 100) -> list[dict[str, Any]]:
|
||||
response = api_request(
|
||||
config,
|
||||
f"/api/swarms/{swarm_id}/logs",
|
||||
query={"limit": limit},
|
||||
)
|
||||
return response["data"]["events"]
|
||||
|
||||
|
||||
def fetch_metrics(config: ApiConfig, swarm_id: str) -> dict[str, Any]:
|
||||
response = api_request(config, f"/api/swarms/{swarm_id}/metrics")
|
||||
return response["data"]
|
||||
|
||||
|
||||
def fetch_agents(base_url: str) -> list[dict[str, Any]]:
|
||||
req = Request(f"{base_url.rstrip('/')}/agents")
|
||||
with urlopen(req, timeout=15) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))["agents"]
|
||||
|
||||
|
||||
def manual_assign(base_url: str, task_id: str, agent_id: str) -> bool:
|
||||
payload = json.dumps({"task_id": task_id, "agent_id": agent_id}).encode("utf-8")
|
||||
req = Request(
|
||||
f"{base_url.rstrip('/')}/tasks/assign",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urlopen(req, timeout=15) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
return bool(data.get("task_id") or data.get("status"))
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def find_event(events: list[dict[str, Any]], event_type: str) -> Optional[dict[str, Any]]:
|
||||
for event in events:
|
||||
if event.get("event_type") == event_type:
|
||||
return event
|
||||
return None
|
||||
|
||||
|
||||
def last_event(events: list[dict[str, Any]], event_type: str) -> Optional[dict[str, Any]]:
|
||||
for event in reversed(events):
|
||||
if event.get("event_type") == event_type:
|
||||
return event
|
||||
return None
|
||||
|
||||
|
||||
def render_html(
|
||||
poster_path: Path,
|
||||
args: argparse.Namespace,
|
||||
swarm_data: dict[str, Any],
|
||||
task: dict[str, Any],
|
||||
metrics: dict[str, Any],
|
||||
started_at: float,
|
||||
halfway_at: float,
|
||||
halfway_observed_at: float,
|
||||
latest_events: list[dict[str, Any]],
|
||||
) -> None:
|
||||
latest_status = task.get("status", "unknown")
|
||||
latest_heartbeat = last_event(latest_events, "task.heartbeat")
|
||||
heartbeat_time = latest_heartbeat.get("occurred_at") if latest_heartbeat else "-"
|
||||
runtime_ratio = metrics.get("budget", {}).get("duration_ratio")
|
||||
ratio_display = f"{runtime_ratio * 100:.0f}%" if isinstance(runtime_ratio, (int, float)) else "50%+"
|
||||
|
||||
html = f"""<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>{args.poster_title}</title>
|
||||
<style>
|
||||
:root {{
|
||||
--bg: #07111f;
|
||||
--panel: rgba(10, 25, 47, 0.84);
|
||||
--panel-strong: rgba(12, 32, 60, 0.95);
|
||||
--line: rgba(125, 211, 252, 0.25);
|
||||
--cyan: #7dd3fc;
|
||||
--teal: #5eead4;
|
||||
--lime: #bef264;
|
||||
--text: #e6f0ff;
|
||||
--muted: #9bb1c8;
|
||||
--warn: #fbbf24;
|
||||
}}
|
||||
* {{ box-sizing: border-box; }}
|
||||
body {{
|
||||
margin: 0;
|
||||
font-family: "Avenir Next", "PingFang SC", "Helvetica Neue", sans-serif;
|
||||
color: var(--text);
|
||||
background:
|
||||
radial-gradient(circle at top left, rgba(45, 212, 191, 0.18), transparent 32%),
|
||||
radial-gradient(circle at top right, rgba(125, 211, 252, 0.24), transparent 28%),
|
||||
linear-gradient(135deg, #050c16 0%, #07111f 42%, #0d1d35 100%);
|
||||
min-height: 100vh;
|
||||
}}
|
||||
.canvas {{
|
||||
width: 1600px;
|
||||
min-height: 900px;
|
||||
margin: 0 auto;
|
||||
padding: 56px;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}}
|
||||
.grid {{
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
background-image:
|
||||
linear-gradient(rgba(125, 211, 252, 0.05) 1px, transparent 1px),
|
||||
linear-gradient(90deg, rgba(125, 211, 252, 0.05) 1px, transparent 1px);
|
||||
background-size: 56px 56px;
|
||||
mask-image: linear-gradient(to bottom, rgba(0,0,0,.75), transparent);
|
||||
pointer-events: none;
|
||||
}}
|
||||
.hero {{
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 28px;
|
||||
align-items: flex-start;
|
||||
margin-bottom: 28px;
|
||||
}}
|
||||
.hero h1 {{
|
||||
margin: 0 0 12px 0;
|
||||
font-size: 68px;
|
||||
line-height: 0.95;
|
||||
letter-spacing: -2px;
|
||||
}}
|
||||
.hero p {{
|
||||
margin: 0;
|
||||
max-width: 800px;
|
||||
color: var(--muted);
|
||||
font-size: 24px;
|
||||
line-height: 1.5;
|
||||
}}
|
||||
.badge {{
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
border: 1px solid var(--line);
|
||||
background: rgba(6, 19, 36, 0.72);
|
||||
border-radius: 999px;
|
||||
padding: 12px 18px;
|
||||
color: var(--cyan);
|
||||
font-size: 16px;
|
||||
letter-spacing: 0.08em;
|
||||
text-transform: uppercase;
|
||||
}}
|
||||
.layout {{
|
||||
display: grid;
|
||||
grid-template-columns: 1.25fr 0.75fr;
|
||||
gap: 28px;
|
||||
}}
|
||||
.panel {{
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 28px;
|
||||
padding: 28px;
|
||||
box-shadow: 0 25px 60px rgba(0, 0, 0, 0.32);
|
||||
backdrop-filter: blur(16px);
|
||||
}}
|
||||
.panel strong {{
|
||||
display: block;
|
||||
font-size: 16px;
|
||||
letter-spacing: 0.08em;
|
||||
text-transform: uppercase;
|
||||
color: var(--cyan);
|
||||
margin-bottom: 14px;
|
||||
}}
|
||||
.cards {{
|
||||
display: grid;
|
||||
grid-template-columns: repeat(3, 1fr);
|
||||
gap: 18px;
|
||||
margin-bottom: 28px;
|
||||
}}
|
||||
.metric {{
|
||||
background: var(--panel-strong);
|
||||
border: 1px solid rgba(190, 242, 100, 0.16);
|
||||
border-radius: 22px;
|
||||
padding: 20px;
|
||||
}}
|
||||
.metric .label {{
|
||||
color: var(--muted);
|
||||
font-size: 15px;
|
||||
margin-bottom: 8px;
|
||||
}}
|
||||
.metric .value {{
|
||||
font-size: 34px;
|
||||
font-weight: 700;
|
||||
letter-spacing: -1px;
|
||||
}}
|
||||
.timeline {{
|
||||
display: grid;
|
||||
gap: 16px;
|
||||
margin-top: 8px;
|
||||
}}
|
||||
.step {{
|
||||
position: relative;
|
||||
padding: 18px 18px 18px 68px;
|
||||
border-radius: 22px;
|
||||
background: rgba(7, 18, 33, 0.9);
|
||||
border: 1px solid rgba(125, 211, 252, 0.12);
|
||||
}}
|
||||
.step::before {{
|
||||
content: "";
|
||||
position: absolute;
|
||||
left: 28px;
|
||||
top: 26px;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border-radius: 50%;
|
||||
background: linear-gradient(135deg, var(--teal), var(--cyan));
|
||||
box-shadow: 0 0 0 8px rgba(94, 234, 212, 0.12);
|
||||
}}
|
||||
.step h3 {{
|
||||
margin: 0 0 8px 0;
|
||||
font-size: 24px;
|
||||
}}
|
||||
.step p {{
|
||||
margin: 0;
|
||||
color: var(--muted);
|
||||
font-size: 16px;
|
||||
line-height: 1.5;
|
||||
}}
|
||||
.code {{
|
||||
font-family: "SF Mono", "JetBrains Mono", monospace;
|
||||
color: var(--lime);
|
||||
word-break: break-all;
|
||||
}}
|
||||
.sidebar {{
|
||||
display: grid;
|
||||
gap: 18px;
|
||||
}}
|
||||
.status {{
|
||||
border-radius: 24px;
|
||||
padding: 22px;
|
||||
background: linear-gradient(160deg, rgba(94, 234, 212, 0.14), rgba(125, 211, 252, 0.08));
|
||||
border: 1px solid rgba(94, 234, 212, 0.25);
|
||||
}}
|
||||
.status .headline {{
|
||||
font-size: 18px;
|
||||
color: var(--muted);
|
||||
margin-bottom: 8px;
|
||||
}}
|
||||
.status .value {{
|
||||
font-size: 44px;
|
||||
font-weight: 700;
|
||||
letter-spacing: -1px;
|
||||
margin-bottom: 10px;
|
||||
}}
|
||||
.status .sub {{
|
||||
color: var(--muted);
|
||||
font-size: 16px;
|
||||
line-height: 1.5;
|
||||
}}
|
||||
.meta-row {{
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
padding: 10px 0;
|
||||
border-bottom: 1px solid rgba(125, 211, 252, 0.08);
|
||||
font-size: 15px;
|
||||
}}
|
||||
.meta-row:last-child {{ border-bottom: none; }}
|
||||
.meta-label {{ color: var(--muted); }}
|
||||
.meta-value {{ text-align: right; max-width: 56%; }}
|
||||
.footer {{
|
||||
margin-top: 22px;
|
||||
color: var(--warn);
|
||||
font-size: 15px;
|
||||
line-height: 1.5;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="canvas">
|
||||
<div class="grid"></div>
|
||||
<div class="hero">
|
||||
<div>
|
||||
<div class="badge">Live Swarm Poster</div>
|
||||
<h1>{args.poster_title}</h1>
|
||||
<p>真实蜂群任务已成功创建,并已记录到“任务开始”和“进行过半”两段里程碑。下面的内容全部来自当前运行中的 swarm 状态,而不是手工拼接。</p>
|
||||
</div>
|
||||
<div class="panel" style="min-width: 360px;">
|
||||
<strong>Demo Goal</strong>
|
||||
<p style="font-size: 18px; color: var(--text); line-height: 1.55;">在目标仓库里开发一个博客系统 MVP,包括文章列表、详情和基础增删改能力。</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="layout">
|
||||
<div class="panel">
|
||||
<div class="cards">
|
||||
<div class="metric">
|
||||
<div class="label">Swarm</div>
|
||||
<div class="value">{swarm_data["swarm_id"]}</div>
|
||||
</div>
|
||||
<div class="metric">
|
||||
<div class="label">Task Status</div>
|
||||
<div class="value">{latest_status}</div>
|
||||
</div>
|
||||
<div class="metric">
|
||||
<div class="label">Budget Progress</div>
|
||||
<div class="value">{ratio_display}</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<strong>Timeline</strong>
|
||||
<div class="timeline">
|
||||
<div class="step">
|
||||
<h3>任务开始</h3>
|
||||
<p>任务已被 agent 领取并进入运行态。开始时间:<span class="code">{to_human_time(started_at)}</span></p>
|
||||
</div>
|
||||
<div class="step">
|
||||
<h3>进行到一半</h3>
|
||||
<p>脚本按 budget 的 50% 自动确认里程碑。预算过半时间:<span class="code">{to_human_time(halfway_at)}</span>,实际记录时间:<span class="code">{to_human_time(halfway_observed_at)}</span></p>
|
||||
</div>
|
||||
<div class="step">
|
||||
<h3>当前执行中</h3>
|
||||
<p>任务仍处于 <span class="code">{latest_status}</span>,最近心跳:<span class="code">{heartbeat_time}</span></p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="sidebar">
|
||||
<div class="status">
|
||||
<div class="headline">Poster Snapshot</div>
|
||||
<div class="value">Mid-Run</div>
|
||||
<div class="sub">这张海报展示的是蜂群任务已经真正启动,并且已经跑到预算半程时的现场状态。</div>
|
||||
</div>
|
||||
|
||||
<div class="panel">
|
||||
<strong>Swarm Meta</strong>
|
||||
<div class="meta-row"><span class="meta-label">Deployment</span><span class="meta-value code">{swarm_data["deployment_id"]}</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Task</span><span class="meta-value code">{task["task_id"]}</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Agent</span><span class="meta-value code">{task.get("assigned_agent_id") or "-"}</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Attempts</span><span class="meta-value">{task.get("attempt", 0)}</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Budget</span><span class="meta-value">{args.budget_seconds}s</span></div>
|
||||
<div class="meta-row"><span class="meta-label">Output File</span><span class="meta-value code">{poster_path.name}</span></div>
|
||||
</div>
|
||||
|
||||
<div class="panel">
|
||||
<strong>Latest Events</strong>
|
||||
<div class="meta-row"><span class="meta-label">1</span><span class="meta-value code">{latest_events[-3]["event_type"] if len(latest_events) >= 3 else "-"}</span></div>
|
||||
<div class="meta-row"><span class="meta-label">2</span><span class="meta-value code">{latest_events[-2]["event_type"] if len(latest_events) >= 2 else "-"}</span></div>
|
||||
<div class="meta-row"><span class="meta-label">3</span><span class="meta-value code">{latest_events[-1]["event_type"] if latest_events else "-"}</span></div>
|
||||
<div class="footer">注:如果后续继续执行,日志与状态还会变化;这张图锁定的是“开始后已过半”的那一刻。</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
poster_path.write_text(html, encoding="utf-8")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
token = read_runtime_token(args.runtime_token)
|
||||
config = ApiConfig(base_url=args.base_url, token=token)
|
||||
|
||||
swarm_data = create_swarm(config, args)
|
||||
swarm_id = swarm_data["swarm_id"]
|
||||
deadline = time.time() + args.timeout_seconds
|
||||
|
||||
started_at = None
|
||||
task_snapshot = None
|
||||
latest_events: list[dict[str, Any]] = []
|
||||
|
||||
while time.time() < deadline:
|
||||
tasks = fetch_tasks(config, swarm_id)
|
||||
latest_events = fetch_logs(config, swarm_id, limit=100)
|
||||
if tasks:
|
||||
task_snapshot = tasks[0]
|
||||
if task_snapshot.get("status") == "pending" and not task_snapshot.get("assigned_agent_id"):
|
||||
for agent in fetch_agents(config.base_url):
|
||||
if agent.get("status") == "idle":
|
||||
if manual_assign(config.base_url, task_snapshot["task_id"], agent["agent_id"]):
|
||||
break
|
||||
if task_snapshot.get("started_at"):
|
||||
started_at = float(task_snapshot["started_at"])
|
||||
break
|
||||
running_event = find_event(latest_events, "task.running")
|
||||
if running_event:
|
||||
started_at = datetime.fromisoformat(
|
||||
running_event["occurred_at"].replace("Z", "+00:00")
|
||||
).timestamp()
|
||||
break
|
||||
time.sleep(args.poll_interval)
|
||||
|
||||
if started_at is None or task_snapshot is None:
|
||||
raise RuntimeError("Task did not reach a running state before timeout.")
|
||||
|
||||
halfway_at = started_at + args.budget_seconds / 2
|
||||
while time.time() < deadline:
|
||||
tasks = fetch_tasks(config, swarm_id)
|
||||
latest_events = fetch_logs(config, swarm_id, limit=100)
|
||||
task_snapshot = tasks[0]
|
||||
if time.time() >= halfway_at:
|
||||
break
|
||||
if task_snapshot.get("status") in {"completed", "failed"}:
|
||||
break
|
||||
time.sleep(args.poll_interval)
|
||||
|
||||
metrics = fetch_metrics(config, swarm_id)
|
||||
halfway_observed_at = time.time()
|
||||
run_id = swarm_id.replace("swarm-", "")
|
||||
poster_path = OUTPUT_DIR / f"swarm-poster-{run_id}.html"
|
||||
summary_path = OUTPUT_DIR / f"swarm-poster-{run_id}.json"
|
||||
|
||||
render_html(
|
||||
poster_path=poster_path,
|
||||
args=args,
|
||||
swarm_data=swarm_data,
|
||||
task=task_snapshot,
|
||||
metrics=metrics,
|
||||
started_at=started_at,
|
||||
halfway_at=halfway_at,
|
||||
halfway_observed_at=halfway_observed_at,
|
||||
latest_events=latest_events,
|
||||
)
|
||||
|
||||
summary = {
|
||||
"swarm": swarm_data,
|
||||
"task": task_snapshot,
|
||||
"metrics": metrics,
|
||||
"started_at": started_at,
|
||||
"halfway_at": halfway_at,
|
||||
"halfway_observed_at": halfway_observed_at,
|
||||
"poster_html": str(poster_path),
|
||||
}
|
||||
summary_path.write_text(json.dumps(summary, indent=2, ensure_ascii=False), encoding="utf-8")
|
||||
print(json.dumps(summary, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
raise SystemExit(main())
|
||||
except KeyboardInterrupt:
|
||||
print("Interrupted.", file=sys.stderr)
|
||||
raise SystemExit(130)
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Keyless stub agent for workflow testing.
|
||||
|
||||
Connects to the orchestrator over WebSocket, registers, and returns canned results WITHOUT
|
||||
calling any LLM (no OPENAI_API_KEY needed). To make the master review loop deterministic, the
|
||||
"testing" task returns a conflicting framework (unittest) on its FIRST execution and an aligned
|
||||
one (pytest) on retry — so the heuristic critic rejects exactly once and then accepts.
|
||||
|
||||
Run standalone against a manually started orchestrator:
|
||||
set "ORCHESTRATOR_URL=ws://localhost:8000"
|
||||
set "AGENT_ID=stub-1"
|
||||
set "AGENT_CAPABILITIES=python,code_generation,testing,pytest,technical-writing,general"
|
||||
python scripts/stub_agent.py
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
|
||||
import websockets
|
||||
|
||||
|
||||
class StubAgent:
|
||||
def __init__(self, orchestrator_url: str, agent_id: str, capabilities: list[str]):
|
||||
self.orchestrator_url = orchestrator_url.rstrip("/")
|
||||
self.agent_id = agent_id
|
||||
self.capabilities = capabilities
|
||||
self.ws = None
|
||||
self.runs: dict[str, int] = {} # task_id -> execution count
|
||||
self.running = True
|
||||
|
||||
def _result_for(self, task_id: str, description: str) -> dict:
|
||||
count = self.runs.get(task_id, 0) + 1
|
||||
self.runs[task_id] = count
|
||||
tid = task_id.lower()
|
||||
if "implementation" in tid:
|
||||
summary = "implemented add(a, b); tests should use pytest"
|
||||
elif "testing" in tid:
|
||||
# First attempt conflicts (unittest); retry aligns with implementation (pytest).
|
||||
summary = "wrote tests with unittest" if count == 1 else "wrote tests with pytest"
|
||||
elif "documentation" in tid:
|
||||
summary = "documented usage of add(a, b)"
|
||||
else:
|
||||
summary = f"completed: {description[:40]}"
|
||||
return {
|
||||
"success": True,
|
||||
"task_id": task_id,
|
||||
"subtasks": [{"status": "completed", "summary": summary, "changes": summary, "files": []}],
|
||||
"awaiting_handoff": False,
|
||||
"agent_id": self.agent_id,
|
||||
"usage": {
|
||||
"model_id": "stub", "model_tokens": 0, "prompt_tokens": 0,
|
||||
"completion_tokens": 0, "model_cost_usd": 0.0, "runtime_seconds": 0.0,
|
||||
},
|
||||
}
|
||||
|
||||
async def send(self, payload: dict):
|
||||
await self.ws.send(json.dumps(payload))
|
||||
|
||||
async def handle(self, msg: dict):
|
||||
msg_type = msg.get("type")
|
||||
if msg_type == "task_assignment":
|
||||
task_id = msg["task_id"]
|
||||
await self.send({"type": "task_accepted", "task_id": task_id, "available_slots": 4})
|
||||
await self.send({"type": "task_start", "agent_id": self.agent_id, "task_id": task_id, "timestamp": time.time()})
|
||||
await asyncio.sleep(0.2) # simulate work
|
||||
result = self._result_for(task_id, msg.get("description", ""))
|
||||
await self.send({
|
||||
"type": "task_complete", "agent_id": self.agent_id,
|
||||
"task_id": task_id, "result": result, "timestamp": time.time(),
|
||||
})
|
||||
elif msg_type == "peer_message" and not msg.get("is_reply"):
|
||||
# Answer an inbound peer query so collaboration round-trips complete.
|
||||
await self.send({
|
||||
"type": "peer_message", "agent_id": self.agent_id,
|
||||
"target_agent_id": msg.get("from_agent_id"), "task_id": msg.get("task_id"),
|
||||
"content": f"stub guidance from {self.agent_id}",
|
||||
"correlation_id": msg.get("correlation_id"), "is_reply": True, "timestamp": time.time(),
|
||||
})
|
||||
|
||||
async def heartbeat(self):
|
||||
while self.running:
|
||||
try:
|
||||
await self.send({
|
||||
"type": "heartbeat", "agent_id": self.agent_id,
|
||||
"timestamp": time.time(), "active_tasks": 0, "available_slots": 4,
|
||||
})
|
||||
except Exception:
|
||||
return
|
||||
await asyncio.sleep(5)
|
||||
|
||||
async def run(self):
|
||||
async with websockets.connect(f"{self.orchestrator_url}/ws/{self.agent_id}") as ws:
|
||||
self.ws = ws
|
||||
await self.send({
|
||||
"type": "register", "agent_id": self.agent_id,
|
||||
"capabilities": self.capabilities, "available_slots": 4, "active_task_ids": [],
|
||||
})
|
||||
hb = asyncio.create_task(self.heartbeat())
|
||||
try:
|
||||
async for raw in ws:
|
||||
await self.handle(json.loads(raw))
|
||||
finally:
|
||||
self.running = False
|
||||
hb.cancel()
|
||||
|
||||
|
||||
async def _main():
|
||||
agent = StubAgent(
|
||||
os.getenv("ORCHESTRATOR_URL", "ws://localhost:8000"),
|
||||
os.getenv("AGENT_ID", "stub-1"),
|
||||
os.getenv("AGENT_CAPABILITIES", "python,code_generation,testing,pytest,technical-writing,general").split(","),
|
||||
)
|
||||
await agent.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(_main())
|
||||
@@ -0,0 +1,156 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Test script for agent components."""
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Add parent directory to path
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
def test_imports():
|
||||
"""Test that all agent modules can be imported."""
|
||||
print("Testing imports...")
|
||||
|
||||
try:
|
||||
from agent.main import Agent
|
||||
print("✓ agent.main imported successfully")
|
||||
except Exception as e:
|
||||
print(f"✗ Failed to import agent.main: {e}")
|
||||
return False
|
||||
|
||||
try:
|
||||
from agent.task_executor import TaskExecutor
|
||||
print("✓ agent.task_executor imported successfully")
|
||||
except Exception as e:
|
||||
print(f"✗ Failed to import agent.task_executor: {e}")
|
||||
return False
|
||||
|
||||
try:
|
||||
from agent.handoff_logic import HandoffDecision, should_handoff
|
||||
print("✓ agent.handoff_logic imported successfully")
|
||||
except Exception as e:
|
||||
print(f"✗ Failed to import agent.handoff_logic: {e}")
|
||||
return False
|
||||
|
||||
try:
|
||||
from agent.git_operations import GitOperations
|
||||
print("✓ agent.git_operations imported successfully")
|
||||
except Exception as e:
|
||||
print(f"✗ Failed to import agent.git_operations: {e}")
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def test_handoff_logic():
|
||||
"""Test handoff decision logic."""
|
||||
print("\nTesting handoff logic...")
|
||||
|
||||
from agent.handoff_logic import should_handoff, estimate_task_complexity
|
||||
|
||||
# Test low complexity task
|
||||
subtask = {
|
||||
"description": "Fix typo in comment",
|
||||
"complexity": "low",
|
||||
"estimated_time": 5,
|
||||
"required_capabilities": ["general"]
|
||||
}
|
||||
decision = should_handoff(subtask, "agent-test")
|
||||
assert not decision.should_handoff, "Low complexity task should not handoff"
|
||||
print("✓ Low complexity task correctly handled")
|
||||
|
||||
# Test high complexity task
|
||||
subtask = {
|
||||
"description": "Refactor entire authentication system",
|
||||
"complexity": "high",
|
||||
"estimated_time": 120,
|
||||
"required_capabilities": ["security", "architecture"]
|
||||
}
|
||||
decision = should_handoff(subtask, "agent-test")
|
||||
assert decision.should_handoff, "High complexity task should handoff"
|
||||
print("✓ High complexity task correctly triggers handoff")
|
||||
|
||||
# Test specialized capabilities
|
||||
subtask = {
|
||||
"description": "Optimize database queries",
|
||||
"complexity": "medium",
|
||||
"estimated_time": 30,
|
||||
"required_capabilities": ["database", "performance"]
|
||||
}
|
||||
decision = should_handoff(subtask, "agent-test")
|
||||
assert decision.should_handoff, "Specialized capabilities should trigger handoff"
|
||||
print("✓ Specialized capabilities correctly trigger handoff")
|
||||
|
||||
# Test complexity estimation
|
||||
assert estimate_task_complexity("Fix typo in README") == "low"
|
||||
assert estimate_task_complexity("Refactor authentication system") == "high"
|
||||
assert estimate_task_complexity("Add new API endpoint") == "medium"
|
||||
print("✓ Complexity estimation working correctly")
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def test_agent_initialization():
|
||||
"""Test agent can be initialized."""
|
||||
print("\nTesting agent initialization...")
|
||||
|
||||
from agent.main import Agent
|
||||
|
||||
# Set dummy API key for testing
|
||||
os.environ["ANTHROPIC_API_KEY"] = "sk-ant-test-key-for-initialization-test"
|
||||
|
||||
try:
|
||||
agent = Agent(
|
||||
orchestrator_url="ws://localhost:8000",
|
||||
agent_id="test-agent",
|
||||
capabilities=["python", "testing"],
|
||||
workspace_dir="/tmp/test-workspace"
|
||||
)
|
||||
print(f"✓ Agent initialized with ID: {agent.agent_id}")
|
||||
print(f"✓ Agent capabilities: {agent.capabilities}")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"✗ Failed to initialize agent: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
"""Run all tests."""
|
||||
print("=" * 60)
|
||||
print("Agent Component Tests")
|
||||
print("=" * 60)
|
||||
|
||||
tests = [
|
||||
("Import Tests", test_imports),
|
||||
("Handoff Logic Tests", test_handoff_logic),
|
||||
("Agent Initialization Tests", test_agent_initialization),
|
||||
]
|
||||
|
||||
results = []
|
||||
for name, test_func in tests:
|
||||
try:
|
||||
result = test_func()
|
||||
results.append((name, result))
|
||||
except Exception as e:
|
||||
print(f"\n✗ {name} failed with exception: {e}")
|
||||
results.append((name, False))
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("Test Results")
|
||||
print("=" * 60)
|
||||
|
||||
for name, result in results:
|
||||
status = "✓ PASS" if result else "✗ FAIL"
|
||||
print(f"{status}: {name}")
|
||||
|
||||
all_passed = all(result for _, result in results)
|
||||
|
||||
if all_passed:
|
||||
print("\n✓ All tests passed!")
|
||||
return 0
|
||||
else:
|
||||
print("\n✗ Some tests failed")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Exercise agent Git workflow against a local temporary repository."""
|
||||
import asyncio
|
||||
import importlib.util
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
GIT_OPS_PATH = ROOT / "agent" / "git_operations.py"
|
||||
|
||||
spec = importlib.util.spec_from_file_location("git_operations", GIT_OPS_PATH)
|
||||
git_operations = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(git_operations)
|
||||
GitOperations = git_operations.GitOperations
|
||||
|
||||
|
||||
def run(command: list[str], cwd: Optional[Path] = None):
|
||||
result = subprocess.run(
|
||||
command,
|
||||
cwd=cwd,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"Command failed: {' '.join(command)}\nSTDOUT:\n{result.stdout}\nSTDERR:\n{result.stderr}"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
async def main():
|
||||
with tempfile.TemporaryDirectory(prefix="swarm-git-test-") as tmp:
|
||||
tmpdir = Path(tmp)
|
||||
source = tmpdir / "source"
|
||||
remote = tmpdir / "remote.git"
|
||||
workspace = tmpdir / "workspace"
|
||||
|
||||
source.mkdir()
|
||||
run(["git", "init", "-b", "main"], cwd=source)
|
||||
run(["git", "config", "user.name", "Test User"], cwd=source)
|
||||
run(["git", "config", "user.email", "test@example.com"], cwd=source)
|
||||
|
||||
(source / "hello.py").write_text(
|
||||
"def hello_world():\n return 'hello'\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
run(["git", "add", "hello.py"], cwd=source)
|
||||
run(["git", "commit", "-m", "Initial commit"], cwd=source)
|
||||
run(["git", "clone", "--bare", str(source), str(remote)])
|
||||
|
||||
os.environ["GIT_BASE_BRANCH"] = "main"
|
||||
git_ops = GitOperations(str(workspace), "test-agent")
|
||||
|
||||
assert await git_ops.clone_workspace(str(remote))
|
||||
assert await git_ops.is_git_workspace()
|
||||
assert await git_ops.create_result_branch("task-123")
|
||||
|
||||
(workspace / "hello.py").write_text(
|
||||
"def hello_world():\n return 'hello from agent'\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
commit_sha = await git_ops.commit_changes("Task task-123: update hello")
|
||||
assert commit_sha
|
||||
|
||||
branch_name = await git_ops.push_results()
|
||||
assert branch_name
|
||||
|
||||
refs = run(["git", "for-each-ref", "--format=%(refname:short)", "refs/heads"], cwd=remote)
|
||||
assert branch_name in refs.stdout.splitlines()
|
||||
|
||||
print("Git workflow test passed")
|
||||
print(f"branch={branch_name}")
|
||||
print(f"commit={commit_sha}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,263 @@
|
||||
"""Smoke tests for the agent_swarm_v4 -> heicode-swarm merge.
|
||||
|
||||
Exercises the new/changed code paths with the gated in-memory (fakeredis) fallback:
|
||||
- redis_client REDIS_FAKE fallback through the rich list API
|
||||
- task_queue.release_task (capacity-rejection requeue, no retry increment)
|
||||
- planner static fallback + build_planner_task_specs mapping & dependency filtering
|
||||
- agent peer-collaboration reply routing and capacity rejection messages
|
||||
"""
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
os.environ["REDIS_FAKE"] = "1" # gated dev/CI in-memory store
|
||||
os.environ.pop("OPENAI_API_KEY", None) # force planner static fallback
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
|
||||
from orchestrator.redis_client import redis_client
|
||||
from orchestrator.agent_registry import agent_registry, AgentStatus
|
||||
from orchestrator.task_queue import task_queue, TaskStatus
|
||||
from orchestrator import main as orch
|
||||
from orchestrator.planner import planner
|
||||
|
||||
# Importing orchestrator.main runs load_dotenv(), which may set a real OPENAI_API_KEY from a
|
||||
# local .env and give the planner a live client. Force it offline so these checks stay
|
||||
# hermetic and deterministic (static plan + heuristic review + concatenated synthesis).
|
||||
planner.client = None
|
||||
|
||||
failures = []
|
||||
|
||||
|
||||
def check(name, cond):
|
||||
print(("PASS" if cond else "FAIL"), "-", name)
|
||||
if not cond:
|
||||
failures.append(name)
|
||||
|
||||
|
||||
async def test_redis_fallback():
|
||||
await redis_client.connect()
|
||||
await redis_client.set("k", "v")
|
||||
check("redis_fallback get/set", (await redis_client.get("k")) == "v")
|
||||
await redis_client.lpush("q", "a")
|
||||
await redis_client.lpush("q", "b")
|
||||
check("redis_fallback list ops", (await redis_client.lrange("q", 0, -1)) == ["b", "a"])
|
||||
await redis_client.lrem("q", 0, "a")
|
||||
check("redis_fallback lrem", (await redis_client.lrange("q", 0, -1)) == ["b"])
|
||||
|
||||
|
||||
async def test_release_task():
|
||||
await agent_registry.register_agent("agent-x", ["general"])
|
||||
task = await task_queue.create_task(description="do thing", task_id="t-rel-1")
|
||||
assigned = await task_queue.assign_task(task.task_id, "agent-x")
|
||||
check("assign_task ok", assigned)
|
||||
released = await task_queue.release_task(task.task_id, agent_id="agent-x")
|
||||
check("release_task ok", released)
|
||||
reloaded = await task_queue.get_task(task.task_id)
|
||||
agent = await agent_registry.get_agent("agent-x")
|
||||
pending = await redis_client.lrange(task_queue.PENDING_QUEUE_KEY, 0, -1)
|
||||
check("release sets PENDING", reloaded.status == TaskStatus.PENDING)
|
||||
check("release no retry increment", reloaded.retry_count == 0)
|
||||
check("release frees agent", agent.status == AgentStatus.IDLE)
|
||||
check("release requeues task", task.task_id in pending)
|
||||
|
||||
|
||||
async def test_planner_fallback():
|
||||
subtasks = await planner.build_plan("swarm-test", "Build a calculator")
|
||||
roles = [s.get("role") for s in subtasks]
|
||||
check("planner static fallback (3 specialists)", roles == ["implementation", "testing", "documentation"])
|
||||
|
||||
run = types.SimpleNamespace(swarm_id="swarm-test", objective="Build a calculator")
|
||||
base_specs = [{"context": {"orchestration_plan": {"x": 1}}}]
|
||||
specs = await orch.build_planner_task_specs(run, {}, base_specs)
|
||||
check("planner specs preserve base context", specs[0]["context"].get("orchestration_plan") == {"x": 1})
|
||||
check("planner specs carry deps", any(s["depends_on"] for s in specs))
|
||||
# Inject a phantom dependency and confirm it is filtered out.
|
||||
specs2 = list(specs)
|
||||
specs2[0]["depends_on"] = ["does-not-exist"]
|
||||
run2 = types.SimpleNamespace(swarm_id="swarm-test", objective="x")
|
||||
|
||||
async def fake_plan(swarm_id, objective):
|
||||
return [
|
||||
{"subtask_id": "a", "description": "A", "role": "implementation", "required_capabilities": ["python"], "depends_on": ["ghost"]},
|
||||
{"subtask_id": "b", "description": "B", "role": "testing", "required_capabilities": ["testing"], "depends_on": ["a"]},
|
||||
]
|
||||
|
||||
orig = planner.build_plan
|
||||
planner.build_plan = fake_plan
|
||||
try:
|
||||
filtered = await orch.build_planner_task_specs(run2, {}, base_specs)
|
||||
finally:
|
||||
planner.build_plan = orig
|
||||
deps_a = next(s for s in filtered if s["task_id"] == "a")["depends_on"]
|
||||
deps_b = next(s for s in filtered if s["task_id"] == "b")["depends_on"]
|
||||
check("phantom dependency filtered", deps_a == [])
|
||||
check("valid dependency kept", deps_b == ["a"])
|
||||
|
||||
|
||||
async def test_agent_peer_routing():
|
||||
from agent.main import Agent
|
||||
|
||||
a = Agent(orchestrator_url="ws://localhost:8000", agent_id="agent-peer", capabilities=["python"])
|
||||
|
||||
sent = []
|
||||
|
||||
async def fake_send(payload):
|
||||
sent.append(payload)
|
||||
|
||||
a.safe_send = fake_send
|
||||
|
||||
# Inbound peer query (no matching waiter) -> agent answers with a reply.
|
||||
await a.handle_peer_message({
|
||||
"type": "peer_message",
|
||||
"from_agent_id": "agent-impl",
|
||||
"task_id": "t1",
|
||||
"content": "please advise",
|
||||
"correlation_id": "corr-1",
|
||||
"is_reply": False,
|
||||
})
|
||||
check("peer query produces a reply", len(sent) == 1 and sent[0]["is_reply"] is True)
|
||||
check("peer reply targets requester", sent[0]["target_agent_id"] == "agent-impl")
|
||||
|
||||
# Inbound reply resolves an outstanding waiter (the requester side).
|
||||
loop = asyncio.get_running_loop()
|
||||
waiter = loop.create_future()
|
||||
a.peer_waiters["corr-2"] = waiter
|
||||
await a.handle_peer_message({
|
||||
"type": "peer_message",
|
||||
"from_agent_id": "agent-impl",
|
||||
"task_id": "t1",
|
||||
"content": "here is guidance",
|
||||
"correlation_id": "corr-2",
|
||||
"is_reply": True,
|
||||
})
|
||||
check("peer reply resolves waiter", waiter.done() and waiter.result()["content"] == "here is guidance")
|
||||
|
||||
# Capacity rejection: fill active_tasks to the limit, then a new assignment is rejected.
|
||||
sent.clear()
|
||||
a.active_tasks = {f"t{i}": None for i in range(a.MAX_CONCURRENT_TASKS)}
|
||||
await a.handle_task_assignment({"task_id": "overflow", "description": "x", "context": {}})
|
||||
check("over-capacity assignment is rejected", len(sent) == 1 and sent[0]["type"] == "task_rejected")
|
||||
|
||||
|
||||
async def test_review_and_synthesis():
|
||||
# planner.review heuristic rejects conflicting test frameworks.
|
||||
conflicting = {
|
||||
"swarm-r-testing": {"result": {"subtasks": [{"summary": "use pytest", "changes": "pytest suite"}]}},
|
||||
"swarm-r-doc": {"result": {"subtasks": [{"summary": "docs say unittest", "changes": "unittest examples"}]}},
|
||||
}
|
||||
verdict = await planner.review("obj", [], conflicting)
|
||||
check("review rejects conflicting frameworks", verdict["accepted"] is False and verdict["retry_tasks"])
|
||||
|
||||
aligned = {"swarm-r-impl": {"result": {"subtasks": [{"summary": "clean implementation", "changes": "added add()"}]}}}
|
||||
verdict2 = await planner.review("obj", [], aligned)
|
||||
check("review accepts aligned results", verdict2["accepted"] is True)
|
||||
|
||||
# synthesize falls back to a deterministic concatenation when no model is configured.
|
||||
synth = await planner.synthesize("obj", aligned)
|
||||
check("synthesize produces a non-empty response", isinstance(synth, str) and "implementation" in synth)
|
||||
|
||||
|
||||
async def test_review_cycle():
|
||||
# Patch out callback/persistence side effects and force a rejection from the critic.
|
||||
orig_save, orig_emit, orig_review = orch.swarm_runtime.save_run, orch.swarm_runtime.emit_event, orch.planner.review
|
||||
|
||||
async def noop(*a, **k):
|
||||
return None
|
||||
|
||||
orch.swarm_runtime.save_run = noop
|
||||
orch.swarm_runtime.emit_event = noop
|
||||
|
||||
task = await task_queue.create_task(description="impl", task_id="t-rev-1")
|
||||
await task_queue.complete_task(task.task_id, '{"summary": "did impl"}')
|
||||
completed = await task_queue.get_task(task.task_id)
|
||||
run = types.SimpleNamespace(
|
||||
swarm_id="swarm-rev", deployment_id="dep-rev", manager_deployment_id="mgr-rev",
|
||||
objective="obj", task_ids=["t-rev-1"], status="completed", metadata={}
|
||||
)
|
||||
|
||||
async def reject(objective, tasks, results):
|
||||
return {"accepted": False, "summary": "needs work", "retry_tasks": ["t-rev-1"]}
|
||||
|
||||
orch.planner.review = reject
|
||||
try:
|
||||
reopened = await orch.maybe_run_review_cycle(run, [completed])
|
||||
check("review cycle reopens rejected task", reopened is True)
|
||||
reloaded = await task_queue.get_task("t-rev-1")
|
||||
check("reopened task is PENDING again", reloaded.status == TaskStatus.PENDING)
|
||||
check("review cycle counter incremented", run.metadata.get("review_cycles") == 1)
|
||||
check("run set back to running", run.status == "running")
|
||||
|
||||
# Exhaust the cycle budget -> no further reopen.
|
||||
run.metadata["review_cycles"] = orch.review_max_cycles()
|
||||
await task_queue.complete_task("t-rev-1", '{"summary": "did impl again"}')
|
||||
completed2 = await task_queue.get_task("t-rev-1")
|
||||
reopened2 = await orch.maybe_run_review_cycle(run, [completed2])
|
||||
check("review cycle respects budget", reopened2 is False)
|
||||
finally:
|
||||
orch.swarm_runtime.save_run = orig_save
|
||||
orch.swarm_runtime.emit_event = orig_emit
|
||||
orch.planner.review = orig_review
|
||||
|
||||
|
||||
async def test_dispatch_context():
|
||||
# A dependency that completed should appear as a dependency artifact; a connected peer
|
||||
# assigned to another task on the run should appear as a peer agent.
|
||||
orch.manager.active_connections["peer-conn"] = object()
|
||||
try:
|
||||
dep = await task_queue.create_task(description="upstream", task_id="dc-dep")
|
||||
await task_queue.complete_task(dep.task_id, '{"summary": "upstream done", "files_modified": ["a.py"]}')
|
||||
peer = await task_queue.create_task(description="peer work", task_id="dc-peer", agent_role="implementation")
|
||||
await task_queue.assign_task(peer.task_id, "peer-conn") if False else None
|
||||
# Manually mark the peer task as owned by the connected agent.
|
||||
peer.assigned_agent_id = "peer-conn"
|
||||
await task_queue._save_task(peer)
|
||||
|
||||
consumer = await task_queue.create_task(
|
||||
description="downstream", task_id="dc-main", depends_on=["dc-dep"], agent_role="testing"
|
||||
)
|
||||
run = types.SimpleNamespace(swarm_id="swarm-dc", objective="ship it", task_ids=["dc-dep", "dc-peer", "dc-main"])
|
||||
ctx = await orch.build_dispatch_context(run, consumer)
|
||||
check("dispatch context injects dependency_artifacts", len(ctx.get("dependency_artifacts", [])) == 1)
|
||||
check("dependency artifact carries summary", ctx["dependency_artifacts"][0]["summary"] == "upstream done")
|
||||
check("dispatch context injects peer_agents", any(p["agent_id"] == "peer-conn" for p in ctx.get("peer_agents", [])))
|
||||
check("dispatch context carries run goal", ctx.get("run_goal") == "ship it")
|
||||
check("dispatch context sets specialist_role from agent_role", ctx.get("specialist_role") == "testing")
|
||||
finally:
|
||||
orch.manager.active_connections.pop("peer-conn", None)
|
||||
|
||||
|
||||
async def test_agent_peer_shares_summary():
|
||||
from agent.main import Agent
|
||||
|
||||
a = Agent(orchestrator_url="ws://localhost:8000", agent_id="agent-impl2", capabilities=["python"])
|
||||
a.last_summary = "implemented add() that raises ValueError on bad input"
|
||||
sent = []
|
||||
|
||||
async def fake_send(payload):
|
||||
sent.append(payload)
|
||||
|
||||
a.safe_send = fake_send
|
||||
await a.answer_peer_query({"from_agent_id": "agent-test", "correlation_id": "c", "is_reply": False})
|
||||
check("peer reply shares last summary", "implemented add()" in sent[0]["content"])
|
||||
|
||||
|
||||
async def main():
|
||||
await test_redis_fallback()
|
||||
await test_release_task()
|
||||
await test_planner_fallback()
|
||||
await test_agent_peer_routing()
|
||||
await test_review_and_synthesis()
|
||||
await test_review_cycle()
|
||||
await test_dispatch_context()
|
||||
await test_agent_peer_shares_summary()
|
||||
print()
|
||||
if failures:
|
||||
print(f"{len(failures)} check(s) FAILED: {failures}")
|
||||
sys.exit(1)
|
||||
print("all merge smoke checks passed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,203 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Test script for orchestrator functionality."""
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
from typing import Optional
|
||||
|
||||
try:
|
||||
import websockets
|
||||
import httpx
|
||||
except ImportError:
|
||||
print("Installing required packages...")
|
||||
import subprocess
|
||||
subprocess.check_call([sys.executable, "-m", "pip", "install", "websockets", "httpx"])
|
||||
import websockets
|
||||
import httpx
|
||||
|
||||
|
||||
ORCHESTRATOR_URL = "http://localhost:8000"
|
||||
ORCHESTRATOR_WS = "ws://localhost:8000"
|
||||
|
||||
|
||||
async def test_health_check():
|
||||
"""Test health check endpoint."""
|
||||
print("\n=== Testing Health Check ===")
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(f"{ORCHESTRATOR_URL}/health")
|
||||
print(f"Status: {response.status_code}")
|
||||
print(f"Response: {response.json()}")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["status"] == "ok"
|
||||
print("✅ Health check passed")
|
||||
|
||||
|
||||
async def test_agent_registration():
|
||||
"""Test agent registration via WebSocket."""
|
||||
print("\n=== Testing Agent Registration ===")
|
||||
|
||||
agent_id = "test-agent-1"
|
||||
uri = f"{ORCHESTRATOR_WS}/ws/{agent_id}"
|
||||
|
||||
async with websockets.connect(uri) as websocket:
|
||||
# Send registration
|
||||
await websocket.send(json.dumps({
|
||||
"type": "register",
|
||||
"capabilities": ["python", "javascript"]
|
||||
}))
|
||||
|
||||
# Receive confirmation
|
||||
response = await websocket.recv()
|
||||
data = json.loads(response)
|
||||
print(f"Registration response: {data}")
|
||||
assert data["type"] == "registered"
|
||||
assert data["agent_id"] == agent_id
|
||||
|
||||
# Verify agent appears in registry
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(f"{ORCHESTRATOR_URL}/agents/{agent_id}")
|
||||
agent_data = response.json()
|
||||
print(f"Agent data: {agent_data}")
|
||||
assert agent_data["agent_id"] == agent_id
|
||||
assert agent_data["status"] == "idle"
|
||||
|
||||
print("✅ Agent registration passed")
|
||||
|
||||
# Test heartbeat
|
||||
print("\n=== Testing Heartbeat ===")
|
||||
await websocket.send(json.dumps({"type": "heartbeat"}))
|
||||
response = await websocket.recv()
|
||||
data = json.loads(response)
|
||||
print(f"Heartbeat response: {data}")
|
||||
assert data["type"] == "heartbeat_ack"
|
||||
print("✅ Heartbeat passed")
|
||||
|
||||
|
||||
async def test_task_creation():
|
||||
"""Test task creation and assignment."""
|
||||
print("\n=== Testing Task Creation ===")
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
# Create task
|
||||
response = await client.post(
|
||||
f"{ORCHESTRATOR_URL}/tasks",
|
||||
json={
|
||||
"description": "Test task",
|
||||
"context": {"test": "data"},
|
||||
"max_retries": 3
|
||||
}
|
||||
)
|
||||
task_data = response.json()
|
||||
print(f"Created task: {task_data}")
|
||||
assert response.status_code == 200
|
||||
assert task_data["status"] == "pending"
|
||||
|
||||
task_id = task_data["task_id"]
|
||||
|
||||
# List tasks
|
||||
response = await client.get(f"{ORCHESTRATOR_URL}/tasks")
|
||||
tasks = response.json()
|
||||
print(f"Total tasks: {len(tasks['tasks'])}")
|
||||
|
||||
print("✅ Task creation passed")
|
||||
return task_id
|
||||
|
||||
|
||||
async def test_handoff():
|
||||
"""Test handoff mechanism between two agents."""
|
||||
print("\n=== Testing Handoff Mechanism ===")
|
||||
|
||||
agent1_id = "test-agent-1"
|
||||
agent2_id = "test-agent-2"
|
||||
|
||||
# Connect both agents
|
||||
uri1 = f"{ORCHESTRATOR_WS}/ws/{agent1_id}"
|
||||
uri2 = f"{ORCHESTRATOR_WS}/ws/{agent2_id}"
|
||||
|
||||
async with websockets.connect(uri1) as ws1, websockets.connect(uri2) as ws2:
|
||||
# Register agent 1
|
||||
await ws1.send(json.dumps({
|
||||
"type": "register",
|
||||
"capabilities": ["python"]
|
||||
}))
|
||||
await ws1.recv()
|
||||
|
||||
# Register agent 2
|
||||
await ws2.send(json.dumps({
|
||||
"type": "register",
|
||||
"capabilities": ["javascript"]
|
||||
}))
|
||||
await ws2.recv()
|
||||
|
||||
# Agent 1 initiates handoff to agent 2
|
||||
await ws1.send(json.dumps({
|
||||
"type": "handoff",
|
||||
"target_agent_id": agent2_id,
|
||||
"task_context": {
|
||||
"task_id": "test-task-123",
|
||||
"description": "Test handoff",
|
||||
"files": ["test.py"]
|
||||
}
|
||||
}))
|
||||
|
||||
# Agent 1 receives handoff confirmation
|
||||
response1 = await ws1.recv()
|
||||
data1 = json.loads(response1)
|
||||
print(f"Agent 1 response: {data1}")
|
||||
assert data1["type"] == "handoff_initiated"
|
||||
|
||||
handoff_id = data1["handoff_id"]
|
||||
|
||||
# Agent 2 receives handoff request
|
||||
response2 = await ws2.recv()
|
||||
data2 = json.loads(response2)
|
||||
print(f"Agent 2 response: {data2}")
|
||||
assert data2["type"] == "handoff_request"
|
||||
assert data2["handoff_id"] == handoff_id
|
||||
|
||||
# Agent 2 accepts handoff
|
||||
await ws2.send(json.dumps({
|
||||
"type": "handoff_accept",
|
||||
"handoff_id": handoff_id
|
||||
}))
|
||||
|
||||
response2 = await ws2.recv()
|
||||
data2 = json.loads(response2)
|
||||
print(f"Agent 2 accept response: {data2}")
|
||||
assert data2["type"] == "handoff_accepted"
|
||||
|
||||
# Verify handoff in history
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(f"{ORCHESTRATOR_URL}/handoffs")
|
||||
handoffs = response.json()
|
||||
print(f"Total handoffs: {len(handoffs['handoffs'])}")
|
||||
assert len(handoffs['handoffs']) > 0
|
||||
|
||||
print("✅ Handoff mechanism passed")
|
||||
|
||||
|
||||
async def main():
|
||||
"""Run all tests."""
|
||||
print("Starting orchestrator tests...")
|
||||
print("Make sure orchestrator is running on localhost:8000")
|
||||
print("Run: kubectl port-forward -n swarm-system svc/orchestrator-service 8000:8000")
|
||||
|
||||
try:
|
||||
await test_health_check()
|
||||
await test_agent_registration()
|
||||
await test_task_creation()
|
||||
await test_handoff()
|
||||
|
||||
print("\n" + "="*50)
|
||||
print("✅ All tests passed!")
|
||||
print("="*50)
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n❌ Test failed: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Lightweight contract checks for the HeiCode Agent Manager runtime bridge."""
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
if "redis.asyncio" not in sys.modules:
|
||||
redis_module = types.ModuleType("redis")
|
||||
redis_asyncio_module = types.ModuleType("redis.asyncio")
|
||||
redis_asyncio_module.Redis = object
|
||||
redis_module.asyncio = redis_asyncio_module
|
||||
sys.modules["redis"] = redis_module
|
||||
sys.modules["redis.asyncio"] = redis_asyncio_module
|
||||
if "httpx" not in sys.modules:
|
||||
httpx_module = types.ModuleType("httpx")
|
||||
|
||||
class AsyncClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
return False
|
||||
|
||||
async def post(self, *args, **kwargs):
|
||||
return types.SimpleNamespace(status_code=204, text="")
|
||||
|
||||
httpx_module.AsyncClient = AsyncClient
|
||||
sys.modules["httpx"] = httpx_module
|
||||
|
||||
from orchestrator.redis_client import redis_client
|
||||
from orchestrator.swarm_runtime import RuntimeValidationError, SwarmRuntime
|
||||
|
||||
|
||||
class FakeRedis:
|
||||
def __init__(self):
|
||||
self.values = {}
|
||||
self.lists = {}
|
||||
|
||||
async def set(self, key, value, ex=None):
|
||||
self.values[key] = value
|
||||
|
||||
async def get(self, key):
|
||||
return self.values.get(key)
|
||||
|
||||
async def keys(self, pattern):
|
||||
prefix = pattern.rstrip("*")
|
||||
return [key for key in self.values if key.startswith(prefix)]
|
||||
|
||||
async def rpush(self, key, *values):
|
||||
self.lists.setdefault(key, []).extend(values)
|
||||
|
||||
async def lrange(self, key, start, end):
|
||||
items = self.lists.get(key, [])
|
||||
if end == -1:
|
||||
return items[start:]
|
||||
return items[start : end + 1]
|
||||
|
||||
|
||||
def valid_body():
|
||||
return {
|
||||
"orchestration_plan": {
|
||||
"objective": "Add a multiply helper and push the branch",
|
||||
"sub_mode": "code",
|
||||
"risk_level": "low",
|
||||
"budget": {"duration_seconds": 3600, "token_limit": 20000},
|
||||
"agents": [
|
||||
{
|
||||
"task_id": "backend-1",
|
||||
"role": "backend",
|
||||
"title": "Backend change",
|
||||
"description": "Implement the helper",
|
||||
"depends_on": [],
|
||||
"resource_grants": [
|
||||
{"secret_ref": "azkv://heicode/git-write-token"}
|
||||
],
|
||||
}
|
||||
],
|
||||
},
|
||||
"callback": {
|
||||
"url": "http://manager.local/api/agnet/callbacks/swarm-events",
|
||||
"subscribed_events": ["task.created"],
|
||||
},
|
||||
"metadata": {
|
||||
"manager_deployment_id": "dep_manager_123",
|
||||
"correlation_id": "corr_123",
|
||||
},
|
||||
"billing_context": {"secret_ref": "azkv://heicode/billing"},
|
||||
"resource_grants": [{"ref": "azkv://heicode/repo-main"}],
|
||||
}
|
||||
|
||||
|
||||
async def main():
|
||||
fake = FakeRedis()
|
||||
redis_client.client = fake
|
||||
runtime = SwarmRuntime()
|
||||
os.environ["ENABLE_SUBTASK_HANDOFF"] = "true"
|
||||
|
||||
body = valid_body()
|
||||
runtime.validate_create_request(body)
|
||||
|
||||
invalid = valid_body()
|
||||
invalid["resource_grants"][0]["ref"] = "plain-token"
|
||||
try:
|
||||
runtime.validate_create_request(invalid)
|
||||
raise AssertionError("plain resource ref was accepted")
|
||||
except RuntimeValidationError:
|
||||
pass
|
||||
|
||||
invalid = valid_body()
|
||||
invalid["metadata"]["api_key"] = "secret-value"
|
||||
try:
|
||||
runtime.validate_create_request(invalid)
|
||||
raise AssertionError("plaintext secret metadata was accepted")
|
||||
except RuntimeValidationError:
|
||||
pass
|
||||
|
||||
run, created = await runtime.get_or_create_run(body, "idem-1", "corr_123")
|
||||
assert created is True
|
||||
same_run, created_again = await runtime.get_or_create_run(body, "idem-1", "corr_123")
|
||||
assert created_again is False
|
||||
assert same_run.swarm_id == run.swarm_id
|
||||
|
||||
tasks = runtime.build_task_descriptions(body)
|
||||
assert tasks[0]["task_id"] == "backend-1"
|
||||
assert tasks[0]["depends_on"] == []
|
||||
assert tasks[0]["workflow_mode"] == "multi_agent"
|
||||
|
||||
await runtime.emit_event(run, "timeline.updated", payload={"summary": "ok"})
|
||||
logs = await runtime.list_events(run.swarm_id)
|
||||
assert logs["events"][0]["event_type"] == "deployment.status_changed"
|
||||
assert logs["events"][-1]["event_type"] == "timeline.updated"
|
||||
|
||||
approval_body = valid_body()
|
||||
approval_body["orchestration_plan"]["risk_level"] = "high"
|
||||
approval_run, _ = await runtime.get_or_create_run(approval_body, "idem-2", "corr_123")
|
||||
approval_id = next(iter(approval_run.approvals))
|
||||
rejected = await runtime.record_approval_decision(
|
||||
approval_run.swarm_id,
|
||||
approval_id,
|
||||
{"decision": "rejected", "reason": "user rejected"},
|
||||
)
|
||||
assert rejected.status == "blocked"
|
||||
|
||||
print("runtime contract checks passed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,132 @@
|
||||
"""End-to-end workflow test: proves the program follows the expected workflow.
|
||||
|
||||
Boots the REAL orchestrator (uvicorn, in-process) on the in-memory store, connects a keyless
|
||||
stub agent over WebSocket, submits one objective, and asserts the full loop:
|
||||
|
||||
decompose -> dispatch to experts -> execute -> master review (with one forced reopen)
|
||||
-> iterate -> synthesize -> deliver
|
||||
|
||||
No OPENAI_API_KEY required: the planner is forced offline (static decomposition + heuristic
|
||||
review + concatenated synthesis), and the stub agent returns canned results that make the
|
||||
critic reject exactly once before accepting.
|
||||
|
||||
Run from the agent_swarm_v5 directory:
|
||||
python scripts/test-workflow-e2e.py
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
# Configure the runtime for a deterministic, hermetic run BEFORE importing the app.
|
||||
os.environ["REDIS_FAKE"] = "1"
|
||||
os.environ["ENABLE_PLANNER_FALLBACK"] = "1"
|
||||
os.environ["ENABLE_REVIEW_LOOP"] = "1"
|
||||
os.environ["MAX_REVIEW_CYCLES"] = "2"
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
sys.path.insert(0, str(ROOT / "scripts"))
|
||||
|
||||
import httpx
|
||||
import uvicorn
|
||||
from orchestrator import main as orch
|
||||
from stub_agent import StubAgent
|
||||
|
||||
# Force the planner/critic/synthesis offline so the workflow is deterministic regardless of any
|
||||
# .env key (load_dotenv runs at import). Quiet the dummy-callback warnings.
|
||||
orch.planner.client = None
|
||||
logging.getLogger("orchestrator.swarm_runtime").setLevel(logging.ERROR)
|
||||
|
||||
PORT = 8123
|
||||
BASE = f"http://127.0.0.1:{PORT}"
|
||||
WS = f"ws://127.0.0.1:{PORT}"
|
||||
|
||||
failures = []
|
||||
|
||||
|
||||
def check(name, cond):
|
||||
print(("PASS" if cond else "FAIL"), "-", name)
|
||||
if not cond:
|
||||
failures.append(name)
|
||||
|
||||
|
||||
async def wait_health(client):
|
||||
for _ in range(150):
|
||||
try:
|
||||
if (await client.get(f"{BASE}/health")).status_code == 200:
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
await asyncio.sleep(0.1)
|
||||
return False
|
||||
|
||||
|
||||
async def main():
|
||||
config = uvicorn.Config(orch.app, host="127.0.0.1", port=PORT, log_level="warning")
|
||||
server = uvicorn.Server(config)
|
||||
server.install_signal_handlers = lambda: None # required when not on the main thread
|
||||
threading.Thread(target=server.run, daemon=True).start()
|
||||
|
||||
agent = StubAgent(WS, "stub-1", [
|
||||
"python", "code_generation", "testing", "pytest", "technical-writing", "general",
|
||||
])
|
||||
agent_task = None
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10) as client:
|
||||
if not await wait_health(client):
|
||||
check("orchestrator started", False)
|
||||
return
|
||||
|
||||
agent_task = asyncio.create_task(agent.run())
|
||||
await asyncio.sleep(0.5) # let the agent register
|
||||
|
||||
body = {
|
||||
"mode": "swarm",
|
||||
"requirement": {"objective": "Write a Python add(a,b) function with tests and docs"},
|
||||
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
||||
"metadata": {"manager_deployment_id": "e2e-1"},
|
||||
}
|
||||
created = (await client.post(f"{BASE}/api/swarms", json=body)).json()
|
||||
dep = created["data"]["deployment_id"]
|
||||
|
||||
# Poll until the run finishes (or times out).
|
||||
status, wf = None, {}
|
||||
for _ in range(200):
|
||||
wf = (await client.get(f"{BASE}/api/swarms/{dep}/workflow")).json()["data"]
|
||||
status = wf.get("status")
|
||||
if status in ("completed", "failed"):
|
||||
break
|
||||
await asyncio.sleep(0.25)
|
||||
|
||||
tasks = (await client.get(f"{BASE}/api/swarms/{dep}/tasks")).json()["data"]["tasks"]
|
||||
events = (await client.get(f"{BASE}/api/swarms/{dep}/logs")).json()["data"]["events"]
|
||||
roles = sorted(t["agent_role"] for t in tasks)
|
||||
summaries = [(e.get("payload") or {}).get("summary", "") or "" for e in events]
|
||||
review_seen = any("Review cycle" in s for s in summaries)
|
||||
|
||||
# ---- workflow assertions ----
|
||||
check("decompose: 3 specialist tasks created", len(tasks) == 3)
|
||||
check("decompose: roles are implementation/testing/documentation",
|
||||
roles == ["documentation", "implementation", "testing"])
|
||||
check("execute: every task completed", bool(tasks) and all(t["status"] == "completed" for t in tasks))
|
||||
check("iterate: at least one master review cycle occurred", review_seen)
|
||||
check("deliver: run reached completed", status == "completed")
|
||||
check("synthesize: a final unified summary is present", bool(wf.get("summary")))
|
||||
finally:
|
||||
agent.running = False
|
||||
if agent_task:
|
||||
agent_task.cancel()
|
||||
server.should_exit = True
|
||||
|
||||
print()
|
||||
if failures:
|
||||
print(f"{len(failures)} workflow check(s) FAILED: {failures}")
|
||||
sys.exit(1)
|
||||
print("workflow follows the expected sequence: decompose -> dispatch -> execute -> review/iterate -> synthesize")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,308 @@
|
||||
[
|
||||
{
|
||||
"task": "Fix syntax error in main.py line 42",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Single file, single line fix. No dependencies or side effects."
|
||||
},
|
||||
{
|
||||
"task": "Add type hints to the calculate_total function",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Single function modification in one file."
|
||||
},
|
||||
{
|
||||
"task": "Update README.md with installation instructions",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Documentation update, no code changes."
|
||||
},
|
||||
{
|
||||
"task": "Rename variable 'data' to 'user_data' in auth.py",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Simple refactoring in single file."
|
||||
},
|
||||
{
|
||||
"task": "Fix import statement missing requests library",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Single line addition at top of file."
|
||||
},
|
||||
{
|
||||
"task": "Add docstring to the process_payment function",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Documentation addition to single function."
|
||||
},
|
||||
{
|
||||
"task": "Remove unused import statements from utils.py",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Simple cleanup in one file."
|
||||
},
|
||||
{
|
||||
"task": "Change default timeout from 30 to 60 seconds",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Single constant value change."
|
||||
},
|
||||
{
|
||||
"task": "Fix indentation error in the for loop",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Formatting fix in single location."
|
||||
},
|
||||
{
|
||||
"task": "Add missing return statement in validate_email function",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Single line addition to fix logic error."
|
||||
},
|
||||
{
|
||||
"task": "Refactor the user authentication module to use JWT tokens",
|
||||
"expectedAgents": 3,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Affects multiple functions in auth module, requires updating login/logout flows and adding token validation."
|
||||
},
|
||||
{
|
||||
"task": "Add input validation to all API endpoints",
|
||||
"expectedAgents": 5,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Multiple endpoints across different route files need validation logic."
|
||||
},
|
||||
{
|
||||
"task": "Implement caching layer using Redis for database queries",
|
||||
"expectedAgents": 4,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Requires Redis setup, cache invalidation logic, and updating multiple query functions."
|
||||
},
|
||||
{
|
||||
"task": "Add logging to all service layer functions",
|
||||
"expectedAgents": 3,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Multiple service files need consistent logging patterns."
|
||||
},
|
||||
{
|
||||
"task": "Migrate database from SQLite to PostgreSQL",
|
||||
"expectedAgents": 6,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Schema migration, connection updates, query syntax changes across multiple files."
|
||||
},
|
||||
{
|
||||
"task": "Implement rate limiting for API endpoints",
|
||||
"expectedAgents": 3,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Middleware implementation and configuration for multiple routes."
|
||||
},
|
||||
{
|
||||
"task": "Add error handling to all async functions",
|
||||
"expectedAgents": 4,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Multiple async functions across different modules need try-catch blocks."
|
||||
},
|
||||
{
|
||||
"task": "Refactor monolithic app.py into separate route modules",
|
||||
"expectedAgents": 5,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Code organization across multiple new files with proper imports."
|
||||
},
|
||||
{
|
||||
"task": "Add pagination to all list endpoints",
|
||||
"expectedAgents": 4,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Multiple endpoints need offset/limit logic and response formatting."
|
||||
},
|
||||
{
|
||||
"task": "Implement user role-based access control",
|
||||
"expectedAgents": 5,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Decorator implementation, role checks across multiple endpoints, database schema update."
|
||||
},
|
||||
{
|
||||
"task": "Add comprehensive unit tests to the entire project",
|
||||
"expectedAgents": 12,
|
||||
"complexity": "high",
|
||||
"reasoning": "Testing all modules, functions, edge cases. Requires understanding entire codebase and writing hundreds of test cases."
|
||||
},
|
||||
{
|
||||
"task": "Refactor entire codebase to follow clean architecture principles",
|
||||
"expectedAgents": 15,
|
||||
"complexity": "high",
|
||||
"reasoning": "Major restructuring: separate layers (domain, application, infrastructure), update all imports, maintain functionality."
|
||||
},
|
||||
{
|
||||
"task": "Implement microservices architecture splitting monolith into 5 services",
|
||||
"expectedAgents": 18,
|
||||
"complexity": "high",
|
||||
"reasoning": "Service separation, API contracts, inter-service communication, deployment configs, data consistency."
|
||||
},
|
||||
{
|
||||
"task": "Add end-to-end tests for all user workflows",
|
||||
"expectedAgents": 10,
|
||||
"complexity": "high",
|
||||
"reasoning": "Multiple user journeys, test setup, teardown, data fixtures, UI interactions."
|
||||
},
|
||||
{
|
||||
"task": "Migrate from REST API to GraphQL",
|
||||
"expectedAgents": 14,
|
||||
"complexity": "high",
|
||||
"reasoning": "Schema definition, resolver implementation, client updates, query optimization across entire API."
|
||||
},
|
||||
{
|
||||
"task": "Implement real-time features using WebSockets",
|
||||
"expectedAgents": 8,
|
||||
"complexity": "high",
|
||||
"reasoning": "WebSocket server setup, connection management, event broadcasting, client integration across features."
|
||||
},
|
||||
{
|
||||
"task": "Add internationalization support for 10 languages",
|
||||
"expectedAgents": 9,
|
||||
"complexity": "high",
|
||||
"reasoning": "Extract all strings, translation files, locale switching, date/number formatting across entire app."
|
||||
},
|
||||
{
|
||||
"task": "Implement comprehensive security audit fixes",
|
||||
"expectedAgents": 11,
|
||||
"complexity": "high",
|
||||
"reasoning": "SQL injection prevention, XSS protection, CSRF tokens, secure headers, input sanitization across all endpoints."
|
||||
},
|
||||
{
|
||||
"task": "Refactor to use async/await throughout the codebase",
|
||||
"expectedAgents": 13,
|
||||
"complexity": "high",
|
||||
"reasoning": "Convert synchronous code to async, update all function signatures, handle async context across entire project."
|
||||
},
|
||||
{
|
||||
"task": "Build CI/CD pipeline with automated testing and deployment",
|
||||
"expectedAgents": 7,
|
||||
"complexity": "high",
|
||||
"reasoning": "Pipeline configuration, test automation, deployment scripts, environment management, rollback strategy."
|
||||
},
|
||||
{
|
||||
"task": "Add a new user registration endpoint",
|
||||
"expectedAgents": 2,
|
||||
"complexity": "low",
|
||||
"reasoning": "Single endpoint with validation and database insert."
|
||||
},
|
||||
{
|
||||
"task": "Update email template for password reset",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Template file modification only."
|
||||
},
|
||||
{
|
||||
"task": "Add environment variable for API key",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Config file update and usage in one location."
|
||||
},
|
||||
{
|
||||
"task": "Fix broken link in navigation menu",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Single URL correction in template."
|
||||
},
|
||||
{
|
||||
"task": "Add favicon to the website",
|
||||
"expectedAgents": 1,
|
||||
"complexity": "low",
|
||||
"reasoning": "Add file and reference in HTML head."
|
||||
},
|
||||
{
|
||||
"task": "Implement search functionality across user profiles",
|
||||
"expectedAgents": 3,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Search endpoint, query optimization, result formatting."
|
||||
},
|
||||
{
|
||||
"task": "Add file upload feature with validation",
|
||||
"expectedAgents": 3,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Upload endpoint, file validation, storage handling, error cases."
|
||||
},
|
||||
{
|
||||
"task": "Implement email notification system",
|
||||
"expectedAgents": 4,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Email service integration, template system, queue management, multiple trigger points."
|
||||
},
|
||||
{
|
||||
"task": "Add OAuth2 authentication with Google and GitHub",
|
||||
"expectedAgents": 5,
|
||||
"complexity": "medium",
|
||||
"reasoning": "OAuth flow implementation, provider integration, user linking, token management."
|
||||
},
|
||||
{
|
||||
"task": "Create admin dashboard with analytics",
|
||||
"expectedAgents": 6,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Dashboard UI, data aggregation queries, charts, multiple admin views."
|
||||
},
|
||||
{
|
||||
"task": "Optimize database queries for performance",
|
||||
"expectedAgents": 5,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Identify slow queries, add indexes, refactor N+1 queries across multiple models."
|
||||
},
|
||||
{
|
||||
"task": "Implement data export feature in CSV and JSON formats",
|
||||
"expectedAgents": 3,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Export endpoints, format serialization, large dataset handling."
|
||||
},
|
||||
{
|
||||
"task": "Add two-factor authentication",
|
||||
"expectedAgents": 4,
|
||||
"complexity": "medium",
|
||||
"reasoning": "TOTP implementation, QR code generation, backup codes, login flow updates."
|
||||
},
|
||||
{
|
||||
"task": "Implement audit logging for all data changes",
|
||||
"expectedAgents": 6,
|
||||
"complexity": "medium",
|
||||
"reasoning": "Audit table design, triggers/hooks across models, query interface, retention policy."
|
||||
},
|
||||
{
|
||||
"task": "Add real-time notifications using Server-Sent Events",
|
||||
"expectedAgents": 4,
|
||||
"complexity": "medium",
|
||||
"reasoning": "SSE endpoint, event broadcasting, client connection management, notification types."
|
||||
},
|
||||
{
|
||||
"task": "Rewrite frontend from jQuery to React",
|
||||
"expectedAgents": 16,
|
||||
"complexity": "high",
|
||||
"reasoning": "Complete frontend rewrite, component architecture, state management, API integration, routing."
|
||||
},
|
||||
{
|
||||
"task": "Implement distributed tracing with OpenTelemetry",
|
||||
"expectedAgents": 8,
|
||||
"complexity": "high",
|
||||
"reasoning": "Instrumentation across all services, trace context propagation, exporter configuration, dashboard setup."
|
||||
},
|
||||
{
|
||||
"task": "Add machine learning model for recommendation system",
|
||||
"expectedAgents": 10,
|
||||
"complexity": "high",
|
||||
"reasoning": "Model training pipeline, feature engineering, inference API, model versioning, A/B testing."
|
||||
},
|
||||
{
|
||||
"task": "Implement event sourcing architecture",
|
||||
"expectedAgents": 14,
|
||||
"complexity": "high",
|
||||
"reasoning": "Event store setup, aggregate design, event handlers, projections, migration from current state."
|
||||
},
|
||||
{
|
||||
"task": "Build mobile app with React Native",
|
||||
"expectedAgents": 12,
|
||||
"complexity": "high",
|
||||
"reasoning": "Mobile app setup, screen components, navigation, API integration, platform-specific features, testing."
|
||||
},
|
||||
{
|
||||
"task": "Implement comprehensive monitoring and alerting",
|
||||
"expectedAgents": 9,
|
||||
"complexity": "high",
|
||||
"reasoning": "Metrics collection, alert rules, dashboard creation, on-call integration, SLO definition."
|
||||
}
|
||||
]
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
K8s 蜂群系统测试脚本
|
||||
测试 Orchestrator API 和系统功能
|
||||
"""
|
||||
|
||||
import requests
|
||||
import time
|
||||
import sys
|
||||
|
||||
# Orchestrator 地址
|
||||
ORCHESTRATOR_URL = "http://52.139.240.116:8000"
|
||||
|
||||
def print_section(title):
|
||||
"""打印分隔线"""
|
||||
print(f"\n{'='*60}")
|
||||
print(f" {title}")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
def test_health():
|
||||
"""测试健康检查"""
|
||||
print_section("1. 健康检查")
|
||||
try:
|
||||
response = requests.get(f"{ORCHESTRATOR_URL}/health", timeout=5)
|
||||
data = response.json()
|
||||
print(f"✅ 状态: {data['status']}")
|
||||
print(f"✅ Redis: {data['redis']}")
|
||||
print(f"✅ 活跃连接: {data['active_connections']}")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"❌ 健康检查失败: {e}")
|
||||
return False
|
||||
|
||||
def test_agents():
|
||||
"""测试 Agent 列表"""
|
||||
print_section("2. Agent 列表")
|
||||
try:
|
||||
response = requests.get(f"{ORCHESTRATOR_URL}/agents", timeout=5)
|
||||
agents = response.json().get("agents", [])
|
||||
print(f"当前 Agent 数量: {len(agents)}")
|
||||
if agents:
|
||||
for agent in agents:
|
||||
print(f" - Agent ID: {agent['agent_id']}")
|
||||
print(f" 状态: {agent['status']}")
|
||||
print(f" 最后心跳: {agent.get('last_heartbeat', 'N/A')}")
|
||||
else:
|
||||
print(" 暂无 Agent 运行")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"❌ 获取 Agent 列表失败: {e}")
|
||||
return False
|
||||
|
||||
def test_tasks():
|
||||
"""测试任务列表"""
|
||||
print_section("3. 任务列表")
|
||||
try:
|
||||
response = requests.get(f"{ORCHESTRATOR_URL}/tasks", timeout=5)
|
||||
tasks = response.json().get("tasks", [])
|
||||
print(f"任务总数: {len(tasks)}")
|
||||
if tasks:
|
||||
for task in tasks:
|
||||
print(f" - 任务 ID: {task['task_id']}")
|
||||
print(f" 状态: {task['status']}")
|
||||
print(f" 描述: {task.get('description', 'N/A')[:50]}...")
|
||||
else:
|
||||
print(" 暂无任务")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"❌ 获取任务列表失败: {e}")
|
||||
return False
|
||||
|
||||
def test_create_task():
|
||||
"""测试创建任务"""
|
||||
print_section("4. 创建测试任务")
|
||||
try:
|
||||
task_data = {
|
||||
"description": "测试任务:修复一个简单的语法错误",
|
||||
"context": {
|
||||
"workspace_id": "test-workspace-001",
|
||||
"required_agents": 1
|
||||
},
|
||||
"max_retries": 3
|
||||
}
|
||||
response = requests.post(
|
||||
f"{ORCHESTRATOR_URL}/tasks",
|
||||
json=task_data,
|
||||
timeout=5
|
||||
)
|
||||
if response.status_code == 200:
|
||||
task = response.json()
|
||||
print(f"✅ 任务创建成功")
|
||||
print(f" 任务 ID: {task['task_id']}")
|
||||
print(f" 状态: {task['status']}")
|
||||
return task['task_id']
|
||||
else:
|
||||
print(f"❌ 任务创建失败: {response.status_code}")
|
||||
print(f" 响应: {response.text}")
|
||||
return None
|
||||
except Exception as e:
|
||||
print(f"❌ 创建任务失败: {e}")
|
||||
return None
|
||||
|
||||
def test_metrics():
|
||||
"""测试 Prometheus 指标"""
|
||||
print_section("5. Prometheus 指标")
|
||||
try:
|
||||
response = requests.get(f"{ORCHESTRATOR_URL}/metrics", timeout=5)
|
||||
metrics = response.text
|
||||
|
||||
# 提取关键指标
|
||||
lines = metrics.split('\n')
|
||||
key_metrics = [
|
||||
'swarm_agents_created_total',
|
||||
'swarm_agents_active',
|
||||
'swarm_tasks_created_total',
|
||||
'swarm_tasks_completed_total'
|
||||
]
|
||||
|
||||
print("关键指标:")
|
||||
for line in lines:
|
||||
for metric in key_metrics:
|
||||
if line.startswith(metric) and not line.startswith('#'):
|
||||
print(f" {line}")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"❌ 获取指标失败: {e}")
|
||||
return False
|
||||
|
||||
def test_redis_connection():
|
||||
"""测试 Redis 连接"""
|
||||
print_section("6. Redis 连接测试")
|
||||
try:
|
||||
# 通过健康检查验证 Redis
|
||||
response = requests.get(f"{ORCHESTRATOR_URL}/health", timeout=5)
|
||||
data = response.json()
|
||||
if data['redis'] == 'connected':
|
||||
print("✅ Redis 连接正常")
|
||||
return True
|
||||
else:
|
||||
print(f"❌ Redis 状态: {data['redis']}")
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"❌ Redis 连接测试失败: {e}")
|
||||
return False
|
||||
|
||||
def main():
|
||||
"""主测试流程"""
|
||||
print("\n" + "="*60)
|
||||
print(" K8s 蜂群系统 - 集成测试")
|
||||
print("="*60)
|
||||
print(f"\nOrchestrator URL: {ORCHESTRATOR_URL}")
|
||||
print(f"测试时间: {time.strftime('%Y-%m-%d %H:%M:%S')}")
|
||||
|
||||
results = []
|
||||
|
||||
# 运行所有测试
|
||||
results.append(("健康检查", test_health()))
|
||||
results.append(("Redis 连接", test_redis_connection()))
|
||||
results.append(("Agent 列表", test_agents()))
|
||||
results.append(("任务列表", test_tasks()))
|
||||
results.append(("创建任务", test_create_task() is not None))
|
||||
results.append(("Prometheus 指标", test_metrics()))
|
||||
|
||||
# 总结
|
||||
print_section("测试总结")
|
||||
passed = sum(1 for _, result in results if result)
|
||||
total = len(results)
|
||||
|
||||
for name, result in results:
|
||||
status = "✅ 通过" if result else "❌ 失败"
|
||||
print(f"{status} - {name}")
|
||||
|
||||
print(f"\n总计: {passed}/{total} 测试通过")
|
||||
|
||||
if passed == total:
|
||||
print("\n🎉 所有测试通过!系统运行正常。")
|
||||
return 0
|
||||
else:
|
||||
print(f"\n⚠️ {total - passed} 个测试失败,请检查日志。")
|
||||
return 1
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user