Initial commit

This commit is contained in:
Songhaoz666
2026-06-08 17:32:34 +08:00
commit d0fa193f79
91 changed files with 19645 additions and 0 deletions
+12
View File
@@ -0,0 +1,12 @@
# CODEOWNERS — 代码所有权与 PR 审查责任绑定
# 这些条目要求对应路径的改动必须经所有者 review。
# 默认:整个 Swarm 仓库
* @Songhaoz666
# 文档 / 标准
/docs/ @Songhaoz666
/README.md @Songhaoz666
# Manager ↔ Swarm 契约相关(如本仓后续新增 docs/integration)
/docs/integration/ @Songhaoz666
+43
View File
@@ -0,0 +1,43 @@
## 变更说明
<!-- 简述本次改动做了什么、为什么 -->
## 影响范围
- [ ] Client
- [ ] Manager
- [ ] Agent / Swarm
- [ ] CodeGW
- [ ] 计费
- [ ] 密钥 / secret_ref
- [ ] 审计
- [ ] 发布链路
- [ ] 文档 / 标准
## 是否读取标准
- [ ] 已读取当前仓 CLAUDE.md
- [ ] 已读取当前仓 PROJECT_STANDARD.md
- [ ] 已读取 heicodeDocs 相关标准
## 是否涉及接口契约
- [ ] 不涉及
- [ ] 涉及,已更新 docs/integration 或相关文档
## 验收方式
<!-- 请说明如何验证,例如:
python scripts/test-runtime-contract.py
python scripts/test-merge-smoke.py
python scripts/test-workflow-e2e.py
-->
## TODO / Mock
- [ ] 没有 TODO / mock
- [ ] 有,说明原因和后续计划
## Release 仓库确认
- [ ] 本 PR 不涉及 release 仓库业务代码
+48
View File
@@ -0,0 +1,48 @@
# Secrets / credentials — never commit
.env
.env.*
secrets/
*.pem
*.key
*.p12
*.pfx
id_rsa
id_ed25519
*_rsa
*_ed25519
# Python
__pycache__/
*.py[cod]
*.pyd
.pytest_cache/
.mypy_cache/
.ruff_cache/
.coverage
.coverage.*
# Virtual environments
.venv/
venv/
env/
ENV/
# Node / desktop-client
node_modules/
desktop-client/dist/
desktop-client/out/
# Build / dist
build/
dist/
*.egg-info/
# Runtime / temp outputs
tmp-workspace/
*.log
# OS / editor
.DS_Store
Thumbs.db
.idea/
.vscode/
+36
View File
@@ -0,0 +1,36 @@
# CLAUDE.md — HeiCode Swarm(执行面 / 运行时)
本仓库(`agent_swarm_v5`,对应 **HeiCode-Swarm**)是蜂群执行面 / 回调 / Swarm Runtime。整体介绍见 [README.md](README.md),工程标准见 [PROJECT_STANDARD.md](PROJECT_STANDARD.md),交付说明见 [docs/DELIVERY.md](docs/DELIVERY.md)。
## 改动前必读
1. 先读本 `CLAUDE.md` 与本仓 `PROJECT_STANDARD.md`。
2. **heicodeDocs 是唯一标准源**;涉及产品/工程/Agent/Swarm/计费/安全/交付标准时,先读取 heicodeDocs 对应标准文件。
3. 跨端或动 Manager ↔ Swarm 接口时,必须先读 `heicode-mananger/docs/heicode.md` 与 `docs/integration/` 契约。
4. 不确定标准时,**先停止修改并要求提供标准路径**;发现文档与代码冲突时,**输出冲突点**,不得自行选择一方。
## 仓库边界
- 只允许修改 Swarm(orchestrator + agent)范围;不得跨仓改 Manager、客户端、release 仓库或 heicodeDocs。
- 一个任务只允许修改任务声明范围内的模块。
- 禁止直接 push / force push `main`;所有改动走 PR,并按 PR 模板声明影响范围。
## 安全与合规
- 禁止把密钥、Token、云凭据、`.env`、证书、私钥写入代码、日志、Markdown 或提交记录。
- 凭据一律经部署环境 / `secret_ref` / Key Vault 注入;`.env` 已被 `.gitignore` 忽略,仅供本地开发。
- `secret_ref`、审批链、鉴权、计费、审计相关改动必须遵循 Manager 审批链与 heicodeDocs 安全规则。
- 禁止只写 TODO / mock / 示例代码就声称完成。
## 架构与关键约束(便于定位)
- **orchestrator/**:FastAPI 编排器。Manager 面接口、HMAC 签名回调、审批链**必须保持契约**。Redis 为权威存储;内存回退仅限 `REDIS_FAKE` / `ALLOW_MEMORY_STORE`(开发/CI)。
- **agent/**:执行单元,**OpenAI 兼容**模型;保留计费/审计归属(`usage` 与 `X-Agent/X-Agnet` 头)。
- **工作流开关默认关闭**:`ENABLE_PLANNER_FALLBACK`、`ENABLE_REVIEW_LOOP`、`ENABLE_SUBTASK_HANDOFF`。
- 提交前必须本地通过:
```
python scripts/test-runtime-contract.py
python scripts/test-merge-smoke.py
python scripts/test-workflow-e2e.py
```
## Agent / Teammate 协作
- 每个 teammate / agent 必须遵循本 `CLAUDE.md` 与 heicodeDocs。
- 不允许 teammate 跨仓自行修改未声明范围内的代码,或绕过 Manager ↔ Swarm 契约、计费、审计、审批链。
- 多 agent 协作的最终总结需说明各 agent 负责范围、修改文件、风险与未完成项。
+26
View File
@@ -0,0 +1,26 @@
FROM python:3.11-slim
# Install Git
RUN apt-get update && \
apt-get install -y git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Set working directory
WORKDIR /app
# Copy agent code
COPY agent/ /app/agent/
# Install Python dependencies
RUN pip install --no-cache-dir -r /app/agent/requirements.txt
# Create workspace directory
RUN mkdir -p /workspace
# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV WORKSPACE_DIR=/workspace
# Run agent
CMD ["python", "-m", "agent.main"]
+16
View File
@@ -0,0 +1,16 @@
FROM python:3.11-slim
WORKDIR /app
# Install dependencies
COPY orchestrator/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy orchestrator code
COPY orchestrator/ ./orchestrator/
# Expose port
EXPOSE 8000
# Run orchestrator
CMD ["python", "-m", "uvicorn", "orchestrator.main:app", "--host", "0.0.0.0", "--port", "8000"]
+37
View File
@@ -0,0 +1,37 @@
# PROJECT_STANDARD.md — HeiCode Swarm 工程标准
本文件约定本仓(`agent_swarm_v5` / HeiCode-Swarm)的工程标准。与 heicodeDocs 冲突时,以 **heicodeDocs** 为准;本文件仅作本仓落地补充。配套:[CLAUDE.md](CLAUDE.md)、[README.md](README.md)、[docs/DELIVERY.md](docs/DELIVERY.md)。
## 运行时与依赖
- Python 3.13+;依赖见 `orchestrator/requirements.txt`、`agent/requirements.txt`。
- 模型后端:**OpenAI 兼容** Chat Completions(`OPENAI_API_KEY` / `OPENAI_API_BASE` / `OPENAI_MODEL`,可指向自定义端点)。
- 持久化:**Redis 为权威存储**。`fakeredis` 内存回退仅限开发/CI,且必须由 `REDIS_FAKE` / `ALLOW_MEMORY_STORE` 显式开启;生产在 Redis 不可用时快速失败。
## 功能开关(默认关闭,保持契约)
- `ENABLE_PLANNER_FALLBACK`:无 Manager 分工时启用 LLM 规划回退。
- `ENABLE_REVIEW_LOOP` / `MAX_REVIEW_CYCLES`:主控评审/重做循环 + 结果汇总。
- `ENABLE_SUBTASK_HANDOFF`:动态子任务移交。
- 开启上述开关不得改变 Manager 面接口、回调、审批链与计费/审计字段语义。
## 安全
- 不得提交密钥/Token/凭据/`.env`/证书/私钥;通过环境 / `secret_ref` / Key Vault 注入。
- K8s 部署:Agent 模型密钥使用 Secret `openai-secret`(Pod 模板)或 `openai-api-key`(Deployment 清单)。
- 保留模型用量归属(`usage`、`X-Agent/X-Agnet` 头)以满足计费/审计。
## 测试与验收(PR 前必须通过)
```
python scripts/test-runtime-contract.py # Manager 契约校验
python scripts/test-merge-smoke.py # 机制级冒烟(评审/协作/汇总等)
python scripts/test-workflow-e2e.py # 端到端工作流(无需密钥)
```
- 新增/修改逻辑应补充对应断言;不得以 TODO / mock 充当完成。
- 涉及 Manager 契约的改动,必须先通过 `test-runtime-contract.py`。
## 部署
- 正式部署走**镜像**:`Dockerfile.orchestrator` / `Dockerfile.agent` + `k8s/` 清单(`orchestrator-deployment.yaml` 使用 `swarm-orchestrator:latest`)。
- `k8s/orchestrator-source-configmap.yaml` 为陈旧快照,请勿用于当前部署(见 `docs/DELIVERY.md` §九)。
## 提交与分支
- 禁止直接 push / force push `main`;所有改动经 PR 合并,至少 1 人 review。
- PR 必须按 `.github/pull_request_template.md` 声明影响范围(Client / Manager / Agent-Swarm / CodeGW / 计费 / 密钥 / 审计 / 发布链路 / 文档)。
- 提交信息清晰描述改动;保持与现有代码风格一致。
+93
View File
@@ -0,0 +1,93 @@
# Agent Swarm(HeiCode Swarm)
一个多智能体「蜂群」系统:用户提出需求后,由主控逻辑自动将其分解为多个子任务,分发给擅长不同领域的专家 Agent 并行完成;专家之间可就重叠领域相互协作;产出汇总后由主控评审是否达标,未达标则退回重做,循环直至生成满意的最终回答。
## 系统架构
```
用户
│
桌面客户端(desktop-client)
│ 提交需求 / 查看状态与结果
▼
Heicode Manager(控制面,外部)
│ 下发编排方案 / 接收带签名回调(审批、计费、审计)
▼
Orchestrator(编排器,FastAPI)
分解 · 派发 · 协作路由 · 评审/重做 · 汇总
├─ Redis(权威状态存储)
└─ WebSocket ┐
▼
Agent · Agent · Agent …(执行单元 / K8s Pod,调用大模型完成任务)
```
## 工作流
```
分解(Plan) → 派发给专家(Dispatch) → 专家执行(Execute)
→ 重叠领域协作 / 移交(Collaborate & Handoff)
→ 主控评审(Review)──不达标──▶ 退回相关任务重做(循环,受上限约束)
└──达标──▶ 汇总为统一回答并交付(Deliver)
```
- **分解**:优先采用 Manager 提供的编排方案;若未提供且开启规划回退(`ENABLE_PLANNER_FALLBACK`),由编排器用大模型把目标拆解为「实现 → 测试 → 文档」等专家子任务。
- **派发**:按「能力匹配 + 剩余容量」将就绪任务下发给已连接的 Agent;派发时把已完成依赖的产物与同伴信息注入上下文。
- **协作 / 移交**:测试、文档等角色可向实现角色咨询以保持语义一致;复杂子任务可移交给更合适的专家。
- **评审 / 重做**:所有任务完成后由评审者判断是否达标(开启 `ENABLE_REVIEW_LOOP`),不达标则退回相关任务重做,受 `MAX_REVIEW_CYCLES` 约束。
- **汇总交付**:通过后将各专家产出汇总为统一、面向用户的最终回答。
## 仓库结构与子文档
| 目录 | 角色 | 文档 |
|---|---|---|
| `orchestrator/` | 编排器:分解、派发、协作、评审、汇总、Manager 对接 | [orchestrator/README.md](orchestrator/README.md) |
| `agent/` | 执行单元:连接编排器、调用大模型完成任务、提交结果 | [agent/README.md](agent/README.md) |
| `desktop-client/` | 桌面客户端:任务提交、复杂度分析、状态展示、结果聚合 | [desktop-client/README.md](desktop-client/README.md) |
| `desktop-client/src/ai/` | 复杂度分析模块:评估规模、推荐 Agent 数量 | [desktop-client/src/ai/README.md](desktop-client/src/ai/README.md) |
| `docs/` | 交付说明(交付物、构建/部署、配置、验收、影响与合规) | [docs/DELIVERY.md](docs/DELIVERY.md) |
| `k8s/` | Kubernetes 部署清单(编排器、Agent、Redis、RBAC、监控等) | — |
| `scripts/` | 部署、契约校验、冒烟与端到端工作流测试脚本 | — |
| `test-data/` | 复杂度分析的标注数据集 | — |
## 快速开始(本地)
需要 Python 3.13+ 环境。本地开发可用受控的内存存储(`REDIS_FAKE=1`,免装 Redis)。
```bash
# 1) 编排器(在本目录 agent_swarm_v5 下运行,开启规划回退与评审循环)
set "REDIS_FAKE=1"
set "ENABLE_PLANNER_FALLBACK=1"
set "ENABLE_REVIEW_LOOP=1"
python -m uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
# 2) 一个 Agent(凭据放在本目录 .env:OPENAI_API_KEY 等)
set "ORCHESTRATOR_URL=ws://localhost:8000"
set "AGENT_ID=worker-1"
set "AGENT_CAPABILITIES=python,code_generation,testing,pytest,technical-writing,general"
set "WORKSPACE_DIR=..\tmp-workspace\worker-1"
python -m agent.main
# 3) 提交一个需求
curl -s -X POST http://localhost:8000/api/swarms -H "Content-Type: application/json" ^
-d "{\"mode\":\"swarm\",\"requirement\":{\"objective\":\"实现 add(a,b) 并补充测试与说明\"},\"callback\":{\"url\":\"http://localhost:9999/cb\",\"subscribed_events\":[]},\"metadata\":{\"manager_deployment_id\":\"dev-1\"}}"
```
随后可通过 `GET /api/swarms/{deployment_id}/workflow` 与 `/logs` 观察分解、派发、评审与汇总过程。各组件的环境变量与接口详见对应子文档。
## 关键设计
- **存储**:Redis 为权威状态存储;仅 `REDIS_FAKE` / `ALLOW_MEMORY_STORE` 开启时才使用进程内回退,生产环境在 Redis 不可用时快速失败。
- **模型**:Agent 与编排器规划/评审均使用 **OpenAI 兼容** API,可指向自定义端点;未配置密钥时规划/评审退化为静态分解与启发式判定。
- **Manager 契约**:面向 Manager 的接口、带签名回调、审批链与计费/审计字段均予以保留。
## 测试
```bash
python scripts/test-runtime-contract.py # Manager 契约校验
python scripts/test-merge-smoke.py # 工作流冒烟测试(评审 / 协作 / 汇总等)
```
## 安全与合规
- **禁止**将密钥、令牌、云凭据写入代码、日志或提交记录;凭据通过 `.env`(已忽略)或部署环境 / `secret_ref` 注入。
- 涉及鉴权、审批链、计费与审计的改动需遵循 Manager 安全规则。
+104
View File
@@ -0,0 +1,104 @@
# Agent 执行器(Agent Runtime)
Agent 是蜂群系统中的执行单元(worker)。每个 Agent 作为独立进程运行(在 Kubernetes 中通常是一个 Pod),通过 WebSocket 连接到 Orchestrator,领取任务、调用大模型完成编码工作,并把结果写回 Git 分支。
## 职责概览
- 通过 WebSocket 连接 Orchestrator,并以自身**能力(capabilities)**注册,例如 `python`、`testing`、`technical-writing`。
- 接收 Orchestrator 下发的任务,使用 **OpenAI 兼容**的大模型完成子任务。
- 在隔离的「按任务工作目录」中生成/修改文件,并提交到独立的结果分支。
- 支持与其它 Agent **协作(peer collaboration)**与**移交(handoff)**。
- 持续上报心跳、状态与执行结果,并暴露 Prometheus 指标。
## 目录结构
```
agent/
├── main.py # Agent 主循环:连接、注册、心跳、消息处理、并发与重连
├── task_executor.py # 任务执行引擎:调用 OpenAI 兼容模型,应用文件改动
├── handoff_logic.py # 移交决策逻辑(复杂度 / 能力 / 预估时长)
├── git_operations.py # Git 克隆、建分支、提交、推送
└── requirements.txt # Python 依赖
```
## 核心模块
### 主循环(`main.py`)
- **连接与重连**:断线后按指数退避自动重连并重新注册。
- **有界并发**:通过信号量限制同时执行的任务数(`MAX_CONCURRENT_TASKS`),并在注册/心跳中上报剩余容量 `available_slots`。
- **任务受理**:对每个分配先做校验,重复任务返回 `task_accepted(status=duplicate)`,超出容量返回 `task_rejected`,由 Orchestrator 重新入队。
- **执行保护**:单任务超时(`TASK_TIMEOUT_SECONDS`)与任务取消(`cancel_task`)。
- **心跳**:每 15 秒上报一次存活与容量信息。
- **优雅退出**:收到 SIGINT/SIGTERM 时停止接单并清理在执行的任务。
### 任务执行器(`task_executor.py`)
- 使用 **OpenAI 兼容 API**(可指向自定义 `base_url`)完成子任务。
- 根据上下文中的**依赖产物(dependency_artifacts)**与**专家角色(specialist_role)**对齐输出:测试与文档以实现产物的行为/异常语义为准。
- 将模型返回的文件改动安全地写入工作目录(带路径越界校验)。
- 记录用量并附带计费/审计归属(`usage` 与 `X-Agent/X-Agnet` 模型归属头),供 Orchestrator 上报。
### 移交逻辑(`handoff_logic.py`)
当子任务满足以下条件时建议移交给更合适的 Agent:
- 复杂度为 `high`;
- 需要当前 Agent 不具备的专业能力;
- 预估耗时超过 60 分钟。
### Git 操作(`git_operations.py`)
- 自动识别仓库根目录(`repo_root`),使按任务子目录也能在父级 Git 仓库中正确执行。
- 为每个任务创建结果分支(形如 `agent/{agent-id}/{task}-{timestamp}`),提交并推送。
## 配置(环境变量)
凭据通过同目录的 `.env`(已被 `.gitignore` 忽略)加载,亦可由部署环境/`secret_ref` 注入。**请勿将密钥写入代码或提交记录。**
```bash
# 模型(OpenAI 兼容)
OPENAI_API_KEY=... # 或 MODEL_API_KEY
OPENAI_API_BASE=https://api.openai.com/v1 # 或 MODEL_API_BASE,可指向自定义端点
OPENAI_MODEL=gpt-4o-mini # 或 MODEL_NAME / MODEL_ID
# 连接与身份
ORCHESTRATOR_URL=ws://localhost:8000 # Orchestrator 的 WebSocket 地址
AGENT_ID=worker-1 # 不填则自动生成
AGENT_CAPABILITIES=python,testing # 逗号分隔的能力列表
WORKSPACE_DIR=/workspace # 工作目录
# 可选
GIT_REPO_URL=https://... # 需在仓库内工作时设置
GIT_USERNAME / GIT_PASSWORD / GIT_TOKEN# 推送凭据
GIT_BASE_BRANCH=main # 结果分支的基线
MAX_CONCURRENT_TASKS=4 # 最大并发任务数
TASK_TIMEOUT_SECONDS=60 # 单任务超时
METRICS_PORT=9000 # 设置后暴露 Prometheus 指标
ENABLE_SUBTASK_HANDOFF=false # 是否启用动态子任务移交
```
## 本地运行
```bash
pip install -r agent/requirements.txt
# 准备 .env(含 OPENAI_API_KEY 等),然后从仓库根目录运行:
set "ORCHESTRATOR_URL=ws://localhost:8000"
set "AGENT_ID=worker-1"
set "AGENT_CAPABILITIES=python,code_generation,testing,pytest,technical-writing,general"
set "WORKSPACE_DIR=...\tmp-workspace\worker-1"
python -m agent.main
```
## 任务执行流程
1. **注册**:连接 Orchestrator 并上报能力与可用容量。
2. **心跳**:每 15 秒上报存活与剩余容量。
3. **受理**:校验分配,必要时拒绝(容量不足)或忽略(重复)。
4. **执行**:在独立的按任务工作目录中调用模型完成子任务;如启用,可向同伴 Agent 咨询或移交。
5. **提交**:若为 Git 工作区,提交并推送到结果分支。
6. **回报**:将结果(含用量)发送回 Orchestrator,并将状态置为空闲。
## 协作(Peer Collaboration)
当上下文提供了 `peer_agents` 时,测试/文档等角色可向实现角色发起咨询;被咨询方会回复自己最近一次完成任务的摘要,帮助各专家在重叠领域保持一致。
## 监控
Agent 暴露的 Prometheus 指标包括:已执行/失败任务数、任务时长、移交次数、重连次数、被拒/重复任务数、当前活跃任务数与 Agent 状态。
+3
View File
@@ -0,0 +1,3 @@
"""Agent package for K8s-based swarm mode."""
__all__: list[str] = []
+219
View File
@@ -0,0 +1,219 @@
"""Git operations for the agent workspace.
Merged from agent_swarm_v4:
- Adds repository-root discovery (`repo_root`) so per-task child workspaces can still
operate inside a parent Git checkout (all git commands run at the repo toplevel)
- Keeps the original branch/commit/push API so the protocol stays compatible
"""
import asyncio
import logging
import os
import shutil
import subprocess
import time
from pathlib import Path
from typing import Optional
from urllib.parse import quote, urlsplit, urlunsplit
logger = logging.getLogger(__name__)
class GitOperations:
"""Handles Git operations for agent workspace."""
def __init__(self, workspace_dir: str, agent_id: str):
self.workspace_dir = str(Path(workspace_dir))
self.agent_id = agent_id
self.result_branch = f"agent-{agent_id}-results"
self.base_branch = os.getenv("GIT_BASE_BRANCH", "main")
def _authenticated_repo_url(self, repo_url: str) -> str:
username = os.getenv("GIT_USERNAME")
password = os.getenv("GIT_PASSWORD") or os.getenv("GIT_TOKEN")
if not username or not password:
return repo_url
parsed = urlsplit(repo_url)
if parsed.scheme not in {"http", "https"} or "@" in parsed.netloc:
return repo_url
userinfo = f"{quote(username)}:{quote(password)}"
return urlunsplit((
parsed.scheme,
f"{userinfo}@{parsed.netloc}",
parsed.path,
parsed.query,
parsed.fragment,
))
async def repo_root(self) -> Optional[str]:
result = await self._run_git_command(
["git", "rev-parse", "--show-toplevel"],
cwd=self.workspace_dir,
)
if result.returncode != 0:
return None
return result.stdout.strip() or None
async def is_git_workspace(self) -> bool:
result = await self._run_git_command(
["git", "rev-parse", "--is-inside-work-tree"],
cwd=self.workspace_dir,
)
return result.returncode == 0 and result.stdout.strip() == "true"
async def clone_workspace(self, repo_url: str) -> bool:
try:
logger.info(f"Cloning repository from {repo_url}")
workspace = Path(self.workspace_dir)
workspace.parent.mkdir(parents=True, exist_ok=True)
if workspace.exists():
if await self.is_git_workspace():
logger.info("Workspace already contains a Git checkout; refreshing existing clone")
return await self._refresh_existing_workspace(repo_url)
if any(workspace.iterdir()):
logger.warning("Workspace directory exists and is not empty; clearing before clone")
self._clear_workspace_dir(workspace)
clone_url = self._authenticated_repo_url(repo_url)
result = await self._run_git_command([
"git", "clone", clone_url, self.workspace_dir
])
if result.returncode != 0:
logger.error(f"Git clone failed: {result.stderr}")
return False
logger.info(f"Successfully cloned repository to {self.workspace_dir}")
await self._configure_git_user()
return True
except Exception as e:
logger.error(f"Error cloning workspace: {e}")
return False
def _clear_workspace_dir(self, workspace: Path):
for child in workspace.iterdir():
if child.is_dir() and not child.is_symlink():
shutil.rmtree(child)
else:
child.unlink()
async def _configure_git_user(self):
cwd = await self.repo_root() or self.workspace_dir
await self._run_git_command([
"git", "config", "user.name", f"Agent {self.agent_id}"
], cwd=cwd)
await self._run_git_command([
"git", "config", "user.email", f"{self.agent_id}@agent.local"
], cwd=cwd)
async def _refresh_existing_workspace(self, repo_url: str) -> bool:
cwd = await self.repo_root() or self.workspace_dir
clone_url = self._authenticated_repo_url(repo_url)
await self._run_git_command(["git", "remote", "set-url", "origin", clone_url], cwd=cwd)
await self._configure_git_user()
fetch = await self._run_git_command(["git", "fetch", "origin"], cwd=cwd)
if fetch.returncode != 0:
logger.error(f"Git fetch failed: {fetch.stderr}")
return False
base_ref = f"origin/{self.base_branch}"
base_check = await self._run_git_command(["git", "rev-parse", "--verify", base_ref], cwd=cwd)
if base_check.returncode == 0:
reset = await self._run_git_command(["git", "reset", "--hard", base_ref], cwd=cwd)
if reset.returncode != 0:
logger.error(f"Git reset failed: {reset.stderr}")
return False
clean = await self._run_git_command(["git", "clean", "-fd"], cwd=cwd)
if clean.returncode != 0:
logger.error(f"Git clean failed: {clean.stderr}")
return False
return True
async def create_result_branch(self, task_id: Optional[str] = None) -> bool:
try:
cwd = await self.repo_root() or self.workspace_dir
if task_id:
safe_task_id = task_id.replace("/", "-")[:12]
timestamp = int(time.time())
self.result_branch = f"agent/{self.agent_id}/{safe_task_id}-{timestamp}"
await self._run_git_command(["git", "fetch", "origin"], cwd=cwd)
base_ref = f"origin/{self.base_branch}"
base_check = await self._run_git_command(["git", "rev-parse", "--verify", base_ref], cwd=cwd)
if base_check.returncode != 0:
base_ref = "HEAD"
logger.warning(f"Base branch origin/{self.base_branch} not found; creating result branch from HEAD")
result = await self._run_git_command(["git", "checkout", "-B", self.result_branch, base_ref], cwd=cwd)
if result.returncode != 0:
logger.error(f"Failed to create branch: {result.stderr}")
return False
logger.info(f"Created result branch: {self.result_branch}")
return True
except Exception as e:
logger.error(f"Error creating result branch: {e}")
return False
async def commit_changes(self, message: str) -> Optional[str]:
try:
cwd = await self.repo_root() or self.workspace_dir
status_result = await self._run_git_command(["git", "status", "--porcelain"], cwd=cwd)
if not status_result.stdout.strip():
logger.info("No changes to commit")
return None
await self._run_git_command(["git", "add", "-A"], cwd=cwd)
commit_result = await self._run_git_command(["git", "commit", "-m", message], cwd=cwd)
if commit_result.returncode != 0:
logger.error(f"Git commit failed: {commit_result.stderr}")
return None
sha_result = await self._run_git_command(["git", "rev-parse", "HEAD"], cwd=cwd)
commit_sha = sha_result.stdout.strip()
logger.info(f"Committed changes: {commit_sha[:8]} - {message}")
return commit_sha
except Exception as e:
logger.error(f"Error committing changes: {e}")
return None
async def push_results(self) -> Optional[str]:
try:
cwd = await self.repo_root() or self.workspace_dir
result = await self._run_git_command(["git", "push", "-u", "origin", self.result_branch], cwd=cwd)
if result.returncode != 0:
logger.error(f"Git push failed: {result.stderr}")
return None
logger.info(f"Pushed results to branch: {self.result_branch}")
return self.result_branch
except Exception as e:
logger.error(f"Error pushing results: {e}")
return None
async def _run_git_command(
self,
command: list[str],
cwd: Optional[str] = None,
) -> subprocess.CompletedProcess:
process = await asyncio.create_subprocess_exec(
*command,
cwd=cwd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await process.communicate()
return subprocess.CompletedProcess(
args=command,
returncode=process.returncode,
stdout=stdout.decode(),
stderr=stderr.decode(),
)
+224
View File
@@ -0,0 +1,224 @@
"""Handoff decision logic for determining when to delegate tasks."""
import logging
from typing import Iterable, List, Optional
from dataclasses import dataclass
logger = logging.getLogger(__name__)
@dataclass
class HandoffDecision:
"""Result of handoff decision analysis."""
should_handoff: bool
reason: str
target_capabilities: List[str]
CAPABILITY_ALIASES = {
"code_generation": {
"code-reading",
"code-editing",
"command-line",
"credentials-handling",
"editing",
"environment-inspection",
"environment-setup",
"file-editing",
"file-operations",
"file-system",
"filesystem",
"read-only-operations",
"reporting",
"repository-inspection",
"shell",
"testing",
"pytest",
"quality-checking",
"attention-to-detail",
"error-handling",
"technical-writing",
"validation",
"verification",
},
"python": {
"pytest",
"testing",
"code-reading",
"code-editing",
"command-line",
"editing",
"file-editing",
"file-operations",
"file-system",
"filesystem",
"reporting",
"shell",
"quality-checking",
"error-handling",
"validation",
"verification",
},
"general": {
"general",
"code-reading",
"file-system",
"filesystem",
"quality-checking",
"attention-to-detail",
},
}
def _expanded_capabilities(capabilities: Iterable[str]) -> set[str]:
"""Return capabilities plus local aliases supported by this agent."""
expanded = {"general", *capabilities}
for capability in capabilities:
expanded.update(CAPABILITY_ALIASES.get(capability, set()))
return expanded
def should_handoff(
subtask: dict,
agent_id: str,
agent_capabilities: Optional[List[str]] = None,
) -> HandoffDecision:
"""Determine if a subtask should be handed off to another agent.
Args:
subtask: Subtask definition with complexity, capabilities, etc.
agent_id: Current agent ID
agent_capabilities: Capabilities advertised by the current agent
Returns:
HandoffDecision: Decision with reasoning
"""
complexity = subtask.get("complexity", "medium")
required_capabilities = subtask.get("required_capabilities", ["general"])
estimated_time = subtask.get("estimated_time", 30)
current_capabilities = _expanded_capabilities(agent_capabilities or ["general"])
# Decision criteria:
# 1. High complexity tasks should be handed off to specialist agents
# 2. Tasks requiring specialized capabilities should go to specialists
# 3. Tasks estimated to take > 60 minutes should be broken down further
# Check complexity threshold
if complexity == "high":
logger.info(f"High complexity subtask detected: {subtask['description']}")
return HandoffDecision(
should_handoff=True,
reason="Task complexity exceeds agent capability threshold",
target_capabilities=required_capabilities
)
# Check for specialized capabilities
specialized_capabilities = [
cap for cap in required_capabilities
if cap not in current_capabilities
]
if specialized_capabilities:
logger.info(f"Specialized capabilities required: {specialized_capabilities}")
return HandoffDecision(
should_handoff=True,
reason=f"Requires specialized capabilities: {', '.join(specialized_capabilities)}",
target_capabilities=specialized_capabilities
)
# Check estimated time
if estimated_time > 60:
logger.info(f"Long-running task detected: {estimated_time} minutes")
return HandoffDecision(
should_handoff=True,
reason=f"Task estimated to take {estimated_time} minutes (threshold: 60)",
target_capabilities=required_capabilities
)
# No handoff needed
return HandoffDecision(
should_handoff=False,
reason="Task within agent capability",
target_capabilities=[]
)
def select_target_agent(
available_agents: List[dict],
required_capabilities: List[str]
) -> str:
"""Select the best agent for a handoff based on capabilities.
Args:
available_agents: List of available agent metadata
required_capabilities: Required capabilities for the task
Returns:
str: Selected agent ID or None if no suitable agent found
"""
# Score each agent based on capability match
best_agent = None
best_score = -1
for agent in available_agents:
agent_capabilities = set(agent.get("capabilities", []))
required_set = set(required_capabilities)
# Calculate match score
matches = len(agent_capabilities.intersection(required_set))
total_required = len(required_set)
if total_required > 0:
score = matches / total_required
else:
score = 0
# Prefer agents with exact capability match
if score > best_score:
best_score = score
best_agent = agent
if best_agent:
logger.info(
f"Selected agent {best_agent['agent_id']} "
f"with score {best_score:.2f} for capabilities {required_capabilities}"
)
return best_agent["agent_id"]
logger.warning(f"No suitable agent found for capabilities: {required_capabilities}")
return None
def estimate_task_complexity(description: str) -> str:
"""Estimate task complexity based on description keywords.
Args:
description: Task description
Returns:
str: "low", "medium", or "high"
"""
description_lower = description.lower()
# High complexity indicators
high_complexity_keywords = [
"refactor", "redesign", "architecture", "migrate",
"optimize", "performance", "security", "scale",
"distributed", "concurrent", "async", "parallel"
]
# Low complexity indicators
low_complexity_keywords = [
"fix typo", "update comment", "rename", "format",
"add log", "simple", "trivial", "quick"
]
# Check for high complexity
if any(keyword in description_lower for keyword in high_complexity_keywords):
return "high"
# Check for low complexity
if any(keyword in description_lower for keyword in low_complexity_keywords):
return "low"
# Default to medium
return "medium"
+629
View File
@@ -0,0 +1,629 @@
"""Agent main loop with WebSocket connection to orchestrator.
Merged from agent_swarm_v4:
- Bounded concurrency, duplicate protection, capacity reporting (available_slots)
- Serialized websocket sends (safe_send) and reconnect-with-reregister logic
- Per-task timeout and graceful task cancellation
- Separate repository root from per-task execution workspace (see git_operations)
- Peer collaboration: request/reply routing fixed relative to v4 (peer_waiters is now
initialized and inbound peer replies are routed back to the waiting coroutine; inbound
peer queries are answered with a lightweight, cost-free acknowledgement)
- OpenAI-only task executor (see task_executor)
- Preserves heicode handoff wiring, Manager-facing metrics, and env-var entrypoint
"""
import asyncio
import json
import logging
import os
import signal
import time
import uuid
from pathlib import Path
from typing import Optional
import websockets
from dotenv import load_dotenv
from pydantic import BaseModel, Field, ValidationError
from prometheus_client import Counter, Gauge, Histogram, start_http_server
from websockets.exceptions import ConnectionClosed
from .git_operations import GitOperations
from .task_executor import TaskExecutor
load_dotenv()
# load_dotenv() above reads a local .env (gitignored) so credentials need not be passed on
# the command line. These defaults just guarantee the vars exist; an empty OPENAI_API_KEY is
# still falsy, so TaskExecutor raises a clear error rather than silently using no key.
# Real deployments supply the key via the environment / secret_ref, NOT a committed file.
os.environ.setdefault("OPENAI_API_KEY", "")
os.environ.setdefault("OPENAI_API_BASE", "https://api.openai.com/v1")
os.environ.setdefault("OPENAI_MODEL", "gpt-4o-mini")
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
)
logger = logging.getLogger(__name__)
TASKS_EXECUTED = Counter("agent_tasks_executed_total", "Total tasks executed")
TASKS_FAILED = Counter("agent_tasks_failed_total", "Total tasks failed")
TASK_DURATION = Histogram("agent_task_duration_seconds", "Task execution duration")
HANDOFFS_INITIATED = Counter("agent_handoffs_initiated_total", "Total handoffs initiated")
AGENT_STATUS = Gauge("agent_status", "Agent status (0=idle, 1=busy, 2=failed)")
WEBSOCKET_RECONNECTS = Counter("agent_websocket_reconnects_total", "WebSocket reconnection attempts")
TASKS_REJECTED = Counter("agent_tasks_rejected_total", "Total tasks rejected due to capacity")
TASKS_DUPLICATE = Counter("agent_tasks_duplicate_total", "Total duplicate task assignments")
ACTIVE_TASKS = Gauge("agent_active_tasks", "Current number of active tasks")
class TaskAssignment(BaseModel):
task_id: str
description: str
context: dict = Field(default_factory=dict)
class AgentRuntimeDisconnected(RuntimeError):
"""Raised when the runtime attempts to send without an active websocket."""
class Agent:
"""Agent that connects to orchestrator and executes tasks."""
MAX_CONCURRENT_TASKS = int(os.getenv("MAX_CONCURRENT_TASKS", "4"))
TASK_TIMEOUT_SECONDS = int(os.getenv("TASK_TIMEOUT_SECONDS", "60"))
HEARTBEAT_INTERVAL_SECONDS = 15
def __init__(
self,
orchestrator_url: str,
agent_id: Optional[str] = None,
capabilities: Optional[list[str]] = None,
workspace_dir: str = "/workspace",
git_repo_url: Optional[str] = None,
):
# Initialize connection state, execution limits, and workspace helpers for this agent runtime.
self.agent_id = agent_id or f"agent-{uuid.uuid4().hex[:8]}"
self.orchestrator_url = orchestrator_url.rstrip("/")
self.capabilities = capabilities or ["general"]
self.workspace_dir = Path(workspace_dir)
self.git_repo_url = git_repo_url
self.websocket: Optional[websockets.WebSocketClientProtocol] = None
self.running = False
self.current_task_id: Optional[str] = None
self.send_lock = asyncio.Lock()
self.task_semaphore = asyncio.Semaphore(self.MAX_CONCURRENT_TASKS)
self.active_tasks: dict[str, asyncio.Task] = {}
self.heartbeat_task: Optional[asyncio.Task] = None
# Outstanding peer-collaboration requests awaiting a reply, keyed by correlation_id.
self.peer_waiters: dict[str, asyncio.Future] = {}
# Summary of this agent's most recently completed task, shared when peers consult it.
self.last_summary: Optional[str] = None
self.workspace_git = GitOperations(str(self.workspace_dir), self.agent_id)
def available_slots(self) -> int:
# Return how many additional tasks this agent can currently accept.
return max(0, self.MAX_CONCURRENT_TASKS - len(self.active_tasks))
def task_workspace(self, task_id: str) -> Path:
# Compute the dedicated per-task working directory inside the agent workspace.
return self.workspace_dir / ".agent_tasks" / task_id
async def safe_send(self, payload: dict):
# Serialize and send a websocket message while holding a lock to prevent concurrent writes.
if not self.websocket:
raise AgentRuntimeDisconnected("websocket unavailable")
async with self.send_lock:
await self.websocket.send(json.dumps(payload))
async def connect(self) -> bool:
# Open a websocket connection to the orchestrator and cache the live socket on success.
try:
self.websocket = await websockets.connect(f"{self.orchestrator_url}/ws/{self.agent_id}")
logger.info(f"Connected to orchestrator at {self.orchestrator_url}")
return True
except Exception as e:
logger.error(f"Failed to connect to orchestrator: {e}")
return False
async def connect_with_retry(self) -> bool:
# Repeatedly attempt to connect with exponential backoff until connected or shutting down.
delay = 1
while self.running:
if await self.connect():
return True
WEBSOCKET_RECONNECTS.inc()
await asyncio.sleep(min(delay, 60))
delay = min(delay * 2, 60)
return False
async def register(self):
# Announce this agent and its current capacity to the orchestrator after connecting.
try:
await self.safe_send({
"type": "register",
"agent_id": self.agent_id,
"capabilities": self.capabilities,
# Backward-compatible extra fields; older orchestrators ignore them.
"available_slots": self.available_slots(),
"active_task_ids": list(self.active_tasks.keys()),
})
logger.info(f"Registered agent {self.agent_id} with capabilities: {self.capabilities}")
return True
except Exception as e:
logger.error(f"Failed to register: {e}")
return False
async def send_heartbeat(self):
# Send a periodic liveness and capacity update to the orchestrator.
await self.safe_send({
"type": "heartbeat",
"agent_id": self.agent_id,
"timestamp": time.time(),
"active_tasks": len(self.active_tasks),
"available_slots": self.available_slots(),
})
async def heartbeat_loop(self):
# Keep sending heartbeat messages until the connection closes or the agent stops running.
while self.running:
try:
await self.send_heartbeat()
except (ConnectionClosed, AgentRuntimeDisconnected):
return
except Exception as e:
logger.error(f"Failed to send heartbeat: {e}")
return
await asyncio.sleep(self.HEARTBEAT_INTERVAL_SECONDS)
async def send_status_update(self, status: str, task_id: Optional[str] = None, message: str = ""):
# Report a human-readable task or agent status transition to the orchestrator.
try:
await self.safe_send({
"type": "status_update",
"agent_id": self.agent_id,
"status": status,
"task_id": task_id,
"message": message,
"timestamp": time.time(),
})
except Exception as e:
logger.error(f"Failed to send status update: {e}")
async def request_peer_collaboration(
self,
task_id: str,
target_agent_id: str,
content: str,
timeout_seconds: float = 20.0,
):
# Ask a peer agent (via the orchestrator) for guidance and wait for its reply.
correlation_id = f"peer-{task_id}-{uuid.uuid4().hex[:8]}"
loop = asyncio.get_running_loop()
waiter = loop.create_future()
self.peer_waiters[correlation_id] = waiter
try:
await self.safe_send({
"type": "peer_message",
"agent_id": self.agent_id,
"target_agent_id": target_agent_id,
"task_id": task_id,
"content": content,
"correlation_id": correlation_id,
"is_reply": False,
"timestamp": time.time(),
})
return await asyncio.wait_for(waiter, timeout=timeout_seconds)
finally:
self.peer_waiters.pop(correlation_id, None)
@staticmethod
def _summarize_result(result: dict) -> Optional[str]:
# Extract a short, human-readable summary from an execution result.
if not isinstance(result, dict):
return None
for subtask_result in reversed(result.get("subtasks", []) or []):
summary = subtask_result.get("summary")
if isinstance(summary, str) and summary.strip():
return summary.strip()
summary = result.get("summary")
return summary.strip() if isinstance(summary, str) and summary.strip() else None
async def answer_peer_query(self, message: dict):
# Respond to an inbound peer query, sharing what this agent has actually done so far.
requester = message.get("from_agent_id") or message.get("agent_id")
correlation_id = message.get("correlation_id")
if not requester or not correlation_id:
return
shared = (
f"My latest result: {self.last_summary}"
if self.last_summary
else "I have no completed result to share yet."
)
reply_content = (
f"From {self.agent_id} (capabilities: {', '.join(self.capabilities)}). {shared} "
"Treat implementation artifacts as the source of truth for behavior and exception semantics."
)
try:
await self.safe_send({
"type": "peer_message",
"agent_id": self.agent_id,
"target_agent_id": requester,
"task_id": message.get("task_id"),
"content": reply_content,
"correlation_id": correlation_id,
"is_reply": True,
"timestamp": time.time(),
})
except Exception as e:
logger.error(f"Failed to answer peer query: {e}")
async def send_task_result(self, task_id: str, success: bool, result: dict):
# Send a completion or failure payload for a finished task execution.
try:
payload = {
"type": "task_complete" if success else "task_failed",
"agent_id": self.agent_id,
"task_id": task_id,
"timestamp": time.time(),
}
if success:
payload["result"] = result
else:
payload["reason"] = result.get("error", "Task failed")
payload["result"] = result
await self.safe_send(payload)
except Exception as e:
logger.error(f"Failed to send task result: {e}")
async def send_blocked_on_handoff(self, task_id: str, result: dict):
# Report that a task is paused because execution delegated work to a child handoff task.
try:
child_task_id = None
for subtask_result in result.get("subtasks", []):
if subtask_result.get("status") == "handed_off":
child_task_id = subtask_result.get("child_task_id")
break
await self.safe_send({
"type": "blocked_on_handoff",
"agent_id": self.agent_id,
"task_id": task_id,
"child_task_id": child_task_id,
"reason": "Waiting on delegated child task",
"result": result,
"timestamp": time.time(),
})
except Exception as e:
logger.error(f"Failed to send blocked_on_handoff: {e}")
async def request_handoff(self, task_id: str, subtask: dict, target_capabilities: list[str]):
# Ask the orchestrator to delegate a discovered subtask to another capable agent.
try:
await self.safe_send({
"type": "handoff_request",
"agent_id": self.agent_id,
"task_id": task_id,
"subtask": subtask,
"target_capabilities": target_capabilities,
"timestamp": time.time(),
})
HANDOFFS_INITIATED.inc()
except Exception as e:
logger.error(f"Failed to request handoff: {e}")
async def execute_assignment(self, assignment: TaskAssignment):
# Execute one accepted assignment, manage workspace/git flow, and publish lifecycle updates.
async with self.task_semaphore:
task_id = assignment.task_id
description = assignment.description
context = assignment.context or {}
self.current_task_id = task_id
ACTIVE_TASKS.set(len(self.active_tasks))
AGENT_STATUS.set(1)
await self.safe_send({
"type": "task_start",
"agent_id": self.agent_id,
"task_id": task_id,
"timestamp": time.time(),
})
await self.send_status_update("busy", task_id, "Starting task execution")
start_time = time.time()
task_workspace = self.task_workspace(task_id)
task_workspace.mkdir(parents=True, exist_ok=True)
git_enabled = False
try:
executor = TaskExecutor(agent_id=self.agent_id, workspace_dir=str(task_workspace))
git_enabled = await self.workspace_git.is_git_workspace()
if git_enabled:
branch_created = await self.workspace_git.create_result_branch(task_id)
if not branch_created:
logger.warning("Failed to create task result branch; continuing without git push")
git_enabled = False
result = await asyncio.wait_for(
executor.execute_task(
task_id=task_id,
description=description,
context={
**context,
"workspace_dir": str(task_workspace),
"repo_workspace_dir": str(self.workspace_dir),
"git_repo_url": self.git_repo_url,
"agent_id": self.agent_id,
},
handoff_callback=self.request_handoff,
agent_capabilities=self.capabilities,
peer_collaboration_callback=self.request_peer_collaboration,
),
timeout=self.TASK_TIMEOUT_SECONDS,
)
awaiting_handoff = result.get("awaiting_handoff", False)
if result.get("success"):
self.last_summary = self._summarize_result(result) or self.last_summary
if result.get("success") and not awaiting_handoff:
if git_enabled:
commit_sha = await self.workspace_git.commit_changes(
message=f"Task {task_id}: {description[:50]}"
)
if commit_sha:
branch_name = await self.workspace_git.push_results()
result["git_branch"] = branch_name
result["commit_sha"] = commit_sha
else:
result["git_skipped"] = "No workspace changes to commit"
else:
result["git_skipped"] = "Workspace is not a Git checkout"
if awaiting_handoff:
await self.send_blocked_on_handoff(task_id, result)
else:
await self.send_task_result(task_id, result.get("success", False), result)
duration = time.time() - start_time
TASK_DURATION.observe(duration)
if result.get("success"):
TASKS_EXECUTED.inc()
else:
TASKS_FAILED.inc()
AGENT_STATUS.set(0)
if awaiting_handoff:
await self.send_status_update("handoff-pending", task_id, "Waiting for delegated child task")
await self.send_status_update("idle", None, "Delegated child task created")
else:
await self.send_status_update("idle", None, "Task completed")
except asyncio.TimeoutError:
TASKS_FAILED.inc()
AGENT_STATUS.set(2)
await self.send_task_result(task_id, False, {"error": "timeout", "success": False})
await self.send_status_update("idle", None, "Task failed: timeout")
except asyncio.CancelledError:
await self.send_task_result(task_id, False, {"error": "cancelled", "success": False})
await self.send_status_update("idle", None, "Task cancelled")
raise
except Exception as e:
logger.error(f"Error executing task {task_id}: {e}")
TASKS_FAILED.inc()
AGENT_STATUS.set(2)
await self.send_task_result(task_id, False, {"error": str(e), "success": False})
await self.send_status_update("idle", None, f"Task failed: {e}")
finally:
self.active_tasks.pop(task_id, None)
ACTIVE_TASKS.set(len(self.active_tasks))
self.current_task_id = None
if not self.active_tasks:
AGENT_STATUS.set(0)
async def cancel_task(self, task_id: str):
# Cancel an actively running asyncio task if the orchestrator requests termination.
task = self.active_tasks.get(task_id)
if task:
task.cancel()
async def handle_task_assignment(self, message: dict):
# Validate an incoming assignment, reject duplicates/capacity overflow, and start execution.
try:
assignment = TaskAssignment(**message)
except ValidationError:
logger.warning(f"invalid task assignment: {message}")
return
if assignment.task_id in self.active_tasks:
TASKS_DUPLICATE.inc()
await self.safe_send({
"type": "task_accepted",
"task_id": assignment.task_id,
"status": "duplicate",
})
return
if len(self.active_tasks) >= self.MAX_CONCURRENT_TASKS:
TASKS_REJECTED.inc()
await self.safe_send({
"type": "task_rejected",
"task_id": assignment.task_id,
"reason": "at_capacity",
"available_slots": self.available_slots(),
})
return
await self.safe_send({
"type": "task_accepted",
"task_id": assignment.task_id,
"available_slots": self.available_slots() - 1,
})
task = asyncio.create_task(self.execute_assignment(assignment))
self.active_tasks[assignment.task_id] = task
ACTIVE_TASKS.set(len(self.active_tasks))
def _resolve_peer_reply(self, message: dict) -> bool:
# Resolve the waiting future for an inbound peer reply; return True if it was a reply.
correlation_id = message.get("correlation_id")
if not correlation_id:
return False
waiter = self.peer_waiters.get(correlation_id)
if waiter and not waiter.done():
waiter.set_result(message)
return True
# A correlation we own but already resolved/timed out: treat as handled reply.
return bool(message.get("is_reply"))
async def handle_peer_message(self, message: dict):
# Route a peer message: resolve our own pending request, or answer an inbound query.
if message.get("is_reply") or message.get("correlation_id") in self.peer_waiters:
handled = self._resolve_peer_reply(message)
if handled:
return
logger.info(
"Received peer query for task %s from %s",
message.get("task_id"),
message.get("from_agent_id") or message.get("agent_id"),
)
await self.answer_peer_query(message)
async def handle_message(self, message: dict):
# Route each inbound orchestrator message to the appropriate handler.
msg_type = message.get("type")
control_messages = {"registered", "heartbeat_ack", "task_completed", "task_failed_ack", "task_blocked_ack"}
if msg_type == "task_assignment":
await self.handle_task_assignment(message)
elif msg_type == "handoff_response":
logger.info(f"Handoff accepted for task {message.get('task_id')}")
elif msg_type == "cancel_task":
await self.cancel_task(message["task_id"])
elif msg_type == "ping":
await self.safe_send({"type": "pong"})
elif msg_type == "peer_message":
await self.handle_peer_message(message)
elif msg_type in control_messages:
logger.debug(f"Received control message: {msg_type}")
else:
logger.warning(f"Unknown message type: {msg_type}")
async def message_loop(self):
# Continuously receive websocket messages and dispatch them until the connection ends.
async for raw in self.websocket:
try:
data = json.loads(raw)
await self.handle_message(data)
except json.JSONDecodeError as e:
logger.error(f"Failed to parse message: {e}")
except Exception as e:
logger.error(f"Error handling message: {e}")
async def shutdown_active_tasks(self):
# Cancel and await all active task coroutines during agent shutdown.
tasks = list(self.active_tasks.values())
for task in tasks:
task.cancel()
if tasks:
await asyncio.gather(*tasks, return_exceptions=True)
def install_signal_handlers(self):
# Register process signal handlers that trigger a graceful runtime shutdown.
def _handle_shutdown():
# Flip the runtime into shutdown mode and close the websocket asynchronously.
logger.info("shutdown signal received")
self.running = False
if self.websocket:
asyncio.create_task(self.websocket.close())
for sig in (signal.SIGINT, signal.SIGTERM):
try:
asyncio.get_running_loop().add_signal_handler(sig, _handle_shutdown)
except NotImplementedError:
# add_signal_handler is unsupported on Windows event loops; skip gracefully.
pass
async def run(self):
# Run the full agent lifecycle: startup, registration, message processing, reconnect, and cleanup.
self.running = True
self.install_signal_handlers()
if os.getenv("METRICS_PORT"):
start_http_server(int(os.getenv("METRICS_PORT", "9000")))
if self.git_repo_url:
logger.info(f"Cloning workspace from {self.git_repo_url}")
if not await self.workspace_git.clone_workspace(self.git_repo_url):
logger.error("Failed to clone workspace, exiting")
return
while self.running:
connected = await self.connect_with_retry()
if not connected:
break
if not await self.register():
logger.error("Failed to register with orchestrator, exiting")
return
self.heartbeat_task = asyncio.create_task(self.heartbeat_loop())
try:
await self.message_loop()
except ConnectionClosed:
logger.warning("connection closed; reconnecting")
except Exception as e:
logger.error(f"Error in message loop: {e}")
finally:
if self.heartbeat_task:
self.heartbeat_task.cancel()
try:
await self.heartbeat_task
except asyncio.CancelledError:
pass
self.heartbeat_task = None
if self.websocket:
try:
await self.safe_send({"type": "deregister", "agent_id": self.agent_id})
except Exception:
pass
try:
await self.websocket.close()
except Exception:
pass
self.websocket = None
if self.running:
await asyncio.sleep(1)
await self.shutdown_active_tasks()
logger.info("Agent shutdown complete")
async def main():
# Build an agent instance from environment configuration and start its runtime loop.
orchestrator_url = os.getenv("ORCHESTRATOR_URL", "ws://localhost:8000")
agent_id = os.getenv("AGENT_ID")
capabilities = os.getenv("AGENT_CAPABILITIES", "general").split(",")
workspace_dir = os.getenv("WORKSPACE_DIR", "/workspace")
git_repo_url = os.getenv("GIT_REPO_URL")
logger.info(f"Starting agent with ID: {agent_id or 'auto-generated'}")
logger.info(f"Capabilities: {capabilities}")
logger.info(f"Orchestrator URL: {orchestrator_url}")
agent = Agent(
orchestrator_url=orchestrator_url,
agent_id=agent_id,
capabilities=capabilities,
workspace_dir=workspace_dir,
git_repo_url=git_repo_url,
)
await agent.run()
if __name__ == "__main__":
asyncio.run(main())
+5
View File
@@ -0,0 +1,5 @@
openai==1.55.3
websockets==13.1
pydantic==2.9.2
python-dotenv==1.0.1
prometheus-client==0.20.0
+480
View File
@@ -0,0 +1,480 @@
"""Task execution engine with OpenAI-compatible LLM provider integration.
Merged from agent_swarm_v4:
- OpenAI-only provider configuration (OPENAI_*/MODEL_* env vars and custom base URLs)
- Peer-collaboration hook and specialist-role alignment prompting
- Preserves model invocation, handoff decision hooks, workspace summarization,
file-application behavior, and the Manager billing/audit usage attribution
(X-Agent/X-Agnet headers, usage payload, billing_source)
"""
import asyncio
import json
import logging
import os
import time
from pathlib import Path
from typing import Callable, Optional
from openai import AsyncOpenAI
from .handoff_logic import should_handoff
logger = logging.getLogger(__name__)
class TaskExecutor:
"""Executes tasks using a configurable OpenAI-compatible API."""
def __init__(self, agent_id: str, workspace_dir: str):
self.agent_id = agent_id
self.workspace_dir = workspace_dir
api_key = (
os.getenv("OPENAI_API_KEY")
or os.getenv("MODEL_API_KEY")
)
if not api_key:
raise ValueError("OPENAI_API_KEY environment variable not set")
self.model = (
os.getenv("OPENAI_MODEL")
or os.getenv("MODEL_NAME")
or os.getenv("MODEL_ID")
or "gpt-4o-mini"
)
self.api_base = (
os.getenv("OPENAI_API_BASE")
or os.getenv("MODEL_API_BASE")
or "https://api.openai.com/v1"
)
self.api_mode = "openai"
self.client = AsyncOpenAI(api_key=api_key, base_url=self.api_base)
self.client_type = "openai"
self.usage = self._empty_usage()
self.current_context: dict = {}
async def execute_task(
self,
task_id: str,
description: str,
context: dict,
handoff_callback: Optional[Callable] = None,
agent_capabilities: Optional[list[str]] = None,
peer_collaboration_callback: Optional[Callable] = None,
) -> dict:
logger.info(f"Executing task {task_id}: {description}")
try:
started_at = time.time()
self.current_context = context or {}
self.usage = self._empty_usage(context)
multi_agent_leaf_mode = self._multi_agent_leaf_mode(context)
allow_handoff = self._allow_dynamic_handoff(context)
if multi_agent_leaf_mode:
subtasks = [{
"description": description,
"complexity": "medium",
"estimated_time": 30,
"dependencies": [],
"required_capabilities": context.get("required_capabilities") or ["general"],
}]
elif self._subtask_handoff_enabled():
subtasks = await self._parse_task(description, context)
else:
subtasks = [{
"description": description,
"complexity": "medium",
"estimated_time": 30,
"dependencies": [],
"required_capabilities": ["general"],
}]
results = []
for subtask in subtasks:
if handoff_callback and self._subtask_handoff_enabled() and allow_handoff:
decision = should_handoff(
subtask,
self.agent_id,
agent_capabilities=agent_capabilities,
)
if decision.should_handoff:
await handoff_callback(
task_id=task_id,
subtask=subtask,
target_capabilities=decision.target_capabilities,
)
results.append({
"subtask": subtask,
"status": "handed_off",
"reason": decision.reason,
"target_capabilities": decision.target_capabilities,
})
continue
results.append(await self._execute_subtask(subtask, task_id, context, peer_collaboration_callback))
success = all(r["status"] in ["completed", "handed_off"] for r in results)
awaiting_handoff = any(r["status"] == "handed_off" for r in results)
return {
"success": success,
"task_id": task_id,
"subtasks": results,
"awaiting_handoff": awaiting_handoff,
"agent_id": self.agent_id,
"usage": self._usage_payload(time.time() - started_at),
}
except Exception as e:
logger.error(f"Error executing task {task_id}: {e}")
return {
"success": False,
"task_id": task_id,
"error": str(e),
"agent_id": self.agent_id,
"usage": self._usage_payload(time.time() - started_at if "started_at" in locals() else 0),
}
finally:
self.current_context = {}
async def _parse_task(self, description: str, context: dict) -> list[dict]:
try:
prompt = f"""You are a task planning assistant. Break down the following programming task into concrete, actionable subtasks.
Task: {description}
Context: {json.dumps(context, indent=2)}
Return a JSON array of subtasks, where each subtask has:
- description: Clear description of what needs to be done
- complexity: \"low\", \"medium\", or \"high\"
- estimated_time: Estimated time in minutes
- dependencies: List of subtask indices this depends on (empty if none)
- required_capabilities: List of capabilities needed
Return ONLY the JSON array, no other text."""
content = await self._complete(prompt, max_tokens=2000)
return json.loads(self._strip_json_fence(content))
except Exception as e:
logger.error(f"Error parsing task: {e}")
return [{
"description": description,
"complexity": "medium",
"estimated_time": 30,
"dependencies": [],
"required_capabilities": ["general"],
}]
async def _execute_subtask(self, subtask: dict, task_id: str, context: dict, peer_collaboration_callback: Optional[Callable]) -> dict:
try:
description = subtask["description"]
workspace_files = self._summarize_workspace()
workspace_context = self._collect_workspace_context()
user_prompt = context.get("user_prompt") or context.get("run_goal") or context.get("root_task_description") or ""
specialist_role = context.get("specialist_role", "general")
dependency_artifacts = context.get("dependency_artifacts") or []
implementation_artifacts = [
artifact for artifact in dependency_artifacts
if "implementation" in (artifact.get("task_id") or "")
]
testing_artifacts = [
artifact for artifact in dependency_artifacts
if "testing" in (artifact.get("task_id") or "")
]
peer_context = []
peer_agents = context.get("peer_agents") or []
should_consult_peers = specialist_role in {"testing", "documentation"}
if peer_collaboration_callback and peer_agents and should_consult_peers:
max_peer_consults = int((context or {}).get("max_peer_consults", 2) or 2)
preferred_roles = []
if specialist_role in {"testing", "documentation"}:
preferred_roles = ["implementation"]
ordered_peers = sorted(
peer_agents,
key=lambda peer: 0 if peer.get("role") in preferred_roles else 1,
)
for peer in ordered_peers[:max_peer_consults]:
try:
reply = await peer_collaboration_callback(
task_id=task_id,
target_agent_id=peer.get("agent_id"),
content=(
f"Specialist role: {specialist_role}. "
f"Please provide guidance and confirm behavior for: {description}. "
f"Original user request: {user_prompt}"
),
timeout_seconds=float((context or {}).get("peer_timeout_seconds", 10.0) or 10.0),
)
peer_context.append(
{
"agent_id": peer.get("agent_id"),
"role": peer.get("role"),
"content": reply.get("content", ""),
}
)
except Exception as exc:
logger.warning(f"Peer collaboration failed with {peer.get('agent_id')}: {exc}")
prompt = f"""You are a programming assistant working in a collaborative agent system.
Original user request:
{user_prompt}
Your specialist role:
{specialist_role}
Task: {description}
Dependency artifacts from other specialists:
{json.dumps(dependency_artifacts, indent=2)}
Implementation artifacts relevant to alignment:
{json.dumps(implementation_artifacts, indent=2)}
Testing artifacts relevant to alignment:
{json.dumps(testing_artifacts, indent=2)}
Workspace directory: {self.workspace_dir}
Current workspace files:
{json.dumps(workspace_files, indent=2)}
Relevant workspace file contents:
{json.dumps(workspace_context, indent=2)}
Peer specialist input:
{json.dumps(peer_context, indent=2)}
Alignment requirements:
- If your role is testing, align your tests with the implementation artifacts and their stated error semantics.
- If your role is documentation, align your docs with both implementation and testing artifacts.
- Do not invent behavior that conflicts with dependency artifacts unless you explicitly surface an error.
- Treat implementation artifacts as the source of truth for API behavior and exception semantics.
- If peer specialist input conflicts with implementation artifacts, prefer implementation semantics and explain the correction in your changes summary.
- If your role is documentation or testing, update only your specialist outputs to converge on implementation behavior unless the implementation artifact is clearly missing or contradictory.
Return your response as JSON with this structure:
{{
\"status\": \"completed\" or \"failed\",
\"summary\": \"Brief description of what was done\",
\"files\": [{{\"path\": \"relative/path.py\", \"action\": \"write\", \"content\": \"complete file content\"}}],
\"changes\": \"Detailed description of changes\",
\"error\": \"Error message if failed, null otherwise\"
}}
Return ONLY the JSON, no other text."""
content = await self._complete(prompt, max_tokens=4000)
result = self._parse_json_response(content)
if result.get("status") == "completed":
apply_result = await self._apply_file_changes(result.get("files", []))
result["files_modified"] = apply_result["files_modified"]
result["files_deleted"] = apply_result["files_deleted"]
if apply_result["errors"]:
result["status"] = "failed"
result["error"] = "; ".join(apply_result["errors"])
result["subtask"] = subtask
return result
except Exception as e:
logger.error(f"Error executing subtask: {e}")
return {
"subtask": subtask,
"status": "failed",
"error": str(e),
"summary": f"Failed to execute: {e}",
}
async def _complete(self, prompt: str, max_tokens: int) -> str:
extra_headers = self._model_attribution_headers()
response = await self.client.chat.completions.create(
model=self.model,
messages=[{"role": "user", "content": prompt}],
max_tokens=max_tokens,
extra_headers=extra_headers or None,
)
self._record_openai_usage(response)
return response.choices[0].message.content or ""
def _empty_usage(self, context: Optional[dict] = None) -> dict:
plan = ((context or {}).get("orchestration_plan") or {})
billing = plan.get("billing_context") or {}
return {
"model_id": self.model,
"model_tokens": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"model_cost_usd": 0.0,
"runtime_seconds": 0.0,
"billing_source": billing.get("provider") or os.getenv("BILLING_SOURCE", "unknown"),
}
def _record_openai_usage(self, response):
usage = getattr(response, "usage", None)
if not usage:
return
prompt_tokens = int(getattr(usage, "prompt_tokens", 0) or 0)
completion_tokens = int(getattr(usage, "completion_tokens", 0) or 0)
total_tokens = int(getattr(usage, "total_tokens", 0) or prompt_tokens + completion_tokens)
self._add_usage(prompt_tokens, completion_tokens, total_tokens)
def _add_usage(self, prompt_tokens: int, completion_tokens: int, total_tokens: int):
self.usage["prompt_tokens"] += prompt_tokens
self.usage["completion_tokens"] += completion_tokens
self.usage["model_tokens"] += total_tokens
input_cost = float(os.getenv("MODEL_INPUT_COST_PER_1M", "0") or 0)
output_cost = float(os.getenv("MODEL_OUTPUT_COST_PER_1M", "0") or 0)
self.usage["model_cost_usd"] += (
prompt_tokens * input_cost / 1_000_000
+ completion_tokens * output_cost / 1_000_000
)
def _usage_payload(self, runtime_seconds: float) -> dict:
usage = dict(self.usage)
usage["runtime_seconds"] = runtime_seconds
return usage
def _model_attribution_headers(self) -> dict:
headers = {}
mapping = {
"manager_deployment_id": ["X-Agent-Manager-Deployment-ID", "X-Agnet-Manager-Deployment-ID"],
"swarm_id": ["X-Agent-Swarm-ID", "X-Agnet-Swarm-ID"],
"task_id": ["X-Agent-Task-ID", "X-Agnet-Task-ID"],
"agent_role": ["X-Agent-Agent-Role", "X-Agnet-Agent-Role"],
"correlation_id": ["X-Correlation-ID"],
"model_id": ["X-Agent-Model-ID", "X-Agnet-Model-ID"],
}
for key, header_names in mapping.items():
value = self.current_context.get(key)
if value:
for header in header_names:
headers[header] = str(value)
headers.setdefault("X-Agent-Model-ID", self.model)
headers.setdefault("X-Agnet-Model-ID", self.model)
return headers
def _subtask_handoff_enabled(self) -> bool:
return os.getenv("ENABLE_SUBTASK_HANDOFF", "false").lower() in {"1", "true", "yes"}
def _multi_agent_leaf_mode(self, context: Optional[dict]) -> bool:
return self._subtask_handoff_enabled() and (context or {}).get("workflow_mode") == "multi_agent"
def _allow_dynamic_handoff(self, context: Optional[dict]) -> bool:
if (context or {}).get("workflow_mode") != "multi_agent":
return True
return bool((context or {}).get("allow_handoff", False))
def _strip_json_fence(self, content: str) -> str:
content = content.strip()
if not content.startswith("```"):
return content
lines = content.split("\n")
if lines and lines[0].startswith("```"):
lines = lines[1:]
if lines and lines[-1].strip() == "```":
lines = lines[:-1]
return "\n".join(lines).strip()
def _parse_json_response(self, content: str) -> dict:
content = self._strip_json_fence(content)
try:
return json.loads(content)
except json.JSONDecodeError:
start = content.find("{")
end = content.rfind("}")
if start == -1 or end == -1 or end <= start:
raise
return json.loads(content[start:end + 1])
def _summarize_workspace(self, max_files: int = 80) -> list[str]:
root = Path(self.workspace_dir)
if not root.exists():
return []
ignored_dirs = {".git", "__pycache__", "node_modules", ".venv", "venv"}
files = []
for path in root.rglob("*"):
if len(files) >= max_files:
break
if not path.is_file():
continue
if any(part in ignored_dirs for part in path.relative_to(root).parts):
continue
files.append(str(path.relative_to(root)))
return sorted(files)
def _collect_workspace_context(self, max_files: int = 20, max_bytes_per_file: int = 6000) -> dict[str, str]:
root = Path(self.workspace_dir)
if not root.exists():
return {}
ignored_dirs = {".git", "__pycache__", "node_modules", ".venv", "venv"}
allowed_suffixes = {".py", ".js", ".ts", ".tsx", ".jsx", ".json", ".md", ".txt", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".sh"}
context = {}
for path in sorted(root.rglob("*")):
if len(context) >= max_files:
break
if not path.is_file():
continue
relative = path.relative_to(root)
if any(part in ignored_dirs for part in relative.parts):
continue
if path.suffix and path.suffix.lower() not in allowed_suffixes:
continue
try:
data = path.read_bytes()[:max_bytes_per_file]
context[str(relative)] = data.decode("utf-8")
except UnicodeDecodeError:
continue
except Exception as e:
logger.warning(f"Failed to read workspace file {relative}: {e}")
return context
def _resolve_workspace_path(self, file_path: str) -> Path:
if not file_path or os.path.isabs(file_path):
raise ValueError(f"Invalid relative path: {file_path}")
root = Path(self.workspace_dir).resolve()
resolved = (root / file_path).resolve()
if root != resolved and root not in resolved.parents:
raise ValueError(f"Path escapes workspace: {file_path}")
return resolved
async def _apply_file_changes(self, files: list[dict]) -> dict:
result = {"files_modified": [], "files_deleted": [], "errors": []}
for file_change in files:
path = file_change.get("path")
action = file_change.get("action", "write")
try:
target = self._resolve_workspace_path(path)
if action == "write":
content = file_change.get("content")
if content is None:
raise ValueError(f"Missing content for {path}")
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(content, encoding="utf-8")
result["files_modified"].append(path)
elif action == "delete":
if target.exists():
target.unlink()
result["files_deleted"].append(path)
else:
raise ValueError(f"Unsupported file action for {path}: {action}")
except Exception as e:
logger.error(f"Failed to apply file change {path}: {e}")
result["errors"].append(f"{path}: {e}")
return result
async def read_file(self, file_path: str) -> Optional[str]:
try:
full_path = os.path.join(self.workspace_dir, file_path)
with open(full_path, "r", encoding="utf-8") as f:
return f.read()
except Exception as e:
logger.error(f"Error reading file {file_path}: {e}")
return None
async def write_file(self, file_path: str, content: str) -> bool:
try:
full_path = self._resolve_workspace_path(file_path)
os.makedirs(os.path.dirname(full_path), exist_ok=True)
with open(full_path, "w", encoding="utf-8") as f:
f.write(content)
logger.info(f"Wrote file: {file_path}")
return True
except Exception as e:
logger.error(f"Error writing file {file_path}: {e}")
return False
@@ -0,0 +1,115 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Swarm Terminal Snapshot</title>
<style>
body {
margin: 0;
background:
radial-gradient(circle at top right, rgba(74, 222, 128, 0.10), transparent 20%),
radial-gradient(circle at top left, rgba(96, 165, 250, 0.10), transparent 24%),
#0a0f14;
color: #d7e3ee;
font-family: Menlo, Monaco, "SFMono-Regular", "JetBrains Mono", monospace;
}
.frame {
width: 1600px;
min-height: 900px;
margin: 0 auto;
padding: 48px;
box-sizing: border-box;
}
.window {
background: #0b1220;
border: 1px solid rgba(148, 163, 184, 0.20);
border-radius: 18px;
overflow: hidden;
box-shadow: 0 30px 80px rgba(0, 0, 0, 0.45);
}
.topbar {
display: flex;
align-items: center;
justify-content: space-between;
padding: 14px 18px;
background: #121a2a;
border-bottom: 1px solid rgba(148, 163, 184, 0.12);
}
.dots {
display: flex;
gap: 8px;
}
.dot {
width: 12px;
height: 12px;
border-radius: 50%;
}
.dot.red { background: #fb7185; }
.dot.yellow { background: #fbbf24; }
.dot.green { background: #4ade80; }
.title {
color: #93a4b8;
font-size: 15px;
letter-spacing: 0.02em;
}
.terminal {
padding: 28px 32px 36px;
font-size: 24px;
line-height: 1.7;
white-space: pre-wrap;
word-break: break-word;
}
.line:nth-child(1),
.line:nth-child(8),
.line:nth-child(13) {
color: #7dd3fc;
}
.line:nth-last-child(2),
.line:nth-last-child(1) {
color: #bef264;
font-weight: 600;
}
</style>
</head>
<body>
<div class="frame">
<div class="window">
<div class="topbar">
<div class="dots">
<span class="dot red"></span>
<span class="dot yellow"></span>
<span class="dot green"></span>
</div>
<div class="title">swarm-terminal-snapshot</div>
<div class="title">swarm-300589052dd0</div>
</div>
<div class="terminal">
<div class='line'>$ python3 scripts/run_swarm_poster_demo.py</div>
<div class='line'>[swarm] deployment_id=runtime-dep-90b592c531e3</div>
<div class='line'>[swarm] swarm_id=swarm-300589052dd0</div>
<div class='line'>[task] task_id=swarm-300589052dd0-task-1</div>
<div class='line'>[task] status=in_progress</div>
<div class='line'>[task] assigned_agent=agent-full-5d6968f886-7cm2l</div>
<div class='line'></div>
<div class='line'>$ curl -H "Authorization: Bearer ***" /api/swarms/{swarm_id}/tasks</div>
<div class='line'>task.started_at = 2026-05-29 10:12:36 UTC</div>
<div class='line'>task.halfway_at = 2026-05-29 10:12:56 UTC</div>
<div class='line'>halfway.observed_at = 2026-05-29 10:12:58 UTC</div>
<div class='line'></div>
<div class='line'>$ curl -H "Authorization: Bearer ***" /api/swarms/{swarm_id}/metrics</div>
<div class='line'>runtime.status = running</div>
<div class='line'>runtime.tasks_total = 1</div>
<div class='line'>runtime.tasks_by_status = {"in_progress": 1}</div>
<div class='line'>runtime.agents_connected = 1</div>
<div class='line'>runtime.budget.duration_seconds = 40</div>
<div class='line'>runtime.budget.duration_ratio = 495.7%</div>
<div class='line'></div>
<div class='line'># milestone</div>
<div class='line'>> 任务开始: 2026-05-29 10:12:36 UTC</div>
<div class='line'>> 任务过半: 2026-05-29 10:12:58 UTC</div>
</div>
</div>
</div>
</body>
</html>
Binary file not shown.

After

Width:  |  Height:  |  Size: 223 KiB

@@ -0,0 +1,285 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Swarm Blog MVP Live Demo</title>
<style>
:root {
--bg: #07111f;
--panel: rgba(10, 25, 47, 0.84);
--panel-strong: rgba(12, 32, 60, 0.95);
--line: rgba(125, 211, 252, 0.25);
--cyan: #7dd3fc;
--teal: #5eead4;
--lime: #bef264;
--text: #e6f0ff;
--muted: #9bb1c8;
--warn: #fbbf24;
}
* { box-sizing: border-box; }
body {
margin: 0;
font-family: "Avenir Next", "PingFang SC", "Helvetica Neue", sans-serif;
color: var(--text);
background:
radial-gradient(circle at top left, rgba(45, 212, 191, 0.18), transparent 32%),
radial-gradient(circle at top right, rgba(125, 211, 252, 0.24), transparent 28%),
linear-gradient(135deg, #050c16 0%, #07111f 42%, #0d1d35 100%);
min-height: 100vh;
}
.canvas {
width: 1600px;
min-height: 900px;
margin: 0 auto;
padding: 56px;
position: relative;
overflow: hidden;
}
.grid {
position: absolute;
inset: 0;
background-image:
linear-gradient(rgba(125, 211, 252, 0.05) 1px, transparent 1px),
linear-gradient(90deg, rgba(125, 211, 252, 0.05) 1px, transparent 1px);
background-size: 56px 56px;
mask-image: linear-gradient(to bottom, rgba(0,0,0,.75), transparent);
pointer-events: none;
}
.hero {
display: flex;
justify-content: space-between;
gap: 28px;
align-items: flex-start;
margin-bottom: 28px;
}
.hero h1 {
margin: 0 0 12px 0;
font-size: 68px;
line-height: 0.95;
letter-spacing: -2px;
}
.hero p {
margin: 0;
max-width: 800px;
color: var(--muted);
font-size: 24px;
line-height: 1.5;
}
.badge {
display: inline-flex;
align-items: center;
gap: 10px;
border: 1px solid var(--line);
background: rgba(6, 19, 36, 0.72);
border-radius: 999px;
padding: 12px 18px;
color: var(--cyan);
font-size: 16px;
letter-spacing: 0.08em;
text-transform: uppercase;
}
.layout {
display: grid;
grid-template-columns: 1.25fr 0.75fr;
gap: 28px;
}
.panel {
background: var(--panel);
border: 1px solid var(--line);
border-radius: 28px;
padding: 28px;
box-shadow: 0 25px 60px rgba(0, 0, 0, 0.32);
backdrop-filter: blur(16px);
}
.panel strong {
display: block;
font-size: 16px;
letter-spacing: 0.08em;
text-transform: uppercase;
color: var(--cyan);
margin-bottom: 14px;
}
.cards {
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 18px;
margin-bottom: 28px;
}
.metric {
background: var(--panel-strong);
border: 1px solid rgba(190, 242, 100, 0.16);
border-radius: 22px;
padding: 20px;
}
.metric .label {
color: var(--muted);
font-size: 15px;
margin-bottom: 8px;
}
.metric .value {
font-size: 34px;
font-weight: 700;
letter-spacing: -1px;
}
.timeline {
display: grid;
gap: 16px;
margin-top: 8px;
}
.step {
position: relative;
padding: 18px 18px 18px 68px;
border-radius: 22px;
background: rgba(7, 18, 33, 0.9);
border: 1px solid rgba(125, 211, 252, 0.12);
}
.step::before {
content: "";
position: absolute;
left: 28px;
top: 26px;
width: 16px;
height: 16px;
border-radius: 50%;
background: linear-gradient(135deg, var(--teal), var(--cyan));
box-shadow: 0 0 0 8px rgba(94, 234, 212, 0.12);
}
.step h3 {
margin: 0 0 8px 0;
font-size: 24px;
}
.step p {
margin: 0;
color: var(--muted);
font-size: 16px;
line-height: 1.5;
}
.code {
font-family: "SF Mono", "JetBrains Mono", monospace;
color: var(--lime);
word-break: break-all;
}
.sidebar {
display: grid;
gap: 18px;
}
.status {
border-radius: 24px;
padding: 22px;
background: linear-gradient(160deg, rgba(94, 234, 212, 0.14), rgba(125, 211, 252, 0.08));
border: 1px solid rgba(94, 234, 212, 0.25);
}
.status .headline {
font-size: 18px;
color: var(--muted);
margin-bottom: 8px;
}
.status .value {
font-size: 44px;
font-weight: 700;
letter-spacing: -1px;
margin-bottom: 10px;
}
.status .sub {
color: var(--muted);
font-size: 16px;
line-height: 1.5;
}
.meta-row {
display: flex;
justify-content: space-between;
gap: 12px;
padding: 10px 0;
border-bottom: 1px solid rgba(125, 211, 252, 0.08);
font-size: 15px;
}
.meta-row:last-child { border-bottom: none; }
.meta-label { color: var(--muted); }
.meta-value { text-align: right; max-width: 56%; }
.footer {
margin-top: 22px;
color: var(--warn);
font-size: 15px;
line-height: 1.5;
}
</style>
</head>
<body>
<div class="canvas">
<div class="grid"></div>
<div class="hero">
<div>
<div class="badge">Live Swarm Poster</div>
<h1>Swarm Blog MVP Live Demo</h1>
<p>真实蜂群任务已成功创建,并已记录到“任务开始”和“进行过半”两段里程碑。下面的内容全部来自当前运行中的 swarm 状态,而不是手工拼接。</p>
</div>
<div class="panel" style="min-width: 360px;">
<strong>Demo Goal</strong>
<p style="font-size: 18px; color: var(--text); line-height: 1.55;">在目标仓库里开发一个博客系统 MVP,包括文章列表、详情和基础增删改能力。</p>
</div>
</div>
<div class="layout">
<div class="panel">
<div class="cards">
<div class="metric">
<div class="label">Swarm</div>
<div class="value">swarm-300589052dd0</div>
</div>
<div class="metric">
<div class="label">Task Status</div>
<div class="value">in_progress</div>
</div>
<div class="metric">
<div class="label">Budget Progress</div>
<div class="value">496%</div>
</div>
</div>
<strong>Timeline</strong>
<div class="timeline">
<div class="step">
<h3>任务开始</h3>
<p>任务已被 agent 领取并进入运行态。开始时间:<span class="code">2026-05-29 10:12:36 UTC</span></p>
</div>
<div class="step">
<h3>进行到一半</h3>
<p>脚本按 budget 的 50% 自动确认里程碑。预算过半时间:<span class="code">2026-05-29 10:12:56 UTC</span>,实际记录时间:<span class="code">2026-05-29 10:12:58 UTC</span></p>
</div>
<div class="step">
<h3>当前执行中</h3>
<p>任务仍处于 <span class="code">in_progress</span>,最近心跳:<span class="code">2026-05-29T10:12:49.319420Z</span></p>
</div>
</div>
</div>
<div class="sidebar">
<div class="status">
<div class="headline">Poster Snapshot</div>
<div class="value">Mid-Run</div>
<div class="sub">这张海报展示的是蜂群任务已经真正启动,并且已经跑到预算半程时的现场状态。</div>
</div>
<div class="panel">
<strong>Swarm Meta</strong>
<div class="meta-row"><span class="meta-label">Deployment</span><span class="meta-value code">runtime-dep-90b592c531e3</span></div>
<div class="meta-row"><span class="meta-label">Task</span><span class="meta-value code">swarm-300589052dd0-task-1</span></div>
<div class="meta-row"><span class="meta-label">Agent</span><span class="meta-value code">agent-full-5d6968f886-7cm2l</span></div>
<div class="meta-row"><span class="meta-label">Attempts</span><span class="meta-value">0</span></div>
<div class="meta-row"><span class="meta-label">Budget</span><span class="meta-value">40s</span></div>
<div class="meta-row"><span class="meta-label">Output File</span><span class="meta-value code">swarm-poster-300589052dd0.html</span></div>
</div>
<div class="panel">
<strong>Latest Events</strong>
<div class="meta-row"><span class="meta-label">1</span><span class="meta-value code">task.running</span></div>
<div class="meta-row"><span class="meta-label">2</span><span class="meta-value code">task.heartbeat</span></div>
<div class="meta-row"><span class="meta-label">3</span><span class="meta-value code">budget.alert</span></div>
<div class="footer">注:如果后续继续执行,日志与状态还会变化;这张图锁定的是“开始后已过半”的那一刻。</div>
</div>
</div>
</div>
</div>
</body>
</html>
@@ -0,0 +1,94 @@
{
"swarm": {
"deployment_id": "runtime-dep-90b592c531e3",
"runtime_deployment_id": "runtime-dep-90b592c531e3",
"manager_deployment_id": "dep_poster_demo_20260529T100940Z",
"swarm_id": "swarm-300589052dd0",
"status": "running",
"created": true
},
"task": {
"task_id": "swarm-300589052dd0-task-1",
"title": "Swarm objective",
"description": "Build a blog system MVP in the configured workspace repository. Deliver a runnable implementation with a simple backend and frontend for posts: list posts, view a post, create, edit, and delete. Keep setup minimal and include short run instructions.",
"status": "in_progress",
"agent_role": "general",
"required_capabilities": [
"general"
],
"depends_on": [],
"parent_task_id": null,
"root_task_id": "swarm-300589052dd0-task-1",
"source": "runtime_bridge",
"task_graph_id": "task-1",
"assigned_agent_id": "agent-full-5d6968f886-7cm2l",
"created_at": 1780049380.5639093,
"started_at": 1780049556.1651127,
"completed_at": null,
"attempt": 0,
"max_retries": 3,
"retry_count": 0,
"blocked_reason": null,
"context": {
"orchestration_plan": {
"objective": "Build a blog system MVP in the configured workspace repository. Deliver a runnable implementation with a simple backend and frontend for posts: list posts, view a post, create, edit, and delete. Keep setup minimal and include short run instructions.",
"sub_mode": "code",
"risk_level": "low",
"budget": {
"duration_seconds": 40,
"token_limit": 120000
},
"agents": [
{
"task_id": "poster-blog-20260529t100940z",
"role": "fullstack",
"title": "Implement blog system MVP",
"description": "Create a runnable blog system MVP in the target repository with post list, post detail, and create/edit/delete flows. Keep the implementation pragmatic and coherent with the existing repo. Commit and push the result branch when changes are ready.",
"depends_on": []
}
]
},
"resource_grants": [],
"sub_mode": "code",
"runtime_deployment_id": "runtime-dep-90b592c531e3",
"swarm_id": "swarm-300589052dd0",
"manager_deployment_id": "dep_poster_demo_20260529T100940Z",
"correlation_id": "corr_poster_demo_20260529T100940Z",
"task_graph_id": "task-1",
"task_title": "Swarm objective",
"depends_on": [],
"agent_role": "general",
"required_capabilities": [
"general"
],
"parent_task_id": null,
"root_task_id": "swarm-300589052dd0-task-1",
"source": "runtime_bridge",
"workflow_mode": "single_agent",
"allow_handoff": false,
"model_id": null
}
},
"metrics": {
"swarm_id": "swarm-300589052dd0",
"window": "15m",
"step": "60s",
"status": "running",
"runtime_duration_seconds": 198.26700401306152,
"tasks_total": 1,
"tasks_by_status": {
"in_progress": 1
},
"agents_connected": 1,
"agents_registered": 1,
"average_task_duration_seconds": 0,
"budget": {
"duration_seconds": 40,
"duration_ratio": 4.956675100326538
}
},
"started_at": 1780049556.1651127,
"halfway_at": 1780049576.1651127,
"halfway_observed_at": 1780049578.828649,
"poster_html": "/Users/mac/Projects/HeiCode-Sawrm/artifacts/swarm-posters/swarm-poster-300589052dd0.html"
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 643 KiB

+109
View File
@@ -0,0 +1,109 @@
#!/bin/bash
# K8s 蜂群系统 - 快速状态检查
echo "======================================"
echo " K8s 蜂群系统 - 状态检查"
echo "======================================"
echo ""
# 颜色定义
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# 检查 kubectl
if ! command -v kubectl &> /dev/null; then
echo -e "${RED}❌ kubectl 未安装${NC}"
exit 1
fi
# 检查集群连接
echo "1. 检查集群连接..."
if kubectl cluster-info &> /dev/null; then
echo -e "${GREEN}✅ 集群连接正常${NC}"
else
echo -e "${RED}❌ 无法连接到集群${NC}"
exit 1
fi
# 检查命名空间
echo ""
echo "2. 检查命名空间..."
if kubectl get namespace swarm-system &> /dev/null; then
echo -e "${GREEN}✅ swarm-system 命名空间存在${NC}"
else
echo -e "${RED}❌ swarm-system 命名空间不存在${NC}"
exit 1
fi
# 检查 Redis
echo ""
echo "3. 检查 Redis..."
REDIS_STATUS=$(kubectl get pod redis-0 -n swarm-system -o jsonpath='{.status.phase}' 2>/dev/null)
if [ "$REDIS_STATUS" = "Running" ]; then
echo -e "${GREEN}✅ Redis 运行中${NC}"
else
echo -e "${RED}❌ Redis 状态: $REDIS_STATUS${NC}"
fi
# 检查 Orchestrator
echo ""
echo "4. 检查 Orchestrator..."
ORCH_READY=$(kubectl get deployment orchestrator -n swarm-system -o jsonpath='{.status.readyReplicas}' 2>/dev/null)
if [ "$ORCH_READY" = "1" ]; then
echo -e "${GREEN}✅ Orchestrator 运行中${NC}"
else
echo -e "${RED}❌ Orchestrator 未就绪${NC}"
fi
# 获取 Orchestrator 外部 IP
echo ""
echo "5. 获取访问地址..."
EXTERNAL_IP=$(kubectl get svc orchestrator-service -n swarm-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}' 2>/dev/null)
if [ -n "$EXTERNAL_IP" ]; then
echo -e "${GREEN}✅ Orchestrator 地址: http://$EXTERNAL_IP:8000${NC}"
# 测试健康检查
echo ""
echo "6. 测试 API 健康检查..."
if curl -s -f "http://$EXTERNAL_IP:8000/health" > /dev/null 2>&1; then
echo -e "${GREEN}✅ API 健康检查通过${NC}"
# 显示健康状态详情
HEALTH=$(curl -s "http://$EXTERNAL_IP:8000/health")
echo " 详情: $HEALTH"
else
echo -e "${YELLOW}⚠️ API 健康检查失败(可能还在启动中)${NC}"
fi
else
echo -e "${YELLOW}⚠️ 等待 LoadBalancer 分配外部 IP...${NC}"
fi
# 显示所有 Pod
echo ""
echo "7. 所有 Pod 状态:"
echo "-----------------------------------"
kubectl get pods -n swarm-system
# 显示服务
echo ""
echo "8. 服务状态:"
echo "-----------------------------------"
kubectl get svc -n swarm-system
# 总结
echo ""
echo "======================================"
echo " 状态检查完成"
echo "======================================"
echo ""
echo "快速命令:"
echo " 查看日志: kubectl logs -n swarm-system deployment/orchestrator -f"
echo " 查看 Pod: kubectl get pods -n swarm-system"
echo " 运行测试: python3 test_system.py"
echo ""
echo "文档:"
echo " 快速入门: 快速入门指南.md"
echo " 完整文档: 部署和使用文档.md"
echo ""
+71
View File
@@ -0,0 +1,71 @@
# HeiCode Swarm 桌面客户端
基于 **Electron + React + TypeScript** 的桌面客户端,是蜂群系统面向用户的入口。它负责提交任务、连接并展示 Orchestrator 的运行状态,分析任务复杂度以推荐 Agent 数量,并对生成结果进行聚合与查看。
## 环境要求
- Node.js 18+ 与 npm
- 可访问的 Kubernetes 集群与有效的 kubeconfig(如使用 K8s 调度)
- 已正确配置的 kubectl
## 安装与开发
```bash
npm install # 安装依赖
npm run build # 编译 TypeScript
npm run dev # 开发模式启动
npm run type-check
npm run lint
```
## 配置
应用按以下顺序加载 kubeconfig:
1. 环境变量 `KUBECONFIG`(若设置)
2. 默认位置 `~/.kube/config`
请确保 kubeconfig 指向正确的集群且凭据有效。
## 目录结构
```
desktop-client/
├── src/
│ ├── main.ts # Electron 主进程
│ ├── App.tsx # React 根组件
│ ├── index.html # 入口页面
│ ├── api/
│ │ └── orchestrator-client.ts # 调用 Orchestrator REST 接口
│ ├── ai/ # 任务复杂度分析(见该目录 README)
│ ├── components/ # 界面组件
│ │ ├── TaskSubmissionForm.tsx # 任务提交
│ │ ├── AgentDashboard.tsx # Agent 与运行状态看板
│ │ ├── ResultViewer.tsx # 结果查看
│ │ └── CostControlDialog.tsx # 成本/规模确认
│ ├── analytics/usage-tracker.ts # 用量统计
│ ├── git/ # Git 工作区与分支合并
│ │ ├── workspace-manager.ts
│ │ └── merge-manager.ts
│ └── k8s/ # K8s 接入
│ ├── kubeconfig-loader.ts # 鉴权/集群配置加载
│ └── auto-scaler.ts # 按复杂度自动伸缩 Agent 规模
├── package.json
├── tsconfig.json
└── tailwind.config.js
```
## 主要能力
- **任务提交**:用户输入需求后,通过 `orchestrator-client` 创建蜂群部署。
- **复杂度分析**:`ai/` 模块评估任务规模并推荐 Agent 数量(详见 `src/ai/README.md`)。
- **成本控制**:超过阈值时通过 `CostControlDialog` 提示并要求确认,避免规模失控。
- **状态展示**:`AgentDashboard` 实时展示 Agent、任务与工作流阶段。
- **结果聚合**:`git/merge-manager` 合并各 Agent 的结果分支,`ResultViewer` 统一查看。
- **弹性伸缩**:`k8s/auto-scaler` 依据复杂度分析结果调整 Agent 数量。
## 工作方式
1. 用户在客户端提交需求并(可选)查看复杂度分析与推荐规模。
2. 客户端调用 Orchestrator 创建部署,Orchestrator 分解任务并调度 Agent 执行。
3. 客户端轮询/订阅运行状态,展示阶段进度与各 Agent 产出。
4. 运行完成后,聚合各结果分支并在结果视图中呈现最终交付物。
+72
View File
@@ -0,0 +1,72 @@
{
"name": "heicode-swarm-desktop",
"version": "0.1.0",
"description": "Desktop client for HeiCode Swarm - K8s-based AI agent orchestration",
"main": "dist/main.js",
"scripts": {
"start": "electron .",
"dev": "concurrently \"npm run build:watch\" \"npm run electron:dev\"",
"build": "tsc",
"build:watch": "tsc --watch",
"electron:dev": "wait-on dist/main.js && electron .",
"package": "electron-builder",
"lint": "eslint src --ext .ts,.tsx",
"type-check": "tsc --noEmit",
"validate-analyzer": "tsc && node dist/ai/validate-analyzer.js"
},
"keywords": [
"electron",
"kubernetes",
"ai",
"swarm",
"orchestration"
],
"author": "HeiCode",
"license": "MIT",
"dependencies": {
"@kubernetes/client-node": "^0.20.0",
"@anthropic-ai/sdk": "^0.20.0",
"isomorphic-git": "^1.25.0",
"zustand": "^4.5.0",
"react": "^18.2.0",
"react-dom": "^18.2.0"
},
"devDependencies": {
"@types/node": "^20.11.0",
"@types/react": "^18.2.0",
"@types/react-dom": "^18.2.0",
"@typescript-eslint/eslint-plugin": "^6.19.0",
"@typescript-eslint/parser": "^6.19.0",
"concurrently": "^8.2.2",
"electron": "^28.0.0",
"electron-builder": "^24.9.1",
"eslint": "^8.56.0",
"typescript": "^5.3.3",
"wait-on": "^7.2.0",
"tailwindcss": "^3.4.0",
"autoprefixer": "^10.4.17",
"postcss": "^8.4.33"
},
"build": {
"appId": "com.heicode.swarm",
"productName": "HeiCode Swarm",
"directories": {
"output": "release"
},
"files": [
"dist/**/*",
"package.json"
],
"mac": {
"category": "public.app-category.developer-tools",
"target": ["dmg", "zip"]
},
"linux": {
"target": ["AppImage", "deb"],
"category": "Development"
},
"win": {
"target": ["nsis", "portable"]
}
}
}
+319
View File
@@ -0,0 +1,319 @@
import React, { useState, useEffect } from 'react';
import { createRoot } from 'react-dom/client';
import { TaskSubmissionForm } from './components/TaskSubmissionForm';
import { AgentDashboard } from './components/AgentDashboard';
import { ResultViewer } from './components/ResultViewer';
import { ComplexityAnalysis, getAnalyzer } from './ai/complexity-analyzer';
import { getOrchestratorClient } from './api/orchestrator-client';
import { AutoScaler } from './k8s/auto-scaler';
import { MergeManager, MergeConflict, ConflictResolution } from './git/merge-manager';
import { WorkspaceManager } from './git/workspace-manager';
import { KubeConfigLoader } from './k8s/kubeconfig-loader';
interface AppState {
status: 'idle' | 'loading' | 'connected' | 'error';
message: string;
currentWorkspaceId: string | null;
mergeConflicts: MergeConflict[];
mergedBranches: string[];
showConflictResolution: boolean;
}
const App: React.FC = () => {
const [state, setState] = useState<AppState>({
status: 'idle',
message: 'Initializing HeiCode Swarm...',
currentWorkspaceId: null,
mergeConflicts: [],
mergedBranches: [],
showConflictResolution: false,
});
const [orchestratorUrl] = useState('http://localhost:8000');
const [autoScaler, setAutoScaler] = useState<AutoScaler | null>(null);
const [workspaceManager] = useState(new WorkspaceManager());
useEffect(() => {
initializeApp();
}, []);
const initializeApp = async () => {
setState(prev => ({ ...prev, status: 'loading', message: 'Connecting to Kubernetes...' }));
try {
// Load kubeconfig
const kubeLoader = new KubeConfigLoader();
kubeLoader.load();
const validation = kubeLoader.validate();
if (!validation.valid) {
throw new Error(`Kubeconfig validation failed: ${validation.error}`);
}
// Test connection
const connectionTest = await kubeLoader.testConnection();
if (!connectionTest.success) {
throw new Error(`K8s connection failed: ${connectionTest.error}`);
}
// Initialize auto-scaler
const scaler = new AutoScaler(kubeLoader.getKubeConfig(), 'default');
setAutoScaler(scaler);
// Initialize complexity analyzer
const apiKey = process.env.ANTHROPIC_API_KEY || '';
if (!apiKey) {
console.warn('ANTHROPIC_API_KEY not set. Complexity analysis will fail.');
}
getAnalyzer({ apiKey });
// Initialize orchestrator client
getOrchestratorClient({ baseUrl: orchestratorUrl });
setState(prev => ({
...prev,
status: 'connected',
message: `Connected to ${validation.cluster}`,
}));
} catch (error) {
setState(prev => ({
...prev,
status: 'error',
message: error instanceof Error ? error.message : 'Initialization failed',
}));
}
};
const handleTaskSubmit = async (taskDescription: string, complexity: ComplexityAnalysis) => {
if (!autoScaler) {
alert('Auto-scaler not initialized');
return;
}
setState(prev => ({ ...prev, status: 'loading', message: 'Creating workspace...' }));
try {
// Create workspace
const workspaceId = `ws-${Date.now()}`;
const workspaceResult = await workspaceManager.initWorkspace(workspaceId);
if (!workspaceResult.success) {
throw new Error(`Failed to create workspace: ${workspaceResult.error}`);
}
const { gitUrl } = workspaceResult.data;
setState(prev => ({ ...prev, message: 'Creating agent pods...' }));
// Scale up agents
const scaleResult = await autoScaler.scaleUp(complexity, {
namespace: 'default',
image: 'heicode-swarm-agent:latest',
orchestratorUrl,
gitUrl,
workspaceId,
});
if (!scaleResult.success) {
throw new Error(`Failed to create agents: ${scaleResult.errors.join(', ')}`);
}
setState(prev => ({ ...prev, message: 'Submitting task to orchestrator...' }));
// Create task in orchestrator
const client = getOrchestratorClient();
await client.createTask(taskDescription, {
workspaceId,
complexity: complexity.complexity,
agentCount: complexity.agentCount,
});
setState(prev => ({
...prev,
status: 'connected',
message: `Task submitted with ${scaleResult.createdPods.length} agents`,
currentWorkspaceId: workspaceId,
}));
} catch (error) {
setState(prev => ({
...prev,
status: 'error',
message: error instanceof Error ? error.message : 'Task submission failed',
}));
}
};
const handleMergeResults = async () => {
if (!state.currentWorkspaceId) {
alert('No active workspace');
return;
}
setState(prev => ({ ...prev, status: 'loading', message: 'Merging agent results...' }));
try {
const repoPath = workspaceManager.getWorkspacePath(state.currentWorkspaceId);
const mergeManager = new MergeManager(repoPath);
// List agent branches
const agentBranches = await mergeManager.listAgentBranches(state.currentWorkspaceId);
if (agentBranches.length === 0) {
throw new Error('No agent branches found');
}
// Merge branches
const mergeResult = await mergeManager.mergeAgentResults('main', agentBranches);
if (mergeResult.conflicts.length > 0) {
// Show conflict resolution UI
setState(prev => ({
...prev,
status: 'connected',
message: `${mergeResult.conflicts.length} conflicts detected`,
mergeConflicts: mergeResult.conflicts,
mergedBranches: mergeResult.mergedBranches,
showConflictResolution: true,
}));
} else {
// Merge successful
setState(prev => ({
...prev,
status: 'connected',
message: 'Merge completed successfully',
mergedBranches: mergeResult.mergedBranches,
showConflictResolution: false,
}));
}
} catch (error) {
setState(prev => ({
...prev,
status: 'error',
message: error instanceof Error ? error.message : 'Merge failed',
}));
}
};
const handleConflictResolve = async (resolutions: ConflictResolution[]) => {
if (!state.currentWorkspaceId) return;
setState(prev => ({ ...prev, status: 'loading', message: 'Applying resolutions...' }));
try {
const repoPath = workspaceManager.getWorkspacePath(state.currentWorkspaceId);
const mergeManager = new MergeManager(repoPath);
const success = await mergeManager.resolveConflicts(resolutions);
if (success) {
setState(prev => ({
...prev,
status: 'connected',
message: 'Conflicts resolved successfully',
mergeConflicts: [],
showConflictResolution: false,
}));
} else {
throw new Error('Failed to apply resolutions');
}
} catch (error) {
setState(prev => ({
...prev,
status: 'error',
message: error instanceof Error ? error.message : 'Resolution failed',
}));
}
};
const handleConflictCancel = () => {
setState(prev => ({
...prev,
mergeConflicts: [],
showConflictResolution: false,
}));
};
return (
<div className="min-h-screen bg-gray-900 text-white">
<header className="bg-gray-800 border-b border-gray-700 px-6 py-4">
<h1 className="text-2xl font-bold">HeiCode Swarm</h1>
<p className="text-gray-400 text-sm">K8s-Based AI Agent Orchestration</p>
</header>
<main className="container mx-auto px-6 py-8">
<div className="bg-gray-800 rounded-lg p-6 shadow-lg mb-8">
<div className="flex items-center space-x-4">
<div className={`w-3 h-3 rounded-full ${
state.status === 'connected' ? 'bg-green-500' :
state.status === 'error' ? 'bg-red-500' :
state.status === 'loading' ? 'bg-yellow-500 animate-pulse' :
'bg-gray-500'
}`} />
<div>
<h2 className="text-lg font-semibold">
{state.status === 'connected' ? 'Connected' :
state.status === 'error' ? 'Error' :
state.status === 'loading' ? 'Loading...' :
'Idle'}
</h2>
<p className="text-gray-400 text-sm">{state.message}</p>
</div>
</div>
</div>
{state.showConflictResolution ? (
<ResultViewer
conflicts={state.mergeConflicts}
mergedBranches={state.mergedBranches}
onResolve={handleConflictResolve}
onCancel={handleConflictCancel}
/>
) : (
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6">
<div className="space-y-6">
<TaskSubmissionForm
onSubmit={handleTaskSubmit}
disabled={state.status !== 'connected'}
/>
{state.currentWorkspaceId && (
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
<h3 className="text-lg font-semibold mb-4">Workspace Actions</h3>
<button
className="w-full bg-purple-600 hover:bg-purple-700 text-white font-semibold px-6 py-2 rounded transition-colors"
onClick={handleMergeResults}
disabled={state.status !== 'connected'}
>
Merge Agent Results
</button>
</div>
)}
</div>
<div>
<AgentDashboard
orchestratorUrl={orchestratorUrl}
workspaceId={state.currentWorkspaceId || undefined}
/>
</div>
</div>
)}
</main>
<footer className="fixed bottom-0 left-0 right-0 bg-gray-800 border-t border-gray-700 px-6 py-3">
<p className="text-gray-400 text-sm text-center">
Phase 5: Desktop Client & Orchestration - Full Integration
</p>
</footer>
</div>
);
};
// Initialize React app
const container = document.getElementById('root');
if (container) {
const root = createRoot(container);
root.render(<App />);
}
export default App;
+79
View File
@@ -0,0 +1,79 @@
# AI 模块 —— 任务复杂度分析
该模块为桌面客户端提供**任务复杂度分析**能力:根据用户需求评估其规模与难度,推荐应投入的 Agent 数量,作为蜂群规模与成本控制的依据。
## 组成
### 1. `complexity-analyzer.ts`
核心分析器,调用大模型对任务进行复杂度评估并给出推荐 Agent 数量。
特性:
- 结构化 JSON 结果解析,并对非 JSON 响应做兜底解析
- Agent 数量校验(1–50)
- 性能监控(目标单次 < 3 秒)
- 支持批量分析
- 单例模式复用客户端
用法:
```typescript
import { ComplexityAnalyzer } from './ai';
const analyzer = new ComplexityAnalyzer({ temperature: 0.3 });
const analysis = await analyzer.analyzeTask('为整个项目补充单元测试');
console.log(`推荐 Agent 数:${analysis.agentCount}`);
console.log(`复杂度:${analysis.complexity}`);
console.log(`理由:${analysis.reasoning}`);
```
### 2. `prompt-builder.ts`
基于带标注的数据集构造 few-shot 提示词。
特性:
- 从 `test-data/complexity-dataset.json` 加载样例
- 按比例选取多样化样例(低 40% / 中 40% / 高 20%)
- 生成完整的 few-shot 提示,批量模式下提供简化提示
- 数据集不可用时回退到内置默认样例
### 3. `validator.ts`
对照标注数据集校验分析器的准确率。
特性与验收目标:
- 全量与快速(10 条抽样)校验
- 准确率(精确匹配、±2 个 Agent 容差)与平均绝对误差(MAE)
- 不匹配项报告与 Markdown 报告生成
- 目标:准确率 ≥ 70%(±2 容差)、MAE ≤ 2、单任务分析 < 3 秒
### 4. `validate-analyzer.ts`
独立的校验脚本,便于在集成前验证分析器质量:
```bash
npm run validate-analyzer
```
## 成本控制
分析结果与 `CostControlDialog` 组件联动,向用户提示资源开销:
- **> 10 个 Agent**:提示并给出资源预估
- **> 15 个 Agent**:要求用户显式确认
- **达到上限**:遵循用户配置的最大值(默认 20)
## 数据集格式
标注数据集 `test-data/complexity-dataset.json` 的样例格式:
```json
[
{
"task": "修复 main.py 第 42 行的语法错误",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "单文件单行修复,无依赖与副作用。"
}
]
```
## 实现要点
1. **Few-shot 学习**:使用数据集中的多样化样例提升判断稳定性。
2. **温度 0.3**:降低随机性以获得更一致的预测。
3. **稳健性**:解析失败时优雅兜底。
4. **先验证后上线**:集成进自动伸缩与任务提交流程前,应先通过校验。
@@ -0,0 +1,120 @@
/**
* Unit tests for complexity analyzer
* Run with: npm test
*/
import { ComplexityAnalyzer } from './complexity-analyzer';
import { loadExamples, selectFewShotExamples, buildComplexityPrompt } from './prompt-builder';
describe('ComplexityAnalyzer', () => {
let analyzer: ComplexityAnalyzer;
beforeEach(() => {
analyzer = new ComplexityAnalyzer({
apiKey: process.env.ANTHROPIC_API_KEY || 'test-key',
model: 'claude-opus-4-7',
});
});
describe('validateAgentCount', () => {
it('should clamp negative values to 1', () => {
const result = (analyzer as any).validateAgentCount(-5);
expect(result).toBe(1);
});
it('should clamp values above 50', () => {
const result = (analyzer as any).validateAgentCount(100);
expect(result).toBe(50);
});
it('should round decimal values', () => {
const result = (analyzer as any).validateAgentCount(3.7);
expect(result).toBe(4);
});
it('should accept valid values', () => {
const result = (analyzer as any).validateAgentCount(10);
expect(result).toBe(10);
});
});
describe('validateComplexity', () => {
it('should accept valid complexity levels', () => {
expect((analyzer as any).validateComplexity('low')).toBe('low');
expect((analyzer as any).validateComplexity('medium')).toBe('medium');
expect((analyzer as any).validateComplexity('high')).toBe('high');
});
it('should default to medium for invalid values', () => {
expect((analyzer as any).validateComplexity('invalid')).toBe('medium');
});
});
describe('inferComplexity', () => {
it('should infer low complexity for 1-2 agents', () => {
expect((analyzer as any).inferComplexity(1)).toBe('low');
expect((analyzer as any).inferComplexity(2)).toBe('low');
});
it('should infer medium complexity for 3-7 agents', () => {
expect((analyzer as any).inferComplexity(3)).toBe('medium');
expect((analyzer as any).inferComplexity(7)).toBe('medium');
});
it('should infer high complexity for 8+ agents', () => {
expect((analyzer as any).inferComplexity(8)).toBe('high');
expect((analyzer as any).inferComplexity(20)).toBe('high');
});
});
});
describe('PromptBuilder', () => {
describe('loadExamples', () => {
it('should load examples from dataset', () => {
const examples = loadExamples();
expect(examples.length).toBeGreaterThan(0);
expect(examples[0]).toHaveProperty('task');
expect(examples[0]).toHaveProperty('expectedAgents');
expect(examples[0]).toHaveProperty('complexity');
expect(examples[0]).toHaveProperty('reasoning');
});
});
describe('selectFewShotExamples', () => {
it('should select requested number of examples', () => {
const examples = loadExamples();
const selected = selectFewShotExamples(examples, 12);
expect(selected.length).toBeLessThanOrEqual(12);
});
it('should include diverse complexity levels', () => {
const examples = loadExamples();
const selected = selectFewShotExamples(examples, 12);
const complexities = selected.map((e) => e.complexity);
expect(complexities).toContain('low');
expect(complexities).toContain('medium');
expect(complexities).toContain('high');
});
});
describe('buildComplexityPrompt', () => {
it('should build prompt with task description', () => {
const prompt = buildComplexityPrompt('Fix syntax error');
expect(prompt).toContain('Fix syntax error');
});
it('should include few-shot examples', () => {
const prompt = buildComplexityPrompt('Test task');
expect(prompt).toContain('Task:');
expect(prompt).toContain('Analysis:');
expect(prompt).toContain('agentCount');
});
it('should include guidelines', () => {
const prompt = buildComplexityPrompt('Test task');
expect(prompt).toContain('Low complexity');
expect(prompt).toContain('Medium complexity');
expect(prompt).toContain('High complexity');
});
});
});
@@ -0,0 +1,191 @@
import Anthropic from '@anthropic-ai/sdk';
import { buildComplexityPrompt } from './prompt-builder';
export interface ComplexityAnalysis {
agentCount: number;
reasoning: string;
complexity: 'low' | 'medium' | 'high';
confidence: number;
}
export interface AnalyzerConfig {
apiKey: string;
model?: string;
maxTokens?: number;
temperature?: number;
}
export class ComplexityAnalyzer {
private client: Anthropic;
private model: string;
private maxTokens: number;
private temperature: number;
constructor(config: AnalyzerConfig) {
this.client = new Anthropic({
apiKey: config.apiKey,
});
this.model = config.model || 'claude-opus-4-7';
this.maxTokens = config.maxTokens || 1024;
this.temperature = config.temperature || 0.3;
}
/**
* Analyze task complexity and determine required agent count
* @param taskDescription - The task to analyze
* @returns ComplexityAnalysis with agent count recommendation
*/
async analyzeTask(taskDescription: string): Promise<ComplexityAnalysis> {
const startTime = Date.now();
try {
const prompt = buildComplexityPrompt(taskDescription);
const response = await this.client.messages.create({
model: this.model,
max_tokens: this.maxTokens,
temperature: this.temperature,
messages: [
{
role: 'user',
content: prompt,
},
],
});
const elapsedTime = Date.now() - startTime;
console.log(`Complexity analysis completed in ${elapsedTime}ms`);
if (elapsedTime > 3000) {
console.warn(`Analysis took ${elapsedTime}ms, exceeding 3s target`);
}
const content = response.content[0];
if (content.type !== 'text') {
throw new Error('Unexpected response type from Claude API');
}
const analysis = this.parseResponse(content.text);
return analysis;
} catch (error) {
console.error('Error analyzing task complexity:', error);
throw new Error(`Failed to analyze task complexity: ${error instanceof Error ? error.message : 'Unknown error'}`);
}
}
/**
* Parse Claude's response into structured ComplexityAnalysis
*/
private parseResponse(responseText: string): ComplexityAnalysis {
try {
// Try to extract JSON from response
const jsonMatch = responseText.match(/\{[\s\S]*\}/);
if (jsonMatch) {
const parsed = JSON.parse(jsonMatch[0]);
return {
agentCount: this.validateAgentCount(parsed.agentCount),
reasoning: parsed.reasoning || 'No reasoning provided',
complexity: this.validateComplexity(parsed.complexity),
confidence: parsed.confidence || 0.8,
};
}
// Fallback: try to extract agent count from text
const agentMatch = responseText.match(/(\d+)\s*agent/i);
if (agentMatch) {
const agentCount = parseInt(agentMatch[1], 10);
return {
agentCount: this.validateAgentCount(agentCount),
reasoning: responseText,
complexity: this.inferComplexity(agentCount),
confidence: 0.6,
};
}
throw new Error('Could not parse agent count from response');
} catch (error) {
console.error('Error parsing response:', responseText);
throw new Error(`Failed to parse complexity analysis: ${error instanceof Error ? error.message : 'Unknown error'}`);
}
}
/**
* Validate and clamp agent count to reasonable range
*/
private validateAgentCount(count: number): number {
if (isNaN(count) || count < 1) {
console.warn(`Invalid agent count ${count}, defaulting to 1`);
return 1;
}
if (count > 50) {
console.warn(`Agent count ${count} exceeds maximum, capping at 50`);
return 50;
}
return Math.round(count);
}
/**
* Validate complexity level
*/
private validateComplexity(complexity: string): 'low' | 'medium' | 'high' {
const normalized = complexity.toLowerCase();
if (normalized === 'low' || normalized === 'medium' || normalized === 'high') {
return normalized as 'low' | 'medium' | 'high';
}
console.warn(`Invalid complexity level ${complexity}, defaulting to medium`);
return 'medium';
}
/**
* Infer complexity from agent count
*/
private inferComplexity(agentCount: number): 'low' | 'medium' | 'high' {
if (agentCount <= 2) return 'low';
if (agentCount <= 7) return 'medium';
return 'high';
}
/**
* Batch analyze multiple tasks
*/
async analyzeBatch(tasks: string[]): Promise<ComplexityAnalysis[]> {
const results: ComplexityAnalysis[] = [];
for (const task of tasks) {
try {
const analysis = await this.analyzeTask(task);
results.push(analysis);
} catch (error) {
console.error(`Failed to analyze task: ${task}`, error);
// Push a default analysis for failed tasks
results.push({
agentCount: 1,
reasoning: `Analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
complexity: 'low',
confidence: 0,
});
}
}
return results;
}
}
/**
* Create a singleton analyzer instance
*/
let analyzerInstance: ComplexityAnalyzer | null = null;
export function getAnalyzer(config?: AnalyzerConfig): ComplexityAnalyzer {
if (!analyzerInstance) {
if (!config) {
throw new Error('Analyzer not initialized. Provide config on first call.');
}
analyzerInstance = new ComplexityAnalyzer(config);
}
return analyzerInstance;
}
export function resetAnalyzer(): void {
analyzerInstance = null;
}
+77
View File
@@ -0,0 +1,77 @@
/**
* Example usage of the complexity analyzer
* This demonstrates how to use the analyzer in the desktop client
*/
import { ComplexityAnalyzer, validateAnalyzer } from './index';
async function exampleUsage() {
// Initialize analyzer
const analyzer = new ComplexityAnalyzer({
apiKey: process.env.ANTHROPIC_API_KEY || 'your-api-key',
model: 'claude-opus-4-7',
temperature: 0.3,
});
// Example 1: Analyze a simple task
console.log('=== Example 1: Simple Task ===');
const simpleTask = 'Fix syntax error in main.py line 42';
const simpleAnalysis = await analyzer.analyzeTask(simpleTask);
console.log(`Task: ${simpleTask}`);
console.log(`Agents needed: ${simpleAnalysis.agentCount}`);
console.log(`Complexity: ${simpleAnalysis.complexity}`);
console.log(`Reasoning: ${simpleAnalysis.reasoning}\n`);
// Example 2: Analyze a medium complexity task
console.log('=== Example 2: Medium Complexity Task ===');
const mediumTask = 'Add input validation to all API endpoints';
const mediumAnalysis = await analyzer.analyzeTask(mediumTask);
console.log(`Task: ${mediumTask}`);
console.log(`Agents needed: ${mediumAnalysis.agentCount}`);
console.log(`Complexity: ${mediumAnalysis.complexity}`);
console.log(`Reasoning: ${mediumAnalysis.reasoning}\n`);
// Example 3: Analyze a complex task
console.log('=== Example 3: Complex Task ===');
const complexTask = 'Add comprehensive unit tests to the entire project';
const complexAnalysis = await analyzer.analyzeTask(complexTask);
console.log(`Task: ${complexTask}`);
console.log(`Agents needed: ${complexAnalysis.agentCount}`);
console.log(`Complexity: ${complexAnalysis.complexity}`);
console.log(`Reasoning: ${complexAnalysis.reasoning}\n`);
// Example 4: Batch analysis
console.log('=== Example 4: Batch Analysis ===');
const tasks = [
'Update README.md with installation instructions',
'Implement caching layer using Redis',
'Migrate from REST API to GraphQL',
];
const batchResults = await analyzer.analyzeBatch(tasks);
batchResults.forEach((result, i: number) => {
console.log(`${i + 1}. ${tasks[i]}: ${result.agentCount} agents (${result.complexity})`);
});
console.log();
// Example 5: Run validation (optional)
if (process.env.RUN_VALIDATION === 'true') {
console.log('=== Example 5: Validation ===');
const validationResult = await validateAnalyzer(analyzer, {
accuracyThreshold: 0.7,
maxMeanAbsoluteError: 2,
});
console.log(`Validation ${validationResult.passed ? 'PASSED' : 'FAILED'}`);
console.log(`Accuracy: ${(validationResult.accuracy * 100).toFixed(1)}%`);
console.log(`MAE: ${validationResult.meanAbsoluteError.toFixed(2)}`);
}
}
// Run if executed directly
if (require.main === module) {
exampleUsage().catch((error) => {
console.error('Error:', error);
process.exit(1);
});
}
export { exampleUsage };
+10
View File
@@ -0,0 +1,10 @@
/**
* AI Module - Task Complexity Analysis
*
* This module provides AI-powered task complexity analysis using Claude Opus 4.7.
* It determines the optimal number of agents needed to complete a given task.
*/
export { ComplexityAnalyzer, ComplexityAnalysis, AnalyzerConfig, getAnalyzer, resetAnalyzer } from './complexity-analyzer';
export { loadExamples, selectFewShotExamples, buildComplexityPrompt, buildSimplifiedPrompt, ComplexityExample } from './prompt-builder';
export { validateAnalyzer, quickValidation, generateValidationReport, ValidationResult, ValidationMismatch, ValidationConfig } from './validator';
+172
View File
@@ -0,0 +1,172 @@
import * as fs from 'fs';
import * as path from 'path';
export interface ComplexityExample {
task: string;
expectedAgents: number;
complexity: 'low' | 'medium' | 'high';
reasoning: string;
}
/**
* Load complexity examples from the test dataset
*/
export function loadExamples(datasetPath?: string): ComplexityExample[] {
const defaultPath = path.join(__dirname, '../../../test-data/complexity-dataset.json');
const filePath = datasetPath || defaultPath;
try {
const fileContent = fs.readFileSync(filePath, 'utf-8');
const examples = JSON.parse(fileContent) as ComplexityExample[];
return examples;
} catch (error) {
console.error(`Failed to load examples from ${filePath}:`, error);
return getDefaultExamples();
}
}
/**
* Get default examples if dataset file is not available
*/
function getDefaultExamples(): ComplexityExample[] {
return [
{
task: 'Fix syntax error in main.py line 42',
expectedAgents: 1,
complexity: 'low',
reasoning: 'Single file, single line fix. No dependencies or side effects.',
},
{
task: 'Add type hints to the calculate_total function',
expectedAgents: 1,
complexity: 'low',
reasoning: 'Single function modification in one file.',
},
{
task: 'Refactor the user authentication module to use JWT tokens',
expectedAgents: 3,
complexity: 'medium',
reasoning: 'Affects multiple functions in auth module, requires updating login/logout flows and adding token validation.',
},
{
task: 'Add input validation to all API endpoints',
expectedAgents: 5,
complexity: 'medium',
reasoning: 'Multiple endpoints across different route files need validation logic.',
},
{
task: 'Add comprehensive unit tests to the entire project',
expectedAgents: 12,
complexity: 'high',
reasoning: 'Testing all modules, functions, edge cases. Requires understanding entire codebase and writing hundreds of test cases.',
},
{
task: 'Refactor entire codebase to follow clean architecture principles',
expectedAgents: 15,
complexity: 'high',
reasoning: 'Major restructuring: separate layers (domain, application, infrastructure), update all imports, maintain functionality.',
},
];
}
/**
* Select diverse examples for few-shot prompting
* Ensures representation across low, medium, and high complexity
*/
export function selectFewShotExamples(
examples: ComplexityExample[],
count: number = 12
): ComplexityExample[] {
const lowExamples = examples.filter((e) => e.complexity === 'low');
const mediumExamples = examples.filter((e) => e.complexity === 'medium');
const highExamples = examples.filter((e) => e.complexity === 'high');
// Aim for balanced distribution: 40% low, 40% medium, 20% high
const lowCount = Math.ceil(count * 0.4);
const mediumCount = Math.ceil(count * 0.4);
const highCount = count - lowCount - mediumCount;
const selected: ComplexityExample[] = [];
// Randomly select from each category
selected.push(...selectRandom(lowExamples, lowCount));
selected.push(...selectRandom(mediumExamples, mediumCount));
selected.push(...selectRandom(highExamples, highCount));
return selected;
}
/**
* Randomly select N items from array
*/
function selectRandom<T>(array: T[], count: number): T[] {
const shuffled = [...array].sort(() => Math.random() - 0.5);
return shuffled.slice(0, Math.min(count, array.length));
}
/**
* Build the few-shot prompt for complexity analysis
*/
export function buildComplexityPrompt(taskDescription: string): string {
const examples = loadExamples();
const fewShotExamples = selectFewShotExamples(examples, 12);
const examplesText = fewShotExamples
.map(
(ex) =>
`Task: "${ex.task}"
Analysis: {
"agentCount": ${ex.expectedAgents},
"complexity": "${ex.complexity}",
"reasoning": "${ex.reasoning}",
"confidence": 0.9
}`
)
.join('\n\n');
return `You are an expert at analyzing software development task complexity and determining how many AI agents are needed to complete the task efficiently.
Your job is to analyze a task description and return a JSON object with:
- agentCount: number of agents needed (1-50)
- complexity: "low", "medium", or "high"
- reasoning: brief explanation of your decision
- confidence: your confidence level (0.0-1.0)
Guidelines for agent count estimation:
- **Low complexity (1-2 agents)**: Single file changes, simple fixes, documentation updates, small additions
- **Medium complexity (3-7 agents)**: Multi-file changes, refactoring modules, adding features across several files, implementing new subsystems
- **High complexity (8-20 agents)**: Major architectural changes, full test suites, large-scale refactoring, migrating technologies, building new systems
Consider these factors:
1. **Scope**: How many files/modules are affected?
2. **Interdependencies**: Do changes require coordination across multiple components?
3. **Complexity**: Does it require deep understanding of the codebase?
4. **Testing**: How much testing is needed?
5. **Risk**: What's the blast radius of potential errors?
Here are examples of task complexity analysis:
${examplesText}
Now analyze this task:
Task: "${taskDescription}"
Return ONLY a JSON object with the analysis. No additional text.`;
}
/**
* Build a simplified prompt for faster analysis (used in batch mode)
*/
export function buildSimplifiedPrompt(taskDescription: string): string {
return `Analyze this software development task and estimate how many AI agents (1-50) are needed to complete it.
Task: "${taskDescription}"
Consider:
- Single file/function changes = 1 agent
- Multi-file changes or module refactoring = 3-7 agents
- Major architectural changes or full test suites = 8-20 agents
Return JSON: {"agentCount": N, "complexity": "low|medium|high", "reasoning": "brief explanation"}`;
}
@@ -0,0 +1,57 @@
#!/usr/bin/env node
/**
* Validation script for complexity analyzer
* Run with: npm run validate-analyzer
*/
import { ComplexityAnalyzer } from './complexity-analyzer';
import { validateAnalyzer, generateValidationReport } from './validator';
import * as fs from 'fs';
import * as path from 'path';
async function main() {
console.log('=== Complexity Analyzer Validation ===\n');
// Check for API key
const apiKey = process.env.ANTHROPIC_API_KEY;
if (!apiKey) {
console.error('Error: ANTHROPIC_API_KEY environment variable not set');
console.error('Please set it with: export ANTHROPIC_API_KEY=your_key_here');
process.exit(1);
}
// Initialize analyzer
console.log('Initializing analyzer with Claude Opus 4.7...');
const analyzer = new ComplexityAnalyzer({
apiKey,
model: 'claude-opus-4-7',
temperature: 0.3,
});
// Run validation
const result = await validateAnalyzer(analyzer, {
accuracyThreshold: 0.7,
maxMeanAbsoluteError: 2,
});
// Generate report
const report = generateValidationReport(result);
const reportPath = path.join(__dirname, '../../../validation-report.md');
fs.writeFileSync(reportPath, report);
console.log(`\nValidation report saved to: ${reportPath}`);
// Exit with appropriate code
if (result.passed) {
console.log('\n✓ Validation PASSED - Analyzer meets acceptance criteria');
process.exit(0);
} else {
console.log('\n✗ Validation FAILED - Analyzer does not meet acceptance criteria');
console.log('Please review mismatches and tune the prompt.');
process.exit(1);
}
}
main().catch((error) => {
console.error('Validation failed with error:', error);
process.exit(1);
});
+247
View File
@@ -0,0 +1,247 @@
import { ComplexityAnalyzer, ComplexityAnalysis } from './complexity-analyzer';
import { loadExamples, ComplexityExample } from './prompt-builder';
export interface ValidationResult {
accuracy: number;
meanAbsoluteError: number;
totalTests: number;
correctPredictions: number;
withinTwoAgents: number;
mismatches: ValidationMismatch[];
passed: boolean;
}
export interface ValidationMismatch {
task: string;
expected: number;
predicted: number;
error: number;
reasoning: string;
}
export interface ValidationConfig {
accuracyThreshold?: number; // Default: 0.7 (70%)
maxMeanAbsoluteError?: number; // Default: 2
datasetPath?: string;
}
/**
* Validate the complexity analyzer against the labeled dataset
*/
export async function validateAnalyzer(
analyzer: ComplexityAnalyzer,
config: ValidationConfig = {}
): Promise<ValidationResult> {
const accuracyThreshold = config.accuracyThreshold || 0.7;
const maxMAE = config.maxMeanAbsoluteError || 2;
console.log('Loading test dataset...');
const examples = loadExamples(config.datasetPath);
console.log(`Loaded ${examples.length} examples`);
console.log('Running validation...');
const startTime = Date.now();
const predictions: ComplexityAnalysis[] = [];
const mismatches: ValidationMismatch[] = [];
let totalError = 0;
let correctPredictions = 0;
let withinTwoAgents = 0;
// Analyze each example
for (let i = 0; i < examples.length; i++) {
const example = examples[i];
console.log(`[${i + 1}/${examples.length}] Analyzing: ${example.task.substring(0, 50)}...`);
try {
const prediction = await analyzer.analyzeTask(example.task);
predictions.push(prediction);
const error = Math.abs(prediction.agentCount - example.expectedAgents);
totalError += error;
// Exact match
if (prediction.agentCount === example.expectedAgents) {
correctPredictions++;
withinTwoAgents++;
}
// Within ±2 agents
else if (error <= 2) {
withinTwoAgents++;
}
// Record mismatches
if (error > 2) {
mismatches.push({
task: example.task,
expected: example.expectedAgents,
predicted: prediction.agentCount,
error,
reasoning: prediction.reasoning,
});
}
} catch (error) {
console.error(`Failed to analyze task: ${example.task}`, error);
// Count as maximum error
totalError += 50;
mismatches.push({
task: example.task,
expected: example.expectedAgents,
predicted: 0,
error: 50,
reasoning: `Analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
});
}
}
const elapsedTime = Date.now() - startTime;
console.log(`Validation completed in ${elapsedTime}ms`);
// Calculate metrics
const totalTests = examples.length;
const accuracy = withinTwoAgents / totalTests;
const meanAbsoluteError = totalError / totalTests;
const passed = accuracy >= accuracyThreshold && meanAbsoluteError <= maxMAE;
const result: ValidationResult = {
accuracy,
meanAbsoluteError,
totalTests,
correctPredictions,
withinTwoAgents,
mismatches,
passed,
};
// Print summary
console.log('\n=== Validation Results ===');
console.log(`Total tests: ${totalTests}`);
console.log(`Exact matches: ${correctPredictions} (${((correctPredictions / totalTests) * 100).toFixed(1)}%)`);
console.log(`Within ±2 agents: ${withinTwoAgents} (${(accuracy * 100).toFixed(1)}%)`);
console.log(`Mean Absolute Error: ${meanAbsoluteError.toFixed(2)}`);
console.log(`Target accuracy: ${(accuracyThreshold * 100).toFixed(0)}%`);
console.log(`Target MAE: ≤${maxMAE}`);
console.log(`Status: ${passed ? '✓ PASSED' : '✗ FAILED'}`);
if (mismatches.length > 0) {
console.log(`\n=== Mismatches (error >2 agents) ===`);
mismatches.slice(0, 10).forEach((m, i) => {
console.log(`\n${i + 1}. Task: ${m.task}`);
console.log(` Expected: ${m.expected}, Predicted: ${m.predicted}, Error: ${m.error}`);
console.log(` Reasoning: ${m.reasoning.substring(0, 100)}...`);
});
if (mismatches.length > 10) {
console.log(`\n... and ${mismatches.length - 10} more mismatches`);
}
}
return result;
}
/**
* Run quick validation on a subset of examples
*/
export async function quickValidation(
analyzer: ComplexityAnalyzer,
sampleSize: number = 10
): Promise<ValidationResult> {
const examples = loadExamples();
// Select diverse sample
const lowExamples = examples.filter((e) => e.complexity === 'low').slice(0, 3);
const mediumExamples = examples.filter((e) => e.complexity === 'medium').slice(0, 4);
const highExamples = examples.filter((e) => e.complexity === 'high').slice(0, 3);
const sample = [...lowExamples, ...mediumExamples, ...highExamples];
console.log(`Running quick validation on ${sample.length} examples...`);
const predictions: ComplexityAnalysis[] = [];
const mismatches: ValidationMismatch[] = [];
let totalError = 0;
let correctPredictions = 0;
let withinTwoAgents = 0;
for (const example of sample) {
try {
const prediction = await analyzer.analyzeTask(example.task);
predictions.push(prediction);
const error = Math.abs(prediction.agentCount - example.expectedAgents);
totalError += error;
if (prediction.agentCount === example.expectedAgents) {
correctPredictions++;
withinTwoAgents++;
} else if (error <= 2) {
withinTwoAgents++;
}
if (error > 2) {
mismatches.push({
task: example.task,
expected: example.expectedAgents,
predicted: prediction.agentCount,
error,
reasoning: prediction.reasoning,
});
}
} catch (error) {
console.error(`Failed to analyze task: ${example.task}`, error);
totalError += 50;
}
}
const totalTests = sample.length;
const accuracy = withinTwoAgents / totalTests;
const meanAbsoluteError = totalError / totalTests;
return {
accuracy,
meanAbsoluteError,
totalTests,
correctPredictions,
withinTwoAgents,
mismatches,
passed: accuracy >= 0.7 && meanAbsoluteError <= 2,
};
}
/**
* Generate a validation report as markdown
*/
export function generateValidationReport(result: ValidationResult): string {
const report = `# Complexity Analyzer Validation Report
## Summary
- **Status**: ${result.passed ? '✓ PASSED' : '✗ FAILED'}
- **Total Tests**: ${result.totalTests}
- **Accuracy (±2 agents)**: ${(result.accuracy * 100).toFixed(1)}%
- **Exact Matches**: ${result.correctPredictions} (${((result.correctPredictions / result.totalTests) * 100).toFixed(1)}%)
- **Mean Absolute Error**: ${result.meanAbsoluteError.toFixed(2)}
## Acceptance Criteria
- ✓ Accuracy ≥70%: ${result.accuracy >= 0.7 ? 'PASS' : 'FAIL'}
- ✓ MAE ≤2: ${result.meanAbsoluteError <= 2 ? 'PASS' : 'FAIL'}
## Mismatches (Error >2 agents)
${result.mismatches.length === 0 ? 'None' : ''}
${result.mismatches
.map(
(m, i) => `
### ${i + 1}. ${m.task}
- **Expected**: ${m.expected} agents
- **Predicted**: ${m.predicted} agents
- **Error**: ${m.error} agents
- **Reasoning**: ${m.reasoning}
`
)
.join('\n')}
---
Generated: ${new Date().toISOString()}
`;
return report;
}
@@ -0,0 +1,342 @@
/**
* Usage Tracker for Agent Analytics and Cost Control
*
* Tracks agent usage, compute time, and estimated costs.
* Provides budget alerts and usage analytics.
*/
export interface AgentUsageEvent {
agentId: string;
taskId: string;
eventType: 'created' | 'started' | 'completed' | 'failed' | 'terminated';
timestamp: number;
metadata?: Record<string, any>;
}
export interface AgentSession {
agentId: string;
taskId: string;
startTime: number;
endTime?: number;
durationSeconds?: number;
podName: string;
nodeType: string;
cpuCores: number;
memoryMB: number;
status: 'running' | 'completed' | 'failed' | 'terminated';
costUSD?: number;
}
export interface UsageMetrics {
totalAgents: number;
activeAgents: number;
completedTasks: number;
failedTasks: number;
totalComputeHours: number;
totalCostUSD: number;
averageTaskDuration: number;
peakConcurrentAgents: number;
}
export interface CostConfig {
cpuCostPerCoreHour: number; // USD per CPU core per hour
memoryCostPerGBHour: number; // USD per GB memory per hour
budgetLimitUSD?: number;
alertThresholds: number[]; // Alert at these percentages of budget
}
export interface BudgetAlert {
timestamp: number;
currentSpend: number;
budgetLimit: number;
percentageUsed: number;
message: string;
severity: 'info' | 'warning' | 'critical';
}
export class UsageTracker {
private sessions: Map<string, AgentSession> = new Map();
private events: AgentUsageEvent[] = [];
private alerts: BudgetAlert[] = [];
private costConfig: CostConfig;
private alertedThresholds: Set<number> = new Set();
constructor(costConfig: CostConfig) {
this.costConfig = costConfig;
}
/**
* Track an agent lifecycle event
*/
trackEvent(event: AgentUsageEvent): void {
this.events.push(event);
switch (event.eventType) {
case 'created':
this.handleAgentCreated(event);
break;
case 'started':
this.handleAgentStarted(event);
break;
case 'completed':
case 'failed':
case 'terminated':
this.handleAgentEnded(event);
break;
}
// Check budget after each event
this.checkBudgetAlerts();
}
private handleAgentCreated(event: AgentUsageEvent): void {
const session: AgentSession = {
agentId: event.agentId,
taskId: event.taskId,
startTime: event.timestamp,
podName: event.metadata?.podName || '',
nodeType: event.metadata?.nodeType || 'standard',
cpuCores: event.metadata?.cpuCores || 2,
memoryMB: event.metadata?.memoryMB || 4096,
status: 'running'
};
this.sessions.set(event.agentId, session);
}
private handleAgentStarted(event: AgentUsageEvent): void {
const session = this.sessions.get(event.agentId);
if (session) {
session.startTime = event.timestamp;
session.status = 'running';
}
}
private handleAgentEnded(event: AgentUsageEvent): void {
const session = this.sessions.get(event.agentId);
if (!session) return;
session.endTime = event.timestamp;
session.durationSeconds = (event.timestamp - session.startTime) / 1000;
session.status = event.eventType === 'completed' ? 'completed' :
event.eventType === 'failed' ? 'failed' : 'terminated';
// Calculate cost
session.costUSD = this.calculateSessionCost(session);
}
/**
* Calculate cost for an agent session
*/
private calculateSessionCost(session: AgentSession): number {
if (!session.durationSeconds) return 0;
const hours = session.durationSeconds / 3600;
const cpuCost = session.cpuCores * this.costConfig.cpuCostPerCoreHour * hours;
const memoryGB = session.memoryMB / 1024;
const memoryCost = memoryGB * this.costConfig.memoryCostPerGBHour * hours;
return cpuCost + memoryCost;
}
/**
* Get current usage metrics
*/
getMetrics(): UsageMetrics {
const allSessions = Array.from(this.sessions.values());
const activeSessions = allSessions.filter(s => s.status === 'running');
const completedSessions = allSessions.filter(s => s.status === 'completed');
const failedSessions = allSessions.filter(s => s.status === 'failed');
const totalComputeSeconds = allSessions.reduce(
(sum, s) => sum + (s.durationSeconds || 0),
0
);
const totalCost = allSessions.reduce(
(sum, s) => sum + (s.costUSD || 0),
0
);
const completedDurations = completedSessions
.map(s => s.durationSeconds || 0)
.filter(d => d > 0);
const averageDuration = completedDurations.length > 0
? completedDurations.reduce((a, b) => a + b, 0) / completedDurations.length
: 0;
// Calculate peak concurrent agents from events
const peakConcurrent = this.calculatePeakConcurrency();
return {
totalAgents: allSessions.length,
activeAgents: activeSessions.length,
completedTasks: completedSessions.length,
failedTasks: failedSessions.length,
totalComputeHours: totalComputeSeconds / 3600,
totalCostUSD: totalCost,
averageTaskDuration: averageDuration,
peakConcurrentAgents: peakConcurrent
};
}
/**
* Calculate peak concurrent agents from event history
*/
private calculatePeakConcurrency(): number {
const timePoints: Array<{ time: number; delta: number }> = [];
for (const session of this.sessions.values()) {
timePoints.push({ time: session.startTime, delta: 1 });
if (session.endTime) {
timePoints.push({ time: session.endTime, delta: -1 });
}
}
timePoints.sort((a, b) => a.time - b.time);
let current = 0;
let peak = 0;
for (const point of timePoints) {
current += point.delta;
peak = Math.max(peak, current);
}
return peak;
}
/**
* Check if budget alerts should be triggered
*/
private checkBudgetAlerts(): void {
if (!this.costConfig.budgetLimitUSD) return;
const metrics = this.getMetrics();
const percentageUsed = (metrics.totalCostUSD / this.costConfig.budgetLimitUSD) * 100;
for (const threshold of this.costConfig.alertThresholds) {
if (percentageUsed >= threshold && !this.alertedThresholds.has(threshold)) {
this.alertedThresholds.add(threshold);
const severity = threshold >= 90 ? 'critical' :
threshold >= 75 ? 'warning' : 'info';
const alert: BudgetAlert = {
timestamp: Date.now(),
currentSpend: metrics.totalCostUSD,
budgetLimit: this.costConfig.budgetLimitUSD,
percentageUsed,
message: `Budget alert: ${percentageUsed.toFixed(1)}% of budget used ($${metrics.totalCostUSD.toFixed(2)} / $${this.costConfig.budgetLimitUSD})`,
severity
};
this.alerts.push(alert);
}
}
}
/**
* Get all budget alerts
*/
getAlerts(): BudgetAlert[] {
return [...this.alerts];
}
/**
* Get recent alerts (last N)
*/
getRecentAlerts(count: number = 10): BudgetAlert[] {
return this.alerts.slice(-count);
}
/**
* Clear all alerts
*/
clearAlerts(): void {
this.alerts = [];
this.alertedThresholds.clear();
}
/**
* Get usage breakdown by task
*/
getUsageByTask(): Map<string, { agents: number; cost: number; duration: number }> {
const taskMap = new Map<string, { agents: number; cost: number; duration: number }>();
for (const session of this.sessions.values()) {
const existing = taskMap.get(session.taskId) || { agents: 0, cost: 0, duration: 0 };
taskMap.set(session.taskId, {
agents: existing.agents + 1,
cost: existing.cost + (session.costUSD || 0),
duration: existing.duration + (session.durationSeconds || 0)
});
}
return taskMap;
}
/**
* Get usage breakdown by time period
*/
getUsageByPeriod(periodMs: number): Array<{ period: number; agents: number; cost: number }> {
const periods = new Map<number, { agents: Set<string>; cost: number }>();
for (const session of this.sessions.values()) {
const periodStart = Math.floor(session.startTime / periodMs) * periodMs;
const existing = periods.get(periodStart) || { agents: new Set(), cost: 0 };
existing.agents.add(session.agentId);
existing.cost += session.costUSD || 0;
periods.set(periodStart, existing);
}
return Array.from(periods.entries())
.map(([period, data]) => ({
period,
agents: data.agents.size,
cost: data.cost
}))
.sort((a, b) => a.period - b.period);
}
/**
* Export usage data for analysis
*/
exportData(): {
sessions: AgentSession[];
events: AgentUsageEvent[];
metrics: UsageMetrics;
alerts: BudgetAlert[];
} {
return {
sessions: Array.from(this.sessions.values()),
events: [...this.events],
metrics: this.getMetrics(),
alerts: [...this.alerts]
};
}
/**
* Reset all tracking data
*/
reset(): void {
this.sessions.clear();
this.events = [];
this.alerts = [];
this.alertedThresholds.clear();
}
}
/**
* Default cost configuration (example AWS pricing)
*/
export const DEFAULT_COST_CONFIG: CostConfig = {
cpuCostPerCoreHour: 0.04, // ~$0.04 per vCPU hour
memoryCostPerGBHour: 0.005, // ~$0.005 per GB hour
budgetLimitUSD: 100, // $100 default budget
alertThresholds: [50, 75, 90, 95] // Alert at 50%, 75%, 90%, 95%
};
@@ -0,0 +1,215 @@
/**
* REST API client for the orchestrator service
*/
export interface AgentMetadata {
agent_id: string;
status: 'idle' | 'busy' | 'handoff' | 'handoff-pending' | 'failed';
capabilities: string[];
current_task_id?: string;
last_heartbeat: number;
created_at: number;
}
export interface TaskMetadata {
task_id: string;
title?: string;
description: string;
status: 'pending' | 'assigned' | 'in_progress' | 'blocked' | 'completed' | 'failed' | 'cancelled';
agent_role?: string;
required_capabilities?: string[];
depends_on?: string[];
parent_task_id?: string;
root_task_id?: string;
source?: string;
assigned_agent_id?: string;
context: Record<string, any>;
created_at: number;
started_at?: number;
completed_at?: number;
attempt?: number;
blocked_reason?: string;
max_retries: number;
retry_count: number;
error?: string;
}
export interface HandoffMetadata {
handoff_id: string;
source_agent_id: string;
target_agent_id: string;
task_context: Record<string, any>;
status: 'pending' | 'completed' | 'failed';
created_at: number;
completed_at?: number;
reason?: string;
}
export interface HealthStatus {
status: string;
redis: string;
active_connections: number;
}
export interface OrchestratorClientConfig {
baseUrl: string;
timeout?: number;
}
export class OrchestratorClient {
private baseUrl: string;
private timeout: number;
constructor(config: OrchestratorClientConfig) {
this.baseUrl = config.baseUrl.replace(/\/$/, ''); // Remove trailing slash
this.timeout = config.timeout || 10000;
}
/**
* Make HTTP request with timeout
*/
private async request<T>(
endpoint: string,
options: RequestInit = {}
): Promise<T> {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), this.timeout);
try {
const response = await fetch(`${this.baseUrl}${endpoint}`, {
...options,
signal: controller.signal,
headers: {
'Content-Type': 'application/json',
...options.headers,
},
});
if (!response.ok) {
const error = await response.text();
throw new Error(`HTTP ${response.status}: ${error}`);
}
return await response.json();
} finally {
clearTimeout(timeoutId);
}
}
/**
* Health check
*/
async health(): Promise<HealthStatus> {
return this.request<HealthStatus>('/health');
}
/**
* List all agents
*/
async listAgents(): Promise<AgentMetadata[]> {
const response = await this.request<{ agents: AgentMetadata[] }>('/agents');
return response.agents;
}
/**
* Get specific agent
*/
async getAgent(agentId: string): Promise<AgentMetadata> {
return this.request<AgentMetadata>(`/agents/${agentId}`);
}
/**
* List idle agents
*/
async listIdleAgents(): Promise<AgentMetadata[]> {
const response = await this.request<{ agents: AgentMetadata[] }>('/agents/idle');
return response.agents;
}
/**
* Create a new task
*/
async createTask(
description: string,
context: Record<string, any> = {},
maxRetries: number = 3
): Promise<TaskMetadata> {
return this.request<TaskMetadata>('/tasks', {
method: 'POST',
body: JSON.stringify({
description,
context,
max_retries: maxRetries,
}),
});
}
/**
* Assign task to agent
*/
async assignTask(taskId: string, agentId: string): Promise<void> {
await this.request('/tasks/assign', {
method: 'POST',
body: JSON.stringify({
task_id: taskId,
agent_id: agentId,
}),
});
}
/**
* List all tasks
*/
async listTasks(status?: string): Promise<TaskMetadata[]> {
const query = status ? `?status=${status}` : '';
const response = await this.request<{ tasks: TaskMetadata[] }>(`/tasks${query}`);
return response.tasks;
}
/**
* Get specific task
*/
async getTask(taskId: string): Promise<TaskMetadata> {
return this.request<TaskMetadata>(`/tasks/${taskId}`);
}
/**
* List handoff history
*/
async listHandoffs(agentId?: string, limit: number = 100): Promise<HandoffMetadata[]> {
const params = new URLSearchParams();
if (agentId) params.append('agent_id', agentId);
params.append('limit', limit.toString());
const query = params.toString() ? `?${params.toString()}` : '';
const response = await this.request<{ handoffs: HandoffMetadata[] }>(`/handoffs${query}`);
return response.handoffs;
}
/**
* Create WebSocket connection for real-time updates
*/
createWebSocket(agentId: string): WebSocket {
const wsUrl = this.baseUrl.replace(/^http/, 'ws');
return new WebSocket(`${wsUrl}/ws/${agentId}`);
}
}
/**
* Singleton instance
*/
let clientInstance: OrchestratorClient | null = null;
export function getOrchestratorClient(config?: OrchestratorClientConfig): OrchestratorClient {
if (!clientInstance) {
if (!config) {
throw new Error('OrchestratorClient not initialized. Provide config on first call.');
}
clientInstance = new OrchestratorClient(config);
}
return clientInstance;
}
export function resetOrchestratorClient(): void {
clientInstance = null;
}
@@ -0,0 +1,225 @@
import React, { useEffect, useState } from 'react';
import { AgentMetadata, getOrchestratorClient } from '../api/orchestrator-client';
export interface AgentDashboardProps {
orchestratorUrl: string;
workspaceId?: string;
}
export const AgentDashboard: React.FC<AgentDashboardProps> = ({
orchestratorUrl,
workspaceId,
}) => {
const [agents, setAgents] = useState<AgentMetadata[]>([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const [connected, setConnected] = useState(false);
useEffect(() => {
const client = getOrchestratorClient({ baseUrl: orchestratorUrl });
// Initial fetch
const fetchAgents = async () => {
try {
const agentList = await client.listAgents();
setAgents(agentList);
setConnected(true);
setError(null);
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to fetch agents');
setConnected(false);
} finally {
setLoading(false);
}
};
fetchAgents();
// Poll for updates every 2 seconds
const interval = setInterval(fetchAgents, 2000);
return () => clearInterval(interval);
}, [orchestratorUrl]);
const getStatusColor = (status: string) => {
switch (status) {
case 'idle':
return 'bg-gray-500';
case 'busy':
return 'bg-blue-500 animate-pulse';
case 'handoff':
case 'handoff-pending':
return 'bg-yellow-500 animate-pulse';
case 'failed':
return 'bg-red-500';
default:
return 'bg-gray-500';
}
};
const getStatusText = (status: string) => {
switch (status) {
case 'idle':
return 'Idle';
case 'busy':
return 'Working';
case 'handoff':
case 'handoff-pending':
return 'Handoff';
case 'failed':
return 'Failed';
default:
return 'Unknown';
}
};
const formatTimestamp = (timestamp: number) => {
const date = new Date(timestamp * 1000);
const now = Date.now();
const diff = now - date.getTime();
if (diff < 60000) {
return 'Just now';
} else if (diff < 3600000) {
return `${Math.floor(diff / 60000)}m ago`;
} else if (diff < 86400000) {
return `${Math.floor(diff / 3600000)}h ago`;
} else {
return date.toLocaleDateString();
}
};
const filteredAgents = workspaceId
? agents.filter(agent => agent.agent_id.includes(workspaceId))
: agents;
if (loading) {
return (
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
<h3 className="text-lg font-semibold mb-4">Agent Dashboard</h3>
<div className="flex items-center justify-center py-8">
<svg className="animate-spin h-8 w-8 text-blue-500" viewBox="0 0 24 24">
<circle
className="opacity-25"
cx="12"
cy="12"
r="10"
stroke="currentColor"
strokeWidth="4"
fill="none"
/>
<path
className="opacity-75"
fill="currentColor"
d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"
/>
</svg>
<span className="ml-3 text-gray-400">Loading agents...</span>
</div>
</div>
);
}
return (
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
<div className="flex items-center justify-between mb-4">
<h3 className="text-lg font-semibold">Agent Dashboard</h3>
<div className="flex items-center space-x-2">
<div className={`w-2 h-2 rounded-full ${connected ? 'bg-green-500' : 'bg-red-500'}`} />
<span className="text-sm text-gray-400">
{connected ? 'Connected' : 'Disconnected'}
</span>
</div>
</div>
{error && (
<div className="bg-red-900/50 border border-red-500 rounded px-4 py-2 text-red-200 text-sm mb-4">
{error}
</div>
)}
<div className="mb-4 grid grid-cols-3 gap-4">
<div className="bg-gray-700 rounded p-3">
<p className="text-xs text-gray-400 mb-1">Total Agents</p>
<p className="text-2xl font-bold text-white">{filteredAgents.length}</p>
</div>
<div className="bg-gray-700 rounded p-3">
<p className="text-xs text-gray-400 mb-1">Active</p>
<p className="text-2xl font-bold text-blue-400">
{filteredAgents.filter(a => a.status === 'busy').length}
</p>
</div>
<div className="bg-gray-700 rounded p-3">
<p className="text-xs text-gray-400 mb-1">Idle</p>
<p className="text-2xl font-bold text-gray-400">
{filteredAgents.filter(a => a.status === 'idle').length}
</p>
</div>
</div>
{filteredAgents.length === 0 ? (
<div className="text-center py-8 text-gray-500">
<p>No agents running</p>
<p className="text-sm mt-2">Submit a task to create agents</p>
</div>
) : (
<div className="space-y-3 max-h-96 overflow-y-auto">
{filteredAgents.map((agent) => (
<div
key={agent.agent_id}
className="bg-gray-700 rounded p-4 hover:bg-gray-650 transition-colors"
>
<div className="flex items-start justify-between">
<div className="flex-1">
<div className="flex items-center space-x-2 mb-2">
<div className={`w-3 h-3 rounded-full ${getStatusColor(agent.status)}`} />
<span className="font-medium text-white truncate">
{agent.agent_id}
</span>
</div>
<div className="space-y-1 text-sm">
<div className="flex items-center justify-between">
<span className="text-gray-400">Status:</span>
<span className="text-gray-300">{getStatusText(agent.status)}</span>
</div>
{agent.current_task_id && (
<div className="flex items-center justify-between">
<span className="text-gray-400">Task:</span>
<span className="text-blue-400 text-xs truncate max-w-xs">
{agent.current_task_id}
</span>
</div>
)}
<div className="flex items-center justify-between">
<span className="text-gray-400">Last Heartbeat:</span>
<span className="text-gray-300">{formatTimestamp(agent.last_heartbeat)}</span>
</div>
{agent.capabilities.length > 0 && (
<div className="mt-2">
<span className="text-gray-400 text-xs">Capabilities:</span>
<div className="flex flex-wrap gap-1 mt-1">
{agent.capabilities.map((cap, idx) => (
<span
key={idx}
className="bg-gray-600 text-gray-300 text-xs px-2 py-0.5 rounded"
>
{cap}
</span>
))}
</div>
</div>
)}
</div>
</div>
</div>
</div>
))}
</div>
)}
</div>
);
};
@@ -0,0 +1,260 @@
import React, { useState } from 'react';
import { ComplexityAnalysis } from '../ai/complexity-analyzer';
export interface CostControlDialogProps {
analysis: ComplexityAnalysis;
maxAgents: number;
onConfirm: () => void;
onCancel: () => void;
onAdjust: (newCount: number) => void;
}
export const CostControlDialog: React.FC<CostControlDialogProps> = ({
analysis,
maxAgents,
onConfirm,
onCancel,
onAdjust,
}) => {
const [adjustedCount, setAdjustedCount] = useState(analysis.agentCount);
const agentCount = analysis.agentCount;
// Determine warning level
const isWarning = agentCount > 10;
const requiresConfirmation = agentCount > 15;
const exceedsMax = agentCount > maxAgents;
// Calculate estimated resource cost
const estimatedCpuHours = (agentCount * 0.5).toFixed(1); // Assume 30min avg per agent
const estimatedCost = (agentCount * 0.05).toFixed(2); // Rough cost estimate
if (!isWarning && !exceedsMax) {
// No warning needed, auto-confirm
return null;
}
return (
<div className="fixed inset-0 bg-black bg-opacity-50 flex items-center justify-center z-50">
<div className="bg-white rounded-lg shadow-xl max-w-2xl w-full mx-4 p-6">
{/* Header */}
<div className="flex items-start mb-4">
<div
className={`flex-shrink-0 w-12 h-12 rounded-full flex items-center justify-center ${
exceedsMax
? 'bg-red-100'
: requiresConfirmation
? 'bg-orange-100'
: 'bg-yellow-100'
}`}
>
<svg
className={`w-6 h-6 ${
exceedsMax
? 'text-red-600'
: requiresConfirmation
? 'text-orange-600'
: 'text-yellow-600'
}`}
fill="none"
stroke="currentColor"
viewBox="0 0 24 24"
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth={2}
d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z"
/>
</svg>
</div>
<div className="ml-4 flex-1">
<h3 className="text-lg font-semibold text-gray-900">
{exceedsMax
? 'Agent Limit Exceeded'
: requiresConfirmation
? 'High Agent Count Confirmation Required'
: 'High Agent Count Warning'}
</h3>
<p className="mt-1 text-sm text-gray-600">
{exceedsMax
? `The task requires ${agentCount} agents, which exceeds your configured maximum of ${maxAgents}.`
: requiresConfirmation
? `This task will create ${agentCount} agents. Please confirm this is intentional.`
: `This task will create ${agentCount} agents, which may consume significant resources.`}
</p>
</div>
</div>
{/* Analysis Details */}
<div className="bg-gray-50 rounded-lg p-4 mb-4">
<h4 className="text-sm font-medium text-gray-900 mb-2">Task Analysis</h4>
<div className="space-y-2 text-sm">
<div className="flex justify-between">
<span className="text-gray-600">Complexity:</span>
<span className="font-medium capitalize">{analysis.complexity}</span>
</div>
<div className="flex justify-between">
<span className="text-gray-600">Recommended Agents:</span>
<span className="font-medium">{agentCount}</span>
</div>
<div className="flex justify-between">
<span className="text-gray-600">Confidence:</span>
<span className="font-medium">{(analysis.confidence * 100).toFixed(0)}%</span>
</div>
</div>
<div className="mt-3 pt-3 border-t border-gray-200">
<p className="text-sm text-gray-700">
<span className="font-medium">Reasoning:</span> {analysis.reasoning}
</p>
</div>
</div>
{/* Resource Estimates */}
<div className="bg-blue-50 rounded-lg p-4 mb-4">
<h4 className="text-sm font-medium text-blue-900 mb-2">Estimated Resource Usage</h4>
<div className="grid grid-cols-2 gap-4 text-sm">
<div>
<span className="text-blue-700">CPU Hours:</span>
<span className="ml-2 font-medium text-blue-900">~{estimatedCpuHours}h</span>
</div>
<div>
<span className="text-blue-700">Estimated Cost:</span>
<span className="ml-2 font-medium text-blue-900">${estimatedCost}</span>
</div>
</div>
</div>
{/* Agent Count Adjustment */}
{!exceedsMax && (
<div className="mb-4">
<label className="block text-sm font-medium text-gray-700 mb-2">
Adjust Agent Count (Optional)
</label>
<div className="flex items-center space-x-4">
<input
type="range"
min="1"
max={maxAgents}
value={adjustedCount}
onChange={(e) => setAdjustedCount(parseInt(e.target.value, 10))}
className="flex-1"
/>
<input
type="number"
min="1"
max={maxAgents}
value={adjustedCount}
onChange={(e) => setAdjustedCount(parseInt(e.target.value, 10))}
className="w-20 px-3 py-2 border border-gray-300 rounded-md text-sm"
/>
</div>
{adjustedCount !== agentCount && (
<p className="mt-2 text-sm text-gray-600">
Adjusting from {agentCount} to {adjustedCount} agents may affect task completion
quality.
</p>
)}
</div>
)}
{/* Actions */}
<div className="flex justify-end space-x-3">
<button
onClick={onCancel}
className="px-4 py-2 text-sm font-medium text-gray-700 bg-white border border-gray-300 rounded-md hover:bg-gray-50 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500"
>
Cancel
</button>
{exceedsMax ? (
<button
onClick={() => {
setAdjustedCount(maxAgents);
onAdjust(maxAgents);
}}
className="px-4 py-2 text-sm font-medium text-white bg-blue-600 rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500"
>
Use Maximum ({maxAgents} agents)
</button>
) : (
<>
{adjustedCount !== agentCount && (
<button
onClick={() => onAdjust(adjustedCount)}
className="px-4 py-2 text-sm font-medium text-white bg-blue-600 rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500"
>
Use {adjustedCount} Agents
</button>
)}
<button
onClick={onConfirm}
className={`px-4 py-2 text-sm font-medium text-white rounded-md focus:outline-none focus:ring-2 focus:ring-offset-2 ${
requiresConfirmation
? 'bg-orange-600 hover:bg-orange-700 focus:ring-orange-500'
: 'bg-yellow-600 hover:bg-yellow-700 focus:ring-yellow-500'
}`}
>
{requiresConfirmation ? 'Confirm' : 'Proceed'} with {agentCount} Agents
</button>
</>
)}
</div>
</div>
</div>
);
};
/**
* Hook to manage cost control dialog state
*/
export function useCostControl(maxAgents: number = 20) {
const [showDialog, setShowDialog] = useState(false);
const [currentAnalysis, setCurrentAnalysis] = useState<ComplexityAnalysis | null>(null);
const [onConfirmCallback, setOnConfirmCallback] = useState<(() => void) | null>(null);
const checkAndConfirm = (
analysis: ComplexityAnalysis,
onConfirm: () => void
): Promise<boolean> => {
return new Promise((resolve) => {
const agentCount = analysis.agentCount;
// Auto-approve if within safe limits
if (agentCount <= 10 && agentCount <= maxAgents) {
onConfirm();
resolve(true);
return;
}
// Show dialog for high counts or exceeding max
setCurrentAnalysis(analysis);
setOnConfirmCallback(() => () => {
onConfirm();
setShowDialog(false);
resolve(true);
});
setShowDialog(true);
});
};
const handleCancel = () => {
setShowDialog(false);
setCurrentAnalysis(null);
setOnConfirmCallback(null);
};
const handleAdjust = (newCount: number) => {
if (currentAnalysis) {
const adjustedAnalysis = { ...currentAnalysis, agentCount: newCount };
setCurrentAnalysis(adjustedAnalysis);
}
};
return {
showDialog,
currentAnalysis,
checkAndConfirm,
handleCancel,
handleConfirm: onConfirmCallback,
handleAdjust,
};
}
@@ -0,0 +1,238 @@
import React, { useState } from 'react';
import { MergeConflict, ConflictResolution } from '../git/merge-manager';
export interface ResultViewerProps {
conflicts: MergeConflict[];
mergedBranches: string[];
onResolve: (resolutions: ConflictResolution[]) => void;
onCancel: () => void;
}
export const ResultViewer: React.FC<ResultViewerProps> = ({
conflicts,
mergedBranches,
onResolve,
onCancel,
}) => {
const [resolutions, setResolutions] = useState<Map<string, ConflictResolution>>(new Map());
const [selectedFile, setSelectedFile] = useState<string | null>(
conflicts.length > 0 ? conflicts[0].filepath : null
);
const handleResolutionChange = (
filepath: string,
resolution: 'ours' | 'theirs' | 'manual',
manualContent?: string
) => {
const newResolutions = new Map(resolutions);
newResolutions.set(filepath, {
filepath,
resolution,
manualContent,
});
setResolutions(newResolutions);
};
const handleResolveAll = () => {
const allResolutions = Array.from(resolutions.values());
// Ensure all conflicts have resolutions
const unresolvedConflicts = conflicts.filter(
conflict => !resolutions.has(conflict.filepath)
);
if (unresolvedConflicts.length > 0) {
alert(`Please resolve all conflicts. ${unresolvedConflicts.length} remaining.`);
return;
}
onResolve(allResolutions);
};
const selectedConflict = conflicts.find(c => c.filepath === selectedFile);
const currentResolution = selectedFile ? resolutions.get(selectedFile) : null;
if (conflicts.length === 0) {
return (
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
<h3 className="text-lg font-semibold mb-4">Merge Results</h3>
<div className="bg-green-900/50 border border-green-500 rounded p-4 mb-4">
<div className="flex items-center space-x-2">
<svg className="w-6 h-6 text-green-400" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M5 13l4 4L19 7" />
</svg>
<span className="text-green-200 font-medium">Merge completed successfully!</span>
</div>
</div>
<div className="space-y-2">
<p className="text-sm text-gray-400">Merged branches:</p>
<div className="bg-gray-700 rounded p-3 space-y-1">
{mergedBranches.map((branch, idx) => (
<div key={idx} className="flex items-center space-x-2">
<svg className="w-4 h-4 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M7.707 3.293a1 1 0 010 1.414L5.414 7H11a7 7 0 017 7v2a1 1 0 11-2 0v-2a5 5 0 00-5-5H5.414l2.293 2.293a1 1 0 11-1.414 1.414l-4-4a1 1 0 010-1.414l4-4a1 1 0 011.414 0z" clipRule="evenodd" />
</svg>
<span className="text-sm text-gray-300 font-mono">{branch}</span>
</div>
))}
</div>
</div>
</div>
);
}
return (
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
<h3 className="text-lg font-semibold mb-4">Resolve Merge Conflicts</h3>
<div className="bg-yellow-900/50 border border-yellow-500 rounded p-4 mb-4">
<div className="flex items-center space-x-2">
<svg className="w-6 h-6 text-yellow-400" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z" />
</svg>
<span className="text-yellow-200 font-medium">
{conflicts.length} conflict{conflicts.length > 1 ? 's' : ''} detected
</span>
</div>
</div>
<div className="grid grid-cols-3 gap-4">
{/* File list */}
<div className="col-span-1 bg-gray-700 rounded p-3 max-h-96 overflow-y-auto">
<p className="text-xs text-gray-400 mb-2 font-medium">Conflicted Files</p>
<div className="space-y-1">
{conflicts.map((conflict) => {
const resolved = resolutions.has(conflict.filepath);
return (
<button
key={conflict.filepath}
className={`w-full text-left px-3 py-2 rounded text-sm transition-colors ${
selectedFile === conflict.filepath
? 'bg-blue-600 text-white'
: 'bg-gray-600 text-gray-300 hover:bg-gray-550'
}`}
onClick={() => setSelectedFile(conflict.filepath)}
>
<div className="flex items-center justify-between">
<span className="truncate font-mono text-xs">{conflict.filepath}</span>
{resolved && (
<svg className="w-4 h-4 text-green-400 flex-shrink-0 ml-2" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
</svg>
)}
</div>
</button>
);
})}
</div>
</div>
{/* Conflict viewer and resolution */}
<div className="col-span-2 space-y-4">
{selectedConflict && (
<>
<div className="bg-gray-700 rounded p-4">
<p className="text-sm font-medium text-gray-300 mb-3">
{selectedConflict.filepath}
</p>
<div className="space-y-3">
<button
className={`w-full text-left p-3 rounded border-2 transition-colors ${
currentResolution?.resolution === 'ours'
? 'border-blue-500 bg-blue-900/30'
: 'border-gray-600 hover:border-gray-500'
}`}
onClick={() => handleResolutionChange(selectedConflict.filepath, 'ours')}
>
<div className="flex items-center justify-between mb-2">
<span className="text-sm font-medium text-gray-300">Use Current Version</span>
{currentResolution?.resolution === 'ours' && (
<svg className="w-5 h-5 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
</svg>
)}
</div>
<pre className="text-xs text-gray-400 bg-gray-800 p-2 rounded overflow-x-auto max-h-32">
{selectedConflict.ours}
</pre>
</button>
<button
className={`w-full text-left p-3 rounded border-2 transition-colors ${
currentResolution?.resolution === 'theirs'
? 'border-blue-500 bg-blue-900/30'
: 'border-gray-600 hover:border-gray-500'
}`}
onClick={() => handleResolutionChange(selectedConflict.filepath, 'theirs')}
>
<div className="flex items-center justify-between mb-2">
<span className="text-sm font-medium text-gray-300">Use Agent Version</span>
{currentResolution?.resolution === 'theirs' && (
<svg className="w-5 h-5 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
</svg>
)}
</div>
<pre className="text-xs text-gray-400 bg-gray-800 p-2 rounded overflow-x-auto max-h-32">
{selectedConflict.theirs}
</pre>
</button>
<div
className={`p-3 rounded border-2 ${
currentResolution?.resolution === 'manual'
? 'border-blue-500 bg-blue-900/30'
: 'border-gray-600'
}`}
>
<div className="flex items-center justify-between mb-2">
<span className="text-sm font-medium text-gray-300">Manual Resolution</span>
{currentResolution?.resolution === 'manual' && (
<svg className="w-5 h-5 text-blue-400" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zm3.707-9.293a1 1 0 00-1.414-1.414L9 10.586 7.707 9.293a1 1 0 00-1.414 1.414l2 2a1 1 0 001.414 0l4-4z" clipRule="evenodd" />
</svg>
)}
</div>
<textarea
className="w-full bg-gray-800 text-gray-300 text-xs font-mono p-2 rounded focus:outline-none focus:ring-2 focus:ring-blue-500"
rows={8}
placeholder="Enter your manual resolution..."
value={currentResolution?.resolution === 'manual' ? currentResolution.manualContent : ''}
onChange={(e) =>
handleResolutionChange(selectedConflict.filepath, 'manual', e.target.value)
}
/>
</div>
</div>
</div>
</>
)}
</div>
</div>
<div className="mt-6 flex justify-between items-center">
<div className="text-sm text-gray-400">
Resolved: {resolutions.size} / {conflicts.length}
</div>
<div className="flex space-x-3">
<button
className="bg-gray-600 hover:bg-gray-700 text-white font-semibold px-6 py-2 rounded transition-colors"
onClick={onCancel}
>
Cancel
</button>
<button
className="bg-green-600 hover:bg-green-700 text-white font-semibold px-6 py-2 rounded disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
onClick={handleResolveAll}
disabled={resolutions.size !== conflicts.length}
>
Apply Resolutions
</button>
</div>
</div>
</div>
);
};
@@ -0,0 +1,165 @@
import React, { useState } from 'react';
import { ComplexityAnalysis, getAnalyzer } from '../ai/complexity-analyzer';
export interface TaskSubmissionFormProps {
onSubmit: (task: string, complexity: ComplexityAnalysis) => void;
disabled?: boolean;
}
export const TaskSubmissionForm: React.FC<TaskSubmissionFormProps> = ({
onSubmit,
disabled = false,
}) => {
const [taskDescription, setTaskDescription] = useState('');
const [analyzing, setAnalyzing] = useState(false);
const [complexity, setComplexity] = useState<ComplexityAnalysis | null>(null);
const [error, setError] = useState<string | null>(null);
const handleAnalyze = async () => {
if (!taskDescription.trim()) {
setError('Please enter a task description');
return;
}
setAnalyzing(true);
setError(null);
try {
const analyzer = getAnalyzer();
const result = await analyzer.analyzeTask(taskDescription);
setComplexity(result);
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to analyze task');
setComplexity(null);
} finally {
setAnalyzing(false);
}
};
const handleSubmit = () => {
if (!complexity) {
setError('Please analyze the task first');
return;
}
onSubmit(taskDescription, complexity);
// Reset form
setTaskDescription('');
setComplexity(null);
setError(null);
};
const getComplexityColor = (level: string) => {
switch (level) {
case 'low':
return 'text-green-400';
case 'medium':
return 'text-yellow-400';
case 'high':
return 'text-red-400';
default:
return 'text-gray-400';
}
};
return (
<div className="bg-gray-800 rounded-lg p-6 shadow-lg">
<h3 className="text-lg font-semibold mb-4">Submit Task</h3>
<div className="space-y-4">
<div>
<label className="block text-sm font-medium text-gray-300 mb-2">
Task Description
</label>
<textarea
className="w-full bg-gray-700 text-white rounded px-4 py-2 focus:outline-none focus:ring-2 focus:ring-blue-500 disabled:opacity-50 disabled:cursor-not-allowed"
rows={6}
placeholder="Describe your programming task in detail..."
value={taskDescription}
onChange={(e) => setTaskDescription(e.target.value)}
disabled={disabled || analyzing}
/>
</div>
{error && (
<div className="bg-red-900/50 border border-red-500 rounded px-4 py-2 text-red-200 text-sm">
{error}
</div>
)}
{complexity && (
<div className="bg-gray-700 rounded p-4 space-y-3">
<div className="flex items-center justify-between">
<span className="text-sm font-medium text-gray-300">Complexity:</span>
<span className={`text-sm font-bold uppercase ${getComplexityColor(complexity.complexity)}`}>
{complexity.complexity}
</span>
</div>
<div className="flex items-center justify-between">
<span className="text-sm font-medium text-gray-300">Recommended Agents:</span>
<span className="text-sm font-bold text-blue-400">{complexity.agentCount}</span>
</div>
<div className="flex items-center justify-between">
<span className="text-sm font-medium text-gray-300">Confidence:</span>
<span className="text-sm font-bold text-gray-300">
{Math.round(complexity.confidence * 100)}%
</span>
</div>
<div className="pt-2 border-t border-gray-600">
<p className="text-xs text-gray-400 mb-1">Analysis:</p>
<p className="text-sm text-gray-300">{complexity.reasoning}</p>
</div>
</div>
)}
<div className="flex space-x-3">
<button
className="flex-1 bg-blue-600 hover:bg-blue-700 text-white font-semibold px-6 py-2 rounded disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
onClick={handleAnalyze}
disabled={disabled || analyzing || !taskDescription.trim()}
>
{analyzing ? (
<span className="flex items-center justify-center">
<svg className="animate-spin h-5 w-5 mr-2" viewBox="0 0 24 24">
<circle
className="opacity-25"
cx="12"
cy="12"
r="10"
stroke="currentColor"
strokeWidth="4"
fill="none"
/>
<path
className="opacity-75"
fill="currentColor"
d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"
/>
</svg>
Analyzing...
</span>
) : (
'Analyze Complexity'
)}
</button>
<button
className="flex-1 bg-green-600 hover:bg-green-700 text-white font-semibold px-6 py-2 rounded disabled:opacity-50 disabled:cursor-not-allowed transition-colors"
onClick={handleSubmit}
disabled={disabled || analyzing || !complexity}
>
Submit Task
</button>
</div>
<p className="text-xs text-gray-500 text-center">
Analyze your task to see complexity and agent requirements before submission
</p>
</div>
</div>
);
};
+333
View File
@@ -0,0 +1,333 @@
/**
* Git merge manager for aggregating agent results with conflict resolution
*/
import * as git from 'isomorphic-git';
import * as fs from 'fs';
import * as path from 'path';
import http from 'isomorphic-git/http/node';
export interface MergeConflict {
filepath: string;
ours: string;
theirs: string;
base?: string;
}
export interface MergeResult {
success: boolean;
conflicts: MergeConflict[];
mergedBranches: string[];
error?: string;
}
export interface ConflictResolution {
filepath: string;
resolution: 'ours' | 'theirs' | 'manual';
manualContent?: string;
}
export class MergeManager {
private repoDir: string;
constructor(repoDir: string) {
this.repoDir = repoDir;
}
/**
* Merge multiple agent result branches into a target branch
*/
async mergeAgentResults(
targetBranch: string,
agentBranches: string[]
): Promise<MergeResult> {
const result: MergeResult = {
success: true,
conflicts: [],
mergedBranches: [],
};
try {
// Checkout target branch
await git.checkout({
fs,
dir: this.repoDir,
ref: targetBranch,
});
// Merge each agent branch sequentially
for (const agentBranch of agentBranches) {
try {
await git.merge({
fs,
dir: this.repoDir,
ours: targetBranch,
theirs: agentBranch,
author: {
name: 'HeiCode Swarm',
email: 'swarm@heicode.local',
},
});
result.mergedBranches.push(agentBranch);
} catch (error) {
// Merge conflict detected
const conflicts = await this.detectConflicts(agentBranch);
result.conflicts.push(...conflicts);
result.success = false;
// Abort the merge to allow manual resolution
await this.abortMerge();
}
}
} catch (error) {
result.success = false;
result.error = error instanceof Error ? error.message : String(error);
}
return result;
}
/**
* Detect merge conflicts between current branch and another branch
*/
async detectConflicts(branchName: string): Promise<MergeConflict[]> {
const conflicts: MergeConflict[] = [];
try {
// Get the status matrix to find conflicted files
const status = await git.statusMatrix({ fs, dir: this.repoDir });
for (const [filepath, HEADStatus, workdirStatus, stageStatus] of status) {
// Check if file has conflicts (different in HEAD, workdir, and stage)
if (HEADStatus !== workdirStatus || workdirStatus !== stageStatus) {
const conflict = await this.getConflictContent(filepath, branchName);
if (conflict) {
conflicts.push(conflict);
}
}
}
} catch (error) {
console.error('Error detecting conflicts:', error);
}
return conflicts;
}
/**
* Get conflict content for a specific file
*/
private async getConflictContent(
filepath: string,
theirBranch: string
): Promise<MergeConflict | null> {
try {
const fullPath = path.join(this.repoDir, filepath);
// Read current (ours) version
const oursContent = fs.existsSync(fullPath)
? fs.readFileSync(fullPath, 'utf-8')
: '';
// Read their version
await git.checkout({
fs,
dir: this.repoDir,
ref: theirBranch,
filepaths: [filepath],
force: true,
});
const theirsContent = fs.existsSync(fullPath)
? fs.readFileSync(fullPath, 'utf-8')
: '';
// Restore our version
await git.checkout({
fs,
dir: this.repoDir,
ref: 'HEAD',
filepaths: [filepath],
force: true,
});
return {
filepath,
ours: oursContent,
theirs: theirsContent,
};
} catch (error) {
console.error(`Error getting conflict content for ${filepath}:`, error);
return null;
}
}
/**
* Resolve conflicts with provided resolutions
*/
async resolveConflicts(resolutions: ConflictResolution[]): Promise<boolean> {
try {
for (const resolution of resolutions) {
const fullPath = path.join(this.repoDir, resolution.filepath);
if (resolution.resolution === 'manual' && resolution.manualContent) {
// Use manual resolution
fs.writeFileSync(fullPath, resolution.manualContent, 'utf-8');
} else if (resolution.resolution === 'ours') {
// Keep our version (already in place, just stage it)
await git.add({ fs, dir: this.repoDir, filepath: resolution.filepath });
} else if (resolution.resolution === 'theirs') {
// Use their version (need to checkout from merge branch)
// This is handled during the merge process
}
// Stage the resolved file
await git.add({ fs, dir: this.repoDir, filepath: resolution.filepath });
}
// Commit the merge
await git.commit({
fs,
dir: this.repoDir,
message: 'Merge agent results with conflict resolution',
author: {
name: 'HeiCode Swarm',
email: 'swarm@heicode.local',
},
});
return true;
} catch (error) {
console.error('Error resolving conflicts:', error);
return false;
}
}
/**
* Abort current merge
*/
async abortMerge(): Promise<void> {
try {
// Reset to HEAD to abort merge
const commits = await git.log({ fs, dir: this.repoDir, depth: 1 });
if (commits.length > 0) {
const headCommit = commits[0].oid;
// Get all files
const status = await git.statusMatrix({ fs, dir: this.repoDir });
for (const [filepath] of status) {
try {
// Restore file to HEAD state
await git.checkout({
fs,
dir: this.repoDir,
ref: headCommit,
filepaths: [filepath],
force: true,
});
} catch (error) {
console.error(`Error restoring ${filepath}:`, error);
}
}
}
} catch (error) {
console.error('Error aborting merge:', error);
}
}
/**
* Get diff between two branches
*/
async getDiff(branch1: string, branch2: string): Promise<string[]> {
const changedFiles: string[] = [];
try {
// Get commits for both branches
const commits1 = await git.log({ fs, dir: this.repoDir, ref: branch1, depth: 1 });
const commits2 = await git.log({ fs, dir: this.repoDir, ref: branch2, depth: 1 });
if (commits1.length === 0 || commits2.length === 0) {
return changedFiles;
}
const oid1 = commits1[0].oid;
const oid2 = commits2[0].oid;
// Compare trees
const tree1 = await git.readTree({ fs, dir: this.repoDir, oid: oid1 });
const tree2 = await git.readTree({ fs, dir: this.repoDir, oid: oid2 });
// Find differences
const files1 = new Set(tree1.tree.map(entry => entry.path));
const files2 = new Set(tree2.tree.map(entry => entry.path));
// Files in branch1 but not in branch2
for (const file of files1) {
if (!files2.has(file)) {
changedFiles.push(file);
}
}
// Files in branch2 but not in branch1
for (const file of files2) {
if (!files1.has(file)) {
changedFiles.push(file);
}
}
// Files in both but with different content
for (const file of files1) {
if (files2.has(file)) {
const entry1 = tree1.tree.find(e => e.path === file);
const entry2 = tree2.tree.find(e => e.path === file);
if (entry1 && entry2 && entry1.oid !== entry2.oid) {
changedFiles.push(file);
}
}
}
} catch (error) {
console.error('Error getting diff:', error);
}
return changedFiles;
}
/**
* Create a merge summary
*/
async createMergeSummary(
targetBranch: string,
agentBranches: string[]
): Promise<{
totalChanges: number;
fileChanges: Map<string, number>;
}> {
const fileChanges = new Map<string, number>();
let totalChanges = 0;
for (const agentBranch of agentBranches) {
const diff = await this.getDiff(targetBranch, agentBranch);
totalChanges += diff.length;
for (const file of diff) {
fileChanges.set(file, (fileChanges.get(file) || 0) + 1);
}
}
return { totalChanges, fileChanges };
}
/**
* List all branches matching a pattern
*/
async listAgentBranches(workspaceId: string): Promise<string[]> {
try {
const branches = await git.listBranches({ fs, dir: this.repoDir });
return branches.filter(branch => branch.startsWith(`agent-${workspaceId}-`));
} catch (error) {
console.error('Error listing branches:', error);
return [];
}
}
}
+316
View File
@@ -0,0 +1,316 @@
import * as git from 'isomorphic-git';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import http from 'isomorphic-git/http/node';
export interface WorkspaceConfig {
workspaceId: string;
repoPath: string;
gitUrl: string;
}
export interface GitOperationResult {
success: boolean;
error?: string;
data?: any;
}
export class WorkspaceManager {
private workspacesRoot: string;
constructor(workspacesRoot?: string) {
this.workspacesRoot = workspacesRoot || path.join(os.homedir(), '.heicode-swarm', 'workspaces');
this.ensureWorkspacesRoot();
}
/**
* Initialize a new bare Git repository for a workspace
*/
public async initWorkspace(workspaceId: string): Promise<GitOperationResult> {
try {
const repoPath = path.join(this.workspacesRoot, workspaceId);
// Create workspace directory
if (!fs.existsSync(repoPath)) {
fs.mkdirSync(repoPath, { recursive: true });
}
// Initialize bare repository
await git.init({
fs,
dir: repoPath,
bare: true,
defaultBranch: 'main'
});
const gitUrl = `file://${repoPath}`;
return {
success: true,
data: {
workspaceId,
repoPath,
gitUrl
}
};
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Clone a workspace repository to a local directory
*/
public async cloneWorkspace(
gitUrl: string,
targetDir: string,
branch: string = 'main'
): Promise<GitOperationResult> {
try {
// Ensure target directory exists
if (!fs.existsSync(targetDir)) {
fs.mkdirSync(targetDir, { recursive: true });
}
await git.clone({
fs,
http,
dir: targetDir,
url: gitUrl,
ref: branch,
singleBranch: true,
depth: 1
});
return { success: true };
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Create a new branch in a repository
*/
public async createBranch(
repoDir: string,
branchName: string,
checkout: boolean = true
): Promise<GitOperationResult> {
try {
// Create branch
await git.branch({
fs,
dir: repoDir,
ref: branchName
});
// Checkout if requested
if (checkout) {
await git.checkout({
fs,
dir: repoDir,
ref: branchName
});
}
return { success: true };
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Commit changes to a repository
*/
public async commit(
repoDir: string,
message: string,
author: { name: string; email: string }
): Promise<GitOperationResult> {
try {
// Stage all changes
const status = await git.statusMatrix({ fs, dir: repoDir });
for (const [filepath, , worktreeStatus] of status) {
if (worktreeStatus !== 1) {
// File is modified or new
await git.add({ fs, dir: repoDir, filepath });
} else if (worktreeStatus === 0) {
// File is deleted
await git.remove({ fs, dir: repoDir, filepath });
}
}
// Commit
const sha = await git.commit({
fs,
dir: repoDir,
message,
author
});
return {
success: true,
data: { sha }
};
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Push changes to remote repository
*/
public async push(
repoDir: string,
remote: string = 'origin',
branch?: string
): Promise<GitOperationResult> {
try {
await git.push({
fs,
http,
dir: repoDir,
remote,
ref: branch
});
return { success: true };
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Merge branches
*/
public async merge(
repoDir: string,
sourceBranch: string,
targetBranch: string
): Promise<GitOperationResult> {
try {
// Checkout target branch
await git.checkout({
fs,
dir: repoDir,
ref: targetBranch
});
// Merge source branch
await git.merge({
fs,
dir: repoDir,
ours: targetBranch,
theirs: sourceBranch,
author: {
name: 'HeiCode Swarm',
email: 'swarm@heicode.local'
}
});
return { success: true };
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* List all branches in a repository
*/
public async listBranches(repoDir: string): Promise<GitOperationResult> {
try {
const branches = await git.listBranches({
fs,
dir: repoDir
});
return {
success: true,
data: { branches }
};
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Get current branch name
*/
public async getCurrentBranch(repoDir: string): Promise<GitOperationResult> {
try {
const branch = await git.currentBranch({
fs,
dir: repoDir
});
return {
success: true,
data: { branch }
};
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Delete a workspace
*/
public async deleteWorkspace(workspaceId: string): Promise<GitOperationResult> {
try {
const repoPath = path.join(this.workspacesRoot, workspaceId);
if (fs.existsSync(repoPath)) {
fs.rmSync(repoPath, { recursive: true, force: true });
}
return { success: true };
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Get workspace path
*/
public getWorkspacePath(workspaceId: string): string {
return path.join(this.workspacesRoot, workspaceId);
}
/**
* Ensure workspaces root directory exists
*/
private ensureWorkspacesRoot(): void {
if (!fs.existsSync(this.workspacesRoot)) {
fs.mkdirSync(this.workspacesRoot, { recursive: true });
}
}
}
+26
View File
@@ -0,0 +1,26 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>HeiCode Swarm</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
sans-serif;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}
</style>
</head>
<body>
<div id="root"></div>
<script src="../dist/App.js"></script>
</body>
</html>
+316
View File
@@ -0,0 +1,316 @@
/**
* K8s auto-scaler for creating and deleting agent pods based on complexity analysis
*/
import * as k8s from '@kubernetes/client-node';
import { ComplexityAnalysis } from '../ai/complexity-analyzer';
export interface AgentPodConfig {
namespace: string;
image: string;
orchestratorUrl: string;
gitUrl: string;
workspaceId: string;
agentId: string;
cpuRequest?: string;
memoryRequest?: string;
cpuLimit?: string;
memoryLimit?: string;
}
export interface ScalingResult {
success: boolean;
createdPods: string[];
deletedPods: string[];
errors: string[];
}
export class AutoScaler {
private kc: k8s.KubeConfig;
private coreApi: k8s.CoreV1Api;
private namespace: string;
constructor(kubeConfig: k8s.KubeConfig, namespace: string = 'default') {
this.kc = kubeConfig;
this.coreApi = this.kc.makeApiClient(k8s.CoreV1Api);
this.namespace = namespace;
}
/**
* Scale up agents based on complexity analysis
*/
async scaleUp(
complexity: ComplexityAnalysis,
baseConfig: Omit<AgentPodConfig, 'agentId'>
): Promise<ScalingResult> {
const result: ScalingResult = {
success: true,
createdPods: [],
deletedPods: [],
errors: [],
};
const agentCount = complexity.agentCount;
for (let i = 0; i < agentCount; i++) {
const agentId = `agent-${baseConfig.workspaceId}-${i}-${Date.now()}`;
const podConfig: AgentPodConfig = {
...baseConfig,
agentId,
};
try {
const podName = await this.createAgentPod(podConfig);
result.createdPods.push(podName);
} catch (error) {
const errorMsg = `Failed to create agent ${agentId}: ${error instanceof Error ? error.message : String(error)}`;
result.errors.push(errorMsg);
result.success = false;
}
}
return result;
}
/**
* Create a single agent pod
*/
async createAgentPod(config: AgentPodConfig): Promise<string> {
const podName = config.agentId;
const podManifest: k8s.V1Pod = {
apiVersion: 'v1',
kind: 'Pod',
metadata: {
name: podName,
namespace: config.namespace,
labels: {
app: 'heicode-swarm-agent',
'workspace-id': config.workspaceId,
'agent-id': config.agentId,
},
},
spec: {
serviceAccountName: 'swarm-agent',
restartPolicy: 'Never',
containers: [
{
name: 'agent',
image: config.image,
env: [
{
name: 'ORCHESTRATOR_URL',
value: config.orchestratorUrl,
},
{
name: 'AGENT_ID',
value: config.agentId,
},
{
name: 'GIT_URL',
value: config.gitUrl,
},
{
name: 'WORKSPACE_ID',
value: config.workspaceId,
},
{
name: 'ANTHROPIC_API_KEY',
valueFrom: {
secretKeyRef: {
name: 'anthropic-api-key',
key: 'api-key',
},
},
},
],
resources: {
requests: {
cpu: config.cpuRequest || '500m',
memory: config.memoryRequest || '512Mi',
},
limits: {
cpu: config.cpuLimit || '1000m',
memory: config.memoryLimit || '1Gi',
},
},
},
],
},
};
try {
await this.coreApi.createNamespacedPod(config.namespace, podManifest);
return podName;
} catch (error) {
throw new Error(`Failed to create pod ${podName}: ${error instanceof Error ? error.message : String(error)}`);
}
}
/**
* Delete agent pods by workspace ID
*/
async deleteAgentPods(workspaceId: string): Promise<ScalingResult> {
const result: ScalingResult = {
success: true,
createdPods: [],
deletedPods: [],
errors: [],
};
try {
// List pods with workspace-id label
const response = await this.coreApi.listNamespacedPod(
this.namespace,
undefined,
undefined,
undefined,
undefined,
`workspace-id=${workspaceId}`
);
const pods = response.body.items;
for (const pod of pods) {
const podName = pod.metadata?.name;
if (!podName) continue;
try {
await this.coreApi.deleteNamespacedPod(podName, this.namespace);
result.deletedPods.push(podName);
} catch (error) {
const errorMsg = `Failed to delete pod ${podName}: ${error instanceof Error ? error.message : String(error)}`;
result.errors.push(errorMsg);
result.success = false;
}
}
} catch (error) {
result.success = false;
result.errors.push(`Failed to list pods: ${error instanceof Error ? error.message : String(error)}`);
}
return result;
}
/**
* Delete a specific agent pod
*/
async deleteAgentPod(agentId: string): Promise<void> {
try {
await this.coreApi.deleteNamespacedPod(agentId, this.namespace);
} catch (error) {
throw new Error(`Failed to delete pod ${agentId}: ${error instanceof Error ? error.message : String(error)}`);
}
}
/**
* List all agent pods for a workspace
*/
async listAgentPods(workspaceId: string): Promise<k8s.V1Pod[]> {
try {
const response = await this.coreApi.listNamespacedPod(
this.namespace,
undefined,
undefined,
undefined,
undefined,
`workspace-id=${workspaceId}`
);
return response.body.items;
} catch (error) {
throw new Error(`Failed to list pods: ${error instanceof Error ? error.message : String(error)}`);
}
}
/**
* Get pod status
*/
async getPodStatus(podName: string): Promise<string | undefined> {
try {
const response = await this.coreApi.readNamespacedPod(podName, this.namespace);
return response.body.status?.phase;
} catch (error) {
throw new Error(`Failed to get pod status: ${error instanceof Error ? error.message : String(error)}`);
}
}
/**
* Wait for pods to be ready
*/
async waitForPodsReady(
workspaceId: string,
timeoutMs: number = 120000
): Promise<{ ready: boolean; pods: string[] }> {
const startTime = Date.now();
while (Date.now() - startTime < timeoutMs) {
const pods = await this.listAgentPods(workspaceId);
const allReady = pods.every(pod => {
const phase = pod.status?.phase;
return phase === 'Running' || phase === 'Succeeded';
});
if (allReady) {
return {
ready: true,
pods: pods.map(p => p.metadata?.name || '').filter(Boolean),
};
}
// Wait 2 seconds before checking again
await new Promise(resolve => setTimeout(resolve, 2000));
}
return {
ready: false,
pods: [],
};
}
/**
* Scale down idle agents
*/
async scaleDownIdle(workspaceId: string, idleThresholdMs: number = 300000): Promise<ScalingResult> {
const result: ScalingResult = {
success: true,
createdPods: [],
deletedPods: [],
errors: [],
};
try {
const pods = await this.listAgentPods(workspaceId);
const now = Date.now();
for (const pod of pods) {
const podName = pod.metadata?.name;
if (!podName) continue;
// Check if pod has been idle (check creation time as proxy)
const creationTime = pod.metadata?.creationTimestamp;
if (!creationTime) continue;
const createdAt = new Date(creationTime).getTime();
const idleTime = now - createdAt;
// If pod is old enough and in Running state, consider it idle
// In production, you'd check actual agent activity via orchestrator
if (idleTime > idleThresholdMs && pod.status?.phase === 'Running') {
try {
await this.deleteAgentPod(podName);
result.deletedPods.push(podName);
} catch (error) {
const errorMsg = `Failed to delete idle pod ${podName}: ${error instanceof Error ? error.message : String(error)}`;
result.errors.push(errorMsg);
result.success = false;
}
}
}
} catch (error) {
result.success = false;
result.errors.push(`Failed to scale down: ${error instanceof Error ? error.message : String(error)}`);
}
return result;
}
}
+171
View File
@@ -0,0 +1,171 @@
import * as k8s from '@kubernetes/client-node';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
export interface KubeConfigValidationResult {
valid: boolean;
error?: string;
context?: string;
cluster?: string;
}
export class KubeConfigLoader {
private kc: k8s.KubeConfig;
constructor() {
this.kc = new k8s.KubeConfig();
}
/**
* Load kubeconfig from default locations or KUBECONFIG env var
* Priority: KUBECONFIG env var > ~/.kube/config
*/
public load(): void {
const kubeconfigPath = this.getKubeConfigPath();
if (!kubeconfigPath) {
throw new Error('No kubeconfig found. Set KUBECONFIG env var or create ~/.kube/config');
}
if (!fs.existsSync(kubeconfigPath)) {
throw new Error(`Kubeconfig file not found at: ${kubeconfigPath}`);
}
try {
this.kc.loadFromFile(kubeconfigPath);
} catch (error) {
throw new Error(`Failed to load kubeconfig: ${error instanceof Error ? error.message : String(error)}`);
}
}
/**
* Validate the loaded kubeconfig
*/
public validate(): KubeConfigValidationResult {
try {
const currentContext = this.kc.getCurrentContext();
if (!currentContext) {
return {
valid: false,
error: 'No current context set in kubeconfig'
};
}
const cluster = this.kc.getCurrentCluster();
if (!cluster) {
return {
valid: false,
error: 'No cluster found for current context'
};
}
const user = this.kc.getCurrentUser();
if (!user) {
return {
valid: false,
error: 'No user found for current context'
};
}
return {
valid: true,
context: currentContext,
cluster: cluster.name
};
} catch (error) {
return {
valid: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Get the KubeConfig instance for use with K8s clients
*/
public getKubeConfig(): k8s.KubeConfig {
return this.kc;
}
/**
* Get the current context name
*/
public getCurrentContext(): string | null {
return this.kc.getCurrentContext();
}
/**
* Get the current cluster name
*/
public getCurrentCluster(): string | null {
const cluster = this.kc.getCurrentCluster();
return cluster ? cluster.name : null;
}
/**
* Test connection to the K8s cluster
*/
public async testConnection(): Promise<{ success: boolean; error?: string }> {
try {
const coreApi = this.kc.makeApiClient(k8s.CoreV1Api);
await coreApi.listNamespace();
return { success: true };
} catch (error) {
return {
success: false,
error: error instanceof Error ? error.message : String(error)
};
}
}
/**
* Get kubeconfig path from env var or default location
*/
private getKubeConfigPath(): string | null {
// Check KUBECONFIG env var first
const kubeconfigEnv = process.env.KUBECONFIG;
if (kubeconfigEnv) {
// KUBECONFIG can contain multiple paths separated by colons
const paths = kubeconfigEnv.split(path.delimiter);
for (const p of paths) {
if (fs.existsSync(p)) {
return p;
}
}
}
// Fall back to default location
const defaultPath = path.join(os.homedir(), '.kube', 'config');
if (fs.existsSync(defaultPath)) {
return defaultPath;
}
return null;
}
}
/**
* Convenience function to load and validate kubeconfig
*/
export async function loadAndValidateKubeConfig(): Promise<{
loader: KubeConfigLoader;
validation: KubeConfigValidationResult;
}> {
const loader = new KubeConfigLoader();
try {
loader.load();
} catch (error) {
return {
loader,
validation: {
valid: false,
error: error instanceof Error ? error.message : String(error)
}
};
}
const validation = loader.validate();
return { loader, validation };
}
+109
View File
@@ -0,0 +1,109 @@
import { app, BrowserWindow } from 'electron';
import * as path from 'path';
import { KubeConfigLoader } from './k8s/kubeconfig-loader';
let mainWindow: BrowserWindow | null = null;
function createWindow(): void {
mainWindow = new BrowserWindow({
width: 1200,
height: 800,
webPreferences: {
nodeIntegration: true,
contextIsolation: false,
preload: path.join(__dirname, 'preload.js')
},
title: 'HeiCode Swarm',
backgroundColor: '#1e1e1e'
});
// In development, load from webpack dev server
// In production, load from built files
if (process.env.NODE_ENV === 'development') {
mainWindow.loadURL('http://localhost:3000');
mainWindow.webContents.openDevTools();
} else {
mainWindow.loadFile(path.join(__dirname, '../renderer/index.html'));
}
mainWindow.on('closed', () => {
mainWindow = null;
});
}
async function initializeApp(): Promise<void> {
try {
// Load and validate kubeconfig on startup
const kubeLoader = new KubeConfigLoader();
kubeLoader.load();
const validation = kubeLoader.validate();
if (!validation.valid) {
console.error('Kubeconfig validation failed:', validation.error);
// Show error dialog to user
const { dialog } = require('electron');
await dialog.showErrorBox(
'Kubernetes Configuration Error',
`Failed to load kubeconfig: ${validation.error}\n\nPlease ensure you have a valid kubeconfig at ~/.kube/config or set the KUBECONFIG environment variable.`
);
app.quit();
return;
}
console.log('Kubeconfig loaded successfully');
console.log('Current context:', validation.context);
console.log('Current cluster:', validation.cluster);
// Test connection
const connectionTest = await kubeLoader.testConnection();
if (!connectionTest.success) {
console.warn('Failed to connect to Kubernetes cluster:', connectionTest.error);
// Don't quit, but warn the user
const { dialog } = require('electron');
await dialog.showMessageBox({
type: 'warning',
title: 'Kubernetes Connection Warning',
message: 'Could not connect to Kubernetes cluster',
detail: `Error: ${connectionTest.error}\n\nThe application will start, but you may not be able to create agent pods.`
});
} else {
console.log('Successfully connected to Kubernetes cluster');
}
createWindow();
} catch (error) {
console.error('Failed to initialize application:', error);
const { dialog } = require('electron');
await dialog.showErrorBox(
'Initialization Error',
`Failed to start application: ${error instanceof Error ? error.message : String(error)}`
);
app.quit();
}
}
// App lifecycle events
app.on('ready', initializeApp);
app.on('window-all-closed', () => {
// On macOS, keep app running until user explicitly quits
if (process.platform !== 'darwin') {
app.quit();
}
});
app.on('activate', () => {
// On macOS, recreate window when dock icon is clicked
if (mainWindow === null) {
createWindow();
}
});
// Handle uncaught exceptions
process.on('uncaughtException', (error) => {
console.error('Uncaught exception:', error);
});
process.on('unhandledRejection', (reason, promise) => {
console.error('Unhandled rejection at:', promise, 'reason:', reason);
});
+10
View File
@@ -0,0 +1,10 @@
module.exports = {
content: [
"./src/**/*.{js,jsx,ts,tsx}",
"./src/index.html"
],
theme: {
extend: {},
},
plugins: [],
}
+22
View File
@@ -0,0 +1,22 @@
{
"compilerOptions": {
"target": "ES2020",
"module": "commonjs",
"lib": ["ES2020", "DOM"],
"jsx": "react",
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"moduleResolution": "node",
"allowSyntheticDefaultImports": true,
"declaration": true,
"declarationMap": true,
"sourceMap": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "release"]
}
+176
View File
@@ -0,0 +1,176 @@
# 交付说明(Delivery Guide)
本文件说明 **Agent Swarm(agent_swarm_v5)** 的交付物、构建/部署方式、配置项、验收方法,以及本次交付对各链路的影响与合规要求。系统总体介绍见根目录 [README.md](../README.md)。
---
## 一、交付物清单
| 类别 | 内容 | 位置 |
|---|---|---|
| 编排器 | FastAPI 编排服务(分解/派发/协作/评审/汇总、Manager 对接) | `orchestrator/` |
| 执行单元 | Agent 运行时(WebSocket、并发、OpenAI 执行、Git 提交) | `agent/` |
| 桌面客户端 | Electron + React 客户端(提交/分析/看板/聚合) | `desktop-client/` |
| 容器镜像定义 | Agent 与编排器的 Dockerfile | `Dockerfile.agent`、`Dockerfile.orchestrator` |
| 部署清单 | Redis、编排器、Agent、RBAC、监控等 K8s 清单 | `k8s/` |
| 部署脚本 | 一键部署脚本 | `scripts/deploy-orchestrator.sh`、`scripts/deploy-agent.sh` |
| 测试 | 契约校验与工作流冒烟测试 | `scripts/test-runtime-contract.py`、`scripts/test-merge-smoke.py` |
| 数据集 | 复杂度分析标注数据集 | `test-data/complexity-dataset.json` |
| 文档 | 各组件 README 与本交付说明 | `README.md`、各目录 `README.md`、`doc/` |
> 交付不包含任何密钥、令牌、`.env` 或证书。凭据由部署环境 / `secret_ref` / Key Vault 注入。
---
## 二、构建
### 容器镜像
```bash
# 编排器镜像
docker build -f Dockerfile.orchestrator -t agent-swarm-orchestrator:<tag> .
# Agent 镜像
docker build -f Dockerfile.agent -t agent-swarm-agent:<tag> .
```
### 桌面客户端
```bash
cd desktop-client
npm install
npm run build
```
---
## 三、部署(Kubernetes)
按以下顺序部署(具体清单见 `k8s/`):
```bash
# 1) 命名空间与 RBAC
kubectl apply -f k8s/test-namespace.yaml
kubectl apply -f k8s/rbac/
# 2) Redis(权威状态存储)
kubectl apply -f k8s/redis-statefulset.yaml
# 3) 编排器
kubectl apply -f k8s/orchestrator-deployment.yaml
# 4) Agent(按需扩缩)
kubectl apply -f k8s/agent-deployment-v2.yaml
```
也可使用脚本:`scripts/deploy-orchestrator.sh`、`scripts/deploy-agent.sh`。
**凭据(必须经 secret_ref / K8s Secret 注入,禁止明文)**:
- 模型访问:`OPENAI_API_KEY`(OpenAI 兼容;如使用自定义端点另配 `OPENAI_API_BASE`)。
- Git 推送:`GIT_USERNAME` / `GIT_PASSWORD`(或 `GIT_TOKEN`)。
- 服务间鉴权与回调签名:`AGENT_RUNTIME_SERVICE_TOKEN`、`AGENT_CALLBACK_SERVICE_TOKEN`、`AGENT_CALLBACK_SIGNING_SECRET`。
> 注意:模型后端为 **OpenAI 兼容**。部署时请确保以 `OPENAI_*` 凭据注入 Agent 与编排器,并据此核对/更新 Secret 与清单中的环境变量。
---
## 四、配置(环境变量与开关)
| 变量 | 作用 | 备注 |
|---|---|---|
| `REDIS_HOST` / `REDIS_PORT` / `REDIS_DB` | Redis 连接 | 生产必需 |
| `REDIS_FAKE` / `ALLOW_MEMORY_STORE` | 内存回退 | **仅开发/CI**,生产禁用 |
| `ENABLE_PLANNER_FALLBACK` | 无 Manager 分工时启用规划回退 | 默认关闭 |
| `ENABLE_REVIEW_LOOP` / `MAX_REVIEW_CYCLES` | 主控评审/重做循环 + 汇总 | 默认关闭 / 默认 2 |
| `ENABLE_SUBTASK_HANDOFF` | 动态子任务移交 | 默认关闭 |
| `OPENAI_API_KEY` / `OPENAI_API_BASE` / `OPENAI_MODEL` | 模型访问 | Agent 与编排器规划/评审共用 |
| `AGENT_RUNTIME_SERVICE_TOKEN` 等 | 鉴权与回调签名 | 与 Manager 契约相关 |
| `MAX_CONCURRENT_TASKS` / `TASK_TIMEOUT_SECONDS` | Agent 并发与超时 | Agent 侧 |
| `OTEL_EXPORTER_OTLP_ENDPOINT` 等 | 链路追踪 | 可选 |
各组件完整变量见对应 README:[orchestrator](../orchestrator/README.md)、[agent](../agent/README.md)。
---
## 五、验收与验证
### 1) 自动化测试(无需 Redis 服务器,也无需模型密钥)
```bash
# 在 agent_swarm_v5 目录下
set "REDIS_FAKE=1"
python scripts/test-runtime-contract.py # Manager 契约校验
python scripts/test-merge-smoke.py # 机制级冒烟测试(评审 / 协作 / 汇总等,32 项)
```
验收标准:两者均通过(contract checks passed / all merge smoke checks passed)。
### 2) 端到端工作流测试(自动、确定性、无需密钥)
```bash
python scripts/test-workflow-e2e.py
```
该脚本在进程内启动**真实编排器**(uvicorn + 内存存储),通过真实 WebSocket 接入一个**无需密钥的桩 Agent**(`scripts/stub_agent.py`),提交一个目标,并逐项断言完整工作流:
| 断言 | 对应工作流步骤 |
|---|---|
| 生成 3 个专家任务且角色为 实现/测试/文档 | 主控**分解**为多专家任务 |
| 每个任务均完成 | 子 Agent**执行** |
| 至少发生一次主控评审循环 | 主控**评审并退回重做** |
| 运行最终 completed | **循环直至达标** |
| 存在统一的最终汇总 | 专家产出**汇总**为最终回答 |
确定性来源:规划/评审/汇总被强制离线(静态分解 + 启发式评审 + 拼接汇总),桩 Agent 在首次「测试」任务返回冲突框架(unittest)、重做时返回对齐框架(pytest),从而稳定地触发**一次**评审重做后通过。
验收标准:输出 `workflow follows the expected sequence: ...`,退出码 0。
### 3) 端到端工作流演示(真实 Agent + 模型)
启动编排器(开启 `REDIS_FAKE`、`ENABLE_PLANNER_FALLBACK`、`ENABLE_REVIEW_LOOP`)与至少一个真实 Agent(`.env` 提供 `OPENAI_API_KEY`),提交需求,按 `GET /api/swarms/{id}/tasks | /logs | /workflow` 确认:分解 → 派发执行 →(评审/重做)→ 汇总交付。详细步骤见根目录 [README.md](../README.md) 的「快速开始」。
> 辅助工具:`scripts/stub_agent.py` 是无需密钥的桩 Agent,可单独运行以对手动启动的编排器做 Tier-3 联调(设置 `ORCHESTRATOR_URL` / `AGENT_ID` / `AGENT_CAPABILITIES` 后 `python scripts/stub_agent.py`)。
---
## 六、交付影响说明
依据组织规范,本次交付影响范围如下(PR 中需复述):
| 链路 | 是否影响 | 说明 |
|---|---|---|
| Client(桌面客户端) | 是 | 提交/分析/看板/聚合 |
| Manager | 否(契约保持) | 面向 Manager 的接口、回调、审批链均保留 |
| Swarm(编排器 + Agent) | 是 | 新增分解/协作/评审/汇总能力,均默认关闭 |
| Agent | 是 | OpenAI 执行、并发、重连、超时、取消、协作 |
| CodeGW | 否 | 无改动 |
| 计费 | 是(字段保留) | 模型用量归属(`usage` 与归属头)保留;模型后端改为 OpenAI 兼容 |
| 密钥 | 是 | 需以 `secret_ref` 注入 `OPENAI_*` 等凭据 |
| 审计 | 否(契约保持) | 事件与签名回调结构保留 |
| 发布链路 | 否 | 不涉及非 dev release 仓库的业务改动 |
---
## 七、安全与合规
- **禁止**将密钥、令牌、云凭据、`.env`、证书写入代码、日志、Markdown 或提交记录。
- 凭据仅经部署环境 / `secret_ref` / Key Vault 注入;`.env` 已被 `.gitignore` 忽略,仅供本地开发。
- 涉及鉴权、审批链、计费、审计的改动须遵循 Manager 安全规则与组织标准。
- 生产环境必须使用真实 Redis;严禁开启内存回退(`REDIS_FAKE` / `ALLOW_MEMORY_STORE`)。
---
## 八、回退与运维
- **回退**:所有新增工作流能力(规划回退、评审循环、子任务移交)均由环境开关控制,关闭后行为回到基础的「Manager 编排 + 单任务执行」语义。
- **可观测性**:编排器与 Agent 暴露 Prometheus 指标(编排器 `GET /metrics`),可接入 `k8s/prometheus-config.yaml` 与 `k8s/grafana-dashboard.json`。
- **健康检查**:`GET /` 与 `GET /health`(含 Redis 连接状态与活跃连接数)。
---
## 九、已知事项与遗留项
以下为已知的陈旧/遗留资产,PR 中应一并说明,并按需清理:
1. **`k8s/orchestrator-source-configmap.yaml` 为陈旧快照(请勿用于当前部署)。**
该 ConfigMap 内嵌的是旧版编排器源码(无规划/评审/汇总)。正式部署走**镜像**方式(`Dockerfile.orchestrator` → `k8s/orchestrator-deployment.yaml`,`swarm-orchestrator:latest`)。如确需源码挂载部署,请从 `orchestrator/` 重新生成:
```bash
kubectl create configmap orchestrator-source --from-file=orchestrator/ --dry-run=client -o yaml
```
否则建议删除该文件。文件顶部已加显著警告。
2. **`scripts/test-agent.py` 为遗留测试脚本。**
它针对旧版 Anthropic 初始化流程(使用占位密钥 `sk-ant-test-...`),与当前 OpenAI 运行时不一致,且**不属于**当前测试套件(`scripts/test-runtime-contract.py`、`scripts/test-merge-smoke.py`、`scripts/test-workflow-e2e.py`)。后续应更新为 OpenAI 流程或移除。
+94
View File
@@ -0,0 +1,94 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: agent-runtime-config
namespace: swarm-system
data:
ORCHESTRATOR_URL: "ws://orchestrator-service:8000"
WORKSPACE_DIR: "/workspace"
GIT_REPO_URL: "git://git-test-server.swarm-system.svc.cluster.local/swarm-test.git"
GIT_BASE_BRANCH: "main"
AGENT_CAPABILITIES: "code_generation,git,python,filesystem,file-operations,editing,validation,verification,shell,reporting,repository-inspection"
ENABLE_SUBTASK_HANDOFF: "false"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: agent-full
namespace: swarm-system
labels:
app: agent-full
component: worker
spec:
replicas: 1
selector:
matchLabels:
app: agent-full
template:
metadata:
labels:
app: agent-full
component: worker
spec:
serviceAccountName: orchestrator-sa
containers:
- name: agent
image: python:3.11-slim
command:
- /bin/bash
- -c
- |
set -euxo pipefail
apt-get update
apt-get install -y git
pip install --no-cache-dir -r /app/agent/requirements.txt
cd /app
python -m agent.main
envFrom:
- configMapRef:
name: agent-runtime-config
env:
- name: AGENT_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: OPENAI_API_KEY
valueFrom:
secretKeyRef:
name: openai-api-key
key: api-key
- name: OPENAI_API_BASE
valueFrom:
secretKeyRef:
name: openai-api-key
key: api-base
optional: true
- name: OPENAI_MODEL
valueFrom:
secretKeyRef:
name: openai-api-key
key: model
optional: true
- name: PYTHONUNBUFFERED
value: "1"
- name: DEBIAN_FRONTEND
value: noninteractive
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
volumeMounts:
- name: agent-source
mountPath: /app/agent
- name: workspace
mountPath: /workspace
volumes:
- name: agent-source
configMap:
name: agent-source
- name: workspace
emptyDir: {}
restartPolicy: Always
+241
View File
@@ -0,0 +1,241 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: agent
namespace: swarm-system
labels:
app: agent
component: worker
spec:
replicas: 1
selector:
matchLabels:
app: agent
template:
metadata:
labels:
app: agent
component: worker
spec:
serviceAccountName: orchestrator-sa
containers:
- name: agent
image: python:3.11-slim
command:
- /bin/bash
- -c
- |
set -e
echo "Installing dependencies..."
pip install --no-cache-dir \
websockets \
prometheus-client \
gitpython \
openai
echo "Setting up agent code..."
mkdir -p /tmp/agent
cd /tmp/agent
echo "Starting agent..."
python3 -c "
import asyncio
import json
import logging
import os
import sys
import time
import uuid
from typing import Optional
import websockets
from websockets.exceptions import ConnectionClosed
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
class SimpleAgent:
def __init__(self, orchestrator_url: str, agent_id: str):
self.orchestrator_url = orchestrator_url
self.agent_id = agent_id
self.api_key = os.getenv('OPENAI_API_KEY')
self.api_base = os.getenv('OPENAI_API_BASE', 'https://api.openai.com/v1')
self.model = os.getenv('OPENAI_MODEL', 'gpt-4o-mini')
self.ws = None
self.running = True
async def connect(self):
logger.info(f'Connecting to orchestrator at {self.orchestrator_url}')
self.ws = await websockets.connect(self.orchestrator_url)
# Register with orchestrator
register_msg = {
'type': 'register',
'agent_id': self.agent_id,
'capabilities': ['code_generation', 'code_review'],
'status': 'idle'
}
await self.ws.send(json.dumps(register_msg))
logger.info(f'Agent {self.agent_id} registered')
async def send_heartbeat(self):
while self.running:
try:
if self.ws:
heartbeat_msg = {
'type': 'heartbeat',
'agent_id': self.agent_id,
'status': 'idle',
'timestamp': time.time()
}
await self.ws.send(json.dumps(heartbeat_msg))
logger.debug(f'Heartbeat sent')
except Exception as e:
logger.error(f'Heartbeat error: {e}')
await asyncio.sleep(15)
async def execute_task(self, task_data):
task_id = task_data.get('task_id')
description = task_data.get('description')
logger.info(f'Executing task {task_id}: {description}')
try:
# Use OpenAI-compatible API
from openai import OpenAI
client = OpenAI(
api_key=self.api_key,
base_url=self.api_base
)
response = client.chat.completions.create(
model=self.model,
messages=[
{'role': 'system', 'content': 'You are a helpful coding assistant. Generate clean, working code.'},
{'role': 'user', 'content': description}
],
max_tokens=2000
)
result = response.choices[0].message.content
# Send result back
result_msg = {
'type': 'task_result',
'task_id': task_id,
'agent_id': self.agent_id,
'status': 'completed',
'result': result,
'timestamp': time.time()
}
await self.ws.send(json.dumps(result_msg))
logger.info(f'Task {task_id} completed')
except Exception as e:
logger.error(f'Task execution failed: {e}')
error_msg = {
'type': 'task_result',
'task_id': task_id,
'agent_id': self.agent_id,
'status': 'failed',
'error': str(e),
'timestamp': time.time()
}
await self.ws.send(json.dumps(error_msg))
async def listen(self):
while self.running:
try:
message = await self.ws.recv()
data = json.loads(message)
msg_type = data.get('type')
logger.info(f'Received message: {msg_type}')
if msg_type == 'task_assignment':
await self.execute_task(data)
elif msg_type == 'ping':
pong_msg = {'type': 'pong', 'agent_id': self.agent_id}
await self.ws.send(json.dumps(pong_msg))
except ConnectionClosed:
logger.warning('Connection closed, reconnecting...')
await asyncio.sleep(5)
await self.connect()
except Exception as e:
logger.error(f'Listen error: {e}')
await asyncio.sleep(1)
async def run(self):
await self.connect()
# Start heartbeat task
heartbeat_task = asyncio.create_task(self.send_heartbeat())
# Start listening
await self.listen()
heartbeat_task.cancel()
async def main():
orchestrator_url = os.getenv('ORCHESTRATOR_URL', 'ws://orchestrator-service:8000/ws')
agent_id = os.getenv('AGENT_ID', f'agent-{uuid.uuid4().hex[:8]}')
logger.info(f'Starting agent {agent_id}')
logger.info(f'Orchestrator URL: {orchestrator_url}')
agent = SimpleAgent(orchestrator_url, agent_id)
await agent.run()
if __name__ == '__main__':
asyncio.run(main())
"
env:
- name: AGENT_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: ORCHESTRATOR_URL
value: "ws://orchestrator-service:8000/ws/$(AGENT_ID)"
- name: OPENAI_API_KEY
valueFrom:
secretKeyRef:
name: openai-api-key
key: api-key
- name: OPENAI_API_BASE
valueFrom:
secretKeyRef:
name: openai-api-key
key: api-base
optional: true
- name: OPENAI_MODEL
valueFrom:
secretKeyRef:
name: openai-api-key
key: model
optional: true
- name: PYTHONUNBUFFERED
value: "1"
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
volumeMounts:
- name: agent-source
mountPath: /app
- name: workspace
mountPath: /workspace
volumes:
- name: agent-source
configMap:
name: agent-source
- name: workspace
emptyDir: {}
restartPolicy: Always
+225
View File
@@ -0,0 +1,225 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: agent
namespace: swarm-system
labels:
app: agent
component: worker
spec:
replicas: 1
selector:
matchLabels:
app: agent
template:
metadata:
labels:
app: agent
component: worker
spec:
serviceAccountName: orchestrator-sa
containers:
- name: agent
image: python:3.11-slim
command:
- /bin/bash
- -c
- |
set -e
echo "Installing dependencies..."
pip install --no-cache-dir \
websockets \
openai \
prometheus-client
echo "Starting agent..."
python3 -c "
import asyncio
import json
import logging
import os
import sys
import time
import uuid
from typing import Optional
import websockets
from websockets.exceptions import ConnectionClosed
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
class SimpleAgent:
def __init__(self, orchestrator_url: str, agent_id: str):
self.orchestrator_url = orchestrator_url
self.agent_id = agent_id
self.api_key = os.getenv('OPENAI_API_KEY')
self.api_base = os.getenv('OPENAI_API_BASE', 'https://api.openai.com/v1')
self.model = os.getenv('OPENAI_MODEL', 'gpt-4o-mini')
self.ws = None
self.running = True
async def connect(self):
logger.info(f'Connecting to orchestrator at {self.orchestrator_url}')
self.ws = await websockets.connect(self.orchestrator_url)
# Register with orchestrator
register_msg = {
'type': 'register',
'agent_id': self.agent_id,
'capabilities': ['code_generation', 'code_review'],
'status': 'idle'
}
await self.ws.send(json.dumps(register_msg))
logger.info(f'Agent {self.agent_id} registered')
async def send_heartbeat(self):
while self.running:
try:
if self.ws:
heartbeat_msg = {
'type': 'heartbeat',
'agent_id': self.agent_id,
'status': 'idle',
'timestamp': time.time()
}
await self.ws.send(json.dumps(heartbeat_msg))
logger.debug(f'Heartbeat sent')
except Exception as e:
logger.error(f'Heartbeat error: {e}')
await asyncio.sleep(15)
async def execute_task(self, task_data):
task_id = task_data.get('task_id')
description = task_data.get('description')
logger.info(f'Executing task {task_id}: {description}')
try:
# Use OpenAI-compatible API
from openai import OpenAI
client = OpenAI(
api_key=self.api_key,
base_url=self.api_base
)
response = client.chat.completions.create(
model=self.model,
messages=[
{'role': 'system', 'content': 'You are a helpful coding assistant. Generate clean, working code.'},
{'role': 'user', 'content': description}
],
max_tokens=2000
)
result = response.choices[0].message.content
# Send result back
result_msg = {
'type': 'task_result',
'task_id': task_id,
'agent_id': self.agent_id,
'status': 'completed',
'result': result,
'timestamp': time.time()
}
await self.ws.send(json.dumps(result_msg))
logger.info(f'Task {task_id} completed')
except Exception as e:
logger.error(f'Task execution failed: {e}')
error_msg = {
'type': 'task_result',
'task_id': task_id,
'agent_id': self.agent_id,
'status': 'failed',
'error': str(e),
'timestamp': time.time()
}
await self.ws.send(json.dumps(error_msg))
async def listen(self):
while self.running:
try:
message = await self.ws.recv()
data = json.loads(message)
msg_type = data.get('type')
logger.info(f'Received message: {msg_type}')
if msg_type == 'task_assignment':
await self.execute_task(data)
elif msg_type == 'ping':
pong_msg = {'type': 'pong', 'agent_id': self.agent_id}
await self.ws.send(json.dumps(pong_msg))
except ConnectionClosed:
logger.warning('Connection closed, reconnecting...')
await asyncio.sleep(5)
await self.connect()
except Exception as e:
logger.error(f'Listen error: {e}')
await asyncio.sleep(1)
async def run(self):
await self.connect()
# Start heartbeat task
heartbeat_task = asyncio.create_task(self.send_heartbeat())
# Start listening
await self.listen()
heartbeat_task.cancel()
async def main():
orchestrator_url = os.getenv('ORCHESTRATOR_URL', 'ws://orchestrator-service:8000/ws')
agent_id = os.getenv('AGENT_ID', f'agent-{uuid.uuid4().hex[:8]}')
logger.info(f'Starting agent {agent_id}')
logger.info(f'Orchestrator URL: {orchestrator_url}')
agent = SimpleAgent(orchestrator_url, agent_id)
await agent.run()
if __name__ == '__main__':
asyncio.run(main())
"
env:
- name: AGENT_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: ORCHESTRATOR_URL
value: "ws://orchestrator-service:8000/ws/$(AGENT_ID)"
- name: OPENAI_API_KEY
valueFrom:
secretKeyRef:
name: openai-api-key
key: api-key
- name: OPENAI_API_BASE
valueFrom:
secretKeyRef:
name: openai-api-key
key: api-base
optional: true
- name: OPENAI_MODEL
valueFrom:
secretKeyRef:
name: openai-api-key
key: model
optional: true
- name: PYTHONUNBUFFERED
value: "1"
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
restartPolicy: Always
+158
View File
@@ -0,0 +1,158 @@
apiVersion: v1
kind: Pod
metadata:
name: agent-${AGENT_ID}
labels:
app: swarm-agent
agent-id: ${AGENT_ID}
spec:
restartPolicy: Never
# Init container to clone workspace
initContainers:
- name: git-clone
image: alpine/git:latest
command:
- sh
- -c
- |
if [ -n "$GIT_REPO_URL" ]; then
echo "Cloning repository from $GIT_REPO_URL"
git clone $GIT_REPO_URL /workspace
else
echo "No GIT_REPO_URL provided, skipping clone"
mkdir -p /workspace
fi
env:
- name: GIT_REPO_URL
valueFrom:
configMapKeyRef:
name: swarm-config
key: git-repo-url
optional: true
volumeMounts:
- name: workspace
mountPath: /workspace
containers:
- name: agent
image: swarm-agent:latest
imagePullPolicy: IfNotPresent
env:
# Agent configuration
- name: AGENT_ID
value: ${AGENT_ID}
- name: AGENT_CAPABILITIES
value: ${AGENT_CAPABILITIES}
# Orchestrator connection
- name: ORCHESTRATOR_URL
value: "ws://orchestrator-service:8000"
# Workspace configuration
- name: WORKSPACE_DIR
value: "/workspace"
- name: GIT_REPO_URL
valueFrom:
configMapKeyRef:
name: swarm-config
key: git-repo-url
optional: true
# Model API key (OpenAI-compatible) — inject via secret_ref, never commit
- name: OPENAI_API_KEY
valueFrom:
secretKeyRef:
name: openai-secret
key: api-key
# Model configuration (OpenAI-compatible)
- name: OPENAI_API_BASE
valueFrom:
configMapKeyRef:
name: swarm-config
key: openai-api-base
optional: true
- name: OPENAI_MODEL
valueFrom:
configMapKeyRef:
name: swarm-config
key: openai-model
optional: true
# Git credentials for push access
- name: GIT_USERNAME
valueFrom:
secretKeyRef:
name: git-credentials
key: username
optional: true
- name: GIT_PASSWORD
valueFrom:
secretKeyRef:
name: git-credentials
key: password
optional: true
volumeMounts:
- name: workspace
mountPath: /workspace
# Resource limits
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "2Gi"
cpu: "1000m"
# Health check
livenessProbe:
exec:
command:
- python
- -c
- "import sys; sys.exit(0)"
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
volumes:
- name: workspace
emptyDir: {}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: swarm-config
data:
git-repo-url: "" # Set this to your Git repository URL
openai-model: "gpt-4o-mini"
openai-api-base: "https://api.openai.com/v1" # Override for an OpenAI-compatible endpoint
---
apiVersion: v1
kind: Secret
metadata:
name: openai-secret
type: Opaque
stringData:
api-key: "" # Set via secret_ref / kubectl — do NOT commit a real key
---
apiVersion: v1
kind: Secret
metadata:
name: git-credentials
type: Opaque
stringData:
username: "" # Set this to your Git username
password: "" # Set this to your Git password or token
+90
View File
@@ -0,0 +1,90 @@
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: agent-image-puller
namespace: swarm-system
labels:
app: agent-image-puller
component: optimization
spec:
selector:
matchLabels:
app: agent-image-puller
template:
metadata:
labels:
app: agent-image-puller
spec:
# Run on all nodes including control plane
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
# Use host network to avoid CNI overhead
hostNetwork: true
initContainers:
# Pre-pull the agent image
- name: pull-agent-image
image: ghcr.io/heicode/swarm-agent:latest
command: ['sh', '-c', 'echo "Agent image pulled successfully"']
resources:
limits:
memory: "128Mi"
cpu: "100m"
# Pre-pull common base images
- name: pull-python-base
image: python:3.11-slim
command: ['sh', '-c', 'echo "Python base image pulled"']
resources:
limits:
memory: "128Mi"
cpu: "100m"
containers:
# Keep-alive container (does nothing but keeps DaemonSet running)
- name: pause
image: gcr.io/google_containers/pause:3.9
resources:
limits:
memory: "32Mi"
cpu: "10m"
requests:
memory: "16Mi"
cpu: "5m"
# Optional: Periodic re-pull to get latest images
- name: periodic-puller
image: docker:24-cli
command:
- /bin/sh
- -c
- |
while true; do
echo "Checking for image updates..."
# This would require docker socket mount in production
# For now, just sleep
sleep 3600 # Re-check every hour
done
resources:
limits:
memory: "64Mi"
cpu: "50m"
volumeMounts:
- name: docker-socket
mountPath: /var/run/docker.sock
readOnly: true
volumes:
- name: docker-socket
hostPath:
path: /var/run/docker.sock
type: Socket
# Ensure DaemonSet runs before agent pods
priorityClassName: system-node-critical
+91
View File
@@ -0,0 +1,91 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: git-test-server
namespace: swarm-system
labels:
app: git-test-server
spec:
replicas: 1
selector:
matchLabels:
app: git-test-server
template:
metadata:
labels:
app: git-test-server
spec:
initContainers:
- name: init-repo
image: alpine:3.20
command:
- /bin/sh
- -c
- |
set -e
apk add --no-cache git >/dev/null
rm -rf /git/swarm-test.git /tmp/swarm-test-src
mkdir -p /tmp/swarm-test-src
cd /tmp/swarm-test-src
git init -b main
git config user.name "Swarm Test"
git config user.email "swarm-test@example.com"
cat > hello.py <<'PY'
def hello_world():
return "hello"
PY
git add hello.py
git commit -m "Initial test repository"
mkdir -p /git
git clone --bare /tmp/swarm-test-src /git/swarm-test.git
git -C /git/swarm-test.git config daemon.receivepack true
touch /git/swarm-test.git/git-daemon-export-ok
volumeMounts:
- name: git-data
mountPath: /git
containers:
- name: git
image: alpine:3.20
command:
- /bin/sh
- -c
- |
set -e
apk add --no-cache git git-daemon >/dev/null
git daemon \
--verbose \
--reuseaddr \
--base-path=/git \
--export-all \
--enable=receive-pack \
/git
ports:
- containerPort: 9418
name: git
volumeMounts:
- name: git-data
mountPath: /git
volumes:
- name: git-data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: git-test-server
namespace: swarm-system
labels:
app: git-test-server
spec:
type: ClusterIP
ports:
- port: 9418
targetPort: git
protocol: TCP
name: git
selector:
app: git-test-server
+488
View File
@@ -0,0 +1,488 @@
{
"dashboard": {
"title": "HeiCode Swarm System Health",
"tags": ["swarm", "k8s", "agents"],
"timezone": "browser",
"schemaVersion": 16,
"version": 1,
"refresh": "10s",
"time": {
"from": "now-1h",
"to": "now"
},
"panels": [
{
"id": 1,
"title": "Active Agents",
"type": "stat",
"gridPos": {"x": 0, "y": 0, "w": 6, "h": 4},
"targets": [
{
"expr": "swarm:agents:active:total",
"legendFormat": "Active Agents"
}
],
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"thresholds": {
"mode": "absolute",
"steps": [
{"value": 0, "color": "green"},
{"value": 10, "color": "yellow"},
{"value": 20, "color": "red"}
]
}
}
}
},
{
"id": 2,
"title": "Task Completion Rate",
"type": "stat",
"gridPos": {"x": 6, "y": 0, "w": 6, "h": 4},
"targets": [
{
"expr": "swarm:tasks:completion_rate:5m",
"legendFormat": "Tasks/sec"
}
],
"fieldConfig": {
"defaults": {
"unit": "ops",
"decimals": 2
}
}
},
{
"id": 3,
"title": "Average Task Duration",
"type": "stat",
"gridPos": {"x": 12, "y": 0, "w": 6, "h": 4},
"targets": [
{
"expr": "swarm:tasks:duration:avg",
"legendFormat": "Avg Duration"
}
],
"fieldConfig": {
"defaults": {
"unit": "s",
"decimals": 1
}
}
},
{
"id": 4,
"title": "Pod Startup Time (P95)",
"type": "stat",
"gridPos": {"x": 18, "y": 0, "w": 6, "h": 4},
"targets": [
{
"expr": "swarm:pod:startup:p95",
"legendFormat": "P95 Startup"
}
],
"fieldConfig": {
"defaults": {
"unit": "s",
"decimals": 1,
"color": {"mode": "thresholds"},
"thresholds": {
"mode": "absolute",
"steps": [
{"value": 0, "color": "green"},
{"value": 30, "color": "yellow"},
{"value": 60, "color": "red"}
]
}
}
}
},
{
"id": 5,
"title": "Agent Count Over Time",
"type": "graph",
"gridPos": {"x": 0, "y": 4, "w": 12, "h": 8},
"targets": [
{
"expr": "swarm:agents:count:by_status",
"legendFormat": "{{status}}"
}
],
"yaxes": [
{"format": "short", "label": "Agents"},
{"format": "short"}
],
"legend": {"show": true, "alignAsTable": true, "rightSide": false}
},
{
"id": 6,
"title": "Task Metrics",
"type": "graph",
"gridPos": {"x": 12, "y": 4, "w": 12, "h": 8},
"targets": [
{
"expr": "rate(swarm_tasks_completed_total[5m])",
"legendFormat": "Completed"
},
{
"expr": "rate(swarm_tasks_failed_total[5m])",
"legendFormat": "Failed"
}
],
"yaxes": [
{"format": "ops", "label": "Tasks/sec"},
{"format": "short"}
]
},
{
"id": 7,
"title": "Handoff Latency Distribution",
"type": "graph",
"gridPos": {"x": 0, "y": 12, "w": 12, "h": 8},
"targets": [
{
"expr": "histogram_quantile(0.50, rate(swarm_handoff_duration_seconds_bucket[5m]))",
"legendFormat": "P50"
},
{
"expr": "histogram_quantile(0.95, rate(swarm_handoff_duration_seconds_bucket[5m]))",
"legendFormat": "P95"
},
{
"expr": "histogram_quantile(0.99, rate(swarm_handoff_duration_seconds_bucket[5m]))",
"legendFormat": "P99"
}
],
"yaxes": [
{"format": "s", "label": "Latency"},
{"format": "short"}
]
},
{
"id": 8,
"title": "Agent Creation Rate",
"type": "graph",
"gridPos": {"x": 12, "y": 12, "w": 12, "h": 8},
"targets": [
{
"expr": "swarm:agents:creation_rate:5m",
"legendFormat": "Creation Rate"
}
],
"yaxes": [
{"format": "ops", "label": "Agents/sec"},
{"format": "short"}
]
},
{
"id": 9,
"title": "System Resource Usage",
"type": "graph",
"gridPos": {"x": 0, "y": 20, "w": 12, "h": 8},
"targets": [
{
"expr": "sum(rate(container_cpu_usage_seconds_total{namespace=\"swarm-system\"}[5m]))",
"legendFormat": "CPU Usage"
},
{
"expr": "sum(container_memory_working_set_bytes{namespace=\"swarm-system\"}) / 1024 / 1024 / 1024",
"legendFormat": "Memory Usage (GB)"
}
],
"yaxes": [
{"format": "short", "label": "Resources"},
{"format": "short"}
]
},
{
"id": 10,
"title": "Redis Operations",
"type": "graph",
"gridPos": {"x": 12, "y": 20, "w": 12, "h": 8},
"targets": [
{
"expr": "rate(redis_commands_processed_total[5m])",
"legendFormat": "Commands/sec"
},
{
"expr": "redis_connected_clients",
"legendFormat": "Connected Clients"
}
],
"yaxes": [
{"format": "ops", "label": "Operations"},
{"format": "short"}
]
},
{
"id": 11,
"title": "Pod Status by Phase",
"type": "piechart",
"gridPos": {"x": 0, "y": 28, "w": 8, "h": 8},
"targets": [
{
"expr": "count by (phase) (kube_pod_status_phase{namespace=\"swarm-system\"})",
"legendFormat": "{{phase}}"
}
]
},
{
"id": 12,
"title": "Error Rate",
"type": "graph",
"gridPos": {"x": 8, "y": 28, "w": 8, "h": 8},
"targets": [
{
"expr": "rate(swarm_errors_total[5m])",
"legendFormat": "{{error_type}}"
}
],
"yaxes": [
{"format": "ops", "label": "Errors/sec"},
{"format": "short"}
],
"alert": {
"conditions": [
{
"evaluator": {"params": [0.1], "type": "gt"},
"operator": {"type": "and"},
"query": {"params": ["A", "5m", "now"]},
"reducer": {"params": [], "type": "avg"},
"type": "query"
}
],
"executionErrorState": "alerting",
"frequency": "60s",
"handler": 1,
"name": "High Error Rate",
"noDataState": "no_data",
"notifications": []
}
},
{
"id": 13,
"title": "Network I/O",
"type": "graph",
"gridPos": {"x": 16, "y": 28, "w": 8, "h": 8},
"targets": [
{
"expr": "rate(container_network_receive_bytes_total{namespace=\"swarm-system\"}[5m])",
"legendFormat": "RX {{pod}}"
},
{
"expr": "rate(container_network_transmit_bytes_total{namespace=\"swarm-system\"}[5m])",
"legendFormat": "TX {{pod}}"
}
],
"yaxes": [
{"format": "Bps", "label": "Bytes/sec"},
{"format": "short"}
]
},
{
"id": 14,
"title": "Top Agents by Duration",
"type": "table",
"gridPos": {"x": 0, "y": 36, "w": 12, "h": 8},
"targets": [
{
"expr": "topk(10, swarm_agent_duration_seconds)",
"format": "table",
"instant": true
}
],
"transformations": [
{
"id": "organize",
"options": {
"excludeByName": {"Time": true},
"indexByName": {},
"renameByName": {
"agent_id": "Agent ID",
"task_id": "Task ID",
"Value": "Duration (s)"
}
}
}
]
},
{
"id": 15,
"title": "Recent Failures",
"type": "table",
"gridPos": {"x": 12, "y": 36, "w": 12, "h": 8},
"targets": [
{
"expr": "topk(10, swarm_agent_failures)",
"format": "table",
"instant": true
}
],
"transformations": [
{
"id": "organize",
"options": {
"excludeByName": {"Time": true},
"indexByName": {},
"renameByName": {
"agent_id": "Agent ID",
"task_id": "Task ID",
"error_type": "Error Type",
"Value": "Count"
}
}
}
]
},
{
"id": 16,
"title": "Orchestrator Health",
"type": "stat",
"gridPos": {"x": 0, "y": 44, "w": 6, "h": 4},
"targets": [
{
"expr": "up{job=\"orchestrator\"}",
"legendFormat": "Status"
}
],
"fieldConfig": {
"defaults": {
"mappings": [
{"type": "value", "value": "0", "text": "DOWN"},
{"type": "value", "value": "1", "text": "UP"}
],
"color": {"mode": "thresholds"},
"thresholds": {
"mode": "absolute",
"steps": [
{"value": 0, "color": "red"},
{"value": 1, "color": "green"}
]
}
}
}
},
{
"id": 17,
"title": "Redis Health",
"type": "stat",
"gridPos": {"x": 6, "y": 44, "w": 6, "h": 4},
"targets": [
{
"expr": "up{job=\"redis\"}",
"legendFormat": "Status"
}
],
"fieldConfig": {
"defaults": {
"mappings": [
{"type": "value", "value": "0", "text": "DOWN"},
{"type": "value", "value": "1", "text": "UP"}
],
"color": {"mode": "thresholds"},
"thresholds": {
"mode": "absolute",
"steps": [
{"value": 0, "color": "red"},
{"value": 1, "color": "green"}
]
}
}
}
},
{
"id": 18,
"title": "Kubernetes Cluster Health",
"type": "stat",
"gridPos": {"x": 12, "y": 44, "w": 6, "h": 4},
"targets": [
{
"expr": "up{job=\"kubernetes-apiservers\"}",
"legendFormat": "API Server"
}
],
"fieldConfig": {
"defaults": {
"mappings": [
{"type": "value", "value": "0", "text": "DOWN"},
{"type": "value", "value": "1", "text": "UP"}
],
"color": {"mode": "thresholds"},
"thresholds": {
"mode": "absolute",
"steps": [
{"value": 0, "color": "red"},
{"value": 1, "color": "green"}
]
}
}
}
},
{
"id": 19,
"title": "Total System Cost (Estimated)",
"type": "stat",
"gridPos": {"x": 18, "y": 44, "w": 6, "h": 4},
"targets": [
{
"expr": "sum(swarm_agent_cost_usd)",
"legendFormat": "Total Cost"
}
],
"fieldConfig": {
"defaults": {
"unit": "currencyUSD",
"decimals": 2
}
}
}
],
"templating": {
"list": [
{
"name": "namespace",
"type": "query",
"query": "label_values(swarm_agent_status, namespace)",
"current": {"text": "swarm-system", "value": "swarm-system"},
"hide": 0,
"includeAll": false,
"multi": false,
"options": [],
"refresh": 1,
"regex": "",
"sort": 0
},
{
"name": "task_id",
"type": "query",
"query": "label_values(swarm_agent_status{namespace=\"$namespace\"}, task_id)",
"current": {"text": "All", "value": "$__all"},
"hide": 0,
"includeAll": true,
"multi": true,
"options": [],
"refresh": 1,
"regex": "",
"sort": 0
}
]
},
"annotations": {
"list": [
{
"datasource": "Prometheus",
"enable": true,
"expr": "ALERTS{alertstate=\"firing\"}",
"iconColor": "red",
"name": "Alerts",
"step": "60s",
"tagKeys": "alertname",
"textFormat": "{{alertname}}",
"titleFormat": "Alert"
}
]
}
}
}
+108
View File
@@ -0,0 +1,108 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: orchestrator-config
namespace: swarm-system
data:
REDIS_HOST: "redis-service"
REDIS_PORT: "6379"
LOG_LEVEL: "INFO"
SWARM_RUNTIME_SOURCE: "heicode-swarm-runtime"
SWARM_RUNTIME_PLATFORM: "aks"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: orchestrator
namespace: swarm-system
labels:
app: orchestrator
spec:
replicas: 1
selector:
matchLabels:
app: orchestrator
template:
metadata:
labels:
app: orchestrator
spec:
serviceAccountName: orchestrator-sa
containers:
- name: orchestrator
image: python:3.11-slim
command: ["/bin/bash", "-c"]
args:
- |
apt-get update && apt-get install -y git
pip install --no-cache-dir -r /app/orchestrator/requirements.txt
cd /app
python -m uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
ports:
- containerPort: 8000
name: http
envFrom:
- configMapRef:
name: orchestrator-config
env:
- name: AGNET_RUNTIME_SERVICE_TOKEN
valueFrom:
secretKeyRef:
name: agnet-runtime-secrets
key: runtime-service-token
optional: true
- name: AGNET_CALLBACK_SERVICE_TOKEN
valueFrom:
secretKeyRef:
name: agnet-runtime-secrets
key: callback-service-token
optional: true
- name: AGNET_CALLBACK_SIGNING_SECRET
valueFrom:
secretKeyRef:
name: agnet-runtime-secrets
key: callback-signing-secret
optional: true
volumeMounts:
- name: orchestrator-code
mountPath: /app/orchestrator
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 5
volumes:
- name: orchestrator-code
configMap:
name: orchestrator-source
---
apiVersion: v1
kind: Service
metadata:
name: orchestrator-service
namespace: swarm-system
labels:
app: orchestrator
spec:
type: LoadBalancer
ports:
- port: 8000
targetPort: 8000
protocol: TCP
name: http
selector:
app: orchestrator
+73
View File
@@ -0,0 +1,73 @@
apiVersion: v1
kind: Service
metadata:
name: orchestrator-service
namespace: swarm-system
labels:
app: orchestrator
spec:
type: ClusterIP
ports:
- port: 8000
targetPort: 8000
name: http
selector:
app: orchestrator
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: orchestrator
namespace: swarm-system
labels:
app: orchestrator
spec:
replicas: 1
selector:
matchLabels:
app: orchestrator
template:
metadata:
labels:
app: orchestrator
spec:
serviceAccountName: swarm-orchestrator
containers:
- name: orchestrator
image: swarm-orchestrator:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8000
name: http
env:
- name: REDIS_HOST
value: "redis-service"
- name: REDIS_PORT
value: "6379"
- name: REDIS_DB
value: "0"
- name: LOG_LEVEL
value: "INFO"
resources:
requests:
memory: "256Mi"
cpu: "200m"
limits:
memory: "512Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
File diff suppressed because it is too large Load Diff
+348
View File
@@ -0,0 +1,348 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-config
namespace: swarm-system
data:
prometheus.yml: |
global:
scrape_interval: 15s
evaluation_interval: 15s
external_labels:
cluster: 'swarm-k8s'
environment: 'production'
# Alerting configuration
alerting:
alertmanagers:
- static_configs:
- targets:
- alertmanager:9093
# Scrape configurations
scrape_configs:
# Prometheus itself
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
# Orchestrator metrics
- job_name: 'orchestrator'
kubernetes_sd_configs:
- role: pod
namespaces:
names:
- swarm-system
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_app]
action: keep
regex: orchestrator
- source_labels: [__meta_kubernetes_pod_name]
target_label: pod
- source_labels: [__meta_kubernetes_namespace]
target_label: namespace
- source_labels: [__address__]
target_label: __address__
regex: ([^:]+)(?::\d+)?
replacement: $1:8000
# Agent pods metrics
- job_name: 'agents'
kubernetes_sd_configs:
- role: pod
namespaces:
names:
- swarm-system
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_app]
action: keep
regex: swarm-agent
- source_labels: [__meta_kubernetes_pod_name]
target_label: pod
- source_labels: [__meta_kubernetes_pod_label_task_id]
target_label: task_id
- source_labels: [__meta_kubernetes_pod_label_agent_id]
target_label: agent_id
- source_labels: [__meta_kubernetes_namespace]
target_label: namespace
- source_labels: [__address__]
target_label: __address__
regex: ([^:]+)(?::\d+)?
replacement: $1:8080
# Redis metrics (via redis_exporter)
- job_name: 'redis'
static_configs:
- targets: ['redis-exporter:9121']
labels:
service: 'redis'
# Kubernetes API server
- job_name: 'kubernetes-apiservers'
kubernetes_sd_configs:
- role: endpoints
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
action: keep
regex: default;kubernetes;https
# Kubernetes nodes
- job_name: 'kubernetes-nodes'
kubernetes_sd_configs:
- role: node
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
# Kubernetes pods (general)
- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_pod_name]
target_label: kubernetes_pod_name
# Recording rules for aggregations
rule_files:
- /etc/prometheus/rules/*.yml
---
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-rules
namespace: swarm-system
data:
swarm_rules.yml: |
groups:
- name: swarm_metrics
interval: 30s
rules:
# Agent count by status
- record: swarm:agents:count:by_status
expr: count by (status) (swarm_agent_status)
# Total active agents
- record: swarm:agents:active:total
expr: count(swarm_agent_status{status="running"})
# Average task duration
- record: swarm:tasks:duration:avg
expr: avg(swarm_task_duration_seconds)
# Task completion rate (last 5m)
- record: swarm:tasks:completion_rate:5m
expr: rate(swarm_tasks_completed_total[5m])
# Agent creation rate (last 5m)
- record: swarm:agents:creation_rate:5m
expr: rate(swarm_agents_created_total[5m])
# Handoff latency p95
- record: swarm:handoff:latency:p95
expr: histogram_quantile(0.95, rate(swarm_handoff_duration_seconds_bucket[5m]))
# Pod startup time p95
- record: swarm:pod:startup:p95
expr: histogram_quantile(0.95, rate(swarm_pod_startup_seconds_bucket[5m]))
- name: swarm_alerts
interval: 30s
rules:
# Alert if too many agents are failing
- alert: HighAgentFailureRate
expr: rate(swarm_agents_failed_total[5m]) > 0.1
for: 5m
labels:
severity: warning
annotations:
summary: "High agent failure rate detected"
description: "Agent failure rate is {{ $value }} failures/sec"
# Alert if orchestrator is down
- alert: OrchestratorDown
expr: up{job="orchestrator"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: "Orchestrator is down"
description: "Orchestrator has been down for more than 1 minute"
# Alert if Redis is down
- alert: RedisDown
expr: up{job="redis"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: "Redis is down"
description: "Redis has been down for more than 1 minute"
# Alert if pod startup is slow
- alert: SlowPodStartup
expr: swarm:pod:startup:p95 > 60
for: 10m
labels:
severity: warning
annotations:
summary: "Pod startup time is slow"
description: "P95 pod startup time is {{ $value }}s (target: <30s)"
# Alert if handoff latency is high
- alert: HighHandoffLatency
expr: swarm:handoff:latency:p95 > 5
for: 5m
labels:
severity: warning
annotations:
summary: "High handoff latency detected"
description: "P95 handoff latency is {{ $value }}s"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus
namespace: swarm-system
spec:
replicas: 1
selector:
matchLabels:
app: prometheus
template:
metadata:
labels:
app: prometheus
spec:
serviceAccountName: prometheus
containers:
- name: prometheus
image: prom/prometheus:v2.45.0
args:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--storage.tsdb.retention.time=30d'
- '--web.enable-lifecycle'
ports:
- containerPort: 9090
name: web
volumeMounts:
- name: config
mountPath: /etc/prometheus
- name: rules
mountPath: /etc/prometheus/rules
- name: storage
mountPath: /prometheus
resources:
requests:
memory: "2Gi"
cpu: "500m"
limits:
memory: "4Gi"
cpu: "2000m"
volumes:
- name: config
configMap:
name: prometheus-config
- name: rules
configMap:
name: prometheus-rules
- name: storage
persistentVolumeClaim:
claimName: prometheus-storage
---
apiVersion: v1
kind: Service
metadata:
name: prometheus
namespace: swarm-system
spec:
type: ClusterIP
ports:
- port: 9090
targetPort: 9090
name: web
selector:
app: prometheus
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prometheus-storage
namespace: swarm-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: prometheus
namespace: swarm-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: prometheus
rules:
- apiGroups: [""]
resources:
- nodes
- nodes/proxy
- services
- endpoints
- pods
verbs: ["get", "list", "watch"]
- apiGroups:
- extensions
resources:
- ingresses
verbs: ["get", "list", "watch"]
- nonResourceURLs: ["/metrics"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: prometheus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: prometheus
subjects:
- kind: ServiceAccount
name: prometheus
namespace: swarm-system
+23
View File
@@ -0,0 +1,23 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: orchestrator-role
namespace: default
labels:
app: heicode-swarm
component: orchestrator
rules:
# Pod management permissions
- apiGroups: [""]
resources: ["pods"]
verbs: ["create", "delete", "list", "watch", "get"]
# Pod log access
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get"]
# Pod status monitoring
- apiGroups: [""]
resources: ["pods/status"]
verbs: ["get"]
+16
View File
@@ -0,0 +1,16 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: orchestrator-rolebinding
namespace: default
labels:
app: heicode-swarm
component: orchestrator
subjects:
- kind: ServiceAccount
name: orchestrator-sa
namespace: default
roleRef:
kind: Role
name: orchestrator-role
apiGroup: rbac.authorization.k8s.io
@@ -0,0 +1,8 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: orchestrator-sa
namespace: default
labels:
app: heicode-swarm
component: orchestrator
+85
View File
@@ -0,0 +1,85 @@
apiVersion: v1
kind: Service
metadata:
name: redis-service
namespace: swarm-system
labels:
app: redis
spec:
ports:
- port: 6379
targetPort: 6379
name: redis
clusterIP: None
selector:
app: redis
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis
namespace: swarm-system
labels:
app: redis
spec:
serviceName: redis-service
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:7-alpine
ports:
- containerPort: 6379
name: redis
command:
- redis-server
- --appendonly
- "yes"
- --appendfsync
- everysec
- --maxmemory-policy
- allkeys-lru
- --maxmemory
- 256mb
volumeMounts:
- name: redis-data
mountPath: /data
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "512Mi"
cpu: "500m"
livenessProbe:
tcpSocket:
port: 6379
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
exec:
command:
- redis-cli
- ping
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
volumeClaimTemplates:
- metadata:
name: redis-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
+235
View File
@@ -0,0 +1,235 @@
# Test namespace and RBAC for HeiCode-Swarm testing
# Provides isolation and resource limits for test runs
---
# Test namespace
apiVersion: v1
kind: Namespace
metadata:
name: heicode-swarm-test
labels:
environment: test
purpose: agent-testing
---
# Resource quota to prevent runaway tests
apiVersion: v1
kind: ResourceQuota
metadata:
name: test-resource-quota
namespace: heicode-swarm-test
spec:
hard:
# Limit total pods to prevent cluster overload
pods: "25"
# CPU limits (enough for 20 agents + orchestrator + redis)
requests.cpu: "22"
limits.cpu: "25"
# Memory limits
requests.memory: "44Gi"
limits.memory: "50Gi"
# Storage limits
persistentvolumeclaims: "5"
requests.storage: "10Gi"
---
# Limit range for individual pods
apiVersion: v1
kind: LimitRange
metadata:
name: test-limit-range
namespace: heicode-swarm-test
spec:
limits:
# Default limits for containers
- type: Container
default:
cpu: "1"
memory: "2Gi"
defaultRequest:
cpu: "500m"
memory: "1Gi"
max:
cpu: "2"
memory: "4Gi"
min:
cpu: "100m"
memory: "128Mi"
# Pod limits
- type: Pod
max:
cpu: "2"
memory: "4Gi"
---
# ServiceAccount for test orchestrator
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-orchestrator-sa
namespace: heicode-swarm-test
labels:
component: orchestrator
environment: test
---
# Role for orchestrator pod management
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: test-orchestrator-role
namespace: heicode-swarm-test
labels:
component: orchestrator
environment: test
rules:
# Pod management permissions
- apiGroups: [""]
resources: ["pods"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
# Pod logs access
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get", "list"]
# Pod exec for debugging (test only)
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
# ConfigMaps for agent configuration
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch"]
# Secrets for API keys
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list"]
# Services for orchestrator/redis
- apiGroups: [""]
resources: ["services"]
verbs: ["get", "list", "watch"]
---
# RoleBinding to grant orchestrator permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: test-orchestrator-rolebinding
namespace: heicode-swarm-test
labels:
component: orchestrator
environment: test
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: test-orchestrator-role
subjects:
- kind: ServiceAccount
name: test-orchestrator-sa
namespace: heicode-swarm-test
---
# ServiceAccount for test agents
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-agent-sa
namespace: heicode-swarm-test
labels:
component: agent
environment: test
---
# Role for agent pods (minimal permissions)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: test-agent-role
namespace: heicode-swarm-test
labels:
component: agent
environment: test
rules:
# Agents can only read their own pod info
- apiGroups: [""]
resources: ["pods"]
verbs: ["get"]
resourceNames: [] # Will be restricted to self via admission controller
# Read ConfigMaps for configuration
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list"]
# Read Secrets for API keys
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
# RoleBinding for agent permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: test-agent-rolebinding
namespace: heicode-swarm-test
labels:
component: agent
environment: test
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: test-agent-role
subjects:
- kind: ServiceAccount
name: test-agent-sa
namespace: heicode-swarm-test
---
# NetworkPolicy to isolate test namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-isolation-policy
namespace: heicode-swarm-test
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress:
# Allow traffic within namespace
- from:
- namespaceSelector:
matchLabels:
environment: test
# Allow traffic from control plane (for kubectl exec, logs)
- from:
- namespaceSelector:
matchLabels:
name: kube-system
egress:
# Allow DNS
- to:
- namespaceSelector:
matchLabels:
name: kube-system
ports:
- protocol: UDP
port: 53
# Allow traffic within namespace
- to:
- namespaceSelector:
matchLabels:
environment: test
# Allow external API calls (model API, OpenAI-compatible, over HTTPS)
- to:
- namespaceSelector: {}
ports:
- protocol: TCP
port: 443
# Allow Git operations
- to:
- namespaceSelector: {}
ports:
- protocol: TCP
port: 22
- protocol: TCP
port: 9418
+70
View File
@@ -0,0 +1,70 @@
# Kind cluster configuration for HeiCode-Swarm local testing
# Optimized for agent pod testing with proper resource isolation
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: heicode-swarm-test
# Multi-node setup for realistic pod scheduling
nodes:
# Control plane node
- role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "node-role=control-plane"
extraPortMappings:
# Expose orchestrator service for desktop client access
- containerPort: 30080
hostPort: 8080
protocol: TCP
# Expose Redis for debugging (optional)
- containerPort: 30379
hostPort: 6379
protocol: TCP
# Worker nodes for agent pods
- role: worker
kubeadmConfigPatches:
- |
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "node-role=worker,workload=agents"
# Resource limits per worker to simulate realistic constraints
extraMounts:
- hostPath: /tmp/heicode-swarm-cache
containerPath: /var/lib/containerd
- role: worker
kubeadmConfigPatches:
- |
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "node-role=worker,workload=agents"
# Networking configuration
networking:
# Disable default CNI to use kindnet (faster startup)
disableDefaultCNI: false
# Pod subnet
podSubnet: "10.244.0.0/16"
# Service subnet
serviceSubnet: "10.96.0.0/12"
# Feature gates for testing
featureGates:
# Enable ephemeral containers for debugging
EphemeralContainers: true
# Runtime configuration
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
endpoint = ["http://localhost:5000"]
[plugins."io.containerd.grpc.v1.cri".containerd]
# Increase max concurrent downloads for faster image pulls
max_concurrent_downloads = 10
+121
View File
@@ -0,0 +1,121 @@
# Orchestrator(蜂群编排器)
基于 FastAPI 的编排器,是蜂群系统的控制核心。它对上对接 Heicode Manager(控制面),对下通过 WebSocket 调度多个 Agent 执行单元,负责任务分解、派发、专家协作、质量评审与结果汇总。
## 能力概览
- **Manager 对接**:提供面向 Manager 的部署/任务/日志/指标/审批等 REST 接口,事件通过带签名的回调上报。
- **WebSocket 调度**:与 Agent 实时双向通信,按能力与容量派发任务。
- **任务图与依赖**:支持任务间依赖(`depends_on`),依赖完成后才派发后继任务。
- **任务规划(可选)**:在 Manager 未给出明确分工时,用大模型把目标拆解为「实现 → 测试 → 文档」等专家子任务。
- **主控评审循环(可选)**:所有任务完成后由「评审者」判断结果是否达标,不达标则把相关任务退回重做,直至通过或达到上限,并对结果做汇总。
- **专家协作**:派发时注入依赖产物与同伴信息,支持 Agent 间消息路由。
- **持久化**:以 Redis 为权威存储;提供仅限开发/CI 的内存回退。
- **故障恢复**:心跳超时检测、任务重派与重试。
## 目录结构
```
orchestrator/
├── main.py # FastAPI 应用:REST 接口、WebSocket、调度循环、评审与汇总
├── swarm_runtime.py # Manager 桥接:运行持久化、事件/回调、审批、任务图构建
├── planner.py # 任务规划 + 主控评审(critic)+ 结果汇总
├── task_queue.py # 任务队列:创建、派发、依赖、重试、重开
├── agent_registry.py # Agent 注册表:注册、心跳、状态
├── handoff_manager.py # 移交协调与历史
├── redis_client.py # 异步 Redis 封装(含受控的内存回退)
├── checkpoint_manager.py / model_tuner.py / tracing.py # 检查点、模型调优、链路追踪
└── requirements.txt
```
## 工作流程
```
分解(Plan) → 派发(Dispatch) → 专家执行(Execute) → 协作/移交(Handoff)
→ 评审(Review,可选循环重做) → 汇总交付(Deliver)
```
- **分解**:优先采用 Manager 提供的编排方案;若未提供且开启了规划回退,则由 `planner` 生成专家子任务图。
- **派发**:调度循环将就绪任务按「能力匹配 + 剩余容量」派发给已连接的空闲 Agent;派发时把已完成依赖的产物与同伴 Agent 信息注入任务上下文。
- **评审循环**:任务全部完成后,`planner.review` 判定是否达标;不达标则 `reopen` 指定任务重做,受 `MAX_REVIEW_CYCLES` 约束;通过后由 `planner.synthesize` 生成统一的最终回答。
## WebSocket 协议(`/ws/{agent_id}`)
Agent → 编排器:`register`(含 `available_slots`)、`heartbeat`、`task_start`、`task_complete`、`task_failed`、`task_accepted`、`task_rejected`、`blocked_on_handoff`、`handoff_request`、`peer_message`、`status_update`。
编排器 → Agent:`task_assignment`、`cancel_task`、`peer_message`(路由转发)、各类确认(`registered`、`heartbeat_ack` 等)。
## REST 接口
**健康与指标**
- `GET /`、`GET /health`、`GET /metrics`(Prometheus)
**Manager 面(含 `/api/swarms`、`/api/agent/swarm/deployments`、`/api/agnet/deployments` 等别名)**
- `POST .../`:创建蜂群部署
- `GET .../{id}`:部署概要
- `GET .../{id}/tasks | /logs | /events | /metrics | /workflow | /diagnostics`
- `POST .../{id}/stop`:停止部署(会向相关 Agent 发送 `cancel_task`)
- `POST .../{id}/approvals/{approval_id}`:接收 Manager 审批决定
**内部/调试**
- `GET /agents`、`/agents/idle`、`/agents/{id}`
- `POST /tasks`、`POST /tasks/assign`、`GET /tasks`、`GET /tasks/{id}`、`GET /handoffs`
## 配置(环境变量)
```bash
# 存储
REDIS_HOST=redis-service
REDIS_PORT=6379
REDIS_DB=0
REDIS_FAKE=1 # 仅开发/CI:使用内存版 fakeredis
ALLOW_MEMORY_STORE=1 # 仅开发/CI:Redis 不可用时回退到内存(生产请勿开启)
# 工作流开关
ENABLE_PLANNER_FALLBACK=1 # 无 Manager 分工时启用规划回退
ENABLE_REVIEW_LOOP=1 # 启用主控评审/重做循环 + 结果汇总
MAX_REVIEW_CYCLES=2 # 评审最大重做轮数
ENABLE_SUBTASK_HANDOFF=false
# 规划/评审用模型(OpenAI 兼容;未配置时退化为静态分解 + 启发式评审)
OPENAI_API_KEY / OPENAI_API_BASE / OPENAI_MODEL
MASTER_REVIEW_MODEL / MAX_SUBTASKS / PLANNER_TIMEOUT_SECONDS
# 安全与回调(与 Manager 契约相关)
AGENT_RUNTIME_SERVICE_TOKEN / AGNET_RUNTIME_SERVICE_TOKEN # 服务间鉴权
AGENT_CALLBACK_SERVICE_TOKEN / AGENT_CALLBACK_SIGNING_SECRET # 回调令牌与 HMAC 签名
# 可观测性
OTEL_EXPORTER_OTLP_ENDPOINT / SWARM_RUNTIME_SOURCE / SWARM_RUNTIME_PLATFORM
```
> 说明:未配置运行时服务令牌时,Manager 面接口进入「非安全开发模式」(不校验鉴权),仅供本地调试。
## 持久化与回退
Redis 为**权威存储**。仅当显式设置 `REDIS_FAKE=1` 或 `ALLOW_MEMORY_STORE=1` 时,才会使用进程内的 `fakeredis` 作为开发/CI 回退;生产环境在 Redis 不可用时会**快速失败**,避免静默丢失持久化与 Manager 状态。
## 故障恢复
- Agent 需每 15 秒心跳;超过 30 秒无心跳判定为失败。
- 失败 Agent 的任务自动退回队列重派;任务失败按 `max_retries` 重试。
- Agent 断连后其在执行的任务会被回收为可重派。
## 本地运行
```bash
pip install -r orchestrator/requirements.txt
# 开发模式(内存回退 + 工作流开关)
set "REDIS_FAKE=1"
set "ENABLE_PLANNER_FALLBACK=1"
set "ENABLE_REVIEW_LOOP=1"
python -m uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
```
## 测试
```bash
python scripts/test-runtime-contract.py # Manager 契约校验
python scripts/test-merge-smoke.py # 工作流冒烟测试(评审/协作/汇总等)
```
+2
View File
@@ -0,0 +1,2 @@
"""Swarm orchestrator package."""
__version__ = "1.0.0"
+149
View File
@@ -0,0 +1,149 @@
"""Agent registry with Redis-backed state management."""
import json
import time
import logging
from typing import Dict, List, Optional
from enum import Enum
from pydantic import BaseModel
from .redis_client import redis_client
logger = logging.getLogger(__name__)
class AgentStatus(str, Enum):
"""Agent status enumeration."""
IDLE = "idle"
BUSY = "busy"
HANDOFF_PENDING = "handoff-pending"
FAILED = "failed"
class AgentMetadata(BaseModel):
"""Agent metadata model."""
agent_id: str
status: AgentStatus
last_heartbeat: float
capabilities: List[str]
current_task_id: Optional[str] = None
class AgentRegistry:
"""Manages agent registration and heartbeat tracking."""
HEARTBEAT_TIMEOUT = 30 # seconds
AGENT_KEY_PREFIX = "agent:"
def __init__(self):
pass
async def register_agent(
self, agent_id: str, capabilities: List[str]
) -> AgentMetadata:
"""Register a new agent."""
metadata = AgentMetadata(
agent_id=agent_id,
status=AgentStatus.IDLE,
last_heartbeat=time.time(),
capabilities=capabilities,
)
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
await redis_client.set(key, metadata.model_dump_json())
logger.info(f"Registered agent {agent_id} with capabilities: {capabilities}")
return metadata
async def deregister_agent(self, agent_id: str):
"""Deregister an agent."""
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
await redis_client.delete(key)
logger.info(f"Deregistered agent {agent_id}")
async def update_heartbeat(self, agent_id: str) -> bool:
"""Update agent heartbeat timestamp."""
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
data = await redis_client.get(key)
if not data:
logger.warning(f"Agent {agent_id} not found for heartbeat update")
return False
metadata = AgentMetadata.model_validate_json(data)
metadata.last_heartbeat = time.time()
await redis_client.set(key, metadata.model_dump_json())
return True
async def update_status(
self, agent_id: str, status: AgentStatus, task_id: Optional[str] = None
) -> bool:
"""Update agent status."""
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
data = await redis_client.get(key)
if not data:
logger.warning(f"Agent {agent_id} not found for status update")
return False
metadata = AgentMetadata.model_validate_json(data)
metadata.status = status
metadata.current_task_id = task_id
await redis_client.set(key, metadata.model_dump_json())
logger.info(f"Updated agent {agent_id} status to {status}")
return True
async def get_agent(self, agent_id: str) -> Optional[AgentMetadata]:
"""Get agent metadata."""
key = f"{self.AGENT_KEY_PREFIX}{agent_id}"
data = await redis_client.get(key)
if not data:
return None
return AgentMetadata.model_validate_json(data)
async def get_all_agents(self) -> List[AgentMetadata]:
"""Get all registered agents."""
pattern = f"{self.AGENT_KEY_PREFIX}*"
keys = await redis_client.keys(pattern)
agents = []
for key in keys:
data = await redis_client.get(key)
if data:
agents.append(AgentMetadata.model_validate_json(data))
return agents
async def get_idle_agents(self) -> List[AgentMetadata]:
"""Get all idle agents."""
all_agents = await self.get_all_agents()
return [agent for agent in all_agents if agent.status == AgentStatus.IDLE]
async def check_failed_agents(self) -> List[str]:
"""Check for agents with expired heartbeats and mark as failed."""
current_time = time.time()
failed_agents = []
all_agents = await self.get_all_agents()
for agent in all_agents:
if agent.status == AgentStatus.FAILED:
continue
time_since_heartbeat = current_time - agent.last_heartbeat
if time_since_heartbeat > self.HEARTBEAT_TIMEOUT:
await self.update_status(agent.agent_id, AgentStatus.FAILED)
failed_agents.append(agent.agent_id)
logger.warning(
f"Agent {agent.agent_id} marked as failed "
f"(no heartbeat for {time_since_heartbeat:.1f}s)"
)
return failed_agents
# Global agent registry instance
agent_registry = AgentRegistry()
+240
View File
@@ -0,0 +1,240 @@
"""
Checkpoint Manager for Task Recovery
Provides partial checkpointing and retry strategies for agent tasks.
Enables recovery from failures without restarting entire workflows.
"""
import json
import time
from typing import Dict, List, Optional, Any
from dataclasses import dataclass, asdict
from datetime import datetime
import redis
import logging
logger = logging.getLogger(__name__)
@dataclass
class Checkpoint:
"""Represents a task checkpoint"""
task_id: str
agent_id: str
checkpoint_id: str
timestamp: float
phase: str # e.g., "analysis", "implementation", "testing"
state: Dict[str, Any] # Serializable state data
files_modified: List[str]
git_commit: Optional[str] = None
metadata: Optional[Dict[str, Any]] = None
@dataclass
class RetryPolicy:
"""Retry strategy configuration"""
max_retries: int = 3
backoff_multiplier: float = 2.0
initial_delay_seconds: float = 1.0
max_delay_seconds: float = 60.0
retry_on_errors: List[str] = None # Error types to retry
class CheckpointManager:
"""Manages task checkpoints and recovery"""
def __init__(self, redis_client: redis.Redis):
self.redis = redis_client
self.checkpoint_ttl = 86400 * 7 # 7 days
def save_checkpoint(self, checkpoint: Checkpoint) -> bool:
"""Save a checkpoint to Redis"""
try:
key = f"checkpoint:{checkpoint.task_id}:{checkpoint.checkpoint_id}"
data = json.dumps(asdict(checkpoint))
self.redis.setex(key, self.checkpoint_ttl, data)
# Add to task's checkpoint list
list_key = f"checkpoints:{checkpoint.task_id}"
self.redis.lpush(list_key, checkpoint.checkpoint_id)
self.redis.expire(list_key, self.checkpoint_ttl)
logger.info(
f"Saved checkpoint {checkpoint.checkpoint_id} for task {checkpoint.task_id}"
)
return True
except Exception as e:
logger.error(f"Failed to save checkpoint: {e}")
return False
def get_checkpoint(self, task_id: str, checkpoint_id: str) -> Optional[Checkpoint]:
"""Retrieve a specific checkpoint"""
try:
key = f"checkpoint:{task_id}:{checkpoint_id}"
data = self.redis.get(key)
if not data:
return None
checkpoint_dict = json.loads(data)
return Checkpoint(**checkpoint_dict)
except Exception as e:
logger.error(f"Failed to retrieve checkpoint: {e}")
return None
def get_latest_checkpoint(self, task_id: str) -> Optional[Checkpoint]:
"""Get the most recent checkpoint for a task"""
try:
list_key = f"checkpoints:{task_id}"
checkpoint_ids = self.redis.lrange(list_key, 0, 0)
if not checkpoint_ids:
return None
checkpoint_id = checkpoint_ids[0].decode('utf-8')
return self.get_checkpoint(task_id, checkpoint_id)
except Exception as e:
logger.error(f"Failed to get latest checkpoint: {e}")
return None
def list_checkpoints(self, task_id: str) -> List[str]:
"""List all checkpoint IDs for a task"""
try:
list_key = f"checkpoints:{task_id}"
checkpoint_ids = self.redis.lrange(list_key, 0, -1)
return [cid.decode('utf-8') for cid in checkpoint_ids]
except Exception as e:
logger.error(f"Failed to list checkpoints: {e}")
return []
def delete_checkpoint(self, task_id: str, checkpoint_id: str) -> bool:
"""Delete a specific checkpoint"""
try:
key = f"checkpoint:{task_id}:{checkpoint_id}"
self.redis.delete(key)
# Remove from list
list_key = f"checkpoints:{task_id}"
self.redis.lrem(list_key, 0, checkpoint_id)
logger.info(f"Deleted checkpoint {checkpoint_id} for task {task_id}")
return True
except Exception as e:
logger.error(f"Failed to delete checkpoint: {e}")
return False
def cleanup_task_checkpoints(self, task_id: str) -> bool:
"""Delete all checkpoints for a task"""
try:
checkpoint_ids = self.list_checkpoints(task_id)
for checkpoint_id in checkpoint_ids:
self.delete_checkpoint(task_id, checkpoint_id)
list_key = f"checkpoints:{task_id}"
self.redis.delete(list_key)
logger.info(f"Cleaned up all checkpoints for task {task_id}")
return True
except Exception as e:
logger.error(f"Failed to cleanup checkpoints: {e}")
return False
class RetryManager:
"""Manages retry logic with exponential backoff"""
def __init__(self, redis_client: redis.Redis):
self.redis = redis_client
def record_attempt(self, task_id: str, agent_id: str, error: Optional[str] = None):
"""Record a task attempt"""
key = f"retry:{task_id}"
attempt_data = {
"agent_id": agent_id,
"timestamp": time.time(),
"error": error
}
self.redis.lpush(key, json.dumps(attempt_data))
self.redis.expire(key, 86400) # 24 hour TTL
def get_attempt_count(self, task_id: str) -> int:
"""Get number of attempts for a task"""
key = f"retry:{task_id}"
return self.redis.llen(key)
def should_retry(self, task_id: str, policy: RetryPolicy) -> bool:
"""Determine if task should be retried"""
attempt_count = self.get_attempt_count(task_id)
return attempt_count < policy.max_retries
def get_retry_delay(self, task_id: str, policy: RetryPolicy) -> float:
"""Calculate delay before next retry (exponential backoff)"""
attempt_count = self.get_attempt_count(task_id)
delay = policy.initial_delay_seconds * (policy.backoff_multiplier ** attempt_count)
return min(delay, policy.max_delay_seconds)
def clear_attempts(self, task_id: str):
"""Clear retry history for a task"""
key = f"retry:{task_id}"
self.redis.delete(key)
class RecoveryCoordinator:
"""Coordinates task recovery from checkpoints"""
def __init__(self, checkpoint_manager: CheckpointManager, retry_manager: RetryManager):
self.checkpoint_manager = checkpoint_manager
self.retry_manager = retry_manager
def recover_task(
self,
task_id: str,
retry_policy: Optional[RetryPolicy] = None
) -> Optional[Dict[str, Any]]:
"""
Attempt to recover a failed task from its latest checkpoint
Returns recovery instructions or None if recovery not possible
"""
if retry_policy is None:
retry_policy = RetryPolicy()
# Check if we should retry
if not self.retry_manager.should_retry(task_id, retry_policy):
logger.warning(f"Task {task_id} exceeded max retries")
return None
# Get latest checkpoint
checkpoint = self.checkpoint_manager.get_latest_checkpoint(task_id)
if not checkpoint:
logger.warning(f"No checkpoint found for task {task_id}")
return None
# Calculate retry delay
delay = self.retry_manager.get_retry_delay(task_id, retry_policy)
recovery_plan = {
"task_id": task_id,
"checkpoint_id": checkpoint.checkpoint_id,
"resume_phase": checkpoint.phase,
"state": checkpoint.state,
"files_modified": checkpoint.files_modified,
"git_commit": checkpoint.git_commit,
"retry_delay_seconds": delay,
"attempt_number": self.retry_manager.get_attempt_count(task_id) + 1
}
logger.info(
f"Recovery plan created for task {task_id} from checkpoint {checkpoint.checkpoint_id}"
)
return recovery_plan
def mark_recovery_success(self, task_id: str):
"""Mark a task as successfully recovered"""
self.retry_manager.clear_attempts(task_id)
logger.info(f"Task {task_id} recovered successfully")
def mark_recovery_failure(self, task_id: str, agent_id: str, error: str):
"""Record a failed recovery attempt"""
self.retry_manager.record_attempt(task_id, agent_id, error)
logger.warning(f"Recovery attempt failed for task {task_id}: {error}")
+165
View File
@@ -0,0 +1,165 @@
"""Handoff manager for task coordination between agents."""
import json
import time
import uuid
import logging
from typing import Dict, Optional
from pydantic import BaseModel
from .redis_client import redis_client
from .agent_registry import agent_registry, AgentStatus
logger = logging.getLogger(__name__)
class HandoffRequest(BaseModel):
"""Handoff request model."""
type: str = "handoff"
source_agent_id: str
target_agent_id: str
task_context: Dict
class HandoffRecord(BaseModel):
"""Handoff history record."""
handoff_id: str
source_agent_id: str
target_agent_id: str
timestamp: float
task_context: Dict
status: str # pending, completed, failed
class HandoffManager:
"""Manages task handoffs between agents."""
HANDOFF_KEY_PREFIX = "handoff:"
def __init__(self):
pass
async def initiate_handoff(
self, source_agent_id: str, target_agent_id: str, task_context: Dict
) -> Optional[str]:
"""Initiate a handoff from source to target agent."""
# Validate source agent exists and is busy
source_agent = await agent_registry.get_agent(source_agent_id)
if not source_agent:
logger.error(f"Source agent {source_agent_id} not found")
return None
# Validate target agent exists and is idle
target_agent = await agent_registry.get_agent(target_agent_id)
if not target_agent:
logger.error(f"Target agent {target_agent_id} not found")
return None
if target_agent.status != AgentStatus.IDLE:
logger.error(
f"Target agent {target_agent_id} is not idle (status: {target_agent.status})"
)
return None
# Create handoff record
handoff_id = str(uuid.uuid4())
record = HandoffRecord(
handoff_id=handoff_id,
source_agent_id=source_agent_id,
target_agent_id=target_agent_id,
timestamp=time.time(),
task_context=task_context,
status="pending",
)
# Store in Redis
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
await redis_client.set(key, record.model_dump_json())
# Update agent statuses
await agent_registry.update_status(source_agent_id, AgentStatus.IDLE)
await agent_registry.update_status(
target_agent_id, AgentStatus.HANDOFF_PENDING, task_context.get("task_id")
)
logger.info(
f"Handoff {handoff_id} initiated: {source_agent_id} -> {target_agent_id}"
)
return handoff_id
async def complete_handoff(self, handoff_id: str) -> bool:
"""Mark handoff as completed."""
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
data = await redis_client.get(key)
if not data:
logger.error(f"Handoff {handoff_id} not found")
return False
record = HandoffRecord.model_validate_json(data)
record.status = "completed"
await redis_client.set(key, record.model_dump_json())
# Update target agent to busy
await agent_registry.update_status(
record.target_agent_id, AgentStatus.BUSY, record.task_context.get("task_id")
)
logger.info(f"Handoff {handoff_id} completed")
return True
async def fail_handoff(self, handoff_id: str, reason: str) -> bool:
"""Mark handoff as failed."""
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
data = await redis_client.get(key)
if not data:
logger.error(f"Handoff {handoff_id} not found")
return False
record = HandoffRecord.model_validate_json(data)
record.status = "failed"
await redis_client.set(key, record.model_dump_json())
# Revert target agent to idle
await agent_registry.update_status(record.target_agent_id, AgentStatus.IDLE)
logger.warning(f"Handoff {handoff_id} failed: {reason}")
return True
async def get_handoff(self, handoff_id: str) -> Optional[HandoffRecord]:
"""Get handoff record by ID."""
key = f"{self.HANDOFF_KEY_PREFIX}{handoff_id}"
data = await redis_client.get(key)
if not data:
return None
return HandoffRecord.model_validate_json(data)
async def get_handoff_history(
self, agent_id: Optional[str] = None, limit: int = 100
) -> list[HandoffRecord]:
"""Get handoff history, optionally filtered by agent."""
pattern = f"{self.HANDOFF_KEY_PREFIX}*"
keys = await redis_client.keys(pattern)
records = []
for key in keys[:limit]:
data = await redis_client.get(key)
if data:
record = HandoffRecord.model_validate_json(data)
if agent_id is None or (
record.source_agent_id == agent_id
or record.target_agent_id == agent_id
):
records.append(record)
# Sort by timestamp descending
records.sort(key=lambda r: r.timestamp, reverse=True)
return records[:limit]
# Global handoff manager instance
handoff_manager = HandoffManager()
+2177
View File
File diff suppressed because it is too large Load Diff
+386
View File
@@ -0,0 +1,386 @@
"""
Complexity Model Tuner
Implements feedback loop and retraining for the complexity analysis model.
Collects actual vs predicted agent counts and adjusts the model over time.
"""
import json
import time
from typing import Dict, List, Optional, Tuple
from dataclasses import dataclass, asdict
from datetime import datetime, timedelta
import redis
import logging
from collections import defaultdict
import statistics
logger = logging.getLogger(__name__)
@dataclass
class ComplexityPrediction:
"""Record of a complexity prediction"""
task_id: str
timestamp: float
task_description: str
predicted_agents: int
predicted_complexity: str # "simple", "moderate", "complex"
confidence: float
model_version: str
@dataclass
class ComplexityActual:
"""Actual outcome of a task"""
task_id: str
timestamp: float
actual_agents: int
actual_duration_seconds: float
success: bool
user_feedback: Optional[str] = None # "too_many", "too_few", "just_right"
user_rating: Optional[int] = None # 1-5 scale
@dataclass
class ModelMetrics:
"""Model performance metrics"""
model_version: str
total_predictions: int
mean_absolute_error: float
accuracy_within_1: float # % predictions within ±1 agent
accuracy_within_2: float # % predictions within ±2 agents
user_satisfaction: float # Average user rating
last_updated: float
class ComplexityModelTuner:
"""Manages complexity model feedback and tuning"""
def __init__(self, redis_client: redis.Redis):
self.redis = redis_client
self.current_model_version = "v1.0"
self.data_ttl = 86400 * 30 # 30 days
def record_prediction(self, prediction: ComplexityPrediction) -> bool:
"""Record a complexity prediction"""
try:
key = f"prediction:{prediction.task_id}"
data = json.dumps(asdict(prediction))
self.redis.setex(key, self.data_ttl, data)
# Add to predictions list
list_key = f"predictions:{prediction.model_version}"
self.redis.lpush(list_key, prediction.task_id)
self.redis.expire(list_key, self.data_ttl)
logger.info(
f"Recorded prediction for task {prediction.task_id}: "
f"{prediction.predicted_agents} agents"
)
return True
except Exception as e:
logger.error(f"Failed to record prediction: {e}")
return False
def record_actual(self, actual: ComplexityActual) -> bool:
"""Record actual task outcome"""
try:
key = f"actual:{actual.task_id}"
data = json.dumps(asdict(actual))
self.redis.setex(key, self.data_ttl, data)
# Add to actuals list
list_key = "actuals:all"
self.redis.lpush(list_key, actual.task_id)
self.redis.expire(list_key, self.data_ttl)
logger.info(
f"Recorded actual for task {actual.task_id}: "
f"{actual.actual_agents} agents, feedback: {actual.user_feedback}"
)
return True
except Exception as e:
logger.error(f"Failed to record actual: {e}")
return False
def get_prediction(self, task_id: str) -> Optional[ComplexityPrediction]:
"""Retrieve a prediction"""
try:
key = f"prediction:{task_id}"
data = self.redis.get(key)
if not data:
return None
return ComplexityPrediction(**json.loads(data))
except Exception as e:
logger.error(f"Failed to get prediction: {e}")
return None
def get_actual(self, task_id: str) -> Optional[ComplexityActual]:
"""Retrieve actual outcome"""
try:
key = f"actual:{task_id}"
data = self.redis.get(key)
if not data:
return None
return ComplexityActual(**json.loads(data))
except Exception as e:
logger.error(f"Failed to get actual: {e}")
return None
def calculate_metrics(
self,
model_version: Optional[str] = None,
limit: int = 1000
) -> Optional[ModelMetrics]:
"""Calculate model performance metrics"""
if model_version is None:
model_version = self.current_model_version
try:
# Get predictions for this model version
list_key = f"predictions:{model_version}"
task_ids = self.redis.lrange(list_key, 0, limit - 1)
if not task_ids:
logger.warning(f"No predictions found for model {model_version}")
return None
errors = []
within_1 = 0
within_2 = 0
ratings = []
for task_id_bytes in task_ids:
task_id = task_id_bytes.decode('utf-8')
prediction = self.get_prediction(task_id)
actual = self.get_actual(task_id)
if not prediction or not actual:
continue
# Calculate error
error = abs(prediction.predicted_agents - actual.actual_agents)
errors.append(error)
# Check accuracy thresholds
if error <= 1:
within_1 += 1
if error <= 2:
within_2 += 1
# Collect user ratings
if actual.user_rating:
ratings.append(actual.user_rating)
if not errors:
logger.warning(f"No matched predictions/actuals for model {model_version}")
return None
total = len(errors)
mae = statistics.mean(errors)
acc_1 = (within_1 / total) * 100
acc_2 = (within_2 / total) * 100
avg_rating = statistics.mean(ratings) if ratings else 0.0
metrics = ModelMetrics(
model_version=model_version,
total_predictions=total,
mean_absolute_error=mae,
accuracy_within_1=acc_1,
accuracy_within_2=acc_2,
user_satisfaction=avg_rating,
last_updated=time.time()
)
# Cache metrics
metrics_key = f"metrics:{model_version}"
self.redis.setex(metrics_key, 3600, json.dumps(asdict(metrics)))
logger.info(
f"Model {model_version} metrics: MAE={mae:.2f}, "
f"Acc±1={acc_1:.1f}%, Acc±2={acc_2:.1f}%, "
f"Satisfaction={avg_rating:.2f}/5"
)
return metrics
except Exception as e:
logger.error(f"Failed to calculate metrics: {e}")
return None
def get_error_patterns(self, limit: int = 100) -> Dict[str, List[Tuple[str, int, int]]]:
"""
Analyze error patterns to identify systematic biases
Returns dict with categories:
- overestimated: tasks where we predicted too many agents
- underestimated: tasks where we predicted too few agents
- accurate: tasks where prediction was close
"""
patterns = {
"overestimated": [],
"underestimated": [],
"accurate": []
}
try:
list_key = "actuals:all"
task_ids = self.redis.lrange(list_key, 0, limit - 1)
for task_id_bytes in task_ids:
task_id = task_id_bytes.decode('utf-8')
prediction = self.get_prediction(task_id)
actual = self.get_actual(task_id)
if not prediction or not actual:
continue
error = prediction.predicted_agents - actual.actual_agents
entry = (
task_id,
prediction.predicted_agents,
actual.actual_agents
)
if error > 1:
patterns["overestimated"].append(entry)
elif error < -1:
patterns["underestimated"].append(entry)
else:
patterns["accurate"].append(entry)
return patterns
except Exception as e:
logger.error(f"Failed to analyze error patterns: {e}")
return patterns
def get_feedback_summary(self, limit: int = 100) -> Dict[str, int]:
"""Summarize user feedback"""
feedback_counts = defaultdict(int)
try:
list_key = "actuals:all"
task_ids = self.redis.lrange(list_key, 0, limit - 1)
for task_id_bytes in task_ids:
task_id = task_id_bytes.decode('utf-8')
actual = self.get_actual(task_id)
if actual and actual.user_feedback:
feedback_counts[actual.user_feedback] += 1
return dict(feedback_counts)
except Exception as e:
logger.error(f"Failed to get feedback summary: {e}")
return {}
def generate_tuning_recommendations(self) -> List[str]:
"""Generate recommendations for model tuning based on data"""
recommendations = []
try:
# Get current metrics
metrics = self.calculate_metrics()
if not metrics:
return ["Insufficient data for recommendations"]
# Check accuracy
if metrics.accuracy_within_1 < 60:
recommendations.append(
f"Low accuracy ({metrics.accuracy_within_1:.1f}%). "
"Consider retraining with more diverse examples."
)
# Check error patterns
patterns = self.get_error_patterns()
overestimated = len(patterns["overestimated"])
underestimated = len(patterns["underestimated"])
total = overestimated + underestimated + len(patterns["accurate"])
if total > 0:
over_pct = (overestimated / total) * 100
under_pct = (underestimated / total) * 100
if over_pct > 40:
recommendations.append(
f"Model overestimates in {over_pct:.1f}% of cases. "
"Consider reducing base agent count or adjusting complexity thresholds."
)
if under_pct > 40:
recommendations.append(
f"Model underestimates in {under_pct:.1f}% of cases. "
"Consider increasing base agent count or lowering complexity thresholds."
)
# Check user satisfaction
if metrics.user_satisfaction < 3.5:
recommendations.append(
f"Low user satisfaction ({metrics.user_satisfaction:.1f}/5). "
"Review user feedback and adjust model accordingly."
)
# Check feedback
feedback = self.get_feedback_summary()
if feedback.get("too_many", 0) > feedback.get("too_few", 0) * 2:
recommendations.append(
"Users frequently report 'too many agents'. "
"Consider reducing default agent counts."
)
elif feedback.get("too_few", 0) > feedback.get("too_many", 0) * 2:
recommendations.append(
"Users frequently report 'too few agents'. "
"Consider increasing default agent counts."
)
if not recommendations:
recommendations.append(
f"Model performing well (MAE={metrics.mean_absolute_error:.2f}, "
f"Acc±1={metrics.accuracy_within_1:.1f}%). Continue monitoring."
)
except Exception as e:
logger.error(f"Failed to generate recommendations: {e}")
recommendations.append(f"Error generating recommendations: {e}")
return recommendations
def export_training_data(self, limit: int = 1000) -> List[Dict]:
"""Export prediction/actual pairs for model retraining"""
training_data = []
try:
list_key = "actuals:all"
task_ids = self.redis.lrange(list_key, 0, limit - 1)
for task_id_bytes in task_ids:
task_id = task_id_bytes.decode('utf-8')
prediction = self.get_prediction(task_id)
actual = self.get_actual(task_id)
if not prediction or not actual:
continue
training_data.append({
"task_description": prediction.task_description,
"predicted_agents": prediction.predicted_agents,
"actual_agents": actual.actual_agents,
"duration_seconds": actual.actual_duration_seconds,
"success": actual.success,
"user_feedback": actual.user_feedback,
"user_rating": actual.user_rating
})
logger.info(f"Exported {len(training_data)} training examples")
return training_data
except Exception as e:
logger.error(f"Failed to export training data: {e}")
return []
+261
View File
@@ -0,0 +1,261 @@
"""LLM task planner (Manager-first fallback).
Ported from agent_swarm_v4. This planner is used ONLY as a fallback to decompose a
swarm objective into specialist subtasks when the Manager's orchestration_plan does
not provide an explicit agent breakdown AND the operator opts in via
``ENABLE_PLANNER_FALLBACK``. It never overrides a Manager-supplied plan.
It degrades gracefully: with no API key or on any error it returns a static
implementation -> testing -> documentation plan, so the runtime never hard-depends on
the model being reachable.
"""
import json
import logging
import os
from typing import List, Dict
logger = logging.getLogger(__name__)
try:
from openai import AsyncOpenAI
except Exception: # pragma: no cover - only when openai is absent
AsyncOpenAI = None
def _planner_timeout() -> float:
try:
return float(os.getenv("PLANNER_TIMEOUT_SECONDS", "45") or 45)
except ValueError:
return 45.0
def _max_subtasks() -> int:
try:
return int(os.getenv("MAX_SUBTASKS", "6") or 6)
except ValueError:
return 6
class Planner:
"""Decomposes an objective into specialist subtask specs."""
def __init__(self):
api_key = os.getenv("OPENAI_API_KEY") or os.getenv("MODEL_API_KEY")
api_base = (
os.getenv("OPENAI_API_BASE")
or os.getenv("MODEL_API_BASE")
or "https://api.openai.com/v1"
)
model = (
os.getenv("OPENAI_MODEL")
or os.getenv("MODEL_NAME")
or os.getenv("MODEL_ID")
or "gpt-4o-mini"
)
self.model = os.getenv("MASTER_REVIEW_MODEL", model)
self.client = (
AsyncOpenAI(api_key=api_key, base_url=api_base)
if (api_key and AsyncOpenAI is not None)
else None
)
def _static_plan(self, run_id: str) -> List[Dict]:
return [
{
"subtask_id": f"{run_id}-implementation",
"description": "Implement the core code required by the user request.",
"required_capabilities": ["python", "code_generation"],
"role": "implementation",
"depends_on": [],
},
{
"subtask_id": f"{run_id}-testing",
"description": "Write tests for the implemented functionality.",
"required_capabilities": ["testing", "pytest"],
"role": "testing",
"depends_on": [f"{run_id}-implementation"],
},
{
"subtask_id": f"{run_id}-documentation",
"description": "Write concise documentation based on the implementation and tests.",
"required_capabilities": ["technical-writing", "general"],
"role": "documentation",
"depends_on": [f"{run_id}-implementation", f"{run_id}-testing"],
},
]
async def build_plan(self, run_id: str, objective: str) -> List[Dict]:
"""Return specialist subtask specs for an objective, or a static fallback."""
fallback = self._static_plan(run_id)
if not self.client:
return fallback
try:
response = await self.client.chat.completions.create(
model=self.model,
messages=[
{
"role": "system",
"content": (
"Break the request into at most 6 specialist subtasks. "
"Return JSON with key 'subtasks'; each subtask has subtask_id, "
"description, role, required_capabilities (list), depends_on (list of subtask_ids)."
),
},
{"role": "user", "content": f"Run id: {run_id}\nObjective: {objective}"},
],
response_format={"type": "json_object"},
timeout=_planner_timeout(),
)
content = response.choices[0].message.content or "{}"
subtasks = (json.loads(content) or {}).get("subtasks")
if not subtasks:
return fallback
return subtasks[: _max_subtasks()]
except Exception as e:
logger.warning(f"Planner LLM call failed ({e}); using static fallback plan")
return fallback
async def review(self, objective: str, tasks: List[Dict], results: Dict) -> Dict:
"""Judge whether the combined specialist results are good enough.
Returns {accepted: bool, summary: str, retry_tasks: [task_id, ...]}. Falls back to a
deterministic consistency heuristic when no model is available or the call fails, so the
review gate degrades safely instead of blocking the run.
"""
artifact_summary = self._summarize_results(results)
if self.client:
try:
response = await self.client.chat.completions.create(
model=self.model,
messages=[
{
"role": "system",
"content": (
"Evaluate whether the specialist results jointly satisfy the objective and are "
"semantically aligned. Compare claimed error behavior, function/API names, and usage "
"examples across specialists. Reject when behavior claims conflict or the objective is "
"unmet. Return JSON with keys: accepted (bool), summary (str), retry_tasks (list of the "
"result keys that must be redone)."
),
},
{
"role": "user",
"content": json.dumps(
{
"objective": objective,
"tasks": tasks,
"results": artifact_summary,
"result_keys": list(results.keys()),
},
ensure_ascii=False,
),
},
],
response_format={"type": "json_object"},
timeout=_planner_timeout(),
)
result = json.loads(response.choices[0].message.content or "{}")
if result:
result.setdefault("accepted", True)
result.setdefault("summary", "accepted")
result.setdefault("retry_tasks", [])
# Only keep retry targets that are real result keys.
result["retry_tasks"] = [t for t in result["retry_tasks"] if t in results]
return result
except Exception as e:
logger.warning(f"Review LLM call failed ({e}); using heuristic consistency check")
consistency = self._heuristic_consistency_check(results)
if not consistency["accepted"]:
return consistency
return {"accepted": True, "summary": "fallback acceptance", "retry_tasks": []}
async def synthesize(self, objective: str, results: Dict) -> str:
"""Compose one coherent answer from the specialist results.
Uses the model when available; otherwise concatenates specialist summaries so a
unified response always exists.
"""
summary = self._summarize_results(results)
deterministic = " | ".join(
f"{key}: {value.get('summary') or value.get('changes') or 'no summary'}"
for key, value in summary.items()
) or objective
if not self.client:
return deterministic
try:
response = await self.client.chat.completions.create(
model=self.model,
messages=[
{
"role": "system",
"content": (
"Synthesize the specialist results into one concise, user-facing answer to the "
"objective. Resolve overlaps in favor of implementation semantics."
),
},
{
"role": "user",
"content": json.dumps(
{"objective": objective, "results": summary}, ensure_ascii=False
),
},
],
timeout=_planner_timeout(),
)
return (response.choices[0].message.content or "").strip() or deterministic
except Exception as e:
logger.warning(f"Synthesis LLM call failed ({e}); using concatenated summary")
return deterministic
def _summarize_results(self, results: Dict) -> Dict:
summary = {}
for task_id, payload in results.items():
result = (payload or {}).get("result", {}) or {}
subtasks = result.get("subtasks", []) or []
files, combined_changes, combined_summary = [], [], []
for item in subtasks:
files.extend(item.get("files_modified", []) or [])
if item.get("changes"):
combined_changes.append(item.get("changes"))
if item.get("summary"):
combined_summary.append(item.get("summary"))
if not combined_summary and result.get("summary"):
combined_summary.append(result.get("summary"))
summary[task_id] = {
"files_modified": files,
"summary": " ".join(combined_summary),
"changes": " ".join(combined_changes),
}
return summary
def _heuristic_consistency_check(self, results: Dict) -> Dict:
combined_text, task_ids = [], []
for task_id, payload in results.items():
task_ids.append(task_id)
result = (payload or {}).get("result", {}) or {}
for item in result.get("subtasks", []) or []:
combined_text.append(item.get("summary", "") or "")
combined_text.append(item.get("changes", "") or "")
for file_item in item.get("files", []) or []:
combined_text.append(file_item.get("content", "") or "")
joined = "\n".join(combined_text).lower()
if ("valueerror" in joined) and ("zerodivisionerror" in joined or "zero division" in joined):
retry = [t for t in task_ids if "documentation" in t or "testing" in t] or task_ids
return {
"accepted": False,
"summary": "conflicting error semantics detected between specialists",
"retry_tasks": retry,
}
if ("pytest" in joined) and ("unittest" in joined):
retry = [t for t in task_ids if "testing" in t or "documentation" in t] or task_ids
return {
"accepted": False,
"summary": "conflicting test framework expectations detected between specialists",
"retry_tasks": retry,
}
return {"accepted": True, "summary": "heuristic consistency acceptance", "retry_tasks": []}
planner = Planner()
+143
View File
@@ -0,0 +1,143 @@
"""Redis client for orchestrator state management.
Redis is the canonical store. An in-memory fallback (backed by the in-process
``fakeredis`` emulator, which faithfully implements the Redis list/hash API) is
available ONLY for local development and CI, and ONLY when explicitly enabled via
``REDIS_FAKE=1`` or ``ALLOW_MEMORY_STORE=1``. In production neither flag is set, so a
Redis outage fails fast on startup instead of silently dropping durability/Manager
state.
"""
import os
from typing import Optional
import logging
try: # redis-py is required in production; guarded so dev/CI can run on fakeredis only.
import redis.asyncio as redis
except Exception: # pragma: no cover - exercised only when redis-py is absent
redis = None
logger = logging.getLogger(__name__)
def _truthy(value: Optional[str]) -> bool:
return (value or "").strip().lower() in {"1", "true", "yes", "on"}
class RedisClient:
"""Async Redis client wrapper for orchestrator operations."""
def __init__(self):
self.client = None
self.host = os.getenv("REDIS_HOST", "redis-service")
self.port = int(os.getenv("REDIS_PORT", "6379"))
self.db = int(os.getenv("REDIS_DB", "0"))
def _fallback_allowed(self) -> bool:
return _truthy(os.getenv("REDIS_FAKE")) or _truthy(os.getenv("ALLOW_MEMORY_STORE"))
def _make_fake_client(self):
"""Return an in-process fakeredis client (dev/CI fallback only)."""
import fakeredis.aioredis as fakeredis # imported lazily; dev/CI dependency
return fakeredis.FakeRedis(decode_responses=True)
async def connect(self):
"""Establish Redis connection, or a gated in-memory fallback for dev/CI."""
# Explicit fake mode (used by local runs and tests) short-circuits real Redis.
if _truthy(os.getenv("REDIS_FAKE")):
self.client = self._make_fake_client()
logger.warning("REDIS_FAKE enabled; using in-memory fakeredis (NOT for production)")
return
try:
if redis is None:
raise RuntimeError("redis-py is not installed")
self.client = redis.Redis(
host=self.host,
port=self.port,
db=self.db,
decode_responses=True,
socket_connect_timeout=5,
socket_keepalive=True,
)
await self.client.ping()
logger.info(f"Connected to Redis at {self.host}:{self.port}")
except Exception as e:
if self._fallback_allowed():
logger.warning(
f"Redis unavailable ({e}); ALLOW_MEMORY_STORE set, using in-memory fakeredis "
"fallback (NOT for production)"
)
self.client = self._make_fake_client()
return
logger.error(f"Failed to connect to Redis: {e}")
raise
async def disconnect(self):
"""Close Redis connection."""
if self.client:
await self.client.close()
logger.info("Disconnected from Redis")
async def set(self, key: str, value: str, ex: Optional[int] = None):
"""Set key-value pair with optional expiration."""
await self.client.set(key, value, ex=ex)
async def get(self, key: str) -> Optional[str]:
"""Get value by key."""
return await self.client.get(key)
async def delete(self, key: str):
"""Delete key."""
await self.client.delete(key)
async def exists(self, key: str) -> bool:
"""Check if key exists."""
return await self.client.exists(key) > 0
async def hset(self, name: str, key: str, value: str):
"""Set hash field."""
await self.client.hset(name, key, value)
async def hget(self, name: str, key: str) -> Optional[str]:
"""Get hash field."""
return await self.client.hget(name, key)
async def hgetall(self, name: str) -> dict:
"""Get all hash fields."""
return await self.client.hgetall(name)
async def hdel(self, name: str, *keys: str):
"""Delete hash fields."""
await self.client.hdel(name, *keys)
async def keys(self, pattern: str) -> list:
"""Get keys matching pattern."""
return await self.client.keys(pattern)
async def lpush(self, key: str, *values: str):
"""Push values to list head."""
await self.client.lpush(key, *values)
async def rpush(self, key: str, *values: str):
"""Push values to list tail."""
await self.client.rpush(key, *values)
async def rpop(self, key: str) -> Optional[str]:
"""Pop value from list tail."""
return await self.client.rpop(key)
async def llen(self, key: str) -> int:
"""Get list length."""
return await self.client.llen(key)
async def lrange(self, key: str, start: int, end: int) -> list:
"""Get list range."""
return await self.client.lrange(key, start, end)
async def lrem(self, key: str, count: int, value: str) -> int:
"""Remove values from list."""
return await self.client.lrem(key, count, value)
# Global Redis client instance
redis_client = RedisClient()
+17
View File
@@ -0,0 +1,17 @@
fastapi==0.115.0
uvicorn[standard]==0.32.0
websockets==13.1
redis==5.2.0
openai==1.55.3
pydantic==2.9.2
python-dotenv==1.0.1
prometheus-client==0.20.0
httpx==0.28.1
# dev/CI only: in-memory Redis emulator for the gated REDIS_FAKE/ALLOW_MEMORY_STORE fallback
fakeredis==2.26.1
opentelemetry-api==1.24.0
opentelemetry-sdk==1.24.0
opentelemetry-exporter-otlp==1.24.0
opentelemetry-instrumentation-redis==0.45b0
opentelemetry-instrumentation-requests==0.45b0
opentelemetry-instrumentation-logging==0.45b0
+781
View File
@@ -0,0 +1,781 @@
"""Agent Manager compatible swarm runtime bridge."""
import asyncio
from copy import deepcopy
import hashlib
import hmac
import json
import logging
import os
import time
import uuid
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional
import httpx
from pydantic import BaseModel, Field
from .redis_client import redis_client
logger = logging.getLogger(__name__)
class CallbackConfig(BaseModel):
"""Callback settings provided by Agent Manager."""
url: Optional[str] = None
signing_secret_ref: Optional[str] = None
subscribed_events: List[str] = Field(default_factory=list)
class SwarmRun(BaseModel):
"""Runtime-owned representation of a swarm run."""
deployment_id: str
swarm_id: str
mode: str = "swarm"
status: str
objective: str
manager_deployment_id: Optional[str] = None
correlation_id: Optional[str] = None
callback: CallbackConfig = Field(default_factory=CallbackConfig)
task_ids: List[str] = Field(default_factory=list)
approvals: Dict[str, Dict[str, Any]] = Field(default_factory=dict)
metadata: Dict[str, Any] = Field(default_factory=dict)
request_body: Dict[str, Any] = Field(default_factory=dict)
created_at: float = Field(default_factory=time.time)
updated_at: float = Field(default_factory=time.time)
class RuntimeValidationError(ValueError):
"""Validation error returned in the Agent Manager runtime envelope."""
def __init__(self, message: str, code: str = "VALIDATION_ERROR"):
super().__init__(message)
self.message = message
self.code = code
class SwarmRuntime:
"""Stores swarm runs and emits Agent Manager callback events."""
RUN_KEY_PREFIX = "swarm:"
TASK_RUN_KEY_PREFIX = "swarm_task:"
IDEMPOTENCY_KEY_PREFIX = "swarm_idempotency:"
EVENT_KEY_PREFIX = "swarm_events:"
def __init__(self):
self.callback_service_token = (
os.getenv("AGENT_CALLBACK_SERVICE_TOKEN")
or os.getenv("AGNET_CALLBACK_SERVICE_TOKEN")
)
self.callback_signing_secret = (
os.getenv("AGENT_CALLBACK_SIGNING_SECRET")
or os.getenv("AGNET_CALLBACK_SIGNING_SECRET")
)
self.runtime_source = os.getenv("SWARM_RUNTIME_SOURCE", "heicode-swarm-runtime")
def multi_agent_workflow_enabled(self, body: Optional[Dict[str, Any]] = None) -> bool:
"""Return whether the DAG-style runtime workflow is enabled."""
if os.getenv("ENABLE_SUBTASK_HANDOFF", "false").lower() not in {"1", "true", "yes"}:
return False
plan = (body or {}).get("orchestration_plan") or {}
agents = plan.get("agents") or (body or {}).get("agents") or []
return len(agents) > 0
async def health(self) -> Dict[str, Any]:
"""Return Agent Manager compatible runtime health."""
await redis_client.client.ping()
return {
"success": True,
"data": {
"status": "healthy",
"service": "heicode-swarm-runtime",
"version": "1.0.0",
"runtime": os.getenv("SWARM_RUNTIME_PLATFORM", "aks"),
"time": self._now_iso(),
"capabilities": [
"swarm.create",
"task.flow",
"handoff.events",
"artifact.events",
"approval.pause_resume",
"deployment.stop",
"runtime.tasks.query",
"runtime.logs.query",
"runtime.events.query",
"runtime.metrics.query",
"runtime.workflow.query",
"runtime.diagnostics.query",
],
},
}
def normalize_create_request(self, body: Dict[str, Any]) -> Dict[str, Any]:
"""Normalize the Manager swarm request into the runtime request shape."""
normalized = deepcopy(body or {})
metadata = dict(normalized.get("metadata") or {})
callback = dict(normalized.get("callback") or {})
requirement = dict(normalized.get("requirement") or {})
model_selection = dict(normalized.get("model_selection") or {})
existing_plan = dict(normalized.get("orchestration_plan") or {})
uses_new_shape = normalized.get("mode") == "swarm" or bool(requirement)
if uses_new_shape:
primary_model = (
model_selection.get("primary_model")
or existing_plan.get("model_id")
or normalized.get("model_id")
)
budget = deepcopy(
normalized.get("budget")
or existing_plan.get("budget")
or {}
)
if not existing_plan.get("objective") and requirement.get("objective"):
existing_plan["objective"] = requirement.get("objective")
existing_plan.setdefault("sub_mode", "swarm")
existing_plan.setdefault("risk_level", normalized.get("risk_level") or "medium")
existing_plan.setdefault("budget", budget)
existing_plan["model_id"] = primary_model
existing_plan["requirement"] = self._redact_sensitive(requirement)
existing_plan["attachments"] = self._redact_sensitive(
requirement.get("attachments") or []
)
existing_plan["constraints"] = self._redact_sensitive(
requirement.get("constraints") or []
)
existing_plan["acceptance_criteria"] = self._redact_sensitive(
requirement.get("acceptance_criteria") or []
)
if not existing_plan.get("agents"):
existing_plan["agents"] = normalized.get("agents") or []
normalized["mode"] = "swarm"
normalized["conversation_id"] = (
normalized.get("conversation_id")
or metadata.get("conversation_id")
)
normalized["requirement"] = requirement
normalized["model_selection"] = model_selection
normalized["model_id"] = primary_model
normalized["orchestration_plan"] = existing_plan
if normalized.get("conversation_id"):
metadata.setdefault("conversation_id", normalized.get("conversation_id"))
if requirement.get("objective"):
metadata.setdefault("objective", requirement.get("objective"))
if model_selection.get("type"):
metadata.setdefault("model_selection_type", model_selection.get("type"))
else:
normalized["orchestration_plan"] = existing_plan
normalized["metadata"] = metadata
normalized["callback"] = callback
return normalized
def validate_create_request(self, body: Dict[str, Any]):
"""Validate the Manager create request before a runtime run is persisted."""
normalized = self.normalize_create_request(body)
plan = normalized.get("orchestration_plan")
metadata = normalized.get("metadata")
callback = normalized.get("callback")
required_paths = {
"orchestration_plan.objective": plan.get("objective") if isinstance(plan, dict) else None,
"callback.url": callback.get("url") if isinstance(callback, dict) else None,
"metadata.manager_deployment_id": metadata.get("manager_deployment_id") if isinstance(metadata, dict) else None,
}
missing = [path for path, value in required_paths.items() if value in (None, "", [], {})]
if missing:
raise RuntimeValidationError(f"Missing required field(s): {', '.join(missing)}")
if normalized.get("mode") not in (None, "swarm"):
raise RuntimeValidationError("mode must be 'swarm'")
if normalized.get("model_selection"):
selection_type = (normalized.get("model_selection") or {}).get("type")
if selection_type not in (None, "primary"):
raise RuntimeValidationError("model_selection.type must be 'primary' for swarm")
billing_secret = (normalized.get("billing_context") or {}).get("secret_ref")
if billing_secret and not self._is_azkv_ref(billing_secret):
raise RuntimeValidationError("billing_context.secret_ref must use azkv://")
self._validate_resource_grants(normalized.get("resource_grants") or [], "resource_grants")
for index, agent in enumerate(plan.get("agents") or []):
self._validate_resource_grants(
agent.get("resource_grants") or [],
f"orchestration_plan.agents[{index}].resource_grants",
)
for index, agent in enumerate(normalized.get("agents") or []):
self._validate_resource_grants(
agent.get("resource_grants") or [],
f"agents[{index}].resource_grants",
)
for path in ("metadata", "resource_grants", "callback"):
value = normalized.get(path)
if value is not None:
self._reject_plaintext_secrets(value, path)
async def get_or_create_run(
self,
body: Dict[str, Any],
idempotency_key: Optional[str],
correlation_id: Optional[str],
) -> tuple[SwarmRun, bool]:
"""Create a swarm run or return the previous run for an idempotency key."""
body = self.normalize_create_request(body)
if idempotency_key:
existing_swarm_id = await redis_client.get(
f"{self.IDEMPOTENCY_KEY_PREFIX}{idempotency_key}"
)
if existing_swarm_id:
existing_run = await self.get_run(existing_swarm_id)
if existing_run:
return existing_run, False
metadata = body.get("metadata") or {}
plan = body.get("orchestration_plan") or {}
callback = CallbackConfig.model_validate(body.get("callback") or {})
swarm_id = f"swarm-{uuid.uuid4().hex[:12]}"
deployment_id = f"runtime-dep-{uuid.uuid4().hex[:12]}"
manager_deployment_id = (
metadata.get("manager_deployment_id")
or metadata.get("heicode_deployment_id")
)
objective = (
plan.get("objective")
or metadata.get("objective")
or "Swarm runtime task"
)
run = SwarmRun(
deployment_id=deployment_id,
swarm_id=swarm_id,
mode="swarm",
status="running",
objective=objective,
manager_deployment_id=manager_deployment_id,
correlation_id=correlation_id or metadata.get("correlation_id"),
callback=callback,
metadata=self._redact_sensitive(metadata),
request_body=self._redact_sensitive(body),
)
if self._requires_approval(body):
approval = self._build_approval(body, run)
run.status = "waiting_approval"
run.approvals[approval["approval_id"]] = approval
await self.save_run(run)
if idempotency_key:
await redis_client.set(
f"{self.IDEMPOTENCY_KEY_PREFIX}{idempotency_key}",
run.swarm_id,
ex=86400,
)
await self.emit_event(
run,
"deployment.status_changed",
payload=self.status_payload(run, phase="Plan"),
)
for approval in run.approvals.values():
await self.emit_event(run, "approval.requested", payload=approval)
return run, True
async def save_run(self, run: SwarmRun):
"""Persist a swarm run."""
run.updated_at = time.time()
await redis_client.set(
f"{self.RUN_KEY_PREFIX}{run.swarm_id}",
run.model_dump_json(),
)
async def get_run(self, swarm_id: str) -> Optional[SwarmRun]:
"""Get a swarm run by id."""
data = await redis_client.get(f"{self.RUN_KEY_PREFIX}{swarm_id}")
if not data:
return None
return SwarmRun.model_validate_json(data)
async def get_run_by_identifier(self, identifier: str) -> Optional[SwarmRun]:
"""Find a run by swarm id, runtime deployment id, or manager deployment id."""
direct = await self.get_run(identifier)
if direct:
return direct
for key in await redis_client.keys(f"{self.RUN_KEY_PREFIX}*"):
data = await redis_client.get(key)
if not data:
continue
run = SwarmRun.model_validate_json(data)
if identifier in {run.deployment_id, run.manager_deployment_id, run.swarm_id}:
return run
return None
async def get_run_for_task(self, task_id: str) -> Optional[SwarmRun]:
"""Find the swarm run that owns a task."""
swarm_id = await redis_client.get(f"{self.TASK_RUN_KEY_PREFIX}{task_id}")
if not swarm_id:
return None
return await self.get_run(swarm_id)
async def attach_task(self, run: SwarmRun, task_id: str):
"""Associate a queue task with a swarm run."""
if task_id not in run.task_ids:
run.task_ids.append(task_id)
await redis_client.set(f"{self.TASK_RUN_KEY_PREFIX}{task_id}", run.swarm_id)
await self.save_run(run)
async def stop_run(self, deployment_id: str, reason: str = "") -> Optional[SwarmRun]:
"""Stop a run by runtime deployment id or swarm id."""
run = await self.find_run_by_deployment_id(deployment_id)
if not run:
return None
run.status = "stopped"
await self.save_run(run)
await self.emit_event(
run,
"deployment.status_changed",
payload=self.status_payload(
run,
phase="Deliver",
reason=reason or "Heicode Manager requested stop",
),
)
return run
async def find_run_by_deployment_id(self, deployment_id: str) -> Optional[SwarmRun]:
"""Find a run by runtime deployment id or swarm id."""
return await self.get_run_by_identifier(deployment_id)
async def record_approval_decision(
self,
swarm_id: str,
approval_id: str,
decision: Dict[str, Any],
) -> Optional[SwarmRun]:
"""Persist an approval decision and update run state."""
run = await self.get_run_by_identifier(swarm_id)
if not run:
return None
approval = run.approvals.get(approval_id, {"approval_id": approval_id})
approval["decision"] = decision.get("decision")
credential_lease = decision.get("credential_lease") or {}
approval["credential_ref"] = (
decision.get("credential_ref") or credential_lease.get("credential_ref")
)
approval["lease_id"] = decision.get("lease_id") or credential_lease.get("lease_id")
approval["lease_expires_at"] = (
decision.get("lease_expires_at") or credential_lease.get("expires_at")
)
approval["decided_at"] = self._now_iso()
run.approvals[approval_id] = self._redact_sensitive(approval)
if decision.get("decision") == "approved":
run.status = "running"
elif decision.get("decision") == "rejected":
run.status = "blocked"
await self.save_run(run)
await self.emit_event(
run,
"deployment.status_changed",
payload=self.status_payload(
run,
phase="Review",
approval_id=approval_id,
decision=decision.get("decision"),
reason=decision.get("reason"),
),
)
if run.status == "blocked":
await self.emit_event(run, "task.blocked", payload={
"approval_id": approval_id,
"reason": decision.get("reason") or "Approval rejected",
"runtime_deployment_id": run.deployment_id,
})
await self.emit_event(run, "timeline.updated", payload={
"summary": "Swarm blocked by approval rejection",
"approval_id": approval_id,
})
return run
async def emit_event(
self,
run: SwarmRun,
event_type: str,
task_id: Optional[str] = None,
agent_instance_id: Optional[str] = None,
payload: Optional[Dict[str, Any]] = None,
artifact: Optional[Dict[str, Any]] = None,
):
"""Record and optionally emit a Manager callback event."""
callback = run.callback
redacted_payload = self._redact_sensitive(payload or {})
redacted_artifact = self._redact_sensitive(artifact) if artifact else None
if redacted_artifact and event_type == "artifact.created":
redacted_payload = {
**redacted_artifact,
**redacted_payload,
}
event_id = f"evt_{uuid.uuid4().hex}"
body: Dict[str, Any] = {
"event_id": event_id,
"idempotency_key": event_id,
"event_type": event_type,
"deployment_id": run.manager_deployment_id or run.deployment_id,
"runtime_deployment_id": run.deployment_id,
"swarm_id": run.swarm_id,
"agent_instance_id": agent_instance_id,
"task_id": task_id,
"occurred_at": self._now_iso(),
"correlation_id": run.correlation_id,
"source": self.runtime_source,
"payload": redacted_payload,
}
if redacted_artifact:
body["artifact"] = redacted_artifact
raw_body = json.dumps(body, ensure_ascii=False, separators=(",", ":"))
await self._store_event(run.swarm_id, raw_body)
if not callback.url:
return
if callback.subscribed_events and event_type not in callback.subscribed_events:
return
headers = {
"Content-Type": "application/json",
"X-Agent-Event-Id": event_id,
"X-Agnet-Event-Id": event_id,
}
if run.correlation_id:
headers["X-Correlation-ID"] = run.correlation_id
if self.callback_service_token:
headers["X-Agent-Service-Token"] = self.callback_service_token
headers["X-Agnet-Service-Token"] = self.callback_service_token
if self.callback_signing_secret:
timestamp = str(int(time.time() * 1000))
signature_payload = f"{timestamp}.{event_id}.{raw_body}"
signature = hmac.new(
self.callback_signing_secret.encode("utf-8"),
signature_payload.encode("utf-8"),
hashlib.sha256,
).hexdigest()
headers["X-Agent-Timestamp"] = timestamp
headers["X-Agent-Signature"] = f"sha256={signature}"
headers["X-Agnet-Timestamp"] = timestamp
headers["X-Agnet-Signature"] = f"sha256={signature}"
if callback.url and (not callback.subscribed_events or event_type in callback.subscribed_events):
await self._upsert_callback_attempt(
run.swarm_id,
event_id,
{
"event_id": event_id,
"event_type": event_type,
"url": callback.url,
"status": "pending",
"attempted_at": self._now_iso(),
},
)
asyncio.create_task(
self._post_callback(run.swarm_id, callback.url, raw_body, headers, event_type, event_id)
)
async def list_events(
self,
swarm_id: str,
limit: int = 100,
cursor: Optional[str] = None,
) -> Dict[str, Any]:
"""Return stored runtime events for a swarm."""
start = int(cursor or 0)
safe_limit = max(1, min(limit, 500))
items = await redis_client.lrange(
f"{self.EVENT_KEY_PREFIX}{swarm_id}",
start,
start + safe_limit - 1,
)
events = [json.loads(item) for item in items]
next_cursor = str(start + safe_limit) if len(events) == safe_limit else None
return {"events": events, "next_cursor": next_cursor}
async def _store_event(self, swarm_id: str, raw_body: str):
await redis_client.rpush(f"{self.EVENT_KEY_PREFIX}{swarm_id}", raw_body)
async def _post_callback(
self,
swarm_id: str,
url: str,
raw_body: str,
headers: Dict[str, str],
event_type: str,
event_id: str,
):
"""Send callback without blocking the agent WebSocket loop."""
if not url:
return
try:
async with httpx.AsyncClient(timeout=10.0) as client:
response = await client.post(url, content=raw_body, headers=headers)
await self._upsert_callback_attempt(
swarm_id,
event_id,
{
"event_id": event_id,
"event_type": event_type,
"url": url,
"status": "delivered" if response.status_code < 400 else "failed",
"response_status": response.status_code,
"attempted_at": self._now_iso(),
},
)
if response.status_code >= 400:
logger.warning(
"Manager callback %s failed with status %s: %s",
event_type,
response.status_code,
response.text[:500],
)
except Exception as exc:
await self._upsert_callback_attempt(
swarm_id,
event_id,
{
"event_id": event_id,
"event_type": event_type,
"url": url,
"status": "failed",
"error": str(exc),
"attempted_at": self._now_iso(),
},
)
logger.warning("Manager callback %s failed: %s", event_type, exc)
def build_task_descriptions(self, body: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Create a simple task graph from a Manager orchestration request."""
body = self.normalize_create_request(body)
plan = body.get("orchestration_plan") or {}
agents = plan.get("agents") or body.get("agents") or []
requirement = body.get("requirement") or {}
objective = (
requirement.get("objective")
or plan.get("objective")
or "Complete swarm objective"
)
multi_agent_enabled = self.multi_agent_workflow_enabled(body)
context = {
"orchestration_plan": self._redact_sensitive(plan),
"resource_grants": self._redact_sensitive(body.get("resource_grants") or []),
"sub_mode": body.get("sub_mode") or plan.get("sub_mode"),
"conversation_id": body.get("conversation_id"),
"mode": body.get("mode") or "swarm",
"requirement": self._redact_sensitive(requirement),
}
if not agents or not multi_agent_enabled:
return [{
"task_id": "task-1",
"title": "Swarm objective",
"description": objective,
"agent_role": "general",
"required_capabilities": ["general"],
"depends_on": [],
"parent_task_id": None,
"root_task_id": "task-1",
"source": "runtime_bridge",
"workflow_mode": "single_agent",
"allow_handoff": False,
"context": context,
}]
tasks = []
for index, agent in enumerate(agents, start=1):
role = agent.get("role") or f"agent-{index}"
title = agent.get("title") or f"{role} task"
description = agent.get("description") or f"[{role}] {objective}"
task_id = agent.get("task_id") or f"task-{index}"
required_capabilities = agent.get("required_capabilities") or [role]
tasks.append({
"task_id": task_id,
"title": title,
"description": description,
"agent_role": role,
"required_capabilities": required_capabilities,
"depends_on": agent.get("depends_on") or [],
"parent_task_id": None,
"root_task_id": task_id,
"source": "runtime_bridge",
"workflow_mode": "multi_agent",
"allow_handoff": True,
"context": {
**context,
"agent_role": role,
"workflow_mode": "multi_agent",
"resource_grants": self._redact_sensitive(
agent.get("resource_grants") or []
),
},
})
return tasks
def task_event_payload(self, task: Any, task_spec: Dict[str, Any]) -> Dict[str, Any]:
"""Return the minimum task graph payload required by Manager."""
return {
"task_id": task.task_id,
"title": task.title or task_spec.get("title", task.description[:80]),
"description": task.description,
"agent_role": task.agent_role or task_spec.get("agent_role", "general"),
"status": task.status.value if hasattr(task.status, "value") else task.status,
"depends_on": task.depends_on or task_spec.get("depends_on") or [],
"parent_task_id": task.parent_task_id or task_spec.get("parent_task_id"),
"root_task_id": task.root_task_id or task_spec.get("root_task_id"),
"required_capabilities": task.required_capabilities or task_spec.get("required_capabilities") or [],
"source": task.source or task_spec.get("source", "runtime_bridge"),
"attempt": getattr(task, "retry_count", 0),
}
def _requires_approval(self, body: Dict[str, Any]) -> bool:
plan = body.get("orchestration_plan") or {}
agile = plan.get("agile_context") or {}
risk = str(plan.get("risk_level") or body.get("risk_level") or "").lower()
return bool(agile.get("requires_user_approval")) or risk == "high"
def status_payload(
self,
run: SwarmRun,
phase: str,
reason: Optional[str] = None,
approval_id: Optional[str] = None,
decision: Optional[str] = None,
deliverable: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""Build the standard runtime status payload."""
payload: Dict[str, Any] = {
"status": run.status,
"runtime_execution_status": run.status,
"runtime_deployment_id": run.deployment_id,
"manager_deployment_id": run.manager_deployment_id,
"phase": phase,
}
if reason:
payload["reason"] = reason
if approval_id:
payload["approval_id"] = approval_id
if decision:
payload["decision"] = decision
if deliverable is not None:
payload["deliverable"] = deliverable
return payload
async def _upsert_callback_attempt(
self,
swarm_id: str,
event_id: str,
attempt: Dict[str, Any],
):
run = await self.get_run(swarm_id)
if not run:
return
callback_attempts = list(run.metadata.get("callback_attempts") or [])
filtered = [item for item in callback_attempts if item.get("event_id") != event_id]
filtered.append(self._redact_sensitive(attempt))
run.metadata["callback_attempts"] = filtered[-50:]
await self.save_run(run)
def _build_approval(self, body: Dict[str, Any], run: SwarmRun) -> Dict[str, Any]:
grants = body.get("resource_grants") or []
grant = grants[0] if grants else {}
return {
"approval_id": f"appr_{uuid.uuid4().hex[:12]}",
"operation": "git.write",
"resource_id": grant.get("resource_id", "repo-main"),
"resource_type": grant.get("resource_type", "git"),
"resource_scope": ",".join(grant.get("permission_scope") or []),
"target_role": grant.get("target_role", "general"),
"risk_level": "high",
"requires_credential": True,
"secret_ref": grant.get("secret_ref") or grant.get("ref"),
"ttl_seconds": 900,
"reason": "High-risk swarm run requires Manager approval",
"runtime_deployment_id": run.deployment_id,
}
def _validate_resource_grants(self, grants: List[Dict[str, Any]], path: str):
for index, grant in enumerate(grants):
for key in ("secret_ref", "ref"):
value = grant.get(key)
if value and not self._is_azkv_ref(value):
raise RuntimeValidationError(f"{path}[{index}].{key} must use azkv://")
def _reject_plaintext_secrets(self, value: Any, path: str):
if isinstance(value, dict):
for key, item in value.items():
child_path = f"{path}.{key}"
key_lower = key.lower()
is_ref_key = key_lower.endswith("_ref") or key_lower == "ref"
if self._looks_sensitive_key(key_lower) and not is_ref_key:
raise RuntimeValidationError(
f"Plaintext secret-like field is not allowed: {child_path}"
)
if is_ref_key and isinstance(item, str) and item and not self._is_azkv_ref(item):
raise RuntimeValidationError(f"{child_path} must use azkv://")
self._reject_plaintext_secrets(item, child_path)
elif isinstance(value, list):
for index, item in enumerate(value):
self._reject_plaintext_secrets(item, f"{path}[{index}]")
def _looks_sensitive_key(self, key: str) -> bool:
return (
any(part in key for part in ("token", "password", "passwd", "secret", "private_key", "api_key"))
or key.endswith("_key")
)
def _is_azkv_ref(self, value: Any) -> bool:
return isinstance(value, str) and value.startswith("azkv://")
def _redact_sensitive(self, value: Any) -> Any:
"""Remove obvious plaintext secrets from callback-safe payloads."""
sensitive_keys = {
"password",
"passwd",
"token",
"api_token",
"access_token",
"refresh_token",
"private_key",
"access_key",
"secret",
"client_secret",
"connection_string",
}
if isinstance(value, dict):
redacted = {}
for key, item in value.items():
key_lower = key.lower()
if key_lower in {"secret_ref", "credential_ref", "signing_secret_ref"}:
redacted[key] = item
elif key_lower in sensitive_keys or key_lower.endswith(("_token", "_secret", "_password", "_key")):
redacted[key] = "[redacted]"
else:
redacted[key] = self._redact_sensitive(item)
return redacted
if isinstance(value, list):
return [self._redact_sensitive(item) for item in value]
return value
def _now_iso(self) -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
swarm_runtime = SwarmRuntime()
+511
View File
@@ -0,0 +1,511 @@
"""Task queue management with dependency-aware failure recovery."""
import json
import time
import uuid
import logging
from typing import Dict, List, Optional
from enum import Enum
from pydantic import BaseModel, Field
from .redis_client import redis_client
from .agent_registry import agent_registry, AgentStatus
logger = logging.getLogger(__name__)
class TaskStatus(str, Enum):
"""Task status enumeration."""
PENDING = "pending"
ASSIGNED = "assigned"
IN_PROGRESS = "in_progress"
BLOCKED = "blocked"
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled"
class Task(BaseModel):
"""Task model."""
task_id: str
title: Optional[str] = None
description: str
status: TaskStatus
agent_role: str = "general"
required_capabilities: List[str] = Field(default_factory=list)
depends_on: List[str] = Field(default_factory=list)
parent_task_id: Optional[str] = None
root_task_id: Optional[str] = None
source: str = "manual"
assigned_agent_id: Optional[str] = None
created_at: float
started_at: Optional[float] = None
completed_at: Optional[float] = None
result: Optional[str] = None
blocked_reason: Optional[str] = None
child_task_ids: List[str] = Field(default_factory=list)
retry_count: int = 0
max_retries: int = 3
context: Dict = Field(default_factory=dict)
class TaskQueue:
"""Manages task assignment and reassignment with failure recovery."""
TASK_KEY_PREFIX = "task:"
PENDING_QUEUE_KEY = "queue:pending"
AGENT_TASK_KEY_PREFIX = "agent_task:"
def __init__(self):
pass
async def _save_task(self, task: Task):
"""Persist a task snapshot."""
key = f"{self.TASK_KEY_PREFIX}{task.task_id}"
await redis_client.set(key, task.model_dump_json())
async def create_task(
self,
description: str,
context: Optional[Dict] = None,
max_retries: int = 3,
task_id: Optional[str] = None,
title: Optional[str] = None,
agent_role: str = "general",
required_capabilities: Optional[List[str]] = None,
depends_on: Optional[List[str]] = None,
parent_task_id: Optional[str] = None,
root_task_id: Optional[str] = None,
source: str = "manual",
enqueue: bool = True,
) -> Task:
"""Create a new task and add to pending queue."""
task = Task(
task_id=task_id or str(uuid.uuid4()),
title=title,
description=description,
status=TaskStatus.PENDING,
agent_role=agent_role,
required_capabilities=required_capabilities or [],
depends_on=depends_on or [],
parent_task_id=parent_task_id,
root_task_id=root_task_id,
source=source,
created_at=time.time(),
max_retries=max_retries,
context=context or {},
)
await self._save_task(task)
if enqueue:
await redis_client.lpush(self.PENDING_QUEUE_KEY, task.task_id)
logger.info(f"Created task {task.task_id}: {description}")
return task
async def add_child_task(self, parent_task_id: str, child_task_id: str):
"""Register a child task on a parent task."""
parent = await self.get_task(parent_task_id)
if not parent:
return
if child_task_id not in parent.child_task_ids:
parent.child_task_ids.append(child_task_id)
await self._save_task(parent)
async def get_ready_pending_task(
self,
agent_capabilities: Optional[List[str]] = None,
) -> Optional[Task]:
"""Return and dequeue the next dispatchable task for an agent."""
pending_ids = await redis_client.lrange(self.PENDING_QUEUE_KEY, 0, -1)
capabilities = set(agent_capabilities or [])
for task_id in pending_ids:
task = await self.get_task(task_id)
if not task:
await self.remove_pending_task(task_id)
continue
if task.status != TaskStatus.PENDING:
await self.remove_pending_task(task_id)
continue
if not await self.is_task_ready(task):
continue
if not self.can_agent_run_task(task, capabilities):
continue
await self.remove_pending_task(task_id)
return task
return None
async def is_task_ready(self, task: Task) -> bool:
"""Return True when all dependencies are terminal and successful."""
if task.status != TaskStatus.PENDING:
return False
for dependency_id in task.depends_on:
dependency = await self.get_task(dependency_id)
if not dependency or dependency.status != TaskStatus.COMPLETED:
return False
return True
def can_agent_run_task(self, task: Task, capabilities: set[str]) -> bool:
"""Return whether an agent capability set satisfies task requirements."""
required = set(task.required_capabilities or [])
if not required:
return True
return required.issubset(capabilities)
async def assign_task(self, task_id: str, agent_id: str) -> bool:
"""Assign a task to an agent."""
# Get task
task = await self.get_task(task_id)
if not task:
logger.error(f"Task {task_id} not found")
return False
if task.status not in [TaskStatus.PENDING, TaskStatus.FAILED]:
logger.error(f"Task {task_id} cannot be assigned (status: {task.status})")
return False
if task.status == TaskStatus.PENDING and not await self.is_task_ready(task):
logger.error(f"Task {task_id} cannot be assigned before dependencies complete")
return False
# Verify agent is idle
agent = await agent_registry.get_agent(agent_id)
if not agent or agent.status != AgentStatus.IDLE:
logger.error(f"Agent {agent_id} is not available for task assignment")
return False
if not self.can_agent_run_task(task, set(agent.capabilities)):
logger.error(f"Agent {agent_id} does not satisfy task {task_id} capabilities")
return False
# Update task
task.status = TaskStatus.ASSIGNED
task.assigned_agent_id = agent_id
task.started_at = time.time()
task.blocked_reason = None
await self._save_task(task)
# Track agent's current task
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{agent_id}"
await redis_client.set(agent_task_key, task_id)
# Update agent status
await agent_registry.update_status(agent_id, AgentStatus.BUSY, task_id)
logger.info(f"Assigned task {task_id} to agent {agent_id}")
return True
async def start_task(self, task_id: str) -> bool:
"""Mark task as in progress."""
task = await self.get_task(task_id)
if not task:
return False
task.status = TaskStatus.IN_PROGRESS
task.blocked_reason = None
await self._save_task(task)
logger.info(f"Task {task_id} started")
return True
async def block_task(
self,
task_id: str,
reason: str = "",
release_agent: bool = True,
) -> Optional[Task]:
"""Mark a task blocked while waiting for delegated child work."""
task = await self.get_task(task_id)
if not task:
return None
if task.status in {TaskStatus.COMPLETED, TaskStatus.FAILED, TaskStatus.CANCELLED}:
return task
previous_agent_id = task.assigned_agent_id
task.status = TaskStatus.BLOCKED
task.blocked_reason = reason or "Waiting for delegated handoff work"
task.assigned_agent_id = None if release_agent else task.assigned_agent_id
await self.remove_pending_task(task_id)
await self._save_task(task)
if release_agent and previous_agent_id:
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{previous_agent_id}"
await redis_client.delete(agent_task_key)
await agent_registry.update_status(previous_agent_id, AgentStatus.IDLE)
logger.info(f"Task {task_id} blocked: {task.blocked_reason}")
return task
async def complete_task(self, task_id: str, result: str = None) -> bool:
"""Mark task as completed."""
task = await self.get_task(task_id)
if not task:
return False
if task.status == TaskStatus.CANCELLED:
logger.warning(f"Ignoring completion for cancelled task {task_id}")
return False
task.status = TaskStatus.COMPLETED
task.completed_at = time.time()
task.blocked_reason = None
if result:
task.result = result
await self._save_task(task)
# Clear agent's current task
if task.assigned_agent_id:
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
await redis_client.delete(agent_task_key)
# Update agent to idle
await agent_registry.update_status(task.assigned_agent_id, AgentStatus.IDLE)
logger.info(f"Task {task_id} completed")
return True
async def fail_task(self, task_id: str, reason: str = "") -> bool:
"""Mark task as failed and handle retry logic."""
task = await self.get_task(task_id)
if not task:
return False
if task.status == TaskStatus.CANCELLED:
logger.warning(f"Ignoring failure for cancelled task {task_id}: {reason}")
return False
task.retry_count += 1
# Clear agent's current task
if task.assigned_agent_id:
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
await redis_client.delete(agent_task_key)
# Update agent to idle
await agent_registry.update_status(task.assigned_agent_id, AgentStatus.IDLE)
# Check if we should retry
if task.retry_count < task.max_retries:
task.status = TaskStatus.PENDING
task.assigned_agent_id = None
task.started_at = None
# Re-add to pending queue
await redis_client.lpush(self.PENDING_QUEUE_KEY, task_id)
logger.warning(
f"Task {task_id} failed (retry {task.retry_count}/{task.max_retries}): {reason}"
)
else:
task.status = TaskStatus.FAILED
task.completed_at = time.time()
logger.error(
f"Task {task_id} permanently failed after {task.retry_count} retries: {reason}"
)
task.blocked_reason = None if task.status == TaskStatus.PENDING else task.blocked_reason
await self._save_task(task)
return True
async def cancel_task(self, task_id: str, reason: str = "") -> bool:
"""Cancel a task without retrying it."""
task = await self.get_task(task_id)
if not task:
return False
if task.status in {TaskStatus.COMPLETED, TaskStatus.FAILED, TaskStatus.CANCELLED}:
logger.info(f"Task {task_id} already terminal ({task.status}); skip cancellation")
return False
if task.assigned_agent_id:
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
await redis_client.delete(agent_task_key)
await self.remove_pending_task(task_id)
task.status = TaskStatus.CANCELLED
task.completed_at = time.time()
task.blocked_reason = None
if reason:
task.result = json.dumps({"cancelled": True, "reason": reason})
await self._save_task(task)
logger.info(f"Task {task_id} cancelled: {reason}")
return True
async def release_task(self, task_id: str, agent_id: Optional[str] = None) -> bool:
"""Return an assigned/in-progress task to the pending queue (e.g. agent rejected it).
Unlike fail_task this does not increment retry_count: a capacity rejection is not
a task failure, just a dispatch that needs to find a different agent.
"""
task = await self.get_task(task_id)
if not task:
return False
if task.status not in {TaskStatus.ASSIGNED, TaskStatus.IN_PROGRESS}:
return False
if agent_id and task.assigned_agent_id and task.assigned_agent_id != agent_id:
return False
previous_agent_id = task.assigned_agent_id
task.status = TaskStatus.PENDING
task.assigned_agent_id = None
task.started_at = None
task.blocked_reason = None
await self._save_task(task)
await self.requeue_task(task_id)
if previous_agent_id:
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{previous_agent_id}"
await redis_client.delete(agent_task_key)
await agent_registry.update_status(previous_agent_id, AgentStatus.IDLE)
logger.info(f"Released task {task_id} back to pending queue")
return True
async def reopen_task(self, task_id: str) -> bool:
"""Re-open a completed task for another round (review loop rejected its result).
Resets the task to PENDING and requeues it without touching retry_count (a review
rejection is a quality decision, not a failure). The review-cycle budget in the
orchestrator bounds how many times this can happen.
"""
task = await self.get_task(task_id)
if not task:
return False
if task.status not in {TaskStatus.COMPLETED, TaskStatus.FAILED}:
return False
if task.assigned_agent_id:
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{task.assigned_agent_id}"
await redis_client.delete(agent_task_key)
task.status = TaskStatus.PENDING
task.assigned_agent_id = None
task.started_at = None
task.completed_at = None
task.blocked_reason = None
await self._save_task(task)
await self.requeue_task(task_id)
logger.info(f"Re-opened task {task_id} for another review cycle")
return True
async def reassign_agent_tasks(self, failed_agent_id: str) -> List[str]:
"""Reassign all tasks from a failed agent back to pending queue."""
# Get agent's current task
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{failed_agent_id}"
task_id = await redis_client.get(agent_task_key)
reassigned_tasks = []
if task_id:
await self.fail_task(task_id, f"Agent {failed_agent_id} failed")
reassigned_tasks.append(task_id)
logger.info(
f"Reassigned {len(reassigned_tasks)} tasks from failed agent {failed_agent_id}"
)
return reassigned_tasks
async def recover_orphaned_tasks(
self,
active_agent_ids: set[str],
stale_after_seconds: int = 30,
) -> List[str]:
"""Recover assigned/in-progress tasks whose agents are no longer connected."""
current_time = time.time()
recoverable_statuses = {TaskStatus.ASSIGNED, TaskStatus.IN_PROGRESS}
recovered_tasks = []
for task in await self.get_all_tasks():
if task.status not in recoverable_statuses:
continue
if not task.assigned_agent_id:
continue
if task.assigned_agent_id in active_agent_ids:
continue
if task.started_at and current_time - task.started_at < stale_after_seconds:
continue
old_agent_id = task.assigned_agent_id
agent_task_key = f"{self.AGENT_TASK_KEY_PREFIX}{old_agent_id}"
await redis_client.delete(agent_task_key)
task.status = TaskStatus.PENDING
task.assigned_agent_id = None
task.started_at = None
if await self.is_task_ready(task):
await self._save_task(task)
await redis_client.lpush(self.PENDING_QUEUE_KEY, task.task_id)
else:
await self._save_task(task)
recovered_tasks.append(task.task_id)
logger.warning(
f"Recovered orphaned task {task.task_id} from inactive agent {old_agent_id}"
)
return recovered_tasks
async def get_task(self, task_id: str) -> Optional[Task]:
"""Get task by ID."""
key = f"{self.TASK_KEY_PREFIX}{task_id}"
data = await redis_client.get(key)
if not data:
return None
return Task.model_validate_json(data)
async def get_next_pending_task(self) -> Optional[Task]:
"""Get next pending task from queue."""
task_id = await redis_client.rpop(self.PENDING_QUEUE_KEY)
if not task_id:
return None
return await self.get_task(task_id)
async def requeue_task(self, task_id: str):
"""Put a task back on the pending queue."""
await redis_client.rpush(self.PENDING_QUEUE_KEY, task_id)
async def remove_pending_task(self, task_id: str):
"""Remove a task from the pending queue if present."""
await redis_client.lrem(self.PENDING_QUEUE_KEY, 0, task_id)
async def get_pending_count(self) -> int:
"""Get count of pending tasks."""
return await redis_client.llen(self.PENDING_QUEUE_KEY)
async def get_dependents(self, task_id: str) -> List[Task]:
"""Return tasks that directly depend on a task."""
return [
task for task in await self.get_all_tasks()
if task_id in task.depends_on
]
async def get_all_tasks(self, status: Optional[TaskStatus] = None) -> List[Task]:
"""Get all tasks, optionally filtered by status."""
pattern = f"{self.TASK_KEY_PREFIX}*"
keys = await redis_client.keys(pattern)
tasks = []
for key in keys:
data = await redis_client.get(key)
if data:
task = Task.model_validate_json(data)
if status is None or task.status == status:
tasks.append(task)
return tasks
# Global task queue instance
task_queue = TaskQueue()
+415
View File
@@ -0,0 +1,415 @@
"""Regression tests for the multi-agent DAG workflow."""
import fnmatch
import os
import unittest
from types import SimpleNamespace
from unittest.mock import patch
from orchestrator import main as orchestrator_main
from orchestrator import swarm_runtime as swarm_runtime_module
from orchestrator import task_queue as task_queue_module
class FakeRedisClient:
"""Tiny in-memory async Redis replacement for task-queue tests."""
def __init__(self):
self.kv = {}
self.lists = {}
async def set(self, key, value, ex=None):
self.kv[key] = value
async def get(self, key):
return self.kv.get(key)
async def delete(self, key):
self.kv.pop(key, None)
self.lists.pop(key, None)
async def keys(self, pattern):
keys = list(self.kv.keys()) + list(self.lists.keys())
return sorted([key for key in keys if fnmatch.fnmatch(key, pattern)])
async def lpush(self, key, *values):
self.lists.setdefault(key, [])
for value in values:
self.lists[key].insert(0, value)
async def rpush(self, key, *values):
self.lists.setdefault(key, [])
self.lists[key].extend(values)
async def rpop(self, key):
items = self.lists.get(key, [])
if not items:
return None
return items.pop()
async def llen(self, key):
return len(self.lists.get(key, []))
async def lrange(self, key, start, end):
items = list(self.lists.get(key, []))
if end == -1:
end = len(items) - 1
return items[start : end + 1]
async def lrem(self, key, count, value):
items = self.lists.get(key, [])
removed = 0
kept = []
for item in items:
if item == value and (count == 0 or removed < count):
removed += 1
continue
kept.append(item)
self.lists[key] = kept
return removed
class MultiAgentWorkflowTests(unittest.IsolatedAsyncioTestCase):
"""Covers DAG task creation, dependency dispatch, and parent-child resolution."""
def setUp(self):
self.fake_redis = FakeRedisClient()
self.redis_patch = patch.object(task_queue_module, "redis_client", self.fake_redis)
self.redis_patch.start()
self.addCleanup(self.redis_patch.stop)
async def test_build_task_descriptions_respects_feature_flag(self):
body = {
"orchestration_plan": {
"objective": "Implement a feature",
"sub_mode": "code",
"risk_level": "low",
"budget": {"duration_seconds": 600, "token_limit": 1000},
"agents": [
{
"task_id": "backend-1",
"role": "backend",
"title": "Backend task",
"description": "Implement backend changes",
"depends_on": [],
},
{
"task_id": "tests-1",
"role": "testing",
"title": "Testing task",
"description": "Add tests",
"depends_on": ["backend-1"],
},
],
}
}
runtime = swarm_runtime_module.SwarmRuntime()
with patch.dict(os.environ, {"ENABLE_SUBTASK_HANDOFF": "false"}, clear=False):
tasks = runtime.build_task_descriptions(body)
self.assertEqual(len(tasks), 1)
self.assertEqual(tasks[0]["workflow_mode"], "single_agent")
with patch.dict(os.environ, {"ENABLE_SUBTASK_HANDOFF": "true"}, clear=False):
tasks = runtime.build_task_descriptions(body)
self.assertEqual(len(tasks), 2)
self.assertEqual(tasks[0]["task_id"], "backend-1")
self.assertEqual(tasks[1]["depends_on"], ["backend-1"])
self.assertEqual(tasks[1]["workflow_mode"], "multi_agent")
async def test_ready_pending_task_respects_dependencies_and_capabilities(self):
parent = await task_queue_module.task_queue.create_task(
task_id="parent-1",
title="Backend",
description="Implement backend",
agent_role="backend",
required_capabilities=["backend"],
source="runtime_bridge",
)
child = await task_queue_module.task_queue.create_task(
task_id="child-1",
title="Tests",
description="Add tests",
agent_role="testing",
required_capabilities=["testing"],
depends_on=[parent.task_id],
root_task_id=parent.task_id,
source="runtime_bridge",
)
task = await task_queue_module.task_queue.get_ready_pending_task(["testing"])
self.assertIsNone(task, "dependent task should not dispatch before parent completion")
task = await task_queue_module.task_queue.get_ready_pending_task(["backend"])
self.assertIsNotNone(task)
self.assertEqual(task.task_id, parent.task_id)
await task_queue_module.task_queue.complete_task(parent.task_id, "done")
task = await task_queue_module.task_queue.get_ready_pending_task(["testing"])
self.assertIsNotNone(task)
self.assertEqual(task.task_id, child.task_id)
async def test_finalize_parent_after_child_success(self):
event_calls = []
async def fake_emit_event(*args, **kwargs):
event_calls.append((args, kwargs))
runtime_patch = patch.object(orchestrator_main.swarm_runtime, "emit_event", fake_emit_event)
runtime_patch.start()
self.addCleanup(runtime_patch.stop)
parent = await task_queue_module.task_queue.create_task(
task_id="parent-1",
description="Parent task",
title="Parent",
agent_role="backend",
source="runtime_bridge",
)
child = await task_queue_module.task_queue.create_task(
task_id="child-1",
description="Child task",
title="Child",
agent_role="testing",
parent_task_id=parent.task_id,
root_task_id=parent.task_id,
source="dynamic_handoff",
)
await task_queue_module.task_queue.add_child_task(parent.task_id, child.task_id)
await task_queue_module.task_queue.block_task(parent.task_id, "Waiting on child")
await task_queue_module.task_queue.complete_task(child.task_id, "child complete")
refreshed_child = await task_queue_module.task_queue.get_task(child.task_id)
run = SimpleNamespace()
await orchestrator_main.finalize_parent_after_child(
run,
refreshed_child,
agent_id="agent-1",
success=True,
summary="Delegated child task completed",
)
refreshed_parent = await task_queue_module.task_queue.get_task(parent.task_id)
self.assertEqual(refreshed_parent.status, task_queue_module.TaskStatus.COMPLETED)
self.assertGreaterEqual(len(event_calls), 2)
async def test_finalize_parent_after_child_failure_is_terminal(self):
async def fake_emit_event(*args, **kwargs):
return None
runtime_patch = patch.object(orchestrator_main.swarm_runtime, "emit_event", fake_emit_event)
runtime_patch.start()
self.addCleanup(runtime_patch.stop)
parent = await task_queue_module.task_queue.create_task(
task_id="parent-2",
description="Parent task",
title="Parent",
agent_role="backend",
source="runtime_bridge",
max_retries=3,
)
child = await task_queue_module.task_queue.create_task(
task_id="child-2",
description="Child task",
title="Child",
agent_role="testing",
parent_task_id=parent.task_id,
root_task_id=parent.task_id,
source="dynamic_handoff",
max_retries=1,
)
await task_queue_module.task_queue.add_child_task(parent.task_id, child.task_id)
await task_queue_module.task_queue.block_task(parent.task_id, "Waiting on child")
await task_queue_module.task_queue.fail_task(child.task_id, "child failed")
refreshed_child = await task_queue_module.task_queue.get_task(child.task_id)
run = SimpleNamespace()
await orchestrator_main.finalize_parent_after_child(
run,
refreshed_child,
agent_id="agent-1",
success=False,
summary="child failed",
)
refreshed_parent = await task_queue_module.task_queue.get_task(parent.task_id)
self.assertEqual(refreshed_parent.status, task_queue_module.TaskStatus.FAILED)
async def test_build_result_artifact_emits_document_without_git_branch(self):
run = SimpleNamespace(
deployment_id="runtime-dep-1",
manager_deployment_id="dep-manager-1",
swarm_id="swarm-1",
)
task = SimpleNamespace(
task_id="task-1",
title="Backend implementation",
agent_role="backend",
parent_task_id=None,
root_task_id="task-1",
context={},
)
result = {
"summary": "Implemented backend endpoints and startup notes",
"files_modified": ["backend/app.py", "README.md"],
"changes": "Added handlers and docs",
"git_skipped": "Workspace is not a Git checkout",
}
artifact = orchestrator_main.build_result_artifact(run, task, result)
self.assertEqual(artifact["artifact_type"], "deployment_manifest")
self.assertEqual(artifact["uri"], "runtime://swarm-1/artifacts/task-1")
self.assertEqual(
artifact["metadata"]["files_modified"],
["backend/app.py", "README.md"],
)
async def test_artifact_created_event_copies_artifact_into_payload(self):
fake_runtime_redis = FakeRedisClient()
runtime_patch = patch.object(
swarm_runtime_module,
"redis_client",
SimpleNamespace(
set=fake_runtime_redis.set,
get=fake_runtime_redis.get,
keys=fake_runtime_redis.keys,
rpush=fake_runtime_redis.rpush,
lrange=fake_runtime_redis.lrange,
),
)
runtime_patch.start()
self.addCleanup(runtime_patch.stop)
runtime = swarm_runtime_module.SwarmRuntime()
run = swarm_runtime_module.SwarmRun(
deployment_id="runtime-dep-1",
swarm_id="swarm-1",
status="running",
objective="Test callback payload",
manager_deployment_id="dep-manager-1",
)
await runtime.emit_event(
run,
"artifact.created",
task_id="task-1",
artifact={
"artifact_id": "art_task-1",
"artifact_type": "document",
"title": "Task result",
"summary": "Generated a result summary",
"uri": "runtime://swarm-1/artifacts/task-1",
},
)
logs = await runtime.list_events(run.swarm_id)
event = logs["events"][-1]
self.assertEqual(event["payload"]["artifact_id"], "art_task-1")
self.assertEqual(event["payload"]["artifact_type"], "document")
self.assertEqual(event["artifact"]["uri"], "runtime://swarm-1/artifacts/task-1")
async def test_validate_create_request_accepts_new_swarm_shape(self):
runtime = swarm_runtime_module.SwarmRuntime()
runtime.validate_create_request({
"mode": "swarm",
"conversation_id": "conv-1",
"requirement": {
"objective": "Build a swarm-delivered feature",
"context": ["ticket-123"],
"attachments": [],
"constraints": ["keep existing API"],
"acceptance_criteria": ["tests pass"],
},
"model_selection": {
"type": "primary",
"primary_model": "gpt-5.4",
},
"metadata": {
"manager_deployment_id": "dep-manager-1",
"correlation_id": "corr-1",
},
"callback": {
"url": "https://manager.example/api/agent/callbacks/runtime-events",
},
})
async def test_validate_create_request_rejects_missing_objective(self):
runtime = swarm_runtime_module.SwarmRuntime()
with self.assertRaises(swarm_runtime_module.RuntimeValidationError):
runtime.validate_create_request({
"mode": "swarm",
"requirement": {},
"metadata": {"manager_deployment_id": "dep-manager-1"},
"callback": {"url": "https://manager.example/callback"},
})
async def test_get_run_by_identifier_supports_manager_deployment_id(self):
fake_runtime_redis = FakeRedisClient()
runtime_patch = patch.object(
swarm_runtime_module,
"redis_client",
SimpleNamespace(
set=fake_runtime_redis.set,
get=fake_runtime_redis.get,
keys=fake_runtime_redis.keys,
rpush=fake_runtime_redis.rpush,
lrange=fake_runtime_redis.lrange,
),
)
runtime_patch.start()
self.addCleanup(runtime_patch.stop)
runtime = swarm_runtime_module.SwarmRuntime()
run = swarm_runtime_module.SwarmRun(
deployment_id="runtime-dep-1",
swarm_id="swarm-1",
status="running",
objective="Test identifier lookup",
manager_deployment_id="dep-manager-1",
)
await runtime.save_run(run)
loaded = await runtime.get_run_by_identifier("dep-manager-1")
self.assertIsNotNone(loaded)
self.assertEqual(loaded.swarm_id, "swarm-1")
async def test_runtime_routes_include_agent_swarm_paths(self):
route_paths = {route.path for route in orchestrator_main.app.routes}
self.assertIn("/api/agent/health", route_paths)
self.assertIn("/api/agent/swarm/deployments", route_paths)
self.assertIn("/api/agent/swarm/deployments/{deployment_id}/events", route_paths)
self.assertIn("/api/agent/swarm/deployments/{deployment_id}/workflow", route_paths)
self.assertIn("/api/agent/swarm/deployments/{deployment_id}/diagnostics", route_paths)
async def test_build_deliverable_fact_marks_summary_only_without_diff(self):
result = {"summary": "Only a plan document"}
deliverable = orchestrator_main.build_deliverable_fact(result)
self.assertFalse(deliverable["has_deliverable"])
self.assertTrue(deliverable["summary_only"])
self.assertFalse(deliverable["has_diff"])
async def test_build_workflow_phases_returns_fixed_phase_order(self):
run = SimpleNamespace(status="running", approvals={})
task = SimpleNamespace(
task_id="task-1",
title="Backend",
agent_role="backend",
status=SimpleNamespace(value="in_progress"),
started_at=10,
completed_at=None,
result=None,
)
phases = orchestrator_main.build_workflow_phases(
run,
[task],
[{"event_type": "task.created", "payload": {}}],
)
self.assertEqual(
[phase["name"] for phase in phases],
["Plan", "Dispatch", "Execute", "Handoff", "Review", "Deliver"],
)
if __name__ == "__main__":
unittest.main()
+329
View File
@@ -0,0 +1,329 @@
"""
OpenTelemetry Distributed Tracing Setup
Provides distributed tracing for agent handoffs and task flows.
Enables end-to-end visibility across the swarm system.
"""
import os
import logging
from typing import Optional, Dict, Any
from contextlib import contextmanager
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.sdk.resources import Resource, SERVICE_NAME, SERVICE_VERSION
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.redis import RedisInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor
from opentelemetry.instrumentation.logging import LoggingInstrumentor
from opentelemetry.trace import Status, StatusCode, SpanKind
logger = logging.getLogger(__name__)
class SwarmTracer:
"""Manages distributed tracing for the swarm system"""
def __init__(
self,
service_name: str,
service_version: str = "1.0.0",
otlp_endpoint: Optional[str] = None,
enable_console: bool = False
):
self.service_name = service_name
self.service_version = service_version
self.otlp_endpoint = otlp_endpoint or os.getenv(
"OTEL_EXPORTER_OTLP_ENDPOINT",
"http://localhost:4317"
)
self.enable_console = enable_console
self._setup_tracing()
def _setup_tracing(self):
"""Initialize OpenTelemetry tracing"""
# Create resource with service information
resource = Resource.create({
SERVICE_NAME: self.service_name,
SERVICE_VERSION: self.service_version,
"deployment.environment": os.getenv("ENVIRONMENT", "production"),
"k8s.namespace": os.getenv("K8S_NAMESPACE", "swarm-system"),
"k8s.pod.name": os.getenv("HOSTNAME", "unknown"),
})
# Create tracer provider
provider = TracerProvider(resource=resource)
# Add OTLP exporter
try:
otlp_exporter = OTLPSpanExporter(endpoint=self.otlp_endpoint)
provider.add_span_processor(BatchSpanProcessor(otlp_exporter))
logger.info(f"OTLP exporter configured: {self.otlp_endpoint}")
except Exception as e:
logger.warning(f"Failed to configure OTLP exporter: {e}")
# Add console exporter for debugging
if self.enable_console:
console_exporter = ConsoleSpanExporter()
provider.add_span_processor(BatchSpanProcessor(console_exporter))
# Set global tracer provider
trace.set_tracer_provider(provider)
# Auto-instrument libraries
self._instrument_libraries()
self.tracer = trace.get_tracer(__name__)
logger.info(f"Tracing initialized for service: {self.service_name}")
def _instrument_libraries(self):
"""Auto-instrument common libraries"""
try:
RedisInstrumentor().instrument()
RequestsInstrumentor().instrument()
LoggingInstrumentor().instrument()
logger.info("Auto-instrumentation enabled")
except Exception as e:
logger.warning(f"Failed to auto-instrument libraries: {e}")
@contextmanager
def trace_operation(
self,
operation_name: str,
attributes: Optional[Dict[str, Any]] = None,
kind: SpanKind = SpanKind.INTERNAL
):
"""
Context manager for tracing an operation
Usage:
with tracer.trace_operation("process_task", {"task_id": "123"}):
# do work
pass
"""
with self.tracer.start_as_current_span(
operation_name,
kind=kind,
attributes=attributes or {}
) as span:
try:
yield span
except Exception as e:
span.set_status(Status(StatusCode.ERROR, str(e)))
span.record_exception(e)
raise
def trace_task_submission(self, task_id: str, task_description: str):
"""Trace task submission"""
with self.trace_operation(
"task.submit",
{
"task.id": task_id,
"task.description": task_description[:100] # Truncate
},
kind=SpanKind.PRODUCER
) as span:
span.add_event("task_submitted")
return span
def trace_agent_creation(self, agent_id: str, task_id: str, pod_name: str):
"""Trace agent pod creation"""
with self.trace_operation(
"agent.create",
{
"agent.id": agent_id,
"task.id": task_id,
"k8s.pod.name": pod_name
}
) as span:
span.add_event("agent_pod_created")
return span
def trace_agent_execution(self, agent_id: str, task_id: str):
"""Trace agent task execution"""
with self.trace_operation(
"agent.execute",
{
"agent.id": agent_id,
"task.id": task_id
}
) as span:
span.add_event("agent_started")
return span
def trace_handoff(
self,
from_agent_id: str,
to_agent_id: str,
task_id: str,
handoff_reason: str
):
"""Trace agent handoff"""
with self.trace_operation(
"agent.handoff",
{
"handoff.from_agent": from_agent_id,
"handoff.to_agent": to_agent_id,
"task.id": task_id,
"handoff.reason": handoff_reason
},
kind=SpanKind.CLIENT
) as span:
span.add_event("handoff_initiated")
return span
def trace_result_aggregation(self, task_id: str, agent_count: int):
"""Trace result aggregation"""
with self.trace_operation(
"result.aggregate",
{
"task.id": task_id,
"agent.count": agent_count
}
) as span:
span.add_event("aggregation_started")
return span
def add_event(self, name: str, attributes: Optional[Dict[str, Any]] = None):
"""Add an event to the current span"""
span = trace.get_current_span()
if span:
span.add_event(name, attributes or {})
def set_attribute(self, key: str, value: Any):
"""Set an attribute on the current span"""
span = trace.get_current_span()
if span:
span.set_attribute(key, value)
def record_error(self, error: Exception):
"""Record an error in the current span"""
span = trace.get_current_span()
if span:
span.set_status(Status(StatusCode.ERROR, str(error)))
span.record_exception(error)
# Singleton instance
_tracer_instance: Optional[SwarmTracer] = None
def initialize_tracing(
service_name: str,
service_version: str = "1.0.0",
otlp_endpoint: Optional[str] = None,
enable_console: bool = False
) -> SwarmTracer:
"""Initialize global tracing instance"""
global _tracer_instance
_tracer_instance = SwarmTracer(
service_name=service_name,
service_version=service_version,
otlp_endpoint=otlp_endpoint,
enable_console=enable_console
)
return _tracer_instance
def get_tracer() -> Optional[SwarmTracer]:
"""Get the global tracer instance"""
return _tracer_instance
# Decorator for tracing functions
def traced(operation_name: Optional[str] = None, **span_attributes):
"""
Decorator to automatically trace a function
Usage:
@traced("my_operation", task_id="123")
def my_function():
pass
"""
def decorator(func):
def wrapper(*args, **kwargs):
tracer = get_tracer()
if not tracer:
return func(*args, **kwargs)
op_name = operation_name or f"{func.__module__}.{func.__name__}"
with tracer.trace_operation(op_name, span_attributes):
return func(*args, **kwargs)
return wrapper
return decorator
# Context propagation helpers
def inject_trace_context(headers: Dict[str, str]) -> Dict[str, str]:
"""
Inject trace context into HTTP headers for propagation
Usage:
headers = inject_trace_context({})
requests.post(url, headers=headers)
"""
from opentelemetry.propagate import inject
inject(headers)
return headers
def extract_trace_context(headers: Dict[str, str]):
"""
Extract trace context from HTTP headers
Usage:
extract_trace_context(request.headers)
"""
from opentelemetry.propagate import extract
return extract(headers)
# Example usage patterns
"""
# In orchestrator/main.py:
from orchestrator.tracing import initialize_tracing, get_tracer
tracer = initialize_tracing(
service_name="swarm-orchestrator",
service_version="1.0.0",
otlp_endpoint="http://otel-collector:4317"
)
# Trace task submission
with tracer.trace_task_submission(task_id, description):
# Submit task logic
pass
# In agent/main.py:
from orchestrator.tracing import initialize_tracing, get_tracer
tracer = initialize_tracing(
service_name="swarm-agent",
service_version="1.0.0"
)
# Trace agent execution
with tracer.trace_agent_execution(agent_id, task_id):
# Execute task
pass
# Trace handoff
with tracer.trace_handoff(from_agent, to_agent, task_id, reason):
# Perform handoff
pass
# Using decorator
@traced("process_subtask", task_id="123")
def process_subtask():
pass
# Manual span management
tracer = get_tracer()
with tracer.trace_operation("custom_operation", {"key": "value"}):
tracer.add_event("checkpoint_reached")
tracer.set_attribute("result_count", 42)
"""
+93
View File
@@ -0,0 +1,93 @@
#!/bin/bash
# Deploy agent to Kubernetes cluster
set -e
echo "=========================================="
echo "Agent Deployment Script"
echo "=========================================="
# Configuration
AGENT_IMAGE="swarm-agent:latest"
AGENT_ID="${AGENT_ID:-agent-$(date +%s)}"
AGENT_CAPABILITIES="${AGENT_CAPABILITIES:-general,python}"
echo "Agent ID: $AGENT_ID"
echo "Agent Capabilities: $AGENT_CAPABILITIES"
# Check required environment variables
if [ -z "$OPENAI_API_KEY" ]; then
echo "Error: OPENAI_API_KEY environment variable not set"
exit 1
fi
# Build Docker image
echo ""
echo "Building Docker image..."
docker build -f Dockerfile.agent -t $AGENT_IMAGE .
# Load image into kind cluster
echo ""
echo "Loading image into kind cluster..."
kind load docker-image $AGENT_IMAGE
# Create secrets if they don't exist
echo ""
echo "Creating/updating secrets..."
# Model API key (OpenAI-compatible)
kubectl create secret generic openai-secret \
--from-literal=api-key="$OPENAI_API_KEY" \
--dry-run=client -o yaml | kubectl apply -f -
# Git credentials (optional)
if [ -n "$GIT_USERNAME" ] && [ -n "$GIT_PASSWORD" ]; then
kubectl create secret generic git-credentials \
--from-literal=username="$GIT_USERNAME" \
--from-literal=password="$GIT_PASSWORD" \
--dry-run=client -o yaml | kubectl apply -f -
else
echo "Note: GIT_USERNAME and GIT_PASSWORD not set, skipping git credentials"
fi
# Create ConfigMap
echo ""
echo "Creating/updating ConfigMap..."
kubectl create configmap swarm-config \
--from-literal=git-repo-url="${GIT_REPO_URL:-}" \
--from-literal=openai-model="${OPENAI_MODEL:-gpt-4o-mini}" \
--from-literal=openai-api-base="${OPENAI_API_BASE:-https://api.openai.com/v1}" \
--dry-run=client -o yaml | kubectl apply -f -
# Deploy agent pod
echo ""
echo "Deploying agent pod..."
export AGENT_ID
export AGENT_CAPABILITIES
envsubst < k8s/agent-pod-template.yaml | kubectl apply -f -
# Wait for pod to be ready
echo ""
echo "Waiting for agent pod to be ready..."
kubectl wait --for=condition=Ready pod/agent-$AGENT_ID --timeout=60s || true
# Show pod status
echo ""
echo "Agent pod status:"
kubectl get pod agent-$AGENT_ID
# Show logs
echo ""
echo "Agent logs (last 20 lines):"
kubectl logs agent-$AGENT_ID --tail=20 || echo "Pod not ready yet"
echo ""
echo "=========================================="
echo "Agent deployment complete!"
echo "=========================================="
echo ""
echo "To view logs:"
echo " kubectl logs -f agent-$AGENT_ID"
echo ""
echo "To delete agent:"
echo " kubectl delete pod agent-$AGENT_ID"
+33
View File
@@ -0,0 +1,33 @@
#!/bin/bash
# Build and deploy orchestrator to Kubernetes
set -e
echo "Building orchestrator Docker image..."
docker build -f Dockerfile.orchestrator -t swarm-orchestrator:latest .
echo "Loading image into kind cluster..."
kind load docker-image swarm-orchestrator:latest --name swarm-cluster
echo "Deploying Redis StatefulSet..."
kubectl apply -f k8s/redis-statefulset.yaml
echo "Waiting for Redis to be ready..."
kubectl wait --for=condition=ready pod -l app=redis -n swarm-system --timeout=120s
echo "Deploying orchestrator..."
kubectl apply -f k8s/orchestrator-deployment.yaml
echo "Waiting for orchestrator to be ready..."
kubectl wait --for=condition=ready pod -l app=orchestrator -n swarm-system --timeout=120s
echo "Orchestrator deployed successfully!"
echo ""
echo "Check status:"
echo " kubectl get pods -n swarm-system"
echo ""
echo "View logs:"
echo " kubectl logs -f -l app=orchestrator -n swarm-system"
echo ""
echo "Port forward to access API:"
echo " kubectl port-forward -n swarm-system svc/orchestrator-service 8000:8000"
+174
View File
@@ -0,0 +1,174 @@
#!/usr/bin/env python3
"""Render a terminal-style HTML snapshot from a swarm poster summary JSON."""
from __future__ import annotations
import argparse
import json
from datetime import datetime, timezone
from pathlib import Path
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Render a terminal-style swarm snapshot HTML.")
parser.add_argument("summary_json", help="Path to the swarm poster summary JSON file.")
parser.add_argument(
"--output-html",
help="Optional output HTML path. Defaults next to the summary file.",
)
return parser.parse_args()
def fmt_ts(ts: float | None) -> str:
if not ts:
return "-"
return datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
def main() -> int:
args = parse_args()
summary_path = Path(args.summary_json).resolve()
data = json.loads(summary_path.read_text(encoding="utf-8"))
swarm = data["swarm"]
task = data["task"]
metrics = data["metrics"]
out_path = (
Path(args.output_html).resolve()
if args.output_html
else summary_path.with_name(summary_path.stem + "-terminal.html")
)
budget_ratio = metrics.get("budget", {}).get("duration_ratio")
budget_display = f"{budget_ratio * 100:.1f}%" if isinstance(budget_ratio, (int, float)) else "-"
transcript = [
"$ python3 scripts/run_swarm_poster_demo.py",
f"[swarm] deployment_id={swarm['deployment_id']}",
f"[swarm] swarm_id={swarm['swarm_id']}",
f"[task] task_id={task['task_id']}",
f"[task] status={task['status']}",
f"[task] assigned_agent={task.get('assigned_agent_id') or '-'}",
"",
"$ curl -H \"Authorization: Bearer ***\" /api/swarms/{swarm_id}/tasks",
f"task.started_at = {fmt_ts(data.get('started_at'))}",
f"task.halfway_at = {fmt_ts(data.get('halfway_at'))}",
f"halfway.observed_at = {fmt_ts(data.get('halfway_observed_at'))}",
"",
"$ curl -H \"Authorization: Bearer ***\" /api/swarms/{swarm_id}/metrics",
f"runtime.status = {metrics.get('status')}",
f"runtime.tasks_total = {metrics.get('tasks_total')}",
f"runtime.tasks_by_status = {json.dumps(metrics.get('tasks_by_status', {}), ensure_ascii=False)}",
f"runtime.agents_connected = {metrics.get('agents_connected')}",
f"runtime.budget.duration_seconds = {metrics.get('budget', {}).get('duration_seconds')}",
f"runtime.budget.duration_ratio = {budget_display}",
"",
"# milestone",
f"> 任务开始: {fmt_ts(data.get('started_at'))}",
f"> 任务过半: {fmt_ts(data.get('halfway_observed_at'))}",
]
lines_html = "\n".join(f"<div class='line'>{line}</div>" for line in transcript)
html = f"""<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Swarm Terminal Snapshot</title>
<style>
body {{
margin: 0;
background:
radial-gradient(circle at top right, rgba(74, 222, 128, 0.10), transparent 20%),
radial-gradient(circle at top left, rgba(96, 165, 250, 0.10), transparent 24%),
#0a0f14;
color: #d7e3ee;
font-family: Menlo, Monaco, "SFMono-Regular", "JetBrains Mono", monospace;
}}
.frame {{
width: 1600px;
min-height: 900px;
margin: 0 auto;
padding: 48px;
box-sizing: border-box;
}}
.window {{
background: #0b1220;
border: 1px solid rgba(148, 163, 184, 0.20);
border-radius: 18px;
overflow: hidden;
box-shadow: 0 30px 80px rgba(0, 0, 0, 0.45);
}}
.topbar {{
display: flex;
align-items: center;
justify-content: space-between;
padding: 14px 18px;
background: #121a2a;
border-bottom: 1px solid rgba(148, 163, 184, 0.12);
}}
.dots {{
display: flex;
gap: 8px;
}}
.dot {{
width: 12px;
height: 12px;
border-radius: 50%;
}}
.dot.red {{ background: #fb7185; }}
.dot.yellow {{ background: #fbbf24; }}
.dot.green {{ background: #4ade80; }}
.title {{
color: #93a4b8;
font-size: 15px;
letter-spacing: 0.02em;
}}
.terminal {{
padding: 28px 32px 36px;
font-size: 24px;
line-height: 1.7;
white-space: pre-wrap;
word-break: break-word;
}}
.line:nth-child(1),
.line:nth-child(8),
.line:nth-child(13) {{
color: #7dd3fc;
}}
.line:nth-last-child(2),
.line:nth-last-child(1) {{
color: #bef264;
font-weight: 600;
}}
</style>
</head>
<body>
<div class="frame">
<div class="window">
<div class="topbar">
<div class="dots">
<span class="dot red"></span>
<span class="dot yellow"></span>
<span class="dot green"></span>
</div>
<div class="title">swarm-terminal-snapshot</div>
<div class="title">{swarm['swarm_id']}</div>
</div>
<div class="terminal">
{lines_html}
</div>
</div>
</div>
</body>
</html>
"""
out_path.write_text(html, encoding="utf-8")
print(out_path)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+660
View File
@@ -0,0 +1,660 @@
#!/usr/bin/env python3
"""Create a real swarm task, wait for start and half-progress, and render a poster page."""
from __future__ import annotations
import argparse
import base64
import json
import os
import subprocess
import sys
import time
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Optional
from urllib.error import HTTPError, URLError
from urllib.parse import urlencode
from urllib.request import Request, urlopen
ROOT = Path(__file__).resolve().parents[1]
OUTPUT_DIR = ROOT / "artifacts" / "swarm-posters"
@dataclass
class ApiConfig:
base_url: str
token: str
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Create a swarm task, wait for real progress milestones, and build a poster.",
)
parser.add_argument(
"--base-url",
default=os.getenv("SWARM_BASE_URL", "http://52.139.240.116:8000"),
help="Base URL of the running swarm runtime.",
)
parser.add_argument(
"--runtime-token",
default=os.getenv("AGNET_RUNTIME_SERVICE_TOKEN"),
help="Runtime bearer token. If omitted, read from the Kubernetes secret.",
)
parser.add_argument(
"--budget-seconds",
type=int,
default=40,
help="Task budget duration used to compute the half-progress milestone.",
)
parser.add_argument(
"--poll-interval",
type=float,
default=3.0,
help="Polling interval in seconds.",
)
parser.add_argument(
"--timeout-seconds",
type=int,
default=240,
help="Maximum time to wait for the half-progress milestone.",
)
parser.add_argument(
"--objective",
default=(
"Build a blog system MVP in the configured workspace repository. "
"Deliver a runnable implementation with a simple backend and frontend for posts: "
"list posts, view a post, create, edit, and delete. Keep setup minimal and include short run instructions."
),
help="High-level objective for the swarm.",
)
parser.add_argument(
"--task-description",
default=(
"Create a runnable blog system MVP in the target repository with post list, "
"post detail, and create/edit/delete flows. Keep the implementation pragmatic and "
"coherent with the existing repo. Commit and push the result branch when changes are ready."
),
help="Concrete task description sent to the agent.",
)
parser.add_argument(
"--poster-title",
default="Swarm Blog MVP Live Demo",
help="Poster title rendered into the HTML.",
)
return parser.parse_args()
def iso_now() -> str:
return datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
def to_human_time(timestamp: Optional[float]) -> str:
if not timestamp:
return "-"
return datetime.fromtimestamp(timestamp, tz=timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
def read_runtime_token(explicit: Optional[str]) -> str:
if explicit:
return explicit
cmd = [
"kubectl",
"get",
"secret",
"-n",
"swarm-system",
"agnet-runtime-secrets",
"-o",
"jsonpath={.data.runtime-service-token}",
]
encoded = subprocess.check_output(cmd, text=True).strip()
if not encoded:
raise RuntimeError("Runtime token was not provided and could not be read from Kubernetes.")
return base64.b64decode(encoded).decode("utf-8")
def api_request(
config: ApiConfig,
path: str,
method: str = "GET",
body: Optional[dict[str, Any]] = None,
headers: Optional[dict[str, str]] = None,
query: Optional[dict[str, Any]] = None,
) -> dict[str, Any]:
query_string = f"?{urlencode(query)}" if query else ""
url = f"{config.base_url.rstrip('/')}{path}{query_string}"
request_headers = {
"Authorization": f"Bearer {config.token}",
"Content-Type": "application/json",
}
if headers:
request_headers.update(headers)
payload = None
if body is not None:
payload = json.dumps(body).encode("utf-8")
req = Request(url, data=payload, headers=request_headers, method=method)
try:
with urlopen(req, timeout=30) as resp:
return json.loads(resp.read().decode("utf-8"))
except HTTPError as exc:
raw = exc.read().decode("utf-8", errors="replace")
raise RuntimeError(f"API request failed with {exc.code}: {raw}") from exc
except URLError as exc:
raise RuntimeError(f"API request failed: {exc}") from exc
def create_swarm(config: ApiConfig, args: argparse.Namespace) -> dict[str, Any]:
request_id = iso_now()
correlation_id = f"corr_poster_demo_{request_id}"
idempotency_key = f"idem_poster_demo_{request_id}"
manager_deployment_id = f"dep_poster_demo_{request_id}"
agents = [
{
"task_id": f"poster-blog-{request_id.lower()}",
"role": "fullstack",
"title": "Implement blog system MVP",
"description": args.task_description,
"depends_on": [],
}
]
body = {
"orchestration_plan": {
"objective": args.objective,
"sub_mode": "code",
"risk_level": "low",
"budget": {
"duration_seconds": args.budget_seconds,
"token_limit": 120000,
},
"agents": agents,
},
"agents": agents,
"callback": {
"url": f"{config.base_url.rstrip('/')}/health",
"subscribed_events": [
"deployment.status_changed",
"task.created",
"task.running",
"task.completed",
"task.failed",
"timeline.updated",
"artifact.created",
],
},
"metadata": {
"manager_deployment_id": manager_deployment_id,
"correlation_id": correlation_id,
},
}
response = api_request(
config,
"/api/swarms",
method="POST",
body=body,
headers={
"X-Correlation-Id": correlation_id,
"X-Idempotency-Key": idempotency_key,
},
)
if not response.get("success"):
raise RuntimeError(f"Swarm creation failed: {json.dumps(response, ensure_ascii=False)}")
return response["data"]
def fetch_tasks(config: ApiConfig, swarm_id: str) -> list[dict[str, Any]]:
response = api_request(config, f"/api/swarms/{swarm_id}/tasks")
return response["data"]["tasks"]
def fetch_logs(config: ApiConfig, swarm_id: str, limit: int = 100) -> list[dict[str, Any]]:
response = api_request(
config,
f"/api/swarms/{swarm_id}/logs",
query={"limit": limit},
)
return response["data"]["events"]
def fetch_metrics(config: ApiConfig, swarm_id: str) -> dict[str, Any]:
response = api_request(config, f"/api/swarms/{swarm_id}/metrics")
return response["data"]
def fetch_agents(base_url: str) -> list[dict[str, Any]]:
req = Request(f"{base_url.rstrip('/')}/agents")
with urlopen(req, timeout=15) as resp:
return json.loads(resp.read().decode("utf-8"))["agents"]
def manual_assign(base_url: str, task_id: str, agent_id: str) -> bool:
payload = json.dumps({"task_id": task_id, "agent_id": agent_id}).encode("utf-8")
req = Request(
f"{base_url.rstrip('/')}/tasks/assign",
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
try:
with urlopen(req, timeout=15) as resp:
data = json.loads(resp.read().decode("utf-8"))
return bool(data.get("task_id") or data.get("status"))
except Exception:
return False
def find_event(events: list[dict[str, Any]], event_type: str) -> Optional[dict[str, Any]]:
for event in events:
if event.get("event_type") == event_type:
return event
return None
def last_event(events: list[dict[str, Any]], event_type: str) -> Optional[dict[str, Any]]:
for event in reversed(events):
if event.get("event_type") == event_type:
return event
return None
def render_html(
poster_path: Path,
args: argparse.Namespace,
swarm_data: dict[str, Any],
task: dict[str, Any],
metrics: dict[str, Any],
started_at: float,
halfway_at: float,
halfway_observed_at: float,
latest_events: list[dict[str, Any]],
) -> None:
latest_status = task.get("status", "unknown")
latest_heartbeat = last_event(latest_events, "task.heartbeat")
heartbeat_time = latest_heartbeat.get("occurred_at") if latest_heartbeat else "-"
runtime_ratio = metrics.get("budget", {}).get("duration_ratio")
ratio_display = f"{runtime_ratio * 100:.0f}%" if isinstance(runtime_ratio, (int, float)) else "50%+"
html = f"""<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>{args.poster_title}</title>
<style>
:root {{
--bg: #07111f;
--panel: rgba(10, 25, 47, 0.84);
--panel-strong: rgba(12, 32, 60, 0.95);
--line: rgba(125, 211, 252, 0.25);
--cyan: #7dd3fc;
--teal: #5eead4;
--lime: #bef264;
--text: #e6f0ff;
--muted: #9bb1c8;
--warn: #fbbf24;
}}
* {{ box-sizing: border-box; }}
body {{
margin: 0;
font-family: "Avenir Next", "PingFang SC", "Helvetica Neue", sans-serif;
color: var(--text);
background:
radial-gradient(circle at top left, rgba(45, 212, 191, 0.18), transparent 32%),
radial-gradient(circle at top right, rgba(125, 211, 252, 0.24), transparent 28%),
linear-gradient(135deg, #050c16 0%, #07111f 42%, #0d1d35 100%);
min-height: 100vh;
}}
.canvas {{
width: 1600px;
min-height: 900px;
margin: 0 auto;
padding: 56px;
position: relative;
overflow: hidden;
}}
.grid {{
position: absolute;
inset: 0;
background-image:
linear-gradient(rgba(125, 211, 252, 0.05) 1px, transparent 1px),
linear-gradient(90deg, rgba(125, 211, 252, 0.05) 1px, transparent 1px);
background-size: 56px 56px;
mask-image: linear-gradient(to bottom, rgba(0,0,0,.75), transparent);
pointer-events: none;
}}
.hero {{
display: flex;
justify-content: space-between;
gap: 28px;
align-items: flex-start;
margin-bottom: 28px;
}}
.hero h1 {{
margin: 0 0 12px 0;
font-size: 68px;
line-height: 0.95;
letter-spacing: -2px;
}}
.hero p {{
margin: 0;
max-width: 800px;
color: var(--muted);
font-size: 24px;
line-height: 1.5;
}}
.badge {{
display: inline-flex;
align-items: center;
gap: 10px;
border: 1px solid var(--line);
background: rgba(6, 19, 36, 0.72);
border-radius: 999px;
padding: 12px 18px;
color: var(--cyan);
font-size: 16px;
letter-spacing: 0.08em;
text-transform: uppercase;
}}
.layout {{
display: grid;
grid-template-columns: 1.25fr 0.75fr;
gap: 28px;
}}
.panel {{
background: var(--panel);
border: 1px solid var(--line);
border-radius: 28px;
padding: 28px;
box-shadow: 0 25px 60px rgba(0, 0, 0, 0.32);
backdrop-filter: blur(16px);
}}
.panel strong {{
display: block;
font-size: 16px;
letter-spacing: 0.08em;
text-transform: uppercase;
color: var(--cyan);
margin-bottom: 14px;
}}
.cards {{
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 18px;
margin-bottom: 28px;
}}
.metric {{
background: var(--panel-strong);
border: 1px solid rgba(190, 242, 100, 0.16);
border-radius: 22px;
padding: 20px;
}}
.metric .label {{
color: var(--muted);
font-size: 15px;
margin-bottom: 8px;
}}
.metric .value {{
font-size: 34px;
font-weight: 700;
letter-spacing: -1px;
}}
.timeline {{
display: grid;
gap: 16px;
margin-top: 8px;
}}
.step {{
position: relative;
padding: 18px 18px 18px 68px;
border-radius: 22px;
background: rgba(7, 18, 33, 0.9);
border: 1px solid rgba(125, 211, 252, 0.12);
}}
.step::before {{
content: "";
position: absolute;
left: 28px;
top: 26px;
width: 16px;
height: 16px;
border-radius: 50%;
background: linear-gradient(135deg, var(--teal), var(--cyan));
box-shadow: 0 0 0 8px rgba(94, 234, 212, 0.12);
}}
.step h3 {{
margin: 0 0 8px 0;
font-size: 24px;
}}
.step p {{
margin: 0;
color: var(--muted);
font-size: 16px;
line-height: 1.5;
}}
.code {{
font-family: "SF Mono", "JetBrains Mono", monospace;
color: var(--lime);
word-break: break-all;
}}
.sidebar {{
display: grid;
gap: 18px;
}}
.status {{
border-radius: 24px;
padding: 22px;
background: linear-gradient(160deg, rgba(94, 234, 212, 0.14), rgba(125, 211, 252, 0.08));
border: 1px solid rgba(94, 234, 212, 0.25);
}}
.status .headline {{
font-size: 18px;
color: var(--muted);
margin-bottom: 8px;
}}
.status .value {{
font-size: 44px;
font-weight: 700;
letter-spacing: -1px;
margin-bottom: 10px;
}}
.status .sub {{
color: var(--muted);
font-size: 16px;
line-height: 1.5;
}}
.meta-row {{
display: flex;
justify-content: space-between;
gap: 12px;
padding: 10px 0;
border-bottom: 1px solid rgba(125, 211, 252, 0.08);
font-size: 15px;
}}
.meta-row:last-child {{ border-bottom: none; }}
.meta-label {{ color: var(--muted); }}
.meta-value {{ text-align: right; max-width: 56%; }}
.footer {{
margin-top: 22px;
color: var(--warn);
font-size: 15px;
line-height: 1.5;
}}
</style>
</head>
<body>
<div class="canvas">
<div class="grid"></div>
<div class="hero">
<div>
<div class="badge">Live Swarm Poster</div>
<h1>{args.poster_title}</h1>
<p>真实蜂群任务已成功创建,并已记录到“任务开始”和“进行过半”两段里程碑。下面的内容全部来自当前运行中的 swarm 状态,而不是手工拼接。</p>
</div>
<div class="panel" style="min-width: 360px;">
<strong>Demo Goal</strong>
<p style="font-size: 18px; color: var(--text); line-height: 1.55;">在目标仓库里开发一个博客系统 MVP,包括文章列表、详情和基础增删改能力。</p>
</div>
</div>
<div class="layout">
<div class="panel">
<div class="cards">
<div class="metric">
<div class="label">Swarm</div>
<div class="value">{swarm_data["swarm_id"]}</div>
</div>
<div class="metric">
<div class="label">Task Status</div>
<div class="value">{latest_status}</div>
</div>
<div class="metric">
<div class="label">Budget Progress</div>
<div class="value">{ratio_display}</div>
</div>
</div>
<strong>Timeline</strong>
<div class="timeline">
<div class="step">
<h3>任务开始</h3>
<p>任务已被 agent 领取并进入运行态。开始时间:<span class="code">{to_human_time(started_at)}</span></p>
</div>
<div class="step">
<h3>进行到一半</h3>
<p>脚本按 budget 的 50% 自动确认里程碑。预算过半时间:<span class="code">{to_human_time(halfway_at)}</span>,实际记录时间:<span class="code">{to_human_time(halfway_observed_at)}</span></p>
</div>
<div class="step">
<h3>当前执行中</h3>
<p>任务仍处于 <span class="code">{latest_status}</span>,最近心跳:<span class="code">{heartbeat_time}</span></p>
</div>
</div>
</div>
<div class="sidebar">
<div class="status">
<div class="headline">Poster Snapshot</div>
<div class="value">Mid-Run</div>
<div class="sub">这张海报展示的是蜂群任务已经真正启动,并且已经跑到预算半程时的现场状态。</div>
</div>
<div class="panel">
<strong>Swarm Meta</strong>
<div class="meta-row"><span class="meta-label">Deployment</span><span class="meta-value code">{swarm_data["deployment_id"]}</span></div>
<div class="meta-row"><span class="meta-label">Task</span><span class="meta-value code">{task["task_id"]}</span></div>
<div class="meta-row"><span class="meta-label">Agent</span><span class="meta-value code">{task.get("assigned_agent_id") or "-"}</span></div>
<div class="meta-row"><span class="meta-label">Attempts</span><span class="meta-value">{task.get("attempt", 0)}</span></div>
<div class="meta-row"><span class="meta-label">Budget</span><span class="meta-value">{args.budget_seconds}s</span></div>
<div class="meta-row"><span class="meta-label">Output File</span><span class="meta-value code">{poster_path.name}</span></div>
</div>
<div class="panel">
<strong>Latest Events</strong>
<div class="meta-row"><span class="meta-label">1</span><span class="meta-value code">{latest_events[-3]["event_type"] if len(latest_events) >= 3 else "-"}</span></div>
<div class="meta-row"><span class="meta-label">2</span><span class="meta-value code">{latest_events[-2]["event_type"] if len(latest_events) >= 2 else "-"}</span></div>
<div class="meta-row"><span class="meta-label">3</span><span class="meta-value code">{latest_events[-1]["event_type"] if latest_events else "-"}</span></div>
<div class="footer">注:如果后续继续执行,日志与状态还会变化;这张图锁定的是“开始后已过半”的那一刻。</div>
</div>
</div>
</div>
</div>
</body>
</html>
"""
poster_path.write_text(html, encoding="utf-8")
def main() -> int:
args = parse_args()
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
token = read_runtime_token(args.runtime_token)
config = ApiConfig(base_url=args.base_url, token=token)
swarm_data = create_swarm(config, args)
swarm_id = swarm_data["swarm_id"]
deadline = time.time() + args.timeout_seconds
started_at = None
task_snapshot = None
latest_events: list[dict[str, Any]] = []
while time.time() < deadline:
tasks = fetch_tasks(config, swarm_id)
latest_events = fetch_logs(config, swarm_id, limit=100)
if tasks:
task_snapshot = tasks[0]
if task_snapshot.get("status") == "pending" and not task_snapshot.get("assigned_agent_id"):
for agent in fetch_agents(config.base_url):
if agent.get("status") == "idle":
if manual_assign(config.base_url, task_snapshot["task_id"], agent["agent_id"]):
break
if task_snapshot.get("started_at"):
started_at = float(task_snapshot["started_at"])
break
running_event = find_event(latest_events, "task.running")
if running_event:
started_at = datetime.fromisoformat(
running_event["occurred_at"].replace("Z", "+00:00")
).timestamp()
break
time.sleep(args.poll_interval)
if started_at is None or task_snapshot is None:
raise RuntimeError("Task did not reach a running state before timeout.")
halfway_at = started_at + args.budget_seconds / 2
while time.time() < deadline:
tasks = fetch_tasks(config, swarm_id)
latest_events = fetch_logs(config, swarm_id, limit=100)
task_snapshot = tasks[0]
if time.time() >= halfway_at:
break
if task_snapshot.get("status") in {"completed", "failed"}:
break
time.sleep(args.poll_interval)
metrics = fetch_metrics(config, swarm_id)
halfway_observed_at = time.time()
run_id = swarm_id.replace("swarm-", "")
poster_path = OUTPUT_DIR / f"swarm-poster-{run_id}.html"
summary_path = OUTPUT_DIR / f"swarm-poster-{run_id}.json"
render_html(
poster_path=poster_path,
args=args,
swarm_data=swarm_data,
task=task_snapshot,
metrics=metrics,
started_at=started_at,
halfway_at=halfway_at,
halfway_observed_at=halfway_observed_at,
latest_events=latest_events,
)
summary = {
"swarm": swarm_data,
"task": task_snapshot,
"metrics": metrics,
"started_at": started_at,
"halfway_at": halfway_at,
"halfway_observed_at": halfway_observed_at,
"poster_html": str(poster_path),
}
summary_path.write_text(json.dumps(summary, indent=2, ensure_ascii=False), encoding="utf-8")
print(json.dumps(summary, ensure_ascii=False, indent=2))
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except KeyboardInterrupt:
print("Interrupted.", file=sys.stderr)
raise SystemExit(130)
+117
View File
@@ -0,0 +1,117 @@
"""Keyless stub agent for workflow testing.
Connects to the orchestrator over WebSocket, registers, and returns canned results WITHOUT
calling any LLM (no OPENAI_API_KEY needed). To make the master review loop deterministic, the
"testing" task returns a conflicting framework (unittest) on its FIRST execution and an aligned
one (pytest) on retry — so the heuristic critic rejects exactly once and then accepts.
Run standalone against a manually started orchestrator:
set "ORCHESTRATOR_URL=ws://localhost:8000"
set "AGENT_ID=stub-1"
set "AGENT_CAPABILITIES=python,code_generation,testing,pytest,technical-writing,general"
python scripts/stub_agent.py
"""
import asyncio
import json
import os
import time
import websockets
class StubAgent:
def __init__(self, orchestrator_url: str, agent_id: str, capabilities: list[str]):
self.orchestrator_url = orchestrator_url.rstrip("/")
self.agent_id = agent_id
self.capabilities = capabilities
self.ws = None
self.runs: dict[str, int] = {} # task_id -> execution count
self.running = True
def _result_for(self, task_id: str, description: str) -> dict:
count = self.runs.get(task_id, 0) + 1
self.runs[task_id] = count
tid = task_id.lower()
if "implementation" in tid:
summary = "implemented add(a, b); tests should use pytest"
elif "testing" in tid:
# First attempt conflicts (unittest); retry aligns with implementation (pytest).
summary = "wrote tests with unittest" if count == 1 else "wrote tests with pytest"
elif "documentation" in tid:
summary = "documented usage of add(a, b)"
else:
summary = f"completed: {description[:40]}"
return {
"success": True,
"task_id": task_id,
"subtasks": [{"status": "completed", "summary": summary, "changes": summary, "files": []}],
"awaiting_handoff": False,
"agent_id": self.agent_id,
"usage": {
"model_id": "stub", "model_tokens": 0, "prompt_tokens": 0,
"completion_tokens": 0, "model_cost_usd": 0.0, "runtime_seconds": 0.0,
},
}
async def send(self, payload: dict):
await self.ws.send(json.dumps(payload))
async def handle(self, msg: dict):
msg_type = msg.get("type")
if msg_type == "task_assignment":
task_id = msg["task_id"]
await self.send({"type": "task_accepted", "task_id": task_id, "available_slots": 4})
await self.send({"type": "task_start", "agent_id": self.agent_id, "task_id": task_id, "timestamp": time.time()})
await asyncio.sleep(0.2) # simulate work
result = self._result_for(task_id, msg.get("description", ""))
await self.send({
"type": "task_complete", "agent_id": self.agent_id,
"task_id": task_id, "result": result, "timestamp": time.time(),
})
elif msg_type == "peer_message" and not msg.get("is_reply"):
# Answer an inbound peer query so collaboration round-trips complete.
await self.send({
"type": "peer_message", "agent_id": self.agent_id,
"target_agent_id": msg.get("from_agent_id"), "task_id": msg.get("task_id"),
"content": f"stub guidance from {self.agent_id}",
"correlation_id": msg.get("correlation_id"), "is_reply": True, "timestamp": time.time(),
})
async def heartbeat(self):
while self.running:
try:
await self.send({
"type": "heartbeat", "agent_id": self.agent_id,
"timestamp": time.time(), "active_tasks": 0, "available_slots": 4,
})
except Exception:
return
await asyncio.sleep(5)
async def run(self):
async with websockets.connect(f"{self.orchestrator_url}/ws/{self.agent_id}") as ws:
self.ws = ws
await self.send({
"type": "register", "agent_id": self.agent_id,
"capabilities": self.capabilities, "available_slots": 4, "active_task_ids": [],
})
hb = asyncio.create_task(self.heartbeat())
try:
async for raw in ws:
await self.handle(json.loads(raw))
finally:
self.running = False
hb.cancel()
async def _main():
agent = StubAgent(
os.getenv("ORCHESTRATOR_URL", "ws://localhost:8000"),
os.getenv("AGENT_ID", "stub-1"),
os.getenv("AGENT_CAPABILITIES", "python,code_generation,testing,pytest,technical-writing,general").split(","),
)
await agent.run()
if __name__ == "__main__":
asyncio.run(_main())
+156
View File
@@ -0,0 +1,156 @@
#!/usr/bin/env python3
"""Test script for agent components."""
import sys
import os
# Add parent directory to path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
def test_imports():
"""Test that all agent modules can be imported."""
print("Testing imports...")
try:
from agent.main import Agent
print("✓ agent.main imported successfully")
except Exception as e:
print(f"✗ Failed to import agent.main: {e}")
return False
try:
from agent.task_executor import TaskExecutor
print("✓ agent.task_executor imported successfully")
except Exception as e:
print(f"✗ Failed to import agent.task_executor: {e}")
return False
try:
from agent.handoff_logic import HandoffDecision, should_handoff
print("✓ agent.handoff_logic imported successfully")
except Exception as e:
print(f"✗ Failed to import agent.handoff_logic: {e}")
return False
try:
from agent.git_operations import GitOperations
print("✓ agent.git_operations imported successfully")
except Exception as e:
print(f"✗ Failed to import agent.git_operations: {e}")
return False
return True
def test_handoff_logic():
"""Test handoff decision logic."""
print("\nTesting handoff logic...")
from agent.handoff_logic import should_handoff, estimate_task_complexity
# Test low complexity task
subtask = {
"description": "Fix typo in comment",
"complexity": "low",
"estimated_time": 5,
"required_capabilities": ["general"]
}
decision = should_handoff(subtask, "agent-test")
assert not decision.should_handoff, "Low complexity task should not handoff"
print("✓ Low complexity task correctly handled")
# Test high complexity task
subtask = {
"description": "Refactor entire authentication system",
"complexity": "high",
"estimated_time": 120,
"required_capabilities": ["security", "architecture"]
}
decision = should_handoff(subtask, "agent-test")
assert decision.should_handoff, "High complexity task should handoff"
print("✓ High complexity task correctly triggers handoff")
# Test specialized capabilities
subtask = {
"description": "Optimize database queries",
"complexity": "medium",
"estimated_time": 30,
"required_capabilities": ["database", "performance"]
}
decision = should_handoff(subtask, "agent-test")
assert decision.should_handoff, "Specialized capabilities should trigger handoff"
print("✓ Specialized capabilities correctly trigger handoff")
# Test complexity estimation
assert estimate_task_complexity("Fix typo in README") == "low"
assert estimate_task_complexity("Refactor authentication system") == "high"
assert estimate_task_complexity("Add new API endpoint") == "medium"
print("✓ Complexity estimation working correctly")
return True
def test_agent_initialization():
"""Test agent can be initialized."""
print("\nTesting agent initialization...")
from agent.main import Agent
# Set dummy API key for testing
os.environ["ANTHROPIC_API_KEY"] = "sk-ant-test-key-for-initialization-test"
try:
agent = Agent(
orchestrator_url="ws://localhost:8000",
agent_id="test-agent",
capabilities=["python", "testing"],
workspace_dir="/tmp/test-workspace"
)
print(f"✓ Agent initialized with ID: {agent.agent_id}")
print(f"✓ Agent capabilities: {agent.capabilities}")
return True
except Exception as e:
print(f"✗ Failed to initialize agent: {e}")
return False
def main():
"""Run all tests."""
print("=" * 60)
print("Agent Component Tests")
print("=" * 60)
tests = [
("Import Tests", test_imports),
("Handoff Logic Tests", test_handoff_logic),
("Agent Initialization Tests", test_agent_initialization),
]
results = []
for name, test_func in tests:
try:
result = test_func()
results.append((name, result))
except Exception as e:
print(f"\n✗ {name} failed with exception: {e}")
results.append((name, False))
print("\n" + "=" * 60)
print("Test Results")
print("=" * 60)
for name, result in results:
status = "✓ PASS" if result else "✗ FAIL"
print(f"{status}: {name}")
all_passed = all(result for _, result in results)
if all_passed:
print("\n✓ All tests passed!")
return 0
else:
print("\n✗ Some tests failed")
return 1
if __name__ == "__main__":
sys.exit(main())
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env python3
"""Exercise agent Git workflow against a local temporary repository."""
import asyncio
import importlib.util
import os
import subprocess
import sys
import tempfile
from pathlib import Path
from typing import Optional
ROOT = Path(__file__).resolve().parents[1]
GIT_OPS_PATH = ROOT / "agent" / "git_operations.py"
spec = importlib.util.spec_from_file_location("git_operations", GIT_OPS_PATH)
git_operations = importlib.util.module_from_spec(spec)
assert spec and spec.loader
spec.loader.exec_module(git_operations)
GitOperations = git_operations.GitOperations
def run(command: list[str], cwd: Optional[Path] = None):
result = subprocess.run(
command,
cwd=cwd,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
if result.returncode != 0:
raise RuntimeError(
f"Command failed: {' '.join(command)}\nSTDOUT:\n{result.stdout}\nSTDERR:\n{result.stderr}"
)
return result
async def main():
with tempfile.TemporaryDirectory(prefix="swarm-git-test-") as tmp:
tmpdir = Path(tmp)
source = tmpdir / "source"
remote = tmpdir / "remote.git"
workspace = tmpdir / "workspace"
source.mkdir()
run(["git", "init", "-b", "main"], cwd=source)
run(["git", "config", "user.name", "Test User"], cwd=source)
run(["git", "config", "user.email", "test@example.com"], cwd=source)
(source / "hello.py").write_text(
"def hello_world():\n return 'hello'\n",
encoding="utf-8",
)
run(["git", "add", "hello.py"], cwd=source)
run(["git", "commit", "-m", "Initial commit"], cwd=source)
run(["git", "clone", "--bare", str(source), str(remote)])
os.environ["GIT_BASE_BRANCH"] = "main"
git_ops = GitOperations(str(workspace), "test-agent")
assert await git_ops.clone_workspace(str(remote))
assert await git_ops.is_git_workspace()
assert await git_ops.create_result_branch("task-123")
(workspace / "hello.py").write_text(
"def hello_world():\n return 'hello from agent'\n",
encoding="utf-8",
)
commit_sha = await git_ops.commit_changes("Task task-123: update hello")
assert commit_sha
branch_name = await git_ops.push_results()
assert branch_name
refs = run(["git", "for-each-ref", "--format=%(refname:short)", "refs/heads"], cwd=remote)
assert branch_name in refs.stdout.splitlines()
print("Git workflow test passed")
print(f"branch={branch_name}")
print(f"commit={commit_sha}")
if __name__ == "__main__":
asyncio.run(main())
+263
View File
@@ -0,0 +1,263 @@
"""Smoke tests for the agent_swarm_v4 -> heicode-swarm merge.
Exercises the new/changed code paths with the gated in-memory (fakeredis) fallback:
- redis_client REDIS_FAKE fallback through the rich list API
- task_queue.release_task (capacity-rejection requeue, no retry increment)
- planner static fallback + build_planner_task_specs mapping & dependency filtering
- agent peer-collaboration reply routing and capacity rejection messages
"""
import asyncio
import os
import sys
import types
from pathlib import Path
os.environ["REDIS_FAKE"] = "1" # gated dev/CI in-memory store
os.environ.pop("OPENAI_API_KEY", None) # force planner static fallback
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from orchestrator.redis_client import redis_client
from orchestrator.agent_registry import agent_registry, AgentStatus
from orchestrator.task_queue import task_queue, TaskStatus
from orchestrator import main as orch
from orchestrator.planner import planner
# Importing orchestrator.main runs load_dotenv(), which may set a real OPENAI_API_KEY from a
# local .env and give the planner a live client. Force it offline so these checks stay
# hermetic and deterministic (static plan + heuristic review + concatenated synthesis).
planner.client = None
failures = []
def check(name, cond):
print(("PASS" if cond else "FAIL"), "-", name)
if not cond:
failures.append(name)
async def test_redis_fallback():
await redis_client.connect()
await redis_client.set("k", "v")
check("redis_fallback get/set", (await redis_client.get("k")) == "v")
await redis_client.lpush("q", "a")
await redis_client.lpush("q", "b")
check("redis_fallback list ops", (await redis_client.lrange("q", 0, -1)) == ["b", "a"])
await redis_client.lrem("q", 0, "a")
check("redis_fallback lrem", (await redis_client.lrange("q", 0, -1)) == ["b"])
async def test_release_task():
await agent_registry.register_agent("agent-x", ["general"])
task = await task_queue.create_task(description="do thing", task_id="t-rel-1")
assigned = await task_queue.assign_task(task.task_id, "agent-x")
check("assign_task ok", assigned)
released = await task_queue.release_task(task.task_id, agent_id="agent-x")
check("release_task ok", released)
reloaded = await task_queue.get_task(task.task_id)
agent = await agent_registry.get_agent("agent-x")
pending = await redis_client.lrange(task_queue.PENDING_QUEUE_KEY, 0, -1)
check("release sets PENDING", reloaded.status == TaskStatus.PENDING)
check("release no retry increment", reloaded.retry_count == 0)
check("release frees agent", agent.status == AgentStatus.IDLE)
check("release requeues task", task.task_id in pending)
async def test_planner_fallback():
subtasks = await planner.build_plan("swarm-test", "Build a calculator")
roles = [s.get("role") for s in subtasks]
check("planner static fallback (3 specialists)", roles == ["implementation", "testing", "documentation"])
run = types.SimpleNamespace(swarm_id="swarm-test", objective="Build a calculator")
base_specs = [{"context": {"orchestration_plan": {"x": 1}}}]
specs = await orch.build_planner_task_specs(run, {}, base_specs)
check("planner specs preserve base context", specs[0]["context"].get("orchestration_plan") == {"x": 1})
check("planner specs carry deps", any(s["depends_on"] for s in specs))
# Inject a phantom dependency and confirm it is filtered out.
specs2 = list(specs)
specs2[0]["depends_on"] = ["does-not-exist"]
run2 = types.SimpleNamespace(swarm_id="swarm-test", objective="x")
async def fake_plan(swarm_id, objective):
return [
{"subtask_id": "a", "description": "A", "role": "implementation", "required_capabilities": ["python"], "depends_on": ["ghost"]},
{"subtask_id": "b", "description": "B", "role": "testing", "required_capabilities": ["testing"], "depends_on": ["a"]},
]
orig = planner.build_plan
planner.build_plan = fake_plan
try:
filtered = await orch.build_planner_task_specs(run2, {}, base_specs)
finally:
planner.build_plan = orig
deps_a = next(s for s in filtered if s["task_id"] == "a")["depends_on"]
deps_b = next(s for s in filtered if s["task_id"] == "b")["depends_on"]
check("phantom dependency filtered", deps_a == [])
check("valid dependency kept", deps_b == ["a"])
async def test_agent_peer_routing():
from agent.main import Agent
a = Agent(orchestrator_url="ws://localhost:8000", agent_id="agent-peer", capabilities=["python"])
sent = []
async def fake_send(payload):
sent.append(payload)
a.safe_send = fake_send
# Inbound peer query (no matching waiter) -> agent answers with a reply.
await a.handle_peer_message({
"type": "peer_message",
"from_agent_id": "agent-impl",
"task_id": "t1",
"content": "please advise",
"correlation_id": "corr-1",
"is_reply": False,
})
check("peer query produces a reply", len(sent) == 1 and sent[0]["is_reply"] is True)
check("peer reply targets requester", sent[0]["target_agent_id"] == "agent-impl")
# Inbound reply resolves an outstanding waiter (the requester side).
loop = asyncio.get_running_loop()
waiter = loop.create_future()
a.peer_waiters["corr-2"] = waiter
await a.handle_peer_message({
"type": "peer_message",
"from_agent_id": "agent-impl",
"task_id": "t1",
"content": "here is guidance",
"correlation_id": "corr-2",
"is_reply": True,
})
check("peer reply resolves waiter", waiter.done() and waiter.result()["content"] == "here is guidance")
# Capacity rejection: fill active_tasks to the limit, then a new assignment is rejected.
sent.clear()
a.active_tasks = {f"t{i}": None for i in range(a.MAX_CONCURRENT_TASKS)}
await a.handle_task_assignment({"task_id": "overflow", "description": "x", "context": {}})
check("over-capacity assignment is rejected", len(sent) == 1 and sent[0]["type"] == "task_rejected")
async def test_review_and_synthesis():
# planner.review heuristic rejects conflicting test frameworks.
conflicting = {
"swarm-r-testing": {"result": {"subtasks": [{"summary": "use pytest", "changes": "pytest suite"}]}},
"swarm-r-doc": {"result": {"subtasks": [{"summary": "docs say unittest", "changes": "unittest examples"}]}},
}
verdict = await planner.review("obj", [], conflicting)
check("review rejects conflicting frameworks", verdict["accepted"] is False and verdict["retry_tasks"])
aligned = {"swarm-r-impl": {"result": {"subtasks": [{"summary": "clean implementation", "changes": "added add()"}]}}}
verdict2 = await planner.review("obj", [], aligned)
check("review accepts aligned results", verdict2["accepted"] is True)
# synthesize falls back to a deterministic concatenation when no model is configured.
synth = await planner.synthesize("obj", aligned)
check("synthesize produces a non-empty response", isinstance(synth, str) and "implementation" in synth)
async def test_review_cycle():
# Patch out callback/persistence side effects and force a rejection from the critic.
orig_save, orig_emit, orig_review = orch.swarm_runtime.save_run, orch.swarm_runtime.emit_event, orch.planner.review
async def noop(*a, **k):
return None
orch.swarm_runtime.save_run = noop
orch.swarm_runtime.emit_event = noop
task = await task_queue.create_task(description="impl", task_id="t-rev-1")
await task_queue.complete_task(task.task_id, '{"summary": "did impl"}')
completed = await task_queue.get_task(task.task_id)
run = types.SimpleNamespace(
swarm_id="swarm-rev", deployment_id="dep-rev", manager_deployment_id="mgr-rev",
objective="obj", task_ids=["t-rev-1"], status="completed", metadata={}
)
async def reject(objective, tasks, results):
return {"accepted": False, "summary": "needs work", "retry_tasks": ["t-rev-1"]}
orch.planner.review = reject
try:
reopened = await orch.maybe_run_review_cycle(run, [completed])
check("review cycle reopens rejected task", reopened is True)
reloaded = await task_queue.get_task("t-rev-1")
check("reopened task is PENDING again", reloaded.status == TaskStatus.PENDING)
check("review cycle counter incremented", run.metadata.get("review_cycles") == 1)
check("run set back to running", run.status == "running")
# Exhaust the cycle budget -> no further reopen.
run.metadata["review_cycles"] = orch.review_max_cycles()
await task_queue.complete_task("t-rev-1", '{"summary": "did impl again"}')
completed2 = await task_queue.get_task("t-rev-1")
reopened2 = await orch.maybe_run_review_cycle(run, [completed2])
check("review cycle respects budget", reopened2 is False)
finally:
orch.swarm_runtime.save_run = orig_save
orch.swarm_runtime.emit_event = orig_emit
orch.planner.review = orig_review
async def test_dispatch_context():
# A dependency that completed should appear as a dependency artifact; a connected peer
# assigned to another task on the run should appear as a peer agent.
orch.manager.active_connections["peer-conn"] = object()
try:
dep = await task_queue.create_task(description="upstream", task_id="dc-dep")
await task_queue.complete_task(dep.task_id, '{"summary": "upstream done", "files_modified": ["a.py"]}')
peer = await task_queue.create_task(description="peer work", task_id="dc-peer", agent_role="implementation")
await task_queue.assign_task(peer.task_id, "peer-conn") if False else None
# Manually mark the peer task as owned by the connected agent.
peer.assigned_agent_id = "peer-conn"
await task_queue._save_task(peer)
consumer = await task_queue.create_task(
description="downstream", task_id="dc-main", depends_on=["dc-dep"], agent_role="testing"
)
run = types.SimpleNamespace(swarm_id="swarm-dc", objective="ship it", task_ids=["dc-dep", "dc-peer", "dc-main"])
ctx = await orch.build_dispatch_context(run, consumer)
check("dispatch context injects dependency_artifacts", len(ctx.get("dependency_artifacts", [])) == 1)
check("dependency artifact carries summary", ctx["dependency_artifacts"][0]["summary"] == "upstream done")
check("dispatch context injects peer_agents", any(p["agent_id"] == "peer-conn" for p in ctx.get("peer_agents", [])))
check("dispatch context carries run goal", ctx.get("run_goal") == "ship it")
check("dispatch context sets specialist_role from agent_role", ctx.get("specialist_role") == "testing")
finally:
orch.manager.active_connections.pop("peer-conn", None)
async def test_agent_peer_shares_summary():
from agent.main import Agent
a = Agent(orchestrator_url="ws://localhost:8000", agent_id="agent-impl2", capabilities=["python"])
a.last_summary = "implemented add() that raises ValueError on bad input"
sent = []
async def fake_send(payload):
sent.append(payload)
a.safe_send = fake_send
await a.answer_peer_query({"from_agent_id": "agent-test", "correlation_id": "c", "is_reply": False})
check("peer reply shares last summary", "implemented add()" in sent[0]["content"])
async def main():
await test_redis_fallback()
await test_release_task()
await test_planner_fallback()
await test_agent_peer_routing()
await test_review_and_synthesis()
await test_review_cycle()
await test_dispatch_context()
await test_agent_peer_shares_summary()
print()
if failures:
print(f"{len(failures)} check(s) FAILED: {failures}")
sys.exit(1)
print("all merge smoke checks passed")
if __name__ == "__main__":
asyncio.run(main())
+203
View File
@@ -0,0 +1,203 @@
#!/usr/bin/env python3
"""Test script for orchestrator functionality."""
import asyncio
import json
import sys
from typing import Optional
try:
import websockets
import httpx
except ImportError:
print("Installing required packages...")
import subprocess
subprocess.check_call([sys.executable, "-m", "pip", "install", "websockets", "httpx"])
import websockets
import httpx
ORCHESTRATOR_URL = "http://localhost:8000"
ORCHESTRATOR_WS = "ws://localhost:8000"
async def test_health_check():
"""Test health check endpoint."""
print("\n=== Testing Health Check ===")
async with httpx.AsyncClient() as client:
response = await client.get(f"{ORCHESTRATOR_URL}/health")
print(f"Status: {response.status_code}")
print(f"Response: {response.json()}")
assert response.status_code == 200
assert response.json()["status"] == "ok"
print("✅ Health check passed")
async def test_agent_registration():
"""Test agent registration via WebSocket."""
print("\n=== Testing Agent Registration ===")
agent_id = "test-agent-1"
uri = f"{ORCHESTRATOR_WS}/ws/{agent_id}"
async with websockets.connect(uri) as websocket:
# Send registration
await websocket.send(json.dumps({
"type": "register",
"capabilities": ["python", "javascript"]
}))
# Receive confirmation
response = await websocket.recv()
data = json.loads(response)
print(f"Registration response: {data}")
assert data["type"] == "registered"
assert data["agent_id"] == agent_id
# Verify agent appears in registry
async with httpx.AsyncClient() as client:
response = await client.get(f"{ORCHESTRATOR_URL}/agents/{agent_id}")
agent_data = response.json()
print(f"Agent data: {agent_data}")
assert agent_data["agent_id"] == agent_id
assert agent_data["status"] == "idle"
print("✅ Agent registration passed")
# Test heartbeat
print("\n=== Testing Heartbeat ===")
await websocket.send(json.dumps({"type": "heartbeat"}))
response = await websocket.recv()
data = json.loads(response)
print(f"Heartbeat response: {data}")
assert data["type"] == "heartbeat_ack"
print("✅ Heartbeat passed")
async def test_task_creation():
"""Test task creation and assignment."""
print("\n=== Testing Task Creation ===")
async with httpx.AsyncClient() as client:
# Create task
response = await client.post(
f"{ORCHESTRATOR_URL}/tasks",
json={
"description": "Test task",
"context": {"test": "data"},
"max_retries": 3
}
)
task_data = response.json()
print(f"Created task: {task_data}")
assert response.status_code == 200
assert task_data["status"] == "pending"
task_id = task_data["task_id"]
# List tasks
response = await client.get(f"{ORCHESTRATOR_URL}/tasks")
tasks = response.json()
print(f"Total tasks: {len(tasks['tasks'])}")
print("✅ Task creation passed")
return task_id
async def test_handoff():
"""Test handoff mechanism between two agents."""
print("\n=== Testing Handoff Mechanism ===")
agent1_id = "test-agent-1"
agent2_id = "test-agent-2"
# Connect both agents
uri1 = f"{ORCHESTRATOR_WS}/ws/{agent1_id}"
uri2 = f"{ORCHESTRATOR_WS}/ws/{agent2_id}"
async with websockets.connect(uri1) as ws1, websockets.connect(uri2) as ws2:
# Register agent 1
await ws1.send(json.dumps({
"type": "register",
"capabilities": ["python"]
}))
await ws1.recv()
# Register agent 2
await ws2.send(json.dumps({
"type": "register",
"capabilities": ["javascript"]
}))
await ws2.recv()
# Agent 1 initiates handoff to agent 2
await ws1.send(json.dumps({
"type": "handoff",
"target_agent_id": agent2_id,
"task_context": {
"task_id": "test-task-123",
"description": "Test handoff",
"files": ["test.py"]
}
}))
# Agent 1 receives handoff confirmation
response1 = await ws1.recv()
data1 = json.loads(response1)
print(f"Agent 1 response: {data1}")
assert data1["type"] == "handoff_initiated"
handoff_id = data1["handoff_id"]
# Agent 2 receives handoff request
response2 = await ws2.recv()
data2 = json.loads(response2)
print(f"Agent 2 response: {data2}")
assert data2["type"] == "handoff_request"
assert data2["handoff_id"] == handoff_id
# Agent 2 accepts handoff
await ws2.send(json.dumps({
"type": "handoff_accept",
"handoff_id": handoff_id
}))
response2 = await ws2.recv()
data2 = json.loads(response2)
print(f"Agent 2 accept response: {data2}")
assert data2["type"] == "handoff_accepted"
# Verify handoff in history
async with httpx.AsyncClient() as client:
response = await client.get(f"{ORCHESTRATOR_URL}/handoffs")
handoffs = response.json()
print(f"Total handoffs: {len(handoffs['handoffs'])}")
assert len(handoffs['handoffs']) > 0
print("✅ Handoff mechanism passed")
async def main():
"""Run all tests."""
print("Starting orchestrator tests...")
print("Make sure orchestrator is running on localhost:8000")
print("Run: kubectl port-forward -n swarm-system svc/orchestrator-service 8000:8000")
try:
await test_health_check()
await test_agent_registration()
await test_task_creation()
await test_handoff()
print("\n" + "="*50)
print("✅ All tests passed!")
print("="*50)
except Exception as e:
print(f"\n❌ Test failed: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
asyncio.run(main())
+153
View File
@@ -0,0 +1,153 @@
"""Lightweight contract checks for the HeiCode Agent Manager runtime bridge."""
import asyncio
import os
import sys
import types
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
if "redis.asyncio" not in sys.modules:
redis_module = types.ModuleType("redis")
redis_asyncio_module = types.ModuleType("redis.asyncio")
redis_asyncio_module.Redis = object
redis_module.asyncio = redis_asyncio_module
sys.modules["redis"] = redis_module
sys.modules["redis.asyncio"] = redis_asyncio_module
if "httpx" not in sys.modules:
httpx_module = types.ModuleType("httpx")
class AsyncClient:
def __init__(self, *args, **kwargs):
pass
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def post(self, *args, **kwargs):
return types.SimpleNamespace(status_code=204, text="")
httpx_module.AsyncClient = AsyncClient
sys.modules["httpx"] = httpx_module
from orchestrator.redis_client import redis_client
from orchestrator.swarm_runtime import RuntimeValidationError, SwarmRuntime
class FakeRedis:
def __init__(self):
self.values = {}
self.lists = {}
async def set(self, key, value, ex=None):
self.values[key] = value
async def get(self, key):
return self.values.get(key)
async def keys(self, pattern):
prefix = pattern.rstrip("*")
return [key for key in self.values if key.startswith(prefix)]
async def rpush(self, key, *values):
self.lists.setdefault(key, []).extend(values)
async def lrange(self, key, start, end):
items = self.lists.get(key, [])
if end == -1:
return items[start:]
return items[start : end + 1]
def valid_body():
return {
"orchestration_plan": {
"objective": "Add a multiply helper and push the branch",
"sub_mode": "code",
"risk_level": "low",
"budget": {"duration_seconds": 3600, "token_limit": 20000},
"agents": [
{
"task_id": "backend-1",
"role": "backend",
"title": "Backend change",
"description": "Implement the helper",
"depends_on": [],
"resource_grants": [
{"secret_ref": "azkv://heicode/git-write-token"}
],
}
],
},
"callback": {
"url": "http://manager.local/api/agnet/callbacks/swarm-events",
"subscribed_events": ["task.created"],
},
"metadata": {
"manager_deployment_id": "dep_manager_123",
"correlation_id": "corr_123",
},
"billing_context": {"secret_ref": "azkv://heicode/billing"},
"resource_grants": [{"ref": "azkv://heicode/repo-main"}],
}
async def main():
fake = FakeRedis()
redis_client.client = fake
runtime = SwarmRuntime()
os.environ["ENABLE_SUBTASK_HANDOFF"] = "true"
body = valid_body()
runtime.validate_create_request(body)
invalid = valid_body()
invalid["resource_grants"][0]["ref"] = "plain-token"
try:
runtime.validate_create_request(invalid)
raise AssertionError("plain resource ref was accepted")
except RuntimeValidationError:
pass
invalid = valid_body()
invalid["metadata"]["api_key"] = "secret-value"
try:
runtime.validate_create_request(invalid)
raise AssertionError("plaintext secret metadata was accepted")
except RuntimeValidationError:
pass
run, created = await runtime.get_or_create_run(body, "idem-1", "corr_123")
assert created is True
same_run, created_again = await runtime.get_or_create_run(body, "idem-1", "corr_123")
assert created_again is False
assert same_run.swarm_id == run.swarm_id
tasks = runtime.build_task_descriptions(body)
assert tasks[0]["task_id"] == "backend-1"
assert tasks[0]["depends_on"] == []
assert tasks[0]["workflow_mode"] == "multi_agent"
await runtime.emit_event(run, "timeline.updated", payload={"summary": "ok"})
logs = await runtime.list_events(run.swarm_id)
assert logs["events"][0]["event_type"] == "deployment.status_changed"
assert logs["events"][-1]["event_type"] == "timeline.updated"
approval_body = valid_body()
approval_body["orchestration_plan"]["risk_level"] = "high"
approval_run, _ = await runtime.get_or_create_run(approval_body, "idem-2", "corr_123")
approval_id = next(iter(approval_run.approvals))
rejected = await runtime.record_approval_decision(
approval_run.swarm_id,
approval_id,
{"decision": "rejected", "reason": "user rejected"},
)
assert rejected.status == "blocked"
print("runtime contract checks passed")
if __name__ == "__main__":
asyncio.run(main())
+132
View File
@@ -0,0 +1,132 @@
"""End-to-end workflow test: proves the program follows the expected workflow.
Boots the REAL orchestrator (uvicorn, in-process) on the in-memory store, connects a keyless
stub agent over WebSocket, submits one objective, and asserts the full loop:
decompose -> dispatch to experts -> execute -> master review (with one forced reopen)
-> iterate -> synthesize -> deliver
No OPENAI_API_KEY required: the planner is forced offline (static decomposition + heuristic
review + concatenated synthesis), and the stub agent returns canned results that make the
critic reject exactly once before accepting.
Run from the agent_swarm_v5 directory:
python scripts/test-workflow-e2e.py
"""
import asyncio
import logging
import os
import sys
import threading
from pathlib import Path
# Configure the runtime for a deterministic, hermetic run BEFORE importing the app.
os.environ["REDIS_FAKE"] = "1"
os.environ["ENABLE_PLANNER_FALLBACK"] = "1"
os.environ["ENABLE_REVIEW_LOOP"] = "1"
os.environ["MAX_REVIEW_CYCLES"] = "2"
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
sys.path.insert(0, str(ROOT / "scripts"))
import httpx
import uvicorn
from orchestrator import main as orch
from stub_agent import StubAgent
# Force the planner/critic/synthesis offline so the workflow is deterministic regardless of any
# .env key (load_dotenv runs at import). Quiet the dummy-callback warnings.
orch.planner.client = None
logging.getLogger("orchestrator.swarm_runtime").setLevel(logging.ERROR)
PORT = 8123
BASE = f"http://127.0.0.1:{PORT}"
WS = f"ws://127.0.0.1:{PORT}"
failures = []
def check(name, cond):
print(("PASS" if cond else "FAIL"), "-", name)
if not cond:
failures.append(name)
async def wait_health(client):
for _ in range(150):
try:
if (await client.get(f"{BASE}/health")).status_code == 200:
return True
except Exception:
pass
await asyncio.sleep(0.1)
return False
async def main():
config = uvicorn.Config(orch.app, host="127.0.0.1", port=PORT, log_level="warning")
server = uvicorn.Server(config)
server.install_signal_handlers = lambda: None # required when not on the main thread
threading.Thread(target=server.run, daemon=True).start()
agent = StubAgent(WS, "stub-1", [
"python", "code_generation", "testing", "pytest", "technical-writing", "general",
])
agent_task = None
try:
async with httpx.AsyncClient(timeout=10) as client:
if not await wait_health(client):
check("orchestrator started", False)
return
agent_task = asyncio.create_task(agent.run())
await asyncio.sleep(0.5) # let the agent register
body = {
"mode": "swarm",
"requirement": {"objective": "Write a Python add(a,b) function with tests and docs"},
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
"metadata": {"manager_deployment_id": "e2e-1"},
}
created = (await client.post(f"{BASE}/api/swarms", json=body)).json()
dep = created["data"]["deployment_id"]
# Poll until the run finishes (or times out).
status, wf = None, {}
for _ in range(200):
wf = (await client.get(f"{BASE}/api/swarms/{dep}/workflow")).json()["data"]
status = wf.get("status")
if status in ("completed", "failed"):
break
await asyncio.sleep(0.25)
tasks = (await client.get(f"{BASE}/api/swarms/{dep}/tasks")).json()["data"]["tasks"]
events = (await client.get(f"{BASE}/api/swarms/{dep}/logs")).json()["data"]["events"]
roles = sorted(t["agent_role"] for t in tasks)
summaries = [(e.get("payload") or {}).get("summary", "") or "" for e in events]
review_seen = any("Review cycle" in s for s in summaries)
# ---- workflow assertions ----
check("decompose: 3 specialist tasks created", len(tasks) == 3)
check("decompose: roles are implementation/testing/documentation",
roles == ["documentation", "implementation", "testing"])
check("execute: every task completed", bool(tasks) and all(t["status"] == "completed" for t in tasks))
check("iterate: at least one master review cycle occurred", review_seen)
check("deliver: run reached completed", status == "completed")
check("synthesize: a final unified summary is present", bool(wf.get("summary")))
finally:
agent.running = False
if agent_task:
agent_task.cancel()
server.should_exit = True
print()
if failures:
print(f"{len(failures)} workflow check(s) FAILED: {failures}")
sys.exit(1)
print("workflow follows the expected sequence: decompose -> dispatch -> execute -> review/iterate -> synthesize")
if __name__ == "__main__":
asyncio.run(main())
+308
View File
@@ -0,0 +1,308 @@
[
{
"task": "Fix syntax error in main.py line 42",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Single file, single line fix. No dependencies or side effects."
},
{
"task": "Add type hints to the calculate_total function",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Single function modification in one file."
},
{
"task": "Update README.md with installation instructions",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Documentation update, no code changes."
},
{
"task": "Rename variable 'data' to 'user_data' in auth.py",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Simple refactoring in single file."
},
{
"task": "Fix import statement missing requests library",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Single line addition at top of file."
},
{
"task": "Add docstring to the process_payment function",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Documentation addition to single function."
},
{
"task": "Remove unused import statements from utils.py",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Simple cleanup in one file."
},
{
"task": "Change default timeout from 30 to 60 seconds",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Single constant value change."
},
{
"task": "Fix indentation error in the for loop",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Formatting fix in single location."
},
{
"task": "Add missing return statement in validate_email function",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Single line addition to fix logic error."
},
{
"task": "Refactor the user authentication module to use JWT tokens",
"expectedAgents": 3,
"complexity": "medium",
"reasoning": "Affects multiple functions in auth module, requires updating login/logout flows and adding token validation."
},
{
"task": "Add input validation to all API endpoints",
"expectedAgents": 5,
"complexity": "medium",
"reasoning": "Multiple endpoints across different route files need validation logic."
},
{
"task": "Implement caching layer using Redis for database queries",
"expectedAgents": 4,
"complexity": "medium",
"reasoning": "Requires Redis setup, cache invalidation logic, and updating multiple query functions."
},
{
"task": "Add logging to all service layer functions",
"expectedAgents": 3,
"complexity": "medium",
"reasoning": "Multiple service files need consistent logging patterns."
},
{
"task": "Migrate database from SQLite to PostgreSQL",
"expectedAgents": 6,
"complexity": "medium",
"reasoning": "Schema migration, connection updates, query syntax changes across multiple files."
},
{
"task": "Implement rate limiting for API endpoints",
"expectedAgents": 3,
"complexity": "medium",
"reasoning": "Middleware implementation and configuration for multiple routes."
},
{
"task": "Add error handling to all async functions",
"expectedAgents": 4,
"complexity": "medium",
"reasoning": "Multiple async functions across different modules need try-catch blocks."
},
{
"task": "Refactor monolithic app.py into separate route modules",
"expectedAgents": 5,
"complexity": "medium",
"reasoning": "Code organization across multiple new files with proper imports."
},
{
"task": "Add pagination to all list endpoints",
"expectedAgents": 4,
"complexity": "medium",
"reasoning": "Multiple endpoints need offset/limit logic and response formatting."
},
{
"task": "Implement user role-based access control",
"expectedAgents": 5,
"complexity": "medium",
"reasoning": "Decorator implementation, role checks across multiple endpoints, database schema update."
},
{
"task": "Add comprehensive unit tests to the entire project",
"expectedAgents": 12,
"complexity": "high",
"reasoning": "Testing all modules, functions, edge cases. Requires understanding entire codebase and writing hundreds of test cases."
},
{
"task": "Refactor entire codebase to follow clean architecture principles",
"expectedAgents": 15,
"complexity": "high",
"reasoning": "Major restructuring: separate layers (domain, application, infrastructure), update all imports, maintain functionality."
},
{
"task": "Implement microservices architecture splitting monolith into 5 services",
"expectedAgents": 18,
"complexity": "high",
"reasoning": "Service separation, API contracts, inter-service communication, deployment configs, data consistency."
},
{
"task": "Add end-to-end tests for all user workflows",
"expectedAgents": 10,
"complexity": "high",
"reasoning": "Multiple user journeys, test setup, teardown, data fixtures, UI interactions."
},
{
"task": "Migrate from REST API to GraphQL",
"expectedAgents": 14,
"complexity": "high",
"reasoning": "Schema definition, resolver implementation, client updates, query optimization across entire API."
},
{
"task": "Implement real-time features using WebSockets",
"expectedAgents": 8,
"complexity": "high",
"reasoning": "WebSocket server setup, connection management, event broadcasting, client integration across features."
},
{
"task": "Add internationalization support for 10 languages",
"expectedAgents": 9,
"complexity": "high",
"reasoning": "Extract all strings, translation files, locale switching, date/number formatting across entire app."
},
{
"task": "Implement comprehensive security audit fixes",
"expectedAgents": 11,
"complexity": "high",
"reasoning": "SQL injection prevention, XSS protection, CSRF tokens, secure headers, input sanitization across all endpoints."
},
{
"task": "Refactor to use async/await throughout the codebase",
"expectedAgents": 13,
"complexity": "high",
"reasoning": "Convert synchronous code to async, update all function signatures, handle async context across entire project."
},
{
"task": "Build CI/CD pipeline with automated testing and deployment",
"expectedAgents": 7,
"complexity": "high",
"reasoning": "Pipeline configuration, test automation, deployment scripts, environment management, rollback strategy."
},
{
"task": "Add a new user registration endpoint",
"expectedAgents": 2,
"complexity": "low",
"reasoning": "Single endpoint with validation and database insert."
},
{
"task": "Update email template for password reset",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Template file modification only."
},
{
"task": "Add environment variable for API key",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Config file update and usage in one location."
},
{
"task": "Fix broken link in navigation menu",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Single URL correction in template."
},
{
"task": "Add favicon to the website",
"expectedAgents": 1,
"complexity": "low",
"reasoning": "Add file and reference in HTML head."
},
{
"task": "Implement search functionality across user profiles",
"expectedAgents": 3,
"complexity": "medium",
"reasoning": "Search endpoint, query optimization, result formatting."
},
{
"task": "Add file upload feature with validation",
"expectedAgents": 3,
"complexity": "medium",
"reasoning": "Upload endpoint, file validation, storage handling, error cases."
},
{
"task": "Implement email notification system",
"expectedAgents": 4,
"complexity": "medium",
"reasoning": "Email service integration, template system, queue management, multiple trigger points."
},
{
"task": "Add OAuth2 authentication with Google and GitHub",
"expectedAgents": 5,
"complexity": "medium",
"reasoning": "OAuth flow implementation, provider integration, user linking, token management."
},
{
"task": "Create admin dashboard with analytics",
"expectedAgents": 6,
"complexity": "medium",
"reasoning": "Dashboard UI, data aggregation queries, charts, multiple admin views."
},
{
"task": "Optimize database queries for performance",
"expectedAgents": 5,
"complexity": "medium",
"reasoning": "Identify slow queries, add indexes, refactor N+1 queries across multiple models."
},
{
"task": "Implement data export feature in CSV and JSON formats",
"expectedAgents": 3,
"complexity": "medium",
"reasoning": "Export endpoints, format serialization, large dataset handling."
},
{
"task": "Add two-factor authentication",
"expectedAgents": 4,
"complexity": "medium",
"reasoning": "TOTP implementation, QR code generation, backup codes, login flow updates."
},
{
"task": "Implement audit logging for all data changes",
"expectedAgents": 6,
"complexity": "medium",
"reasoning": "Audit table design, triggers/hooks across models, query interface, retention policy."
},
{
"task": "Add real-time notifications using Server-Sent Events",
"expectedAgents": 4,
"complexity": "medium",
"reasoning": "SSE endpoint, event broadcasting, client connection management, notification types."
},
{
"task": "Rewrite frontend from jQuery to React",
"expectedAgents": 16,
"complexity": "high",
"reasoning": "Complete frontend rewrite, component architecture, state management, API integration, routing."
},
{
"task": "Implement distributed tracing with OpenTelemetry",
"expectedAgents": 8,
"complexity": "high",
"reasoning": "Instrumentation across all services, trace context propagation, exporter configuration, dashboard setup."
},
{
"task": "Add machine learning model for recommendation system",
"expectedAgents": 10,
"complexity": "high",
"reasoning": "Model training pipeline, feature engineering, inference API, model versioning, A/B testing."
},
{
"task": "Implement event sourcing architecture",
"expectedAgents": 14,
"complexity": "high",
"reasoning": "Event store setup, aggregate design, event handlers, projections, migration from current state."
},
{
"task": "Build mobile app with React Native",
"expectedAgents": 12,
"complexity": "high",
"reasoning": "Mobile app setup, screen components, navigation, API integration, platform-specific features, testing."
},
{
"task": "Implement comprehensive monitoring and alerting",
"expectedAgents": 9,
"complexity": "high",
"reasoning": "Metrics collection, alert rules, dashboard creation, on-call integration, SLO definition."
}
]
+183
View File
@@ -0,0 +1,183 @@
#!/usr/bin/env python3
"""
K8s 蜂群系统测试脚本
测试 Orchestrator API 和系统功能
"""
import requests
import time
import sys
# Orchestrator 地址
ORCHESTRATOR_URL = "http://52.139.240.116:8000"
def print_section(title):
"""打印分隔线"""
print(f"\n{'='*60}")
print(f" {title}")
print(f"{'='*60}\n")
def test_health():
"""测试健康检查"""
print_section("1. 健康检查")
try:
response = requests.get(f"{ORCHESTRATOR_URL}/health", timeout=5)
data = response.json()
print(f"✅ 状态: {data['status']}")
print(f"✅ Redis: {data['redis']}")
print(f"✅ 活跃连接: {data['active_connections']}")
return True
except Exception as e:
print(f"❌ 健康检查失败: {e}")
return False
def test_agents():
"""测试 Agent 列表"""
print_section("2. Agent 列表")
try:
response = requests.get(f"{ORCHESTRATOR_URL}/agents", timeout=5)
agents = response.json().get("agents", [])
print(f"当前 Agent 数量: {len(agents)}")
if agents:
for agent in agents:
print(f" - Agent ID: {agent['agent_id']}")
print(f" 状态: {agent['status']}")
print(f" 最后心跳: {agent.get('last_heartbeat', 'N/A')}")
else:
print(" 暂无 Agent 运行")
return True
except Exception as e:
print(f"❌ 获取 Agent 列表失败: {e}")
return False
def test_tasks():
"""测试任务列表"""
print_section("3. 任务列表")
try:
response = requests.get(f"{ORCHESTRATOR_URL}/tasks", timeout=5)
tasks = response.json().get("tasks", [])
print(f"任务总数: {len(tasks)}")
if tasks:
for task in tasks:
print(f" - 任务 ID: {task['task_id']}")
print(f" 状态: {task['status']}")
print(f" 描述: {task.get('description', 'N/A')[:50]}...")
else:
print(" 暂无任务")
return True
except Exception as e:
print(f"❌ 获取任务列表失败: {e}")
return False
def test_create_task():
"""测试创建任务"""
print_section("4. 创建测试任务")
try:
task_data = {
"description": "测试任务:修复一个简单的语法错误",
"context": {
"workspace_id": "test-workspace-001",
"required_agents": 1
},
"max_retries": 3
}
response = requests.post(
f"{ORCHESTRATOR_URL}/tasks",
json=task_data,
timeout=5
)
if response.status_code == 200:
task = response.json()
print(f"✅ 任务创建成功")
print(f" 任务 ID: {task['task_id']}")
print(f" 状态: {task['status']}")
return task['task_id']
else:
print(f"❌ 任务创建失败: {response.status_code}")
print(f" 响应: {response.text}")
return None
except Exception as e:
print(f"❌ 创建任务失败: {e}")
return None
def test_metrics():
"""测试 Prometheus 指标"""
print_section("5. Prometheus 指标")
try:
response = requests.get(f"{ORCHESTRATOR_URL}/metrics", timeout=5)
metrics = response.text
# 提取关键指标
lines = metrics.split('\n')
key_metrics = [
'swarm_agents_created_total',
'swarm_agents_active',
'swarm_tasks_created_total',
'swarm_tasks_completed_total'
]
print("关键指标:")
for line in lines:
for metric in key_metrics:
if line.startswith(metric) and not line.startswith('#'):
print(f" {line}")
return True
except Exception as e:
print(f"❌ 获取指标失败: {e}")
return False
def test_redis_connection():
"""测试 Redis 连接"""
print_section("6. Redis 连接测试")
try:
# 通过健康检查验证 Redis
response = requests.get(f"{ORCHESTRATOR_URL}/health", timeout=5)
data = response.json()
if data['redis'] == 'connected':
print("✅ Redis 连接正常")
return True
else:
print(f"❌ Redis 状态: {data['redis']}")
return False
except Exception as e:
print(f"❌ Redis 连接测试失败: {e}")
return False
def main():
"""主测试流程"""
print("\n" + "="*60)
print(" K8s 蜂群系统 - 集成测试")
print("="*60)
print(f"\nOrchestrator URL: {ORCHESTRATOR_URL}")
print(f"测试时间: {time.strftime('%Y-%m-%d %H:%M:%S')}")
results = []
# 运行所有测试
results.append(("健康检查", test_health()))
results.append(("Redis 连接", test_redis_connection()))
results.append(("Agent 列表", test_agents()))
results.append(("任务列表", test_tasks()))
results.append(("创建任务", test_create_task() is not None))
results.append(("Prometheus 指标", test_metrics()))
# 总结
print_section("测试总结")
passed = sum(1 for _, result in results if result)
total = len(results)
for name, result in results:
status = "✅ 通过" if result else "❌ 失败"
print(f"{status} - {name}")
print(f"\n总计: {passed}/{total} 测试通过")
if passed == total:
print("\n🎉 所有测试通过!系统运行正常。")
return 0
else:
print(f"\n⚠️ {total - passed} 个测试失败,请检查日志。")
return 1
if __name__ == "__main__":
sys.exit(main())