Merge pull request #26 from xmindlab-heicode/feat/decentralized-swarm-rework
去中心化蜂群重构:播种+自组织为唯一行为(Refs #6 #7 #8 #11 #12 #18)
This commit is contained in:
+38
-10
@@ -78,19 +78,47 @@ jobs:
|
|||||||
env: { REDIS_FAKE: "1" }
|
env: { REDIS_FAKE: "1" }
|
||||||
run: python scripts/test-quality.py
|
run: python scripts/test-quality.py
|
||||||
|
|
||||||
- name: ACO decision engine (Group A)
|
- name: Decision-engine pheromone library (τ)
|
||||||
env: { REDIS_FAKE: "1" }
|
env: { REDIS_FAKE: "1" }
|
||||||
run: python scripts/test-decision-engine.py
|
run: python scripts/test-decision-engine.py
|
||||||
|
|
||||||
# Same e2e workflow, but through the probabilistic ACO dispatch path (seeded).
|
- name: Dispatch scoring formulas
|
||||||
- name: End-to-end workflow test (ACO dispatch on)
|
|
||||||
env: { REDIS_FAKE: "1", ENABLE_ACO_DISPATCH: "1", ACO_SEED: "42" }
|
|
||||||
run: python scripts/test-workflow-e2e.py
|
|
||||||
|
|
||||||
- name: Dispatch scoring formulas (#9 unit)
|
|
||||||
env: { REDIS_FAKE: "1" }
|
env: { REDIS_FAKE: "1" }
|
||||||
run: python scripts/test-dispatch-score.py
|
run: python scripts/test-dispatch-score.py
|
||||||
|
|
||||||
- name: Dispatch scoring integration (#9, scored matchmaking)
|
# --- decentralized swarm flow (the only flow; primitives are unconditional) ---
|
||||||
env: { REDIS_FAKE: "1", ENABLE_DISPATCH_SCORE: "1" }
|
- name: Swarm seeder (#6)
|
||||||
run: python scripts/test-dispatch-scored.py
|
run: python scripts/test-swarm-seed.py
|
||||||
|
|
||||||
|
- name: Swarm self-selection dispatch
|
||||||
|
env: { REDIS_FAKE: "1" }
|
||||||
|
run: python scripts/test-swarm-dispatch.py
|
||||||
|
|
||||||
|
- name: Swarm autonomous task generation (#7)
|
||||||
|
env: { REDIS_FAKE: "1", AGENT_PROPOSAL_BUDGET: "3" }
|
||||||
|
run: python scripts/test-swarm-autonomous.py
|
||||||
|
|
||||||
|
- name: Swarm task competition (#8)
|
||||||
|
env: { REDIS_FAKE: "1" }
|
||||||
|
run: python scripts/test-swarm-competition.py
|
||||||
|
|
||||||
|
- name: Swarm cross-review (#11)
|
||||||
|
env: { REDIS_FAKE: "1" }
|
||||||
|
run: python scripts/test-swarm-cross-review.py
|
||||||
|
|
||||||
|
- name: Swarm convergence (#12)
|
||||||
|
env: { REDIS_FAKE: "1" }
|
||||||
|
run: python scripts/test-swarm-convergence.py
|
||||||
|
|
||||||
|
- name: Swarm health guard
|
||||||
|
env: { REDIS_FAKE: "1" }
|
||||||
|
run: python scripts/test-swarm-guard.py
|
||||||
|
|
||||||
|
# Pure-module unit tests for the swarm primitives (formulas/policies, infra-free).
|
||||||
|
- name: Swarm primitive modules (unit)
|
||||||
|
env: { REDIS_FAKE: "1" }
|
||||||
|
run: |
|
||||||
|
python scripts/test-autonomous-tasks.py
|
||||||
|
python scripts/test-task-competition.py
|
||||||
|
python scripts/test-cross-review.py
|
||||||
|
python scripts/test-convergence.py
|
||||||
|
|||||||
@@ -22,9 +22,9 @@
|
|||||||
## 架构与关键约束(便于定位)
|
## 架构与关键约束(便于定位)
|
||||||
- **orchestrator/**:FastAPI 编排器。Manager 面接口、HMAC 签名回调、审批链**必须保持契约**。Redis 为权威存储;内存回退仅限 `REDIS_FAKE` / `ALLOW_MEMORY_STORE`(开发/CI)。
|
- **orchestrator/**:FastAPI 编排器。Manager 面接口、HMAC 签名回调、审批链**必须保持契约**。Redis 为权威存储;内存回退仅限 `REDIS_FAKE` / `ALLOW_MEMORY_STORE`(开发/CI)。
|
||||||
- **agent/**:执行单元,**OpenAI 兼容**模型;保留计费/审计归属(`usage` 与 `X-Agent/X-Agnet` 头)。
|
- **agent/**:执行单元,**OpenAI 兼容**模型;保留计费/审计归属(`usage` 与 `X-Agent/X-Agnet` 头)。
|
||||||
- **工作流开关默认关闭**:`ENABLE_PLANNER_FALLBACK`、`ENABLE_REVIEW_LOOP`、`ENABLE_SUBTASK_HANDOFF`、`ENABLE_QUALITY_EVAL`、`ENABLE_ACO_DISPATCH`、`ENABLE_DISPATCH_SCORE`。
|
- **去中心化蜂群是唯一行为(cutover 已完成)**:本仓**就是蜂群运行时**,模式选择(single/chain/sub/swarm)在仓外——**无「启用 swarm」开关**。流程:编排器**播种**单一种子任务 → Agent 感知共享池**自选**(信息素 τ + 能力/负载/预算,`swarm_dispatch`)→ 经 `task_proposal` **自主分解**(#7)→ 经 `task_bid/yield/takeover` **竞争/接管**(#8)→ **同伴交叉评审**(≥2 评审者,#11)→ **收敛**(`termination_reason`,#12)。这些原语**全部无条件生效**(无 feature flag)。已删除:Master `planner` 分解兜底、贪心/ACO/scored 派发模式、单 critic Master 评审环及其开关。`master_agent.synthesize` 作为汇总工具保留。设计见 `docs/swarm/decentralized-rework-plan.md`。
|
||||||
- **ACO 决策引擎**(`orchestrator/decision_engine.py`):信息素**学习常开**(被动观察,不改行为);**概率选择仅在** `ENABLE_ACO_DISPATCH=1` 时生效(改变派发顺序,CI 用 `ACO_SEED` 固定随机数)。设计见 `docs/benchmark/decision-engine.md`。
|
- **仅剩的真实开关**:`ENABLE_QUALITY_EVAL` + `HEICODE_SANDBOX_ISOLATED`(Group B 代码沙箱,见下)、`ENABLE_SUBTASK_HANDOFF`(agent 侧子任务移交)。`decision_engine`(信息素 τ)与 `dispatch_score`(可解释打分)现为 `swarm_dispatch` 复用的库,无独立模式开关。
|
||||||
- **调度评分**(`orchestrator/dispatch_score.py`,issue #9):`ENABLE_DISPATCH_SCORE=1` 启用**任务为中心的可解释打分匹配**——为每个就绪任务按能力/历史成功(τ)/负载/预算压力/权限等≥4 个非能力维度给候选 Agent 打分并择优,产出 `dispatch.decision_made` 内部记录(候选明细+排除原因,存 `SwarmRun.dispatch_decisions`,**非 Manager 事件**,可审计/回放)。默认关;与 `ENABLE_ACO_DISPATCH` 互斥(择一)。设计见 `docs/scheduling/dispatch-score-schema.md`。
|
- **待办(P-guard)**:检测「swarm 无法运作」并列原因(无 Agent/无模型/依赖死锁/预算耗尽/种子无法分解)——唯一保留的非正常路径处理,最后编写。
|
||||||
- **代码测试沙箱**(`orchestrator/sandbox.py`):会**执行模型生成代码**,OS 级隔离边界 = K8s Pod。**Fail-closed 双门控**:需同时 `ENABLE_QUALITY_EVAL=1`(开功能)+ `HEICODE_SANDBOX_ISOLATED=1`(显式确认运行在隔离 Pod);二者缺一则启动拒绝/运行时抛 `SandboxIsolationError`,不执行任何代码。安全模型见 `docs/integration/security-boundary.md §8.1`。**`HEICODE_SANDBOX_ISOLATED` 只允许在真正隔离的 Pod 或 ephemeral CI/test runner 中设置。**
|
- **代码测试沙箱**(`orchestrator/sandbox.py`):会**执行模型生成代码**,OS 级隔离边界 = K8s Pod。**Fail-closed 双门控**:需同时 `ENABLE_QUALITY_EVAL=1`(开功能)+ `HEICODE_SANDBOX_ISOLATED=1`(显式确认运行在隔离 Pod);二者缺一则启动拒绝/运行时抛 `SandboxIsolationError`,不执行任何代码。安全模型见 `docs/integration/security-boundary.md §8.1`。**`HEICODE_SANDBOX_ISOLATED` 只允许在真正隔离的 Pod 或 ephemeral CI/test runner 中设置。**
|
||||||
- 提交前必须本地通过:
|
- 提交前必须本地通过:
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Agent Swarm(HeiCode Swarm)
|
# Agent Swarm(HeiCode Swarm)
|
||||||
|
|
||||||
一个多智能体「蜂群」系统:用户提出需求后,由**主控 Agent(Master Agent,`orchestrator/master_agent.py`)**自动将其分解为多个子任务,分发给擅长不同领域的专家 Agent 并行完成;专家之间可就重叠领域相互协作;产出汇总后由主控 Agent 评审是否达标,未达标则退回重做,循环直至生成满意的最终回答。主控 Agent 负责「分解 / 评审决策 / 汇总」的认知决策,编排器负责执行其决策(派发、重开任务、持久化、事件)。
|
一个**去中心化自组织**的多智能体「蜂群」运行时。本仓**就是 swarm 运行时本身**(模式 single/chain/sub/swarm 的选择在仓外);无中央主控分解派发,秩序与质量是**涌现**的。流程:编排器把用户需求**播种**为单一种子任务 → 各 Agent 感知共享池(任务池 + 信息素 τ)**自选**最适任务(`swarm_dispatch`)→ 经 `task_proposal` **自主分解**出后续子任务(bottom-up,非主控分解)→ 经 `task_bid/yield/takeover` **竞争/接管** → **同伴交叉评审**(≥2 评审者,取代单一评审者)→ **收敛**判定并给出 `termination_reason`,最后**汇总**为统一回答。设计与历史见 [docs/swarm/decentralized-rework-plan.md](docs/swarm/decentralized-rework-plan.md)。
|
||||||
|
|
||||||
> ⚠️ **能力边界(主链路接入状态)**
|
> ⚠️ **能力边界(主链路接入状态)**
|
||||||
> 本仓当前是一个**可运行的多 Agent 工作流运行系统**,**尚未**作为 Heicode 主链路的正式 Runtime Backend 接入。下表区分能力状态;契约见 [docs/integration/](docs/integration/),量化标准见 [docs/benchmark/](docs/benchmark/)。
|
> 本仓当前是一个**可运行的多 Agent 工作流运行系统**,**尚未**作为 Heicode 主链路的正式 Runtime Backend 接入。下表区分能力状态;契约见 [docs/integration/](docs/integration/),量化标准见 [docs/benchmark/](docs/benchmark/)。
|
||||||
@@ -13,6 +13,8 @@
|
|||||||
> | Benchmark 自证(`Benchmark_Agent`、`S_swarm`、`G_E`、`G_E,c`、治理/协作/通信/鲁棒性指标、baseline 对比、telemetry 架构) | 🔴 规划中(标准见 `docs/benchmark/`,采集器尚未落地) |
|
> | Benchmark 自证(`Benchmark_Agent`、`S_swarm`、`G_E`、`G_E,c`、治理/协作/通信/鲁棒性指标、baseline 对比、telemetry 架构) | 🔴 规划中(标准见 `docs/benchmark/`,采集器尚未落地) |
|
||||||
>
|
>
|
||||||
> 在以上「待接入 / 规划中」项目完成并经对应 Team 验收前,本文与各子文档**不得宣称**「已接入主链路」或「已具备完整 Agent Swarm 工程能力」。
|
> 在以上「待接入 / 规划中」项目完成并经对应 Team 验收前,本文与各子文档**不得宣称**「已接入主链路」或「已具备完整 Agent Swarm 工程能力」。
|
||||||
|
>
|
||||||
|
> 🔄 **去中心化重构:核心 cutover 已完成(Path B)**。已从 Master 中心化切换为「播种 + 自组织」蜂群并设为**唯一**流程:删除了 planner 主控分解、贪心/ACO/scored 派发模式、单 critic 评审环及其全部 `ENABLE_*` 开关;播种/自选/自主分解/竞争/交叉评审/收敛**无条件生效**。仅剩待办为 **P-guard**(检测「swarm 无法运作」并列原因)。进度见 [docs/swarm/decentralized-rework-plan.md](docs/swarm/decentralized-rework-plan.md)。下文若有「主控 Agent 分解」字样为历史描述,以本段与计划文档为准。
|
||||||
|
|
||||||
## 系统架构
|
## 系统架构
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# 产品定位说明(Product Positioning)
|
||||||
|
|
||||||
|
> 状态:**已选定 Path B 并完成去中心化重构(owner 指示)**。本文回应工单 #6「蜂群模式产品定义偏差」:原实现为 **Master 中心化**,heicodeDocs 蜂群定义为**弱中心**;现已按 **Path B** 重构为弱中心自组织并设为**唯一行为**(无开关;计划见 [swarm/decentralized-rework-plan.md](./swarm/decentralized-rework-plan.md))。
|
||||||
|
>
|
||||||
|
> **决策记录与 #6 关闭口径**:路径选择的依据 = owner 在工单/会话中的指示 + 本重构 PR + 本文 + [swarm-definition-gap.md](./swarm-definition-gap.md)。但**尚无形式 ARB 决策记录的链接**。故重构 PR 以 **`Refs #6`**(不自动关闭)引用本单;待补 ARB 记录链接(或 owner 明确接受上述断言为准)后由 owner 关闭 #6。两条路径取舍保留如下以备追溯。
|
||||||
|
>
|
||||||
|
> 配套:[`swarm-definition-gap.md`](./swarm-definition-gap.md)(逐条差距与文件/行号引用)、[README.md](../README.md)(能力边界表)、[`integration/runtime-contract.md`](./integration/runtime-contract.md)、[`benchmark/IMPORTANT-metric-coverage-gaps.md`](./benchmark/IMPORTANT-metric-coverage-gaps.md)。
|
||||||
|
|
||||||
|
## 1. 本系统当前「是什么」
|
||||||
|
|
||||||
|
- 一个**可运行的多 Agent 工作流运行时**(multi-agent workflow runtime):分解 → 派发 → 执行 → peer 协作/移交 → 评审/重做 → 汇总,仓内可端到端跑通(见 README 工作流)。
|
||||||
|
- 一个 **Master 中心化**编排系统:认知决策(分解 / 评审 / 汇总)集中于单一主控实体 `MasterAgent`(`orchestrator/master_agent.py`),由编排器 `task_dispatch_loop`(`orchestrator/main.py:290`)执行其决策。
|
||||||
|
- 任务规划在无 Manager 分工时由**单一 planner 回退**生成(`orchestrator/planner.py:62-85` 静态计划、`:87-116` LLM 计划),受 `ENABLE_PLANNER_FALLBACK` 门控。
|
||||||
|
|
||||||
|
## 2. 本系统当前「不是什么」
|
||||||
|
|
||||||
|
- **不是** Heicode 主链路的正式 Runtime Backend:尚未按 `heicode-am-contract` 注册接入(README 能力边界表标 🟡「待接入」;`docs/integration/runtime-contract.md`)。
|
||||||
|
- **不是 heicodeDocs 定义的弱中心蜂群**:当前不存在「Agent 感知共享态势 → 自主生成/竞争任务 → peer 交叉评审 → 自组织收敛」的去中心机制。派发是中心驱动的能力子集匹配(`orchestrator/task_queue.py:181-186` `can_agent_run_task`、`:140-168` 候选枚举),评审是中心化单点裁决(`planner.py:118-171` `review`)。
|
||||||
|
- **不是**已自证「优于基线」的蜂群:`gain / S_swarm / Benchmark_Agent` 仍 `NaN`(`benchmark/IMPORTANT-metric-coverage-gaps.md` §2)。
|
||||||
|
- 详见 [`swarm-definition-gap.md`](./swarm-definition-gap.md)。
|
||||||
|
|
||||||
|
## 3. 两条 ARB 路径(取舍并列,不预设结论)
|
||||||
|
|
||||||
|
| 维度 | Path A:对齐文档到现状 | Path B:构建弱中心能力 |
|
||||||
|
|---|---|---|
|
||||||
|
| 做什么 | 把对外定位/文档收敛为「Master 中心化多 Agent 工作流运行时」,不再宣称弱中心蜂群 | 落地弱中心机制,使实现向 heicodeDocs 定义收敛(对应工单 #7/#8/#9/#11/#12) |
|
||||||
|
| 改动面 | 仅文档与定位口径,**零代码** | 编排器派发/评审/态势/收敛的实质性架构改造 |
|
||||||
|
| 成本 | 低(数小时级文档工作) | 高(跨多工单的架构与验证工作,含基线对比) |
|
||||||
|
| 风险 | 放弃「弃中心蜂群」对外叙事;需确认与 heicodeDocs 标准不冲突(冲突须按 CLAUDE.md 输出冲突点,不得自行选边) | 周期长、引入分布式一致性/收敛终止/防活锁等新风险;需 Benchmark 自证 |
|
||||||
|
| 诚实性影响 | 立刻消除「宣称弱中心、实现中心化」的偏差(组织规则 #9) | 偏差在 Path B 完成并验收前持续存在,期间文档须保持 🟡/🔴 标注 |
|
||||||
|
| 可逆性 | 高(后续仍可转 Path B) | 中(架构改造沉没成本较高) |
|
||||||
|
|
||||||
|
> 说明:两条路径并非互斥的二选一终局——可先取 Path A 止血定位偏差,再按工单排期推进 Path B。但**采用哪条、是否并行、排期与验收门**均属产品/架构决策,不在本仓 docs 自行决定。
|
||||||
|
|
||||||
|
## 4. 待裁定项(ARB 决策,非本仓自决)
|
||||||
|
|
||||||
|
- [x] **已选定 Path B(owner 指示)并完成重构** —— 见 `swarm/decentralized-rework-plan.md`(swarm 已是唯一行为,无开关)。
|
||||||
|
- [ ] **关闭 #6 前置**:补一个**形式 ARB 决策记录链接**(或 owner 明确接受「owner 指示 + 本 PR + 本文」为决策记录)。在此之前重构 PR 用 `Refs #6` 不自动关闭。
|
||||||
|
- [ ] 若选 Path A:确认收敛后的对外定位口径与 heicodeDocs 标准无冲突(如有冲突,按 CLAUDE.md 输出冲突点交标准源裁定)。
|
||||||
|
- [ ] 若选 Path B:确认 #7/#8/#9/#11/#12 的范围、依赖顺序与 Benchmark 验收门(基线对比,见 `benchmark/IMPORTANT-metric-coverage-gaps.md` §5)。
|
||||||
|
|
||||||
|
> 本文不代行 ARB 裁定;上述路径选择**仍为 OPEN**,由人类 owner 决定。
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# 蜂群定义差距分析(Swarm Definition Gap)
|
||||||
|
|
||||||
|
> 状态:**现状差距分析**。逐条比对 heicodeDocs「弱中心蜂群」定义与本仓**当前 Master 中心化实现**,引用具体文件/行号,并标注每条差距由哪个工单(#7/#8/#9/#11/#12)关闭。回应工单 #6。
|
||||||
|
>
|
||||||
|
> 配套:[`product-positioning.md`](./product-positioning.md)、[README.md](../README.md) 能力边界表、[`benchmark/decision-engine.md`](./benchmark/decision-engine.md)、[`benchmark/IMPORTANT-metric-coverage-gaps.md`](./benchmark/IMPORTANT-metric-coverage-gaps.md)。
|
||||||
|
>
|
||||||
|
> 标准源声明:heicodeDocs 为唯一标准源。本文引用的「弱中心蜂群」四要素(共享态势感知 / 自主生成·竞争任务 / peer 交叉评审 / 自组织收敛)若与 heicodeDocs 表述存在歧义,以 heicodeDocs 为准;如发现文档与代码冲突,按 CLAUDE.md 要求输出冲突点,不自行选边。
|
||||||
|
|
||||||
|
## 1. 一句话差距
|
||||||
|
|
||||||
|
heicodeDocs 期望 Agent **自主感知共享态势、竞争/认领任务、互相交叉评审、自组织收敛**;本仓所有这些决策点目前都**集中在单一主控 + 中心编排循环**:Agent 是被动接收任务的执行单元,不感知全局态势,不竞争任务,不互评,收敛由中心评审单点裁决。
|
||||||
|
|
||||||
|
## 2. 差距对照表
|
||||||
|
|
||||||
|
| # | 弱中心蜂群定义(heicodeDocs) | 本仓当前实现(中心化) | 证据(文件:行) | 差距 | 关闭工单 |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| G1 | Agent **感知共享态势/全局状态**,据此自主行动 | Agent 仅被动收 `task_assignment`;全局态势由编排器在派发时单向注入上下文,Agent 无共享态势视图 | `orchestrator/main.py:361-366`(单向下发)、`main.py:186-222` `build_dispatch_context`(中心注入依赖产物/peer 信息)、`master_agent.py:1-17`(认知集中于 master) | Agent 无共享态势感知,无法据全局自主决策 | #11 |
|
||||||
|
| G2 | Agent **自主生成/竞争任务**(任务由群体涌现,非中心下发) | 任务由中心生成:Manager 分工优先,否则单一 planner 回退(静态/LLM 计划),Agent 不生成任务 | `orchestrator/planner.py:62-85`(静态计划)、`planner.py:87-116` `build_plan`、`master_agent.py:35-39` `plan` | 任务来源中心化,无群体涌现/竞争 | #7 |
|
||||||
|
| G3 | Agent **竞争认领**就绪任务(多 Agent 对同一任务竞标/抢占) | 中心派发循环按到达顺序遍历空闲 Agent,对每个 Agent 取能力匹配的第一个就绪任务;非 Agent 主动竞争 | `orchestrator/main.py:290-332` `task_dispatch_loop`、`task_queue.py:114-138` `get_ready_pending_task`、`task_queue.py:181-186` `can_agent_run_task`(能力子集匹配) | 单边、中心驱动的能力子集匹配,无双边竞争 | #8、#9 |
|
||||||
|
| G4 | **peer 交叉评审**(Agent 之间互评结果,质量由群体判定) | 评审为中心化单点裁决:master 调 planner.review 给出 accepted/retry_tasks;peer 间仅消息路由协作,不互评质量 | `orchestrator/planner.py:118-171` `review`、`master_agent.py:41-53` `review_and_decide`、`docs/peer-communication-logistics.md`(peer 仅协作非互评) | 质量裁决单点中心化,无 peer 交叉评审 | #12 |
|
||||||
|
| G5 | **自组织收敛**(群体在无中心调度下趋于终态) | 收敛由中心驱动:评审不达标→中心 `reopen_task` 重开,受 `MAX_REVIEW_CYCLES` 中心上限约束;终态由编排器刷新判定 | `orchestrator/task_queue.py:398-423` `reopen_task`、`orchestrator/main.py:376+` `refresh_swarm_run_status`、`planner.py:118-171`(中心裁决驱动循环) | 收敛逻辑中心化,非群体自组织 | #11、#12 |
|
||||||
|
|
||||||
|
> 注:`ENABLE_ACO_DISPATCH`(`main.py:309-328`)开启时引入 τ/η/P 概率派发,但仍是**中心循环按到达顺序对单个 Agent 采样候选任务**(Option A 单边),不构成 Agent 间竞争或去中心;学习常开但选择门控(见 `benchmark/decision-engine.md`)。它**不关闭** G3 的双边竞争差距。
|
||||||
|
|
||||||
|
## 3. 结论
|
||||||
|
|
||||||
|
- 上述 G1–G5 共同构成工单 #6 所指「实现仍为 Master 中心化、与弱中心蜂群定义不一致」的具体证据。
|
||||||
|
- 每条差距的关闭归属对应工单(#7/#8/#9/#11/#12);是否推进、以及按 Path A / Path B 哪条路径处理,属 ARB 决策,见 [`product-positioning.md`](./product-positioning.md) §4。
|
||||||
|
- 在差距关闭并经对应验收前,对外文档**不得宣称**本仓已是弱中心蜂群(组织规则 #9,不伪造已实现能力)。
|
||||||
@@ -0,0 +1,132 @@
|
|||||||
|
# Agent 自主任务生成:提案协议(Issue #7)
|
||||||
|
|
||||||
|
> 状态:**已接入实时 WS 主循环(去中心化重构 P3)**。`main.py: handle_task_proposal` + WS `task_proposal` 分支:Agent 提案 → 审查(置信度/去重合并/每 run 预算 `AGENT_PROPOSAL_BUDGET`)→ ACCEPT 即入池为真实 PENDING 任务(带 lineage,`source=agent_proposed`,发标准 `task.created`);提案生命周期存 `run.metadata["proposals"]`(内部遥测)。feature flag `ENABLE_AGENT_TASK_PROPOSALS`(构建期;cutover 转无条件)。集成测试 `scripts/test-swarm-autonomous.py`。本文是该机制的唯一入口。
|
||||||
|
> 实现:`orchestrator/autonomous_tasks.py`(纯模块,无 Redis / WS / FastAPI 依赖)。
|
||||||
|
> 测试:`scripts/test-autonomous-tasks.py`。
|
||||||
|
> 配套:任务模型见 `orchestrator/task_queue.py`;事件流 schema 见 [`../integration/event-schema.md`](../integration/event-schema.md);派发回退对照 [`../benchmark/decision-engine.md`](../benchmark/decision-engine.md)。
|
||||||
|
|
||||||
|
## 1. 问题与边界
|
||||||
|
|
||||||
|
Issue #7:执行单元(Agent)在工作中会**发现**当前共享状态缺失的工作(例如实现已落地但无测试、缺文档、需要补一个后续步骤),但运行时此前没有任何机制让 Agent **提出**或**补全**任务。本机制补上「自底向上」的任务来源。
|
||||||
|
|
||||||
|
**关键边界(务必先读)**:
|
||||||
|
|
||||||
|
- **是 Agent 在提案,不是 Master 在生成任务。** Master/planner 的自顶向下分解路径**已在去中心化重构中删除**;本机制(自底向上提案)是蜂群**唯一**的分解路径。提案恒带 `source="agent_proposed"` 与 `proposed_by_agent_id`;`assert_not_master_origin()` 在运行时拒绝任何 proposer/trigger 名含 `master`/`planner` 的提案,防止 Master 借此路径「洗白」任务来源。
|
||||||
|
- **无条件接入(无开关)。** 该机制是蜂群运行时的固有行为,**无 `ENABLE_*` 开关**(本仓即 swarm 运行时,模式选择在仓外);`main.py` 的 WS `task_proposal` 分支与 `handle_task_proposal` 始终生效。
|
||||||
|
- **Agent 只提案,编排器才决策。** Agent 给出自评置信度 `proposal_confidence`,但「收 / 拒 / 并」由编排器侧的策略函数 `review_proposal` 依据置信度阈值、提案预算、与现存任务去重来裁决——不调用任何模型。
|
||||||
|
|
||||||
|
## 2. 数据模型(`orchestrator/autonomous_tasks.py`)
|
||||||
|
|
||||||
|
### 2.1 `TaskProposal`
|
||||||
|
|
||||||
|
| 字段 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `proposal_id` | `prop-<hex>`,自动生成 |
|
||||||
|
| `proposed_by_agent_id` | **必填**,提案的执行单元;为空抛错 |
|
||||||
|
| `source` | 恒为 `"agent_proposed"`(`__post_init__` 硬锁,外部覆盖无效) |
|
||||||
|
| `title` / `description` / `agent_role` / `required_capabilities` / `depends_on` | 拟建任务的规格 |
|
||||||
|
| `proposal_reason` | Agent 为何提出(人/审计可读) |
|
||||||
|
| `proposal_confidence` | Agent 自评 0..1,构造时夹紧到 `[0,1]` |
|
||||||
|
| `lineage` | `ProposalLineage`:`origin_task_id` / `trigger_event` / `shared_state_snapshot` |
|
||||||
|
| `status` | `ProposalStatus` 枚举 |
|
||||||
|
| `merged_into_task_id` | 合并时记录并入的现存任务 id |
|
||||||
|
| `decision_reason` | `review_proposal` 写入的裁决理由 |
|
||||||
|
|
||||||
|
### 2.2 `ProposalStatus`(状态机)
|
||||||
|
|
||||||
|
```
|
||||||
|
proposed ──accept──▶ accepted ──ingest──▶ (新建 PENDING 任务)
|
||||||
|
│
|
||||||
|
├──reject──▶ rejected
|
||||||
|
├──merge───▶ merged(并入现存任务,不新建)
|
||||||
|
└──(超时/未处理)──▶ expired # 枚举已定义,过期判定由集成方按预算/时限实现
|
||||||
|
```
|
||||||
|
|
||||||
|
`ProposalLineage` 三要素让被提案任务端到端可审计:来自哪个原始任务(`origin_task_id`)、被什么事件触发(`trigger_event`)、Agent 基于哪份共享状态快照推理(`shared_state_snapshot`,需脱敏)。
|
||||||
|
|
||||||
|
## 3. 策略与摄取
|
||||||
|
|
||||||
|
### 3.1 `review_proposal(proposal, policy, existing_tasks) -> ReviewOutcome`
|
||||||
|
|
||||||
|
纯函数,按序裁决,返回 `ProposalDecision`(`accept`/`reject`/`merge`)并就地更新 `proposal.status` 与 `decision_reason`:
|
||||||
|
|
||||||
|
1. **置信度下限**:`proposal_confidence < policy.min_confidence` → **REJECT**。
|
||||||
|
2. **去重**:与某个**存活**(pending/assigned/in_progress/blocked)现存任务描述相似度 ≥ `dedup_similarity_threshold` → **MERGE** 进该任务(不新建)。相似度用 `description_similarity`(词集 Jaccard,确定性、无模型调用、可复现)。**已完成任务不作为去重目标**——允许 Agent 提后续轮次。
|
||||||
|
3. **预算**:`policy.remaining_proposal_budget <= 0` → **REJECT**。
|
||||||
|
4. 否则 → **ACCEPT**。
|
||||||
|
|
||||||
|
`ProposalPolicy` 字段(`min_confidence` / `auto_accept_confidence` / `remaining_proposal_budget` / `dedup_similarity_threshold`)全部显式,集成方应从 run 的 `orchestration_plan.budget` 取值,不从模型读取。
|
||||||
|
|
||||||
|
### 3.2 `ingest_accepted_proposal(proposal, *, swarm_id, root_task_id) -> dict`
|
||||||
|
|
||||||
|
仅接受 **ACCEPTED** 提案(否则抛错),再次校验 `assert_not_master_origin`,产出与 `task_queue.create_task` 对齐的 kwargs 形 spec:`description` / `agent_role` / `required_capabilities` / `depends_on` / `parent_task_id` / `root_task_id` / `source="agent_proposed"` / `context`。完整 lineage 写入 `context["proposal"]`(提案人、理由、置信度、origin/trigger、快照),新任务由 `create_task` 落为 `PENDING`。默认以 `origin_task_id` 为 `parent_task_id`,将新任务挂入原 run。
|
||||||
|
|
||||||
|
> 本函数只**产出 spec**,由集成方调用 `task_queue.create_task(**spec)` 入队——模块本身不碰 Redis/队列,以保持纯净与可测。
|
||||||
|
|
||||||
|
### 3.3 生命周期事件
|
||||||
|
|
||||||
|
`build_proposal_event(event_type, proposal, **extra)` 与 `build_lifecycle_events_for_outcome(proposal, outcome)` 产出扁平 payload:恒先 `task.proposal_submitted`(提案这一动作本身可审计),再按裁决追加 `task.proposal_accepted` / `task.proposal_rejected` / `task.proposal_merged`(合并事件带 `merge_target_task_id`)。
|
||||||
|
|
||||||
|
事件类型常量:`EVENT_SUBMITTED` / `EVENT_ACCEPTED` / `EVENT_REJECTED` / `EVENT_MERGED`。
|
||||||
|
|
||||||
|
## 4. 接入现状(已无条件接入)
|
||||||
|
|
||||||
|
**已落地于 `orchestrator/main.py`**(去中心化重构,无开关):WS `task_proposal` 分支 → `handle_task_proposal`:构造 `TaskProposal` → `review_proposal`(置信度/去重合并/每 run 预算 `AGENT_PROPOSAL_BUDGET`)→ ACCEPT 即 `task_queue.create_task` 入队为真实 PENDING 任务(带 lineage、`source=agent_proposed`、发标准 `task.created`);提案生命周期存 `run.metadata["proposals"]`(**内部遥测,非 Manager 事件**)。这是蜂群唯一的分解路径。集成测试 `scripts/test-swarm-autonomous.py`。
|
||||||
|
|
||||||
|
模块本身(`autonomous_tasks.py`)仍是纯策略/数据层(无 Redis/WS/模型):`TaskProposal`/lineage/状态枚举、`review_proposal` 三路裁决、`ingest_accepted_proposal` 映射、生命周期事件、`assert_not_master_origin` 守卫。
|
||||||
|
|
||||||
|
**仍为后续**:`expired` 过期判定仅定义枚举未实现;`Task`/`SwarmRun` 未加持久化提案字段(提案存于 `run.metadata`);真实 Agent 执行器基于 LLM 决定「提哪些子任务」为后续(`scripts/stub_agent.py` 已演示种子→提案)。
|
||||||
|
|
||||||
|
## 5. Agent 端发送(`agent/main.py`)
|
||||||
|
|
||||||
|
新增一个方法(与 `request_handoff` 同风格),由执行器在发现缺口时调用:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def propose_task(self, *, description, reason, confidence, origin_task_id,
|
||||||
|
trigger_event, shared_state_snapshot, agent_role="general",
|
||||||
|
required_capabilities=None, depends_on=None, title=None):
|
||||||
|
await self.safe_send({
|
||||||
|
"type": "task_proposal", "agent_id": self.agent_id,
|
||||||
|
"title": title, "description": description, "proposal_reason": reason,
|
||||||
|
"proposal_confidence": confidence, "agent_role": agent_role,
|
||||||
|
"required_capabilities": required_capabilities or [], "depends_on": depends_on or [],
|
||||||
|
"origin_task_id": origin_task_id, "trigger_event": trigger_event,
|
||||||
|
"shared_state_snapshot": shared_state_snapshot, "timestamp": time.time(),
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
并在 `handle_message` 的控制消息集合中加入 `task_proposal_ack`。
|
||||||
|
|
||||||
|
### 5.3 新增 Task 字段(`orchestrator/task_queue.py`,可选增强)
|
||||||
|
|
||||||
|
去重与审计已可由 `source="agent_proposed"` + `context["proposal"]` 承载,**无需**新增 Task 字段即可工作。如需一等公民查询,可选增(默认不破坏现有契约):
|
||||||
|
|
||||||
|
- `Task.proposal_id: Optional[str] = None`
|
||||||
|
- `Task.proposed_by_agent_id: Optional[str] = None`
|
||||||
|
|
||||||
|
### 5.4 新增 SwarmRun 字段(`orchestrator/swarm_runtime.py`)
|
||||||
|
|
||||||
|
提案预算与提案台账建议落到 run 上(与 `collaboration` / `decisions` 同为内部遥测,**非** Manager 事件):
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 已提案任务台账(内部遥测,不入 HM 事件注册表)。
|
||||||
|
proposals: List[Dict[str, Any]] = Field(default_factory=list)
|
||||||
|
```
|
||||||
|
|
||||||
|
预算可暂存于既有 `metadata["proposal_budget"]`(如 §5.1 所示),无需新增字段;若要显式化可加 `proposal_budget: int = 3`。配套可加 `record_proposal(run, proposal_dict)` 方法(仿 `record_decision`)。
|
||||||
|
|
||||||
|
### 5.5 影响范围声明(PR 模板用)
|
||||||
|
|
||||||
|
- Client:否(除非同时落 §5.2 Agent 端)。
|
||||||
|
- Manager:**接口不变**;提案事件为内部遥测,未改 HM 事件注册表 / 回调 / 审批链 / 计费 / 审计字段语义。
|
||||||
|
- agent_swarm:是(新增 WS `task_proposal` 分支、派发入队、SwarmRun 遥测字段)。
|
||||||
|
- 密钥 / 计费 / 审计 / 发布链路:不涉及。
|
||||||
|
|
||||||
|
## 6. 测试
|
||||||
|
|
||||||
|
```
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
..\.venv\Scripts\python.exe scripts/test-autonomous-tasks.py
|
||||||
|
```
|
||||||
|
|
||||||
|
模块级、无需 Redis / WS / 模型密钥:模拟 Agent 基于共享状态快照提案 → 策略 accept/reject/merge → ACCEPTED 提案产出带完整 lineage 的新 PENDING 任务 spec + 状态机转移 + 生命周期事件。当前 47 项断言全部通过。
|
||||||
@@ -0,0 +1,129 @@
|
|||||||
|
# 蜂群收敛协议(共识 / 冲突消解 / 终止函数)
|
||||||
|
|
||||||
|
> 状态:**已无条件接入 `refresh_swarm_run_status`(去中心化重构,无开关)**。每次 run 终态都产 `ConvergenceReport` 存 `run.metadata["convergence"]`,并把 `termination_reason` 附到 `timeline.updated`。**当前为解释性**——产出 `termination_reason`/共识/冲突,但**不覆盖 `run.status`**(今天 next_status 与报告对 completed/failed 一致);authoritative 状态覆盖为后续。
|
||||||
|
>
|
||||||
|
> 对应 Issue #12:「蜂群收敛机制缺失:缺少共识、冲突消解与终止函数,无法解释 Swarm 为什么结束」。
|
||||||
|
>
|
||||||
|
> 实现:`orchestrator/convergence.py`(纯计算,无 I/O)+ `main.py: compute_convergence_report`(装配 run_state,shadow 落库);单测 `scripts/test-convergence.py` + 集成测试 `scripts/test-swarm-convergence.py`。计划见 [decentralized-rework-plan.md](./decentralized-rework-plan.md)。
|
||||||
|
|
||||||
|
## 0. 问题:今天「为什么结束」是不可解释的
|
||||||
|
|
||||||
|
当前一次 run 走到终态,**唯一**判定在 `orchestrator/main.py:refresh_swarm_run_status`:
|
||||||
|
|
||||||
|
- 所有已知任务到达终态后,`next_status = "failed"`(任一任务 `FAILED`)否则 `"completed"`;
|
||||||
|
- 可选地(仅 `ENABLE_REVIEW_LOOP`)先跑主控评审,被拒则重开任务再 finalize。
|
||||||
|
|
||||||
|
也就是说,**今天的终止 = 「所有任务做完 + (可选)评审通过」**,没有:
|
||||||
|
|
||||||
|
- **共识模型**:没有任何「多少产出处于一致状态」的度量;
|
||||||
|
- **一等公民的冲突检测 / 消解**:两个 Agent 对同一文件写出不同内容、依赖未满足却标记完成等,均无显式识别;
|
||||||
|
- **可机读的终止原因**:`completed` / `failed` 无法区分「质量达标」「预算耗尽」「轮次上限」「被风险阻断」。
|
||||||
|
|
||||||
|
本协议补齐上述三者。**已无条件接入**(无开关),但当前**作为解释层**:产出报告 + `termination_reason` 存 run,**不覆盖** run 的终态语义(规则 #9:authoritative 覆盖为后续)。
|
||||||
|
|
||||||
|
## 1. 设计原则
|
||||||
|
|
||||||
|
- **纯函数、可测**:`evaluate_convergence(run_state)` 不做任何 I/O,不修改 run;调用方(`main.py: compute_convergence_report`)负责持久化。
|
||||||
|
- **无条件(无开关)**:`refresh_swarm_run_status` 每次终态都计算报告——本仓即 swarm 运行时,无 `ENABLE_*` 门控。
|
||||||
|
- **不伪造信号**(规则 #9):无质量评分 ≠ 通过;无测试信号 ≠ 失败;无预算 ≠ 充足。缺信号一律退出判定,不补 0、不补 100。
|
||||||
|
- **解释优先于干预**:本版**只解释**「为什么结束」,不自动改写终态、不自动 merge 冲突产物。
|
||||||
|
|
||||||
|
## 2. 数据结构(`orchestrator/convergence.py`)
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass
|
||||||
|
class ConvergenceReport:
|
||||||
|
status: ConvergenceStatus # running / converged / failed / blocked
|
||||||
|
termination_reason: TerminationReason | None # 终态必有,running 时为 None
|
||||||
|
consensus_score: float # [0,100],无冲突牵连的已完成产出占比
|
||||||
|
conflicts: list[Conflict] # 全部检出冲突
|
||||||
|
resolved_conflicts: list[Conflict] # 已消解
|
||||||
|
unresolved_risks: list[dict] # 阻断性风险 + 未消解硬冲突
|
||||||
|
budget_state: dict # 预算限额 / 消耗 / exhausted
|
||||||
|
quality_state: dict # fixture 评分 / 阈值 / reached
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.1 终止原因枚举(`TerminationReason`)
|
||||||
|
|
||||||
|
| 值 | 含义 | 触发输入 |
|
||||||
|
|---|---|---|
|
||||||
|
| `quality_reached` | 质量 / 验收门达标 | `quality.test_pass_rate ≥ acceptance_threshold`(默认 1.0) |
|
||||||
|
| `budget_exhausted` | token / 成本 / 时长预算耗尽 | `usage ≥ budget` 任一限额 |
|
||||||
|
| `max_rounds_reached` | 评审 / 重做轮次达上限 | `review_cycles ≥ max_review_cycles` |
|
||||||
|
| `risk_blocked` | 被未消解阻断性风险终止 | 输入阻断风险或未消解硬冲突 |
|
||||||
|
| `tasks_completed` | **诚实回退**:仅「所有任务做完」 | 以上均不适用——**即今天唯一真实信号** |
|
||||||
|
|
||||||
|
> `tasks_completed` 明确对应「现状」:当没有质量评分、预算、轮次或风险信号时,我们能说的**只有**「任务都做完了」,不冒充更强的解释。
|
||||||
|
|
||||||
|
### 2.2 冲突类型(`ConflictType`)
|
||||||
|
|
||||||
|
| 类型 | 检测器 | 判据 |
|
||||||
|
|---|---|---|
|
||||||
|
| `artifact_mismatch` | `detect_artifact_mismatch` | 两个已完成任务对同一 `files_modified` 路径记录了不同 `file_hashes`/`file_contents` 指纹 |
|
||||||
|
| `test_failure` | `detect_test_failures` | 任务结果 `tests_passed is False` 或 `test_pass_rate < 1` |
|
||||||
|
| `review_disagreement` | `detect_review_disagreement` | 主控评审 `accepted=False`(沿用 `master_agent.review_and_decide` 的 verdict 形状) |
|
||||||
|
| `dependency_inconsistency` | `detect_dependency_inconsistency` | 任务已完成,但其 `depends_on` 依赖在同 run 内未完成 |
|
||||||
|
|
||||||
|
## 3. 终止函数判定顺序
|
||||||
|
|
||||||
|
`evaluate_convergence(run_state)` 依次判断,**先匹配先返回**:
|
||||||
|
|
||||||
|
1. 无任务 / 任一任务处于 `pending|assigned|in_progress` → `RUNNING`(无 reason)。
|
||||||
|
2. 存在阻断风险或未消解硬冲突 → `BLOCKED` + `risk_blocked`。
|
||||||
|
3. 仅有 `blocked` 任务(无阻断风险) → `RUNNING`(与 `refresh_swarm_run_status` 中「blocked 但有活跃子任务保持 running」对齐)。
|
||||||
|
4. 任一任务 `failed` → `FAILED`,原因取(预算耗尽→`budget_exhausted`;否则轮次到顶→`max_rounds_reached`;否则 `tasks_completed`)。
|
||||||
|
5. 全部完成且预算耗尽 → `CONVERGED` + `budget_exhausted`。
|
||||||
|
6. 全部完成且轮次到顶 → `CONVERGED` + `max_rounds_reached`。
|
||||||
|
7. 全部完成且质量达标 → `CONVERGED` + `quality_reached`。
|
||||||
|
8. 其余(全部完成) → `CONVERGED` + `tasks_completed`(诚实回退)。
|
||||||
|
|
||||||
|
**不变式**:任何终态(converged/failed/blocked)必带且仅带一个 `termination_reason`;`running` 必为 `None`。单测逐条断言。
|
||||||
|
|
||||||
|
### 3.1 共识分(`compute_consensus_score`)
|
||||||
|
|
||||||
|
```
|
||||||
|
consensus = 100 × (未被任何冲突牵连的已完成任务数 / 已完成任务总数)
|
||||||
|
```
|
||||||
|
|
||||||
|
无已完成任务 → `0.0`。这是一个**可解释的真实代理量**(仅来自任务状态 + 检出冲突),不是模型主观打分。
|
||||||
|
|
||||||
|
### 3.2 消解(`resolve_conflicts`)——保守首版
|
||||||
|
|
||||||
|
- `artifact_mismatch`:仅当评审 `accepted=True`(评审隐式选定了胜出版本)才标记 resolved;否则 unresolved。
|
||||||
|
- `review_disagreement`:verdict 给出可操作 `retry_tasks` 时标记 resolved(蜂群可通过重开任务行动);空泛拒绝无可操作项 → unresolved。
|
||||||
|
- `test_failure` / `dependency_inconsistency`:视为硬阻断,本版**不自动消解**,留给重做循环或人工。
|
||||||
|
|
||||||
|
每个冲突就地写回 `resolved` 与 `resolution`(消解方式或无法消解的原因)。
|
||||||
|
|
||||||
|
## 4. 事件载荷构造器
|
||||||
|
|
||||||
|
按 `SwarmRuntime.emit_event(run, event_type, payload=...)` 约定,每个构造器返回 `(event_type, payload)`,payload 为 JSON 安全 dict:
|
||||||
|
|
||||||
|
| 函数 | event_type | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| `event_convergence_started` | `convergence.started` | 收敛评估开始 |
|
||||||
|
| `event_conflict_detected` | `conflict.detected` | 每检出一个冲突 |
|
||||||
|
| `event_conflict_resolved` | `conflict.resolved` | 每消解一个冲突 |
|
||||||
|
| `event_consensus_updated` | `consensus.updated` | 当前共识分与冲突计数 |
|
||||||
|
| `event_convergence_reached` | `convergence.reached` | 终态成功,附 `termination_reason` |
|
||||||
|
| `event_convergence_failed` | `convergence.failed` | 终态失败/阻断,附 `termination_reason` |
|
||||||
|
|
||||||
|
> ⚠️ 上述六个 `event_type` **尚未登记进 Manager 事件契约**(heicode `agent_callback.go`)。在订阅或回调消费前,必须先按下方集成说明走 Manager 侧契约登记,否则属于未登记事件。
|
||||||
|
|
||||||
|
## 5. 测试
|
||||||
|
|
||||||
|
```
|
||||||
|
..\.venv\Scripts\python.exe scripts\test-convergence.py
|
||||||
|
```
|
||||||
|
|
||||||
|
模块级、无 WS/Redis/模型。覆盖:两个冲突产物 → 检出 artifact_mismatch + review_disagreement;review_disagreement 可消解、artifact_mismatch 未消解;产出带具体 `termination_reason` 的 `ConvergenceReport`;**断言每个终态都带 termination_reason**;断言至少一个冲突「检出 + 消解」闭环;以及五个终止原因各自的终态用例。全部 PASS。
|
||||||
|
|
||||||
|
## 6. 接入现状与诚实差距(规则 #9)
|
||||||
|
|
||||||
|
**已接入**(`main.py: compute_convergence_report` + `refresh_swarm_run_status`,无开关):每次 run 终态装配 `run_state`(tasks/budget/usage/quality/review_cycles)→ `evaluate_convergence` → 报告存 `run.metadata["convergence"]`,`termination_reason` 附到 `timeline.updated` 载荷。
|
||||||
|
|
||||||
|
**诚实差距**:
|
||||||
|
- **解释性,非权威**:报告**不覆盖** `run.status`(今天 next_status 与报告对 completed/failed 一致)。authoritative 模式(如 `BLOCKED`→置 run `blocked`)改变 Manager 面终态语义,须先过 `scripts/test-runtime-contract.py` + 契约评审——列为后续。
|
||||||
|
- **质量/预算/风险输入有条件**:`quality` 依赖 Group B fixture 评分(绑定 fixture 时);`budget`/`usage` 需 run 提供;`risks` 需上游注入。缺失时相应原因不触发,回退 `tasks_completed`(不伪造)。
|
||||||
|
- **消解为保守首版**:不做自动 merge / 自动选胜,硬冲突留待重做或人工。
|
||||||
|
- **收敛事件不进 Manager 流**:六个 `convergence.*`/`conflict.*`/`consensus.*` 事件**构建器已实现但不经 `emit_event` 外发**(未在 Manager `agent_callback.go` 注册;与 `swarm.health` 同策略,避免向订阅全部的回调投递未登记事件)。登记后方可启用 Manager 侧发送。`termination_reason` 以**新增可选字段**附在 `timeline.updated`,对旧消费方向后兼容。
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# 去中心化蜂群重构计划(Decentralized Swarm Rework)
|
||||||
|
|
||||||
|
> 状态:**进行中(先建核心管线 → 切换为唯一路径 → 最后写守卫)**。
|
||||||
|
> 依据:去中心化自组织蜂群模型(信息素/stigmergy)+ heicodeDocs 弱中心定义 + 复审工单 #6/#7/#8/#9/#11/#12。
|
||||||
|
> 关联:[product-positioning.md](../product-positioning.md)(本计划即 ARB **Path B**)、[swarm-definition-gap.md](../swarm-definition-gap.md)、各模块协议文档。
|
||||||
|
|
||||||
|
## 0. 范围与边界(重要)
|
||||||
|
|
||||||
|
- **本仓 = 蜂群运行时本身**。模式选择(single/chain/sub/**swarm**)由项目**其他部分**负责,不在本仓。
|
||||||
|
- **因此本仓不设「启用 swarm」开关**——swarm 不是本仓的一个可选模式,而是本仓的**全部行为**。
|
||||||
|
- **删除一切非 swarm 路径**:Master 上游分解(planner fallback)、贪心/单 Agent 兜底、派发「模式」开关等**作为本仓的替代路径一律移除**。
|
||||||
|
- **唯一保留的兜底是「守卫(guard)」**:检测「swarm 无法正常运作」并列出原因(如无可用 Agent、无模型、依赖死锁、预算耗尽)。守卫**在整套程序构建完成后最后编写**。
|
||||||
|
- **Manager↔Swarm 契约保留**:Manager 仍可经 `/api/swarms` 下发;若提供 `agents` 明细,作为种子集;否则由 objective 播种。契约本身不删。
|
||||||
|
|
||||||
|
## 1. 目标形态(要把系统改成什么)
|
||||||
|
|
||||||
|
从「**Master 中心化**:主控分解 → 派发 → 评审」改为「**去中心化自组织**:编排器只播种 + 托管共享状态,Agent 自组织」。秩序与质量是**涌现**的,不是命令出来的。
|
||||||
|
|
||||||
|
核心循环(每个 Agent 重复直到收敛):
|
||||||
|
|
||||||
|
```
|
||||||
|
感知(perceive) → 决策(decide) → 执行(act) → 信息素沉积(stigmergy) → 收敛检查(converge)
|
||||||
|
```
|
||||||
|
|
||||||
|
- **播种器(Seeder)**:把用户 prompt 变成 1~少数**种子任务**写入共享池;**不做完整主控分解、不做中心派发**。(取代 Master planner 的「指挥」角色)
|
||||||
|
- **Agent 自选/自生成/竞争**:从共享池**自选**任务(信息素 τ + 启发式 η 驱动),可**自主生成**后续子任务(#7),多 Agent 可**竞争/让渡**同一任务(#8)。
|
||||||
|
- **质量靠同伴交叉评审 / 留出测试**:**不是自评通过**(自评只用于 Agent 自己路由);验收由**同伴 cross-review**(#11)或 held-out 测试(Group B)裁定。
|
||||||
|
- **轻量仲裁者(Arbiter)**:**仅在**(a) Agent 间冲突、(b) 最终收敛/验收时介入;产出 `termination_reason`(#12)。**不得退化为逐任务的 Master。**
|
||||||
|
- **信息素(τ pheromone)为主协调媒介**:成功/质量强化路径、随时间衰减——**该引擎已存在**(`decision_engine.py`)。
|
||||||
|
|
||||||
|
三种协调通道(无中心调度器):**stigmergy(信息素,主)** / **broadcast(消息总线)** / **handoff(交接)**。任务带 **DAG 依赖**(`Task.depends_on`)。
|
||||||
|
|
||||||
|
## 2. 现状 → 目标 映射(已有积木 vs 缺口)
|
||||||
|
|
||||||
|
| 文章/标准概念 | 本仓现状 | 本次动作 |
|
||||||
|
|---|---|---|
|
||||||
|
| 信息素 / stigmergy | ✅ `decision_engine.py` τ trail(沉积+衰减,Redis,学习常开) | 升为**主协调**,非可选 |
|
||||||
|
| 共享环境 / 任务池 | ✅ `task_queue`(Redis)+ `SwarmRun` | 暴露为 Agent 可感知的共享状态(#7) |
|
||||||
|
| 去中心化自选 | 🟡 ACO/scored 仍由编排器循环驱动 | 转向 **Agent 拉取自选**(#9/#10 复用打分) |
|
||||||
|
| 自主任务生成 | 🟡 `autonomous_tasks.py`(已建未接) | **接线**(#7) |
|
||||||
|
| 竞争/竞价 | 🟡 `task_competition.py`(已建未接) | **接线**(#8) |
|
||||||
|
| 同伴交叉评审 | 🟡 `cross_review.py`(已建未接) | **接线**(#11),取代单评审 |
|
||||||
|
| 收敛 + 终止原因 | 🟡 `convergence.py`(已建未接) | **接线**进 `refresh_swarm_run_status`(#12) |
|
||||||
|
| Handoff | ✅ `handoff_manager` | 保留 |
|
||||||
|
| DAG 依赖 | ✅ `Task.depends_on` | 保留 |
|
||||||
|
| 「编排器只播种、不调度」 | ❌ 今天相反(planner + master_agent 指挥) | **核心改造**(#6 Path B:播种器化) |
|
||||||
|
|
||||||
|
## 3. 分阶段计划(先建核心管线 → 切换为唯一路径 → 守卫)
|
||||||
|
|
||||||
|
**无 `ENABLE_SWARM_MODE` 总开关**(见 §0)。构建期为避免半成品成为唯一路径而打断仓库,旧 Master 路径**临时保留**,新管线逐件建好并单测;待核心管线(播种 + 自主分解 + 自选 + 交叉评审 + 收敛)齐备后,在 **P-cutover** 一次性切换为唯一路径并删除旧路径、改写测试。构建期的临时内部开关仅为「建到一半不破仓」,**在 P-cutover 全部移除**,不作为产品特性。
|
||||||
|
|
||||||
|
- **P1 收敛/仲裁(最终门,#12)**:`convergence.py` 接入 `refresh_swarm_run_status`,产 `ConvergenceReport` + `termination_reason`。先 **shadow**(存 `run.metadata`,不改 `run.status`),cutover 时转 **authoritative**(收敛裁定决定终态)。✅ 已建 shadow。
|
||||||
|
- **P2 播种器(#6 Path B 关键)**:`build_seed_task_specs` 把 objective 播为单一种子任务;**不调用 master_agent.plan**。✅ 函数已建(cutover 时成为唯一任务创建路径)。
|
||||||
|
- **P3 自主任务生成(#7)**:✅ 编排器侧已接入(`main.py: handle_task_proposal` + WS `task_proposal` 分支 + `ENABLE_AGENT_TASK_PROPOSALS`):Agent 提案 → 审查/去重/预算 → 入池为真实任务(带 lineage)。集成测试 `test-swarm-autonomous.py`。**剩余(Agent 侧)**:执行单元基于种子/共享态用 LLM 决定提哪些子任务(在 cutover e2e 用会提案的 stub agent 验证)。
|
||||||
|
- **P4 竞争/让渡(#8)**:✅ 已接入(`main.py` WS `task_bid`/`task_yield`/`task_takeover_request` 分支 + `handle_task_bid`/`arbitrate_and_assign`/`handle_task_yield`/`handle_task_takeover` + `ENABLE_TASK_COMPETITION`):竞价→τ 加权确定性仲裁→`finalize_dispatch` 分派;让渡复用 `release_task`;接管须 decisive。集成测试 `test-swarm-competition.py`。
|
||||||
|
- **P5 同伴交叉评审(#11)**:✅ 已接入(`main.py` `review_decision` 分支 + `handle_review_decision` + `run_cross_review` + `ENABLE_CROSS_REVIEW`):收集 ≥2 同伴独立评审→`aggregate_reviews` 仲裁(分歧记录、安全偏向)→拒绝则 reopen 返工目标 + 归因;在 `refresh_swarm_run_status` 先于单 critic Master 评审。集成测试 `test-swarm-cross-review.py`。cutover 时**取代** Master 评审。
|
||||||
|
- **P6 去中心化自选**:✅ 已建(`main.py: swarm_dispatch` + `ENABLE_SWARM_DISPATCH`):每个空闲 Agent 感知共享池、按 capability+τ+load+budget **自选**最适任务(统一 #9 可解释打分 + #10 信息素 τ),记录可解释 `dispatch.decision_made`。集成测试 `test-swarm-dispatch.py`。cutover 时成为**唯一**派发,删除 greedy/ACO/scored 与各模式开关。
|
||||||
|
- **P-cutover 切换为唯一路径**:✅ 已完成。`swarm_dispatch` 为唯一派发(删 greedy/ACO/scored 分支 + `scored_matchmake`);`build_seed_task_specs` 为唯一任务创建(删 planner-fallback `build_planner_task_specs`/`planner_fallback_enabled`);删单 critic Master 评审环(`maybe_run_review_cycle`/`review_loop_enabled`),`run_cross_review` 为唯一评审;收敛/提案/竞争/评审原语全部**无条件**(移除全部 `ENABLE_*` 构建开关);`test-workflow-e2e` 改写为 seed→自选→自主分解→执行→收敛全流程(stub agent 改为感知种子后提案分解),`test-merge-smoke` 的 planner/单评审用例改写为 seeder/cross-review;CI 同步。`master_agent.synthesize` 作为汇总工具保留。
|
||||||
|
- **P-guard 守卫(最后)**:✅ 已完成。`orchestrator/guard.py: diagnose`(纯函数)检测 `NO_AGENTS_CONNECTED`/`NO_CAPABLE_AGENT`/`DEPENDENCY_DEADLOCK`/`BUDGET_EXHAUSTED`/`SEED_UNDECOMPOSED` 并给出可读原因;`main.py: assess_swarm_health` 在派发环检测到「有待办却本 tick 无任何分派」时诊断受影响 run,存 `run.metadata["health"]` 并在不健康时发内部 `swarm.health` 事件(仅诊断,不改 run)。测试 `test-swarm-guard.py`。**这是唯一保留的非正常路径处理。**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 重构完成(P1–P6 + cutover + guard 全部落地)
|
||||||
|
|
||||||
|
去中心化蜂群已是本仓**唯一**行为:播种 → 自选 → 自主分解 → 竞争/接管 → 同伴交叉评审 → 收敛,外加健康守卫。无 `ENABLE_*` 模式开关;旧 Master 中心化路径(planner 分解、贪心/ACO/scored 派发、单 critic 评审)已删除。剩余为后续打磨(如 convergence authoritative 状态覆盖、真实多 Agent 竞争 e2e、benchmark Group C 度量去中心化是否更优)。
|
||||||
|
|
||||||
|
## 4. 契约要点(接线时遵守)
|
||||||
|
|
||||||
|
- **冲突检测**:artifact 不一致 / 测试失败 / 评审分歧 / 依赖不一致(`convergence.py` 已定义)。
|
||||||
|
- **终止原因**:`quality_reached` / `budget_exhausted` / `max_rounds_reached` / `risk_blocked` / `tasks_completed`(终态必带其一)。
|
||||||
|
- **交叉评审**:≥2 独立评审者;分歧→仲裁;安全偏向(平票→拒绝);返工归因供 `P_rework`。
|
||||||
|
- **Manager 契约**:新事件类型(`task.proposal_*`/`task.bid_*`/`review.*`/`convergence.*`/`dispatch.decision_made`)**默认只进 Swarm 内部状态/事件流,不进 Manager 回调注册表**,除非在 `event-schema.md §4` 注册并与 HM 联调(跨端,单列)。
|
||||||
|
|
||||||
|
## 5. 诚实边界(本次重构**不**主张的)
|
||||||
|
|
||||||
|
- **不动 benchmark 验收**:`G_E`/`Benchmark_Agent` 仍需真实模型 run(Group C/#13)。
|
||||||
|
- **去中心化是否优于中心化未证**:是经验命题,需 Group C 对比 harness;但本仓**不因此保留中心化兜底**——本仓职责就是 swarm,模式取舍在仓外。
|
||||||
|
- **前端可见状态**为跨端(#18):本仓只产出事件/状态,UI 由 Frontend 团队消费。
|
||||||
|
- **cutover 后删除 Master/planner/贪心兜底**:Master 不再是「中心化备选」,其 `plan/review/synthesize` 中仍被 swarm 流程复用的部分(如汇总)保留为 swarm 内的工具函数,但**不再作为独立的中心化派发/分解路径**。
|
||||||
|
|
||||||
|
## 6. 参考实现:OpenAI Swarm(github.com/openai/swarm)映射
|
||||||
|
|
||||||
|
OpenAI Swarm 是轻量、教学型多 Agent 编排框架。其核心原语印证并细化本次设计:
|
||||||
|
|
||||||
|
| OpenAI Swarm 原语 | 含义 | 本仓映射 |
|
||||||
|
|---|---|---|
|
||||||
|
| **Handoff**:函数 `return another_agent` 转移控制 | 无中心调度器;Agent 运行时自行决定把控制权交给谁 | **去中心化协调的核心**:Agent 结果可声明「交给哪个角色/能力」(`handoff_manager` + #8 接管),取代中心派发 |
|
||||||
|
| **`Result(value, agent, context_variables)`** | 一次函数返回可同时携带:输出 + handoff 目标 + 共享态更新 | 任务结果可同时携带:产物 + handoff/接管目标(#8) + 共享态写入 + **提案的后续任务(#7)** |
|
||||||
|
| **`context_variables`**(贯穿的共享 dict) | 黑板/共享状态 | 本仓共享状态 = Redis run 状态 + 任务池 + **信息素图(τ)**;Agent 感知它来决策 |
|
||||||
|
| **`client.run()` 循环 + `max_turns`** | 无状态循环:补全→执行工具→按 handoff 切换→更新共享态→无函数调用则停 | run 生命周期 + **收敛裁定(#12)**;`max_turns` 正是用户要的 **守卫**雏形(防死循环/无进展) |
|
||||||
|
| **动态 instructions(context_variables)** | 提示随共享态变化 | 派发上下文 `build_dispatch_context` 已按角色/依赖产物动态拼装 |
|
||||||
|
|
||||||
|
**采纳**:handoff 作为去中心化协调原语;run 循环 + `max_turns` 作为收敛 + 守卫的骨架;`Result` 的「输出+handoff+共享态+提案」四合一作为任务结果契约。
|
||||||
|
|
||||||
|
**差异(本仓是 Swarm 的超集)**:OpenAI Swarm 为单进程、同一时刻一个活动 Agent、同步 handoff;本仓为**多 WS Agent 并发** + **stigmergy 信息素自选**(Swarm 没有)。即:**Swarm 的 handoff 原语 + 蜂群文章的并行/信息素**。模型经本仓网关,非直连 OpenAI。
|
||||||
|
|
||||||
|
## 7. 受影响文件(预期)
|
||||||
|
|
||||||
|
- 代码:`orchestrator/main.py`(派发环/`refresh_swarm_run_status`/WS 分支/播种/cutover 删旧路径)、`orchestrator/swarm_runtime.py`(状态字段)、接入 `autonomous_tasks/task_competition/cross_review/convergence/dispatch_score/decision_engine`、新增 `guard`(最后)。
|
||||||
|
- 文档:本计划、`product-positioning.md`(Path B)、`swarm-definition-gap.md`、各协议文档(状态对齐)、`README.md`、`CLAUDE.md`、`docs/TESTING.md`。
|
||||||
|
- 测试:各阶段 hermetic 测试 + 改写 `test-workflow-e2e`/`test-merge-smoke` 为 swarm 流程 + CI。
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
# Review Loop 协议:从 Supervisor Retry 到蜂群交叉验证(issue #11)
|
||||||
|
|
||||||
|
> 状态:**已接入在线 finalize 路径(去中心化重构 P5)**。`main.py` 新增 `review_decision` WS 分支与 `handle_review_decision`(收集 ≥2 同伴独立评审)+ `run_cross_review`(聚合仲裁→分歧记录→拒绝则 reopen 返工目标 + 归因),在 `refresh_swarm_run_status` 中先于单 critic Master 评审。feature flag `ENABLE_CROSS_REVIEW`(构建期;cutover 取代 Master 评审)。集成测试 `scripts/test-swarm-cross-review.py`,模块单测 `scripts/test-cross-review.py`。
|
||||||
|
>
|
||||||
|
> 对应 issue #11 —「Review Loop 等价于 Supervisor Retry:评审重做机制未形成蜂群协同验证闭环」。
|
||||||
|
>
|
||||||
|
> 实现:`orchestrator/cross_review.py`(纯模块,仅依赖标准库)。单测:`scripts/test-cross-review.py`。
|
||||||
|
> 配套:[`../benchmark/swarm-metrics-schema.md`](../benchmark/swarm-metrics-schema.md)(`P_rework` / `Reward`)、`orchestrator/main.py`(现有评审循环)、`orchestrator/master_agent.py`、`orchestrator/planner.py`。
|
||||||
|
|
||||||
|
## 1. 现状:今天的评审循环是「单评审 + 重试」
|
||||||
|
|
||||||
|
当 `ENABLE_REVIEW_LOOP=1` 时,`orchestrator/main.py` 在 run 完成前调用一次主控评审(`maybe_run_review_cycle`):
|
||||||
|
|
||||||
|
1. `master_agent.review_and_decide(objective, tasks, results)` 委托 `planner.review(...)`;
|
||||||
|
2. `planner.review` 用**单个** LLM(或无模型时的启发式一致性检查)返回 `{accepted, summary, retry_tasks}`;
|
||||||
|
3. 若 `accepted=False`,编排器把 `retry_tasks` 里的任务 `reopen_task` 重新入队,`review_cycles += 1`,run 退回 `running`,受 `MAX_REVIEW_CYCLES`(默认 2)约束;
|
||||||
|
4. 重做完成后再次 finalize;预算耗尽或无可重做项即接受。
|
||||||
|
|
||||||
|
**本质:这是一个 Supervisor Retry。** 由**单一权威**(主控)判 pass/fail 并指派重做,没有第二个独立意见、没有记录分歧、没有结构化的「为什么重做 / 谁引入的缺陷」。因此:
|
||||||
|
|
||||||
|
- 不构成蜂群式**交叉验证闭环**(cross-validation)——验证仍是中心化的一票否决;
|
||||||
|
- 无法为基准 `P_rework`(见 §4)提供**可归因**的输入:每次重做只是一个无差别的 retry 计数,无法区分是需求 / 实现 / 测试 / 文档 / 协作哪一环引入。
|
||||||
|
|
||||||
|
## 2. 目标:多评审交叉验证 + 重做归因
|
||||||
|
|
||||||
|
`orchestrator/cross_review.py` 提供一个**纯协议层**(无 Redis / 无 WebSocket / 无模型调用),把「单评审 pass/fail」升级为「≥2 独立评审 → 检测分歧 → 仲裁 → 结构化重做归因」:
|
||||||
|
|
||||||
|
| 能力 | 今天(Supervisor Retry) | 交叉验证(本模块) |
|
||||||
|
|---|---|---|
|
||||||
|
| 评审者数量 | 1(主控) | ≥ 2 独立评审者(`aggregate_reviews` 强制) |
|
||||||
|
| 分歧 | 不存在概念 | `disagreement` 显式检测并记录 |
|
||||||
|
| 仲裁 | 主控单方裁定 | `majority` / `weighted`,平票安全偏向拒绝 |
|
||||||
|
| 证据 | `summary` 一行 | `evidence` / `failed_criteria` / `affected_tasks` 结构化 |
|
||||||
|
| 重做归因 | 仅 `retry_tasks` | `rework_reason` / `root_cause` / `source_task_id` / `introduced_by_agent_id` |
|
||||||
|
| 基准输入 | 无差别 retry 计数 | 按 `ReworkCategory` 分类的 `P_rework` 输入 |
|
||||||
|
|
||||||
|
### 2.1 数据结构
|
||||||
|
|
||||||
|
- **`ReviewDecision`**(单评审者的结构化裁决)
|
||||||
|
- `verdict`:`"pass"` / `"fail"`(构造时校验,非法即 `ValueError`);
|
||||||
|
- `reviewer_agent_id`(必填)、`evidence[]`、`failed_criteria[]`、`affected_tasks[]`、`recommended_rework[]`;
|
||||||
|
- `confidence`(自评 0–1)、`weight`(仲裁权重,如角色信任 / 资历)、`summary`。
|
||||||
|
|
||||||
|
- **`AggregatedVerdict`**(交叉验证结果)
|
||||||
|
- `accepted`、`method`(实际使用的仲裁法)、`disagreement`、`pass_votes`/`fail_votes`、`rework_targets[]`、`decisions[]`、`summary`。
|
||||||
|
|
||||||
|
- **`ReworkAttribution`**(一次重做的归因)
|
||||||
|
- `target_task_id`、`rework_reason`、`root_cause`(`ReworkCategory`)、`source_task_id`、`introduced_by_agent_id`、`detected_by_agent_id`、`evidence[]`。
|
||||||
|
|
||||||
|
- **`ReworkCategory`**:`requirement` / `implementation` / `test` / `doc` / `collaboration` / `unknown`。
|
||||||
|
- `unknown` 是显式取值:**无信号不伪造原因**(组织诚信规则 #9)。
|
||||||
|
|
||||||
|
### 2.2 仲裁规则(`aggregate_reviews`)
|
||||||
|
|
||||||
|
1. **强制 ≥ 2 评审者**——单评审者就是 Supervisor Retry,不是交叉验证,少于 2 抛 `ValueError`;
|
||||||
|
2. **检测分歧**:pass 与 fail 同时出现 → `disagreement=True`,并写入 `summary`;
|
||||||
|
3. **仲裁**:
|
||||||
|
- `majority`:fail 多于 pass → 拒绝;**平票安全偏向拒绝**(绝不静默接受分裂裁决);
|
||||||
|
- `weighted`:比较 pass 侧与 fail 侧的 `Σ(weight·confidence)`,重侧胜,平局 → 拒绝;
|
||||||
|
4. **合并重做目标**:取所有 **fail 评审者** 的 `recommended_rework ∪ affected_tasks`,去重保序;
|
||||||
|
5. **接受条件**:仲裁非拒绝 **且** 无重做目标。
|
||||||
|
|
||||||
|
### 2.3 重做分类(`classify_rework`)
|
||||||
|
|
||||||
|
确定性关键词打分(与 `planner._heuristic_consistency_check` 同族,可解释、无模型):
|
||||||
|
|
||||||
|
- 各 `ReworkCategory` 有关键词信号集;命中即加分,最高分胜;
|
||||||
|
- **分歧偏置**:当 `disagreement=True` 时,`collaboration` 既 +1 票**又赢平票**——评审者分裂本身就是跨专家一致性缺口的证据;
|
||||||
|
- 全无信号 → `unknown`(不伪造)。
|
||||||
|
|
||||||
|
## 3. 接入现状(已无条件接入,取代单评审)
|
||||||
|
|
||||||
|
> 同伴交叉评审是蜂群**唯一**的评审路径,**无开关**:去中心化重构已**删除**单 critic 主控评审环(`maybe_run_review_cycle` / `review_loop_enabled`)。`main.py: run_cross_review` 在 `refresh_swarm_run_status` 中无条件运行(<2 评审时为 no-op,放行收敛)。
|
||||||
|
|
||||||
|
### 3.1 评审者来源(≥ 2 独立意见)
|
||||||
|
|
||||||
|
同伴 Agent 经 WS `review_decision` 消息提交独立评审 → `main.py: handle_review_decision` 累积到 `run.metadata["reviews"]`;`run_cross_review` 在收齐 ≥ 2 条时 `aggregate_reviews(...)` 仲裁。`reviewer_agent_id` 取提交者,`weight` 可按角色信任赋值。`weighted` 法用 `weight·confidence` 比较,平票安全偏向拒绝。
|
||||||
|
|
||||||
|
### 3.2 状态与事件
|
||||||
|
|
||||||
|
- **状态**:`run.metadata["cross_review"]` 存 `AggregatedVerdict.to_dict()`,`run.metadata["rework_attributions"]` 存 `[ReworkAttribution.to_dict()]`,与 `review_cycles` 并存(`MAX_REVIEW_CYCLES` 预算复用);拒绝则 `reopen_task` 返工目标、run 退回 `running`。
|
||||||
|
- **事件不进 Manager 流**:`review.*` / `rework.*` 的 payload builder 已实现但**不经 `emit_event` 外发**(未在 Manager `agent_callback.go` 注册;与 `swarm.health`/`convergence.*` 同策略,避免向订阅全部的回调投递未登记事件)。重开通过既有 `timeline.updated` 反映。登记后方可启用 Manager 侧发送。
|
||||||
|
|
||||||
|
### 3.4 不变量
|
||||||
|
|
||||||
|
- 不改 Manager 面接口、HMAC 回调、审批链、计费 / 审计字段语义;
|
||||||
|
- 信息素学习(`decision_engine`)、沙箱双门控(`quality`)等其它开关不受影响;
|
||||||
|
- 纯协议层无副作用:可在无密钥、无 Redis、无 WS 的环境单测(见 §5)。
|
||||||
|
|
||||||
|
## 4. 与基准 `P_rework` / `Reward` 的关系
|
||||||
|
|
||||||
|
基准执行层(`swarm-metrics-schema.md` §2):
|
||||||
|
|
||||||
|
```
|
||||||
|
R = ... − w₈·P_rework (w₈ = 0.05)
|
||||||
|
P_rework = ReworkCount/TotalTasks×100
|
||||||
|
```
|
||||||
|
|
||||||
|
schema 标注 `P_rework` 当前是「retry 派生、已知低估口径」。本模块把每次重做升级为带 `root_cause` 的 `ReworkAttribution`:
|
||||||
|
|
||||||
|
- `ReworkCount` 可由 `rework.requested` 事件数(或 `rework_attributions` 长度)精确计数,而非从 `retry_count` 反推;
|
||||||
|
- 可**按 `ReworkCategory` 分桶**(requirement/impl/test/doc/collaboration),让采集器区分重做归属的阶段,而不是一个无差别 retry 数;
|
||||||
|
- `collaboration` 类重做(评审者分歧驱动)正是「蜂群协同验证」要暴露的信号——它对应 `S_collaboration`(§3 蜂群层)的反面证据。
|
||||||
|
|
||||||
|
> 接入采集是后续工作:本任务仅提供结构化输入与协议;`benchmark/collectors/run_collector.py` 的消费留待接入 PR,且须遵守 schema 的「无信号 → NaN,不伪造」口径。
|
||||||
|
|
||||||
|
## 5. 测试
|
||||||
|
|
||||||
|
```
|
||||||
|
..\.venv\Scripts\python.exe scripts\test-cross-review.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Hermetic(无 WS / 无模型 / 无 Redis),覆盖:单评审者被拒绝;两评审者分歧检测与 majority/weighted 仲裁;一致通过即接受;三评审者多数否决与重做目标去重;重做归因与 `ReworkCategory` 分类(含分歧偏置 collaboration、无信号 → unknown);四个事件 payload builder;非法 verdict / 缺评审者 id 的构造校验。
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
# Agent 任务竞争协议(Task Competition Protocol)
|
||||||
|
|
||||||
|
解决 issue #8:「Agent 自主竞争机制缺失:任务无法被抢占、竞价、协商或重新接管」。
|
||||||
|
|
||||||
|
本文档是该机制的**唯一入口**。机制实现于 `orchestrator/task_competition.py`(纯仲裁),**已接入 WS 主循环(去中心化重构 P4)**:`main.py` 新增 `task_bid`/`task_yield`/`task_takeover_request` 分支与 `handle_task_bid`/`arbitrate_and_assign`/`handle_task_yield`/`handle_task_takeover`——竞价收集→确定性仲裁(τ 加权)→择优 `finalize_dispatch` 分派;让渡复用 `release_task`;接管须 decisive 胜出方可重分派。仲裁/竞价/让渡审计存 `run.metadata`(内部遥测)。feature flag `ENABLE_TASK_COMPETITION`(构建期;cutover 转无条件)。集成测试 `scripts/test-swarm-competition.py`。模块单测 `scripts/test-task-competition.py`。
|
||||||
|
|
||||||
|
## 1. 背景与缺口
|
||||||
|
|
||||||
|
当前派发是**单向拉取**:空闲 Agent 拉取就绪任务(`task_queue.get_ready_pending_task`),由到达顺序固定 Agent 一侧;ACO 决策引擎(`decision_engine.py`)在 `ENABLE_ACO_DISPATCH=1` 时按 `P_i = τ^α·η^β / Σ` 采样**任务**,但仍是「Agent 挑任务」的单边匹配。缺失的是 Agent 之间围绕**同一个任务**的主动博弈:
|
||||||
|
|
||||||
|
- **竞价(bid)**:多个 Agent 同时声明「我能做这个任务」,附上自评置信度、成本、耗时、风险。
|
||||||
|
- **让渡(yield)**:持有任务的 Agent 主动释放,给出理由,并可推荐接手者。
|
||||||
|
- **接管(takeover)**:另一个 Agent 请求从当前持有者手中接过任务(持有者卡住、或请求者更合适)。
|
||||||
|
- **仲裁(arbitrate)**:一个**确定性、可审计**的裁决器在竞价者中选出赢家,并记录理由与落败者。
|
||||||
|
|
||||||
|
## 2. 接入现状(已无条件接入,无开关)
|
||||||
|
|
||||||
|
竞争协议是蜂群固有行为,**无 `ENABLE_*` 开关**(本仓即 swarm 运行时)。`main.py` 始终生效的 WS 分支 `task_bid` / `task_yield` / `task_takeover_request` → `handle_task_bid`(累积竞价到 `run.metadata["bids"]`)/ `arbitrate_and_assign`(τ 加权确定性仲裁 → `finalize_dispatch` 择优分派)/ `handle_task_yield`(复用 `release_task`)/ `handle_task_takeover`(须 decisive 胜出方可重分派)。仲裁/竞价/让渡审计存 `run.metadata`(**内部遥测,非 Manager 事件**)。`task_competition.py` 模块本身仍是纯逻辑 + 数据模型(不碰 Redis/WS/计费/审批链)。
|
||||||
|
|
||||||
|
## 3. 消息 / 数据模型(`orchestrator/task_competition.py`)
|
||||||
|
|
||||||
|
| 模型 | 字段 | 含义 |
|
||||||
|
|---|---|---|
|
||||||
|
| `TaskBid` | `task_id, agent_id, confidence, estimated_cost, estimated_time, risk_score, reason, capabilities, current_load` | 一次竞价。`confidence/risk_score∈[0,1]`;`estimated_cost`=占 run 预算比例;`estimated_time`=秒;`current_load`=在飞任务数 |
|
||||||
|
| `TaskYield` | `task_id, agent_id, release_with_reason, recommend_agent?` | 主动让渡 + 理由 + 可选推荐接手者 |
|
||||||
|
| `TaskTakeoverRequest` | `task_id, requesting_agent_id, current_agent_id?, reason, bid?` | 接管请求,可携带 `bid` 以便与持有者在同一标准下被仲裁 |
|
||||||
|
| `ArbitrationScore` | `agent_id, total, components{}` | 单个竞价者的**逐项打分明细**(审计) |
|
||||||
|
| `TaskArbitrationResult` | `task_id, winner_agent_id, reason, decisive, scores[], losers[], policy, arbitrated_at` | 裁决结果:赢家、人读理由、是否「明确」、全量分数、落败者、所用策略 |
|
||||||
|
|
||||||
|
## 4. 仲裁算法(`arbitrate(bids, policy, *, required_capabilities, historical_success)`)
|
||||||
|
|
||||||
|
**纯函数,确定性。** 每个竞价独立打分(无共享可变状态),然后按 `(total 降序, agent_id 升序)` 稳定排序,头部即赢家。`agent_id` 兜底排序**消除了对输入顺序和字典迭代顺序的依赖** —— 相同输入恒得相同赢家、相同分数、相同有序落败列表。
|
||||||
|
|
||||||
|
每项分量先归一化到 `[0,1]` 再乘策略权重;「越低越好」的字段(成本/耗时/风险/负载)转为余量 `1 - 归一值`,使「越高越好」统一成立:
|
||||||
|
|
||||||
|
| 分量 | 来源 | 默认权重 |
|
||||||
|
|---|---|---|
|
||||||
|
| `capability` | 竞价 `capabilities` 对任务 `required_capabilities` 的覆盖率 | 0.25 |
|
||||||
|
| `historical_success`(**τ**) | 复用 ACO 信息素 trail 的「挣来的声誉」`pheromone:{role}:{id}`;缺失 → 文档化中性 **0.5**(对所有竞价同值,不扭曲排序,同 `decision_engine` 对缺失 confidence 的处理) | 0.25 |
|
||||||
|
| `confidence` | 竞价自评置信度 | 0.20 |
|
||||||
|
| `budget` | 成本余量(越便宜越高) | 0.10 |
|
||||||
|
| `risk` | `1 - risk_score` | 0.10 |
|
||||||
|
| `time` | 速度余量(越快越高) | 0.05 |
|
||||||
|
| `load` | `current_load` 余量(越闲越高) | 0.05 |
|
||||||
|
|
||||||
|
权重由 `ArbitrationPolicy` 提供,非负、无需归一(相对比较)。`decisive_margin`(默认 0.02):当头两名分差低于它时,结果标记 `decisive=False`,理由中**建议转 Manager 审核**而非自动分配 —— 仲裁仍是确定的(赢家=稳定排序头部),但把「过于接近的平局」交回人工/审批链,符合 heicodeDocs 安全与审批约束。
|
||||||
|
|
||||||
|
**τ 是真实输入**:测试断言「其余完全相同、仅声誉不同」时高 τ 者胜,证明 `historical_success` 真正改变结果而非装饰字段。
|
||||||
|
|
||||||
|
## 5. 事件载荷构建器(builders)
|
||||||
|
|
||||||
|
下列函数**只构建 payload dict,不发射**。形状对齐既有 HM 事件(`task_id` + 角色/id + 人读 `summary`):
|
||||||
|
|
||||||
|
| 函数 | event_type |
|
||||||
|
|---|---|
|
||||||
|
| `bid_submitted_event(bid)` | `task.bid_submitted` |
|
||||||
|
| `yielded_event(yield_msg)` | `task.yielded` |
|
||||||
|
| `takeover_requested_event(req)` | `task.takeover_requested` |
|
||||||
|
| `arbitrated_event(result)` | `task.arbitrated`(携带全量 `scores` 与 `losers` 审计) |
|
||||||
|
|
||||||
|
## 6. 测试
|
||||||
|
|
||||||
|
`scripts/test-task-competition.py`:**模块级、无 Redis / WS / 模型**(`task_competition` 无副作用,直接测真实仲裁数学)。覆盖:两 Agent 同任务竞价→更优者胜并记录可审计理由与落败者;输入乱序 / 重复调用结果字节一致(确定性);τ 单因子决胜;死平局不 decisive 且兜底确定 + 建议审核;空竞价;让渡(带理由 + 推荐);接管请求并与持有者仲裁;自定义策略权重改变结果;全部事件 builder。
|
||||||
|
|
||||||
|
运行(在 `agent_swarm_v6` 下,先装依赖):
|
||||||
|
|
||||||
|
```
|
||||||
|
pip install -r orchestrator/requirements.txt
|
||||||
|
..\.venv\Scripts\python.exe scripts/test-task-competition.py
|
||||||
|
```
|
||||||
|
|
||||||
|
结果:35/35 PASS(`ALL PASSED`)。
|
||||||
|
|
||||||
|
## 集成现状(已接入)
|
||||||
|
|
||||||
|
已落地于 `orchestrator/main.py`(无开关):
|
||||||
|
|
||||||
|
1. **WS 入站分支**(紧邻 `peer_message`/`handoff_request`):`task_bid` → `handle_task_bid`(按 agent 去重累积到 `run.metadata["bids"]`);`task_yield` → `handle_task_yield`(复用 `task_queue.release_task`,不增 retry_count);`task_takeover_request` → `handle_task_takeover`(请求者 bid 与持有者中性 bid 一并送 `arbitrate`)。
|
||||||
|
2. **仲裁分派**:`arbitrate_and_assign(run, task_id)` 取 `run.metadata["bids"]` → `arbitrate` → 赢家经 `finalize_dispatch` 分派;审计存 `run.metadata["arbitrations"]/["yields"]`。
|
||||||
|
3. **τ 注入**:`historical_success` 由 `decision_engine.get_tau(...)` 逐竞价者取值(经 `normalize_tau`)传入 `arbitrate`,与自选/ACO 共用同一声誉源。
|
||||||
|
4. **事件不进 Manager 流**:`task.bid_submitted/yielded/takeover_requested/arbitrated` 的 builder 已实现但**不经 `emit_event` 外发**(未在 `agent_callback.go` 注册;与 `peer_message`/`swarm.health` 同策略——仅内部遥测)。登记后方可启用 Manager 侧发送。
|
||||||
|
5. **审批 / 计费 / 审计(后续硬约束)**:`decisive=False`(接近平局)或接管已分配任务时,按 heicodeDocs 应走 Manager 审批链而非自动改派;当前实现仅在 decisive 胜出时改派,平票不改派(记录待 review)。任何改派保留 `usage` 与 `X-Agent/X-Agnet` 归属。专用「竞价窗口」收集期(定时触发 `arbitrate_and_assign`)为后续优化。
|
||||||
@@ -0,0 +1,394 @@
|
|||||||
|
"""Agent-proposed autonomous task generation (issue #7).
|
||||||
|
|
||||||
|
This module lets an *execution unit* (an agent) propose a NEW task it discovered while
|
||||||
|
working — e.g. it notices the shared run state is missing tests, docs, or a follow-up step —
|
||||||
|
and lets the orchestrator decide whether to accept, reject, or merge that proposal into the
|
||||||
|
runtime task graph.
|
||||||
|
|
||||||
|
Design boundaries (read before extending):
|
||||||
|
|
||||||
|
- This is the AGENT proposing, NOT a Master/planner generating work. The planner fallback
|
||||||
|
(``master_agent.plan`` / ``ENABLE_PLANNER_FALLBACK``) is a top-down decomposition done by a
|
||||||
|
controller. This path is strictly bottom-up: the proposal carries ``source="agent_proposed"``
|
||||||
|
and ``proposed_by_agent_id``, and ``review_proposal``/``ingest_accepted_proposal`` MUST NOT be
|
||||||
|
driven by master_agent. ``assert_not_master_origin`` enforces that at runtime.
|
||||||
|
- PURE module: no Redis, no WebSocket, no FastAPI imports. The functions take plain data in and
|
||||||
|
return plain data out, so the orchestrator integrator wires them into the live loop and the
|
||||||
|
hermetic test exercises them at the module level without any infrastructure.
|
||||||
|
- UNCONDITIONAL: agent task proposals are the swarm's ONLY decomposition path — there is no
|
||||||
|
enable flag (this repo is the swarm runtime; see docs/swarm/decentralized-rework-plan.md). The
|
||||||
|
WS ``task_proposal`` branch + ``handle_task_proposal`` in main.py are always active.
|
||||||
|
- HONESTY (rule #9): what this module implements — the proposal model, the review policy
|
||||||
|
(confidence/budget/dedup), the accepted->task-spec mapping with full lineage, and the
|
||||||
|
lifecycle-event builders. Wiring into the live loop is in main.py (see
|
||||||
|
``docs/swarm/autonomous-task-generation.md``).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from dataclasses import dataclass, field, asdict
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Dict, List, Optional, Sequence
|
||||||
|
|
||||||
|
|
||||||
|
# --- constants --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
# The Task.source value a proposal-derived task carries. Distinct from "manual"/"planner"/
|
||||||
|
# "runtime_bridge"/"dynamic_handoff" so the Manager/audit can attribute bottom-up tasks.
|
||||||
|
PROPOSED_SOURCE = "agent_proposed"
|
||||||
|
|
||||||
|
# Lifecycle event types. Internal runtime events (NOT registered Heicode Manager events): an
|
||||||
|
# integrator either maps them onto an existing HM event or records them as telemetry. Kept here
|
||||||
|
# so producers and the collector share one vocabulary. See doc §事件.
|
||||||
|
EVENT_SUBMITTED = "task.proposal_submitted"
|
||||||
|
EVENT_ACCEPTED = "task.proposal_accepted"
|
||||||
|
EVENT_REJECTED = "task.proposal_rejected"
|
||||||
|
EVENT_MERGED = "task.proposal_merged"
|
||||||
|
|
||||||
|
|
||||||
|
class ProposalStatus(str, Enum):
|
||||||
|
"""Lifecycle status of an agent-submitted task proposal."""
|
||||||
|
PROPOSED = "proposed"
|
||||||
|
ACCEPTED = "accepted"
|
||||||
|
REJECTED = "rejected"
|
||||||
|
MERGED = "merged"
|
||||||
|
EXPIRED = "expired"
|
||||||
|
|
||||||
|
|
||||||
|
class ProposalDecision(str, Enum):
|
||||||
|
"""Outcome of the review policy for a single proposal."""
|
||||||
|
ACCEPT = "accept"
|
||||||
|
REJECT = "reject"
|
||||||
|
MERGE = "merge"
|
||||||
|
|
||||||
|
|
||||||
|
# --- data models ------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ProposalLineage:
|
||||||
|
"""Provenance of a proposal: where it came from and what triggered it.
|
||||||
|
|
||||||
|
This is what makes a proposed task auditable end-to-end: the originating task, the event
|
||||||
|
that prompted the agent to propose, and the shared-state snapshot the agent reasoned over.
|
||||||
|
"""
|
||||||
|
origin_task_id: Optional[str] = None
|
||||||
|
trigger_event: Optional[str] = None
|
||||||
|
# A redaction-safe snapshot of the shared run state the agent based the proposal on
|
||||||
|
# (e.g. completed task summaries, open gaps, pending task ids). Free-form on purpose; the
|
||||||
|
# dedup/policy logic only reads a few well-known keys (see review_proposal).
|
||||||
|
shared_state_snapshot: Dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TaskProposal:
|
||||||
|
"""A NEW task an agent proposes based on shared run state.
|
||||||
|
|
||||||
|
``source`` is fixed to PROPOSED_SOURCE and ``proposed_by_agent_id`` is required: this is the
|
||||||
|
agent proposing, never a Master. ``proposal_confidence`` is the agent's own 0..1 estimate that
|
||||||
|
the task is worth doing; the orchestrator policy (review_proposal), not the agent, decides.
|
||||||
|
"""
|
||||||
|
proposed_by_agent_id: str
|
||||||
|
description: str
|
||||||
|
proposal_reason: str
|
||||||
|
proposal_confidence: float = 0.0
|
||||||
|
title: Optional[str] = None
|
||||||
|
agent_role: str = "general"
|
||||||
|
required_capabilities: List[str] = field(default_factory=list)
|
||||||
|
depends_on: List[str] = field(default_factory=list)
|
||||||
|
lineage: ProposalLineage = field(default_factory=ProposalLineage)
|
||||||
|
proposal_id: str = field(default_factory=lambda: f"prop-{uuid.uuid4().hex[:12]}")
|
||||||
|
source: str = PROPOSED_SOURCE
|
||||||
|
status: ProposalStatus = ProposalStatus.PROPOSED
|
||||||
|
created_at: float = field(default_factory=time.time)
|
||||||
|
# The id of an existing task this proposal was merged into (set on MERGE).
|
||||||
|
merged_into_task_id: Optional[str] = None
|
||||||
|
# Human/policy-readable note attached by review_proposal (why it was rejected, what it merged
|
||||||
|
# into, etc.). Carried into the lifecycle event payload.
|
||||||
|
decision_reason: Optional[str] = None
|
||||||
|
|
||||||
|
def __post_init__(self):
|
||||||
|
# Hard-pin the source: even if a caller passes something else, a TaskProposal is by
|
||||||
|
# definition agent-proposed. This keeps Task.source attribution honest.
|
||||||
|
self.source = PROPOSED_SOURCE
|
||||||
|
if not self.proposed_by_agent_id:
|
||||||
|
raise ValueError("TaskProposal requires proposed_by_agent_id (an agent, not a Master)")
|
||||||
|
# Clamp confidence into [0, 1] so a bad client value cannot defeat the threshold.
|
||||||
|
try:
|
||||||
|
self.proposal_confidence = max(0.0, min(1.0, float(self.proposal_confidence)))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
self.proposal_confidence = 0.0
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
"""Serialize to a plain dict (enums -> values) for events / transport / storage."""
|
||||||
|
data = asdict(self)
|
||||||
|
data["status"] = self.status.value
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ProposalPolicy:
|
||||||
|
"""Acceptance policy for the review step.
|
||||||
|
|
||||||
|
All thresholds are explicit so the integrator can source them from the run's
|
||||||
|
orchestration_plan/budget. Nothing here is read from the model — the agent only *proposes*.
|
||||||
|
"""
|
||||||
|
# Minimum agent confidence to consider accepting at all.
|
||||||
|
min_confidence: float = 0.6
|
||||||
|
# Confidence at/above which a unique proposal is accepted outright. Between min_confidence
|
||||||
|
# and this band, a non-duplicate proposal is still accepted; this field is kept so an
|
||||||
|
# integrator can split "auto-accept" from "queue for human review" later.
|
||||||
|
auto_accept_confidence: float = 0.6
|
||||||
|
# Remaining proposal budget for the run (how many more proposed tasks may be enqueued).
|
||||||
|
# 0 => budget exhausted => reject. Integrator decrements this as it enqueues.
|
||||||
|
remaining_proposal_budget: int = 3
|
||||||
|
# Similarity ratio (0..1) at/above which a proposal is treated as a duplicate of an existing
|
||||||
|
# task and MERGED rather than enqueued as new.
|
||||||
|
dedup_similarity_threshold: float = 0.8
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ExistingTaskRef:
|
||||||
|
"""The minimal view of an existing task the dedup check needs.
|
||||||
|
|
||||||
|
A plain dataclass (not the Pydantic Task) so this module stays infra-free; the integrator
|
||||||
|
builds these from task_queue.get_all_tasks().
|
||||||
|
"""
|
||||||
|
task_id: str
|
||||||
|
description: str
|
||||||
|
agent_role: str = "general"
|
||||||
|
status: str = "pending"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ReviewOutcome:
|
||||||
|
"""Result of review_proposal: the decision plus the (mutated) proposal and a reason."""
|
||||||
|
decision: ProposalDecision
|
||||||
|
proposal: TaskProposal
|
||||||
|
reason: str
|
||||||
|
# Set when decision is MERGE: the existing task id the proposal folds into.
|
||||||
|
merge_target_task_id: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
# --- dedup helper -----------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _normalize_words(text: str) -> set:
|
||||||
|
"""Lowercase token set used for cheap, dependency-free description similarity."""
|
||||||
|
return {w for w in "".join(c.lower() if c.isalnum() else " " for c in (text or "")).split() if w}
|
||||||
|
|
||||||
|
|
||||||
|
def description_similarity(a: str, b: str) -> float:
|
||||||
|
"""Jaccard similarity of word sets in two descriptions (0..1).
|
||||||
|
|
||||||
|
Deliberately simple and deterministic (no embeddings, no model call) so the dedup decision is
|
||||||
|
reproducible and testable. Matches the spirit of the heuristic match score in decision_engine.
|
||||||
|
"""
|
||||||
|
wa, wb = _normalize_words(a), _normalize_words(b)
|
||||||
|
if not wa or not wb:
|
||||||
|
return 0.0
|
||||||
|
inter = len(wa & wb)
|
||||||
|
union = len(wa | wb)
|
||||||
|
return inter / union if union else 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def find_duplicate(
|
||||||
|
proposal: TaskProposal,
|
||||||
|
existing_tasks: Sequence[ExistingTaskRef],
|
||||||
|
threshold: float,
|
||||||
|
) -> Optional[ExistingTaskRef]:
|
||||||
|
"""Return the most similar non-terminal existing task above ``threshold``, else None.
|
||||||
|
|
||||||
|
Only live tasks (pending/assigned/in_progress/blocked) are dedup targets — re-proposing work
|
||||||
|
similar to a *completed* task is allowed (the agent may legitimately want a follow-up round).
|
||||||
|
"""
|
||||||
|
live = {"pending", "assigned", "in_progress", "blocked"}
|
||||||
|
best: Optional[ExistingTaskRef] = None
|
||||||
|
best_sim = threshold
|
||||||
|
for task in existing_tasks:
|
||||||
|
if task.status not in live:
|
||||||
|
continue
|
||||||
|
sim = description_similarity(proposal.description, task.description)
|
||||||
|
if sim >= best_sim:
|
||||||
|
best_sim = sim
|
||||||
|
best = task
|
||||||
|
return best
|
||||||
|
|
||||||
|
|
||||||
|
# --- core policy ------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def review_proposal(
|
||||||
|
proposal: TaskProposal,
|
||||||
|
policy: ProposalPolicy,
|
||||||
|
existing_tasks: Optional[Sequence[ExistingTaskRef]] = None,
|
||||||
|
) -> ReviewOutcome:
|
||||||
|
"""Decide whether to accept / reject / merge an agent-submitted proposal.
|
||||||
|
|
||||||
|
Pure function. Order of checks:
|
||||||
|
1. Confidence floor: below ``policy.min_confidence`` => REJECT.
|
||||||
|
2. Dedup: if it closely matches a live existing task => MERGE into that task.
|
||||||
|
3. Budget: if the run's remaining proposal budget is exhausted => REJECT.
|
||||||
|
4. Otherwise => ACCEPT.
|
||||||
|
|
||||||
|
Mutates and returns the proposal with its new status / decision_reason so the caller can
|
||||||
|
persist it and emit the matching lifecycle event. This NEVER calls a model or a Master — the
|
||||||
|
agent proposed; the orchestrator policy disposes.
|
||||||
|
"""
|
||||||
|
existing_tasks = existing_tasks or []
|
||||||
|
|
||||||
|
if proposal.proposal_confidence < policy.min_confidence:
|
||||||
|
proposal.status = ProposalStatus.REJECTED
|
||||||
|
proposal.decision_reason = (
|
||||||
|
f"confidence {proposal.proposal_confidence:.2f} < min {policy.min_confidence:.2f}"
|
||||||
|
)
|
||||||
|
return ReviewOutcome(ProposalDecision.REJECT, proposal, proposal.decision_reason)
|
||||||
|
|
||||||
|
duplicate = find_duplicate(proposal, existing_tasks, policy.dedup_similarity_threshold)
|
||||||
|
if duplicate is not None:
|
||||||
|
proposal.status = ProposalStatus.MERGED
|
||||||
|
proposal.merged_into_task_id = duplicate.task_id
|
||||||
|
proposal.decision_reason = f"duplicate of existing task {duplicate.task_id}"
|
||||||
|
return ReviewOutcome(
|
||||||
|
ProposalDecision.MERGE,
|
||||||
|
proposal,
|
||||||
|
proposal.decision_reason,
|
||||||
|
merge_target_task_id=duplicate.task_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
if policy.remaining_proposal_budget <= 0:
|
||||||
|
proposal.status = ProposalStatus.REJECTED
|
||||||
|
proposal.decision_reason = "proposal budget exhausted for this run"
|
||||||
|
return ReviewOutcome(ProposalDecision.REJECT, proposal, proposal.decision_reason)
|
||||||
|
|
||||||
|
proposal.status = ProposalStatus.ACCEPTED
|
||||||
|
proposal.decision_reason = (
|
||||||
|
f"accepted (confidence {proposal.proposal_confidence:.2f} >= {policy.min_confidence:.2f})"
|
||||||
|
)
|
||||||
|
return ReviewOutcome(ProposalDecision.ACCEPT, proposal, proposal.decision_reason)
|
||||||
|
|
||||||
|
|
||||||
|
def ingest_accepted_proposal(
|
||||||
|
proposal: TaskProposal,
|
||||||
|
*,
|
||||||
|
swarm_id: Optional[str] = None,
|
||||||
|
root_task_id: Optional[str] = None,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Map an ACCEPTED proposal into a pending-task spec for the integrator to enqueue.
|
||||||
|
|
||||||
|
Returns a kwargs-shaped dict aligned with ``task_queue.create_task`` (description, agent_role,
|
||||||
|
required_capabilities, depends_on, parent_task_id, root_task_id, source, context). The
|
||||||
|
integrator calls ``task_queue.create_task(**spec_to_create_task_kwargs(spec))`` (or maps it
|
||||||
|
through ``create_tasks_for_run``). The new task starts PENDING — create_task already sets that.
|
||||||
|
|
||||||
|
The full lineage is carried into ``context`` so the proposed task is auditable: which agent
|
||||||
|
proposed it, why, from which origin task, on which trigger, and the shared-state snapshot.
|
||||||
|
Raises if the proposal is not in ACCEPTED status — merged/rejected proposals never become tasks.
|
||||||
|
"""
|
||||||
|
if proposal.status != ProposalStatus.ACCEPTED:
|
||||||
|
raise ValueError(
|
||||||
|
f"ingest_accepted_proposal requires ACCEPTED status, got {proposal.status.value}"
|
||||||
|
)
|
||||||
|
assert_not_master_origin(proposal)
|
||||||
|
|
||||||
|
# parent/root: a proposal born from an origin task is a child of it by default; otherwise it
|
||||||
|
# is a new root. The integrator may override root_task_id to thread it into an existing run.
|
||||||
|
parent = proposal.lineage.origin_task_id
|
||||||
|
root = root_task_id or proposal.lineage.origin_task_id or None
|
||||||
|
|
||||||
|
return {
|
||||||
|
"task_id": None, # let create_task mint a uuid; integrator may prefix with swarm_id
|
||||||
|
"title": proposal.title or proposal.description[:80],
|
||||||
|
"description": proposal.description,
|
||||||
|
"agent_role": proposal.agent_role,
|
||||||
|
"required_capabilities": list(proposal.required_capabilities),
|
||||||
|
"depends_on": list(proposal.depends_on),
|
||||||
|
"parent_task_id": parent,
|
||||||
|
"root_task_id": root,
|
||||||
|
"source": PROPOSED_SOURCE,
|
||||||
|
"context": {
|
||||||
|
"source": PROPOSED_SOURCE,
|
||||||
|
"agent_role": proposal.agent_role,
|
||||||
|
"swarm_id": swarm_id,
|
||||||
|
"proposal": {
|
||||||
|
"proposal_id": proposal.proposal_id,
|
||||||
|
"proposed_by_agent_id": proposal.proposed_by_agent_id,
|
||||||
|
"proposal_reason": proposal.proposal_reason,
|
||||||
|
"proposal_confidence": proposal.proposal_confidence,
|
||||||
|
"origin_task_id": proposal.lineage.origin_task_id,
|
||||||
|
"trigger_event": proposal.lineage.trigger_event,
|
||||||
|
"shared_state_snapshot": dict(proposal.lineage.shared_state_snapshot),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --- lifecycle events -------------------------------------------------------------------
|
||||||
|
|
||||||
|
def build_proposal_event(event_type: str, proposal: TaskProposal, **extra: Any) -> Dict[str, Any]:
|
||||||
|
"""Build one proposal-lifecycle event payload.
|
||||||
|
|
||||||
|
Returns a flat payload dict (the shape the orchestrator passes to
|
||||||
|
``swarm_runtime.emit_event(run, event_type, payload=...)``). ``event_type`` must be one of
|
||||||
|
the EVENT_* constants. The integrator owns wrapping this in the HM event envelope.
|
||||||
|
"""
|
||||||
|
valid = {EVENT_SUBMITTED, EVENT_ACCEPTED, EVENT_REJECTED, EVENT_MERGED}
|
||||||
|
if event_type not in valid:
|
||||||
|
raise ValueError(f"unknown proposal event_type {event_type!r}; expected one of {valid}")
|
||||||
|
payload: Dict[str, Any] = {
|
||||||
|
"event_type": event_type,
|
||||||
|
"proposal_id": proposal.proposal_id,
|
||||||
|
"proposed_by_agent_id": proposal.proposed_by_agent_id,
|
||||||
|
"source": PROPOSED_SOURCE,
|
||||||
|
"status": proposal.status.value,
|
||||||
|
"title": proposal.title or proposal.description[:80],
|
||||||
|
"proposal_reason": proposal.proposal_reason,
|
||||||
|
"proposal_confidence": proposal.proposal_confidence,
|
||||||
|
"origin_task_id": proposal.lineage.origin_task_id,
|
||||||
|
"trigger_event": proposal.lineage.trigger_event,
|
||||||
|
"decision_reason": proposal.decision_reason,
|
||||||
|
"merged_into_task_id": proposal.merged_into_task_id,
|
||||||
|
}
|
||||||
|
payload.update(extra)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def build_lifecycle_events_for_outcome(
|
||||||
|
proposal: TaskProposal,
|
||||||
|
outcome: ReviewOutcome,
|
||||||
|
) -> List[Dict[str, Any]]:
|
||||||
|
"""Build the ordered event list for a reviewed proposal: submitted -> decision event.
|
||||||
|
|
||||||
|
Always emits ``task.proposal_submitted`` first (the agent's act of proposing is itself an
|
||||||
|
auditable fact), then the decision event matching the policy outcome.
|
||||||
|
"""
|
||||||
|
events = [build_proposal_event(EVENT_SUBMITTED, proposal)]
|
||||||
|
decision_event = {
|
||||||
|
ProposalDecision.ACCEPT: EVENT_ACCEPTED,
|
||||||
|
ProposalDecision.REJECT: EVENT_REJECTED,
|
||||||
|
ProposalDecision.MERGE: EVENT_MERGED,
|
||||||
|
}[outcome.decision]
|
||||||
|
extra: Dict[str, Any] = {}
|
||||||
|
if outcome.merge_target_task_id:
|
||||||
|
extra["merge_target_task_id"] = outcome.merge_target_task_id
|
||||||
|
events.append(build_proposal_event(decision_event, proposal, **extra))
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
# --- guardrail --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def assert_not_master_origin(proposal: TaskProposal) -> None:
|
||||||
|
"""Fail loudly if a proposal looks like it came from a Master/planner, not an agent.
|
||||||
|
|
||||||
|
Issue #7 is explicitly about the AGENT proposing. A Master generating tasks is the existing
|
||||||
|
planner path and must not be laundered through this module (it would mis-attribute source and
|
||||||
|
bypass the planner's own contract). We reject any proposer id / trigger that names a master or
|
||||||
|
planner. This is a cheap, explicit invariant — not a substitute for auth.
|
||||||
|
"""
|
||||||
|
proposer = (proposal.proposed_by_agent_id or "").lower()
|
||||||
|
trigger = (proposal.lineage.trigger_event or "").lower()
|
||||||
|
banned = ("master", "planner")
|
||||||
|
if any(tok in proposer for tok in banned) or any(tok in trigger for tok in banned):
|
||||||
|
raise ValueError(
|
||||||
|
"TaskProposal must originate from an executing agent, not a Master/planner "
|
||||||
|
f"(proposed_by_agent_id={proposal.proposed_by_agent_id!r}, "
|
||||||
|
f"trigger_event={proposal.lineage.trigger_event!r})"
|
||||||
|
)
|
||||||
@@ -0,0 +1,622 @@
|
|||||||
|
"""Swarm convergence protocol — consensus, conflict resolution, and a termination function.
|
||||||
|
|
||||||
|
Issue #12: today a swarm run reaches a terminal state in
|
||||||
|
`orchestrator/main.py:refresh_swarm_run_status` purely by task bookkeeping —
|
||||||
|
"all known tasks reached a terminal state, and (optionally, behind
|
||||||
|
ENABLE_REVIEW_LOOP) the master critic accepted the work". There is NO explicit
|
||||||
|
convergence model: no consensus score, no first-class conflict detection /
|
||||||
|
resolution pass, and no machine-readable `termination_reason` that explains WHY
|
||||||
|
the swarm stopped. A reader of the run cannot tell "quality reached" apart from
|
||||||
|
"budget exhausted" apart from "blocked by an unresolved risk".
|
||||||
|
|
||||||
|
This module adds that missing model as a PURE, side-effect-free layer:
|
||||||
|
|
||||||
|
* `evaluate_convergence(run_state) -> ConvergenceReport` derives a run status,
|
||||||
|
a single explanatory `termination_reason`, a consensus score, and the
|
||||||
|
detected / resolved / unresolved conflicts from plain inputs (task states,
|
||||||
|
review verdicts, budget, risks). It performs NO I/O and never mutates the
|
||||||
|
run — the caller decides whether/when to persist or emit.
|
||||||
|
* Conflict detectors for artifact mismatch, test failure, review disagreement,
|
||||||
|
and dependency inconsistency, plus a `resolve_conflicts` pass that marks each
|
||||||
|
conflict resolved or unresolved.
|
||||||
|
* Event-payload builders matching the SwarmRuntime emit_event convention
|
||||||
|
(Manager `event_type` + plain dict payload).
|
||||||
|
|
||||||
|
UNCONDITIONAL: `evaluate_convergence` is wired into `refresh_swarm_run_status` and runs on every
|
||||||
|
terminal swarm run (no enable flag — this repo is the swarm runtime). It produces the report +
|
||||||
|
`termination_reason` stored on the run and surfaced on `timeline.updated`.
|
||||||
|
|
||||||
|
Honesty (org rule #9): the report is currently EXPLANATORY — it derives `termination_reason` /
|
||||||
|
consensus / conflicts from real run inputs but does NOT override `run.status` (today next_status
|
||||||
|
from task bookkeeping and the report agree on completed/failed). Authoritative status-override is a
|
||||||
|
documented follow-on.
|
||||||
|
|
||||||
|
Code is English; the companion design doc is Simplified Chinese (docs/ style).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
|
||||||
|
# --- enums ----------------------------------------------------------------
|
||||||
|
|
||||||
|
class TerminationReason(str, Enum):
|
||||||
|
"""Why the swarm stopped converging.
|
||||||
|
|
||||||
|
Exactly one is attached to every terminal ConvergenceReport. `tasks_completed`
|
||||||
|
is the honest fallback that mirrors today's only real termination signal
|
||||||
|
("all tasks done + review passed"); the others require their respective
|
||||||
|
inputs (quality grade, budget state, round counter, blocking risk) to be
|
||||||
|
present, and are advisory until those inputs are wired in.
|
||||||
|
"""
|
||||||
|
|
||||||
|
QUALITY_REACHED = "quality_reached" # acceptance/quality gate satisfied
|
||||||
|
BUDGET_EXHAUSTED = "budget_exhausted" # token / cost / duration budget spent
|
||||||
|
MAX_ROUNDS_REACHED = "max_rounds_reached" # review/redo cycle cap hit
|
||||||
|
RISK_BLOCKED = "risk_blocked" # an unresolved blocking risk stopped the run
|
||||||
|
TASKS_COMPLETED = "tasks_completed" # fallback: all tasks terminal, nothing else to explain
|
||||||
|
|
||||||
|
|
||||||
|
class ConflictType(str, Enum):
|
||||||
|
"""Kinds of inter-agent disagreement the swarm can detect."""
|
||||||
|
|
||||||
|
ARTIFACT_MISMATCH = "artifact_mismatch" # two tasks wrote conflicting content to one path
|
||||||
|
TEST_FAILURE = "test_failure" # a task's reported tests did not pass
|
||||||
|
REVIEW_DISAGREEMENT = "review_disagreement" # master critic rejected accepted-looking work
|
||||||
|
DEPENDENCY_INCONSISTENCY = "dependency_inconsistency" # task completed before/without its dependency
|
||||||
|
|
||||||
|
|
||||||
|
class ConvergenceStatus(str, Enum):
|
||||||
|
"""Run-level convergence verdict.
|
||||||
|
|
||||||
|
Maps onto SwarmRun.status: CONVERGED->completed, FAILED->failed,
|
||||||
|
BLOCKED->blocked, RUNNING->running (not yet terminal).
|
||||||
|
"""
|
||||||
|
|
||||||
|
RUNNING = "running"
|
||||||
|
CONVERGED = "converged"
|
||||||
|
FAILED = "failed"
|
||||||
|
BLOCKED = "blocked"
|
||||||
|
|
||||||
|
|
||||||
|
# --- data structures ------------------------------------------------------
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Conflict:
|
||||||
|
"""One detected disagreement. `resolved` is set by resolve_conflicts."""
|
||||||
|
|
||||||
|
conflict_id: str
|
||||||
|
type: ConflictType
|
||||||
|
description: str
|
||||||
|
task_ids: List[str] = field(default_factory=list)
|
||||||
|
detail: Dict[str, Any] = field(default_factory=dict)
|
||||||
|
resolved: bool = False
|
||||||
|
resolution: Optional[str] = None # how it was resolved, or why it could not be
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"conflict_id": self.conflict_id,
|
||||||
|
"type": self.type.value,
|
||||||
|
"description": self.description,
|
||||||
|
"task_ids": list(self.task_ids),
|
||||||
|
"detail": dict(self.detail),
|
||||||
|
"resolved": self.resolved,
|
||||||
|
"resolution": self.resolution,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ConvergenceReport:
|
||||||
|
"""Machine-readable explanation of why (and whether) a swarm converged."""
|
||||||
|
|
||||||
|
status: ConvergenceStatus
|
||||||
|
termination_reason: Optional[TerminationReason] # None while still RUNNING
|
||||||
|
consensus_score: float # [0,100]; share of work in agreement
|
||||||
|
conflicts: List[Conflict] = field(default_factory=list)
|
||||||
|
resolved_conflicts: List[Conflict] = field(default_factory=list)
|
||||||
|
unresolved_risks: List[Dict[str, Any]] = field(default_factory=list)
|
||||||
|
budget_state: Dict[str, Any] = field(default_factory=dict)
|
||||||
|
quality_state: Dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def is_terminal(self) -> bool:
|
||||||
|
return self.status in {
|
||||||
|
ConvergenceStatus.CONVERGED,
|
||||||
|
ConvergenceStatus.FAILED,
|
||||||
|
ConvergenceStatus.BLOCKED,
|
||||||
|
}
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"status": self.status.value,
|
||||||
|
"termination_reason": (
|
||||||
|
self.termination_reason.value if self.termination_reason else None
|
||||||
|
),
|
||||||
|
"consensus_score": self.consensus_score,
|
||||||
|
"conflicts": [c.to_dict() for c in self.conflicts],
|
||||||
|
"resolved_conflicts": [c.to_dict() for c in self.resolved_conflicts],
|
||||||
|
"unresolved_risks": [dict(r) for r in self.unresolved_risks],
|
||||||
|
"budget_state": dict(self.budget_state),
|
||||||
|
"quality_state": dict(self.quality_state),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --- terminal status constants (avoid importing TaskStatus to keep this pure) ---
|
||||||
|
|
||||||
|
_TASK_TERMINAL_OK = {"completed"}
|
||||||
|
_TASK_TERMINAL_FAIL = {"failed"}
|
||||||
|
_TASK_ACTIVE = {"pending", "assigned", "in_progress"}
|
||||||
|
_TASK_BLOCKED = {"blocked"}
|
||||||
|
|
||||||
|
|
||||||
|
def _task_status(task: Any) -> str:
|
||||||
|
"""Normalize a task's status to a plain lowercase string.
|
||||||
|
|
||||||
|
Accepts a plain dict (test inputs), a pydantic Task, or an enum value.
|
||||||
|
"""
|
||||||
|
if isinstance(task, dict):
|
||||||
|
status = task.get("status")
|
||||||
|
else:
|
||||||
|
status = getattr(task, "status", None)
|
||||||
|
value = getattr(status, "value", status)
|
||||||
|
return str(value or "").lower()
|
||||||
|
|
||||||
|
|
||||||
|
def _task_id(task: Any) -> str:
|
||||||
|
if isinstance(task, dict):
|
||||||
|
return str(task.get("task_id") or "")
|
||||||
|
return str(getattr(task, "task_id", "") or "")
|
||||||
|
|
||||||
|
|
||||||
|
def _task_result(task: Any) -> Dict[str, Any]:
|
||||||
|
"""Pull a structured result dict off a task (dict or model). Empty if absent."""
|
||||||
|
if isinstance(task, dict):
|
||||||
|
result = task.get("result")
|
||||||
|
else:
|
||||||
|
result = getattr(task, "result", None)
|
||||||
|
if isinstance(result, dict):
|
||||||
|
return result
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
# --- conflict detectors ---------------------------------------------------
|
||||||
|
|
||||||
|
def detect_artifact_mismatch(tasks: List[Any]) -> List[Conflict]:
|
||||||
|
"""Two completed tasks claim to have written different content to the same path.
|
||||||
|
|
||||||
|
Looks at each task result's `files_modified` (paths) and the optional
|
||||||
|
per-path `file_contents` / `file_hashes` map. If two tasks touch the same
|
||||||
|
path with a differing recorded content/hash, that is an artifact mismatch.
|
||||||
|
"""
|
||||||
|
conflicts: List[Conflict] = []
|
||||||
|
# path -> list of (task_id, fingerprint)
|
||||||
|
by_path: Dict[str, List[tuple]] = {}
|
||||||
|
for task in tasks:
|
||||||
|
if _task_status(task) not in _TASK_TERMINAL_OK:
|
||||||
|
continue
|
||||||
|
result = _task_result(task)
|
||||||
|
tid = _task_id(task)
|
||||||
|
contents = result.get("file_contents") or {}
|
||||||
|
hashes = result.get("file_hashes") or {}
|
||||||
|
for path in result.get("files_modified") or []:
|
||||||
|
fingerprint = contents.get(path, hashes.get(path))
|
||||||
|
by_path.setdefault(path, []).append((tid, fingerprint))
|
||||||
|
|
||||||
|
for path, entries in by_path.items():
|
||||||
|
if len(entries) < 2:
|
||||||
|
continue
|
||||||
|
fingerprints = {fp for _, fp in entries if fp is not None}
|
||||||
|
if len(fingerprints) > 1:
|
||||||
|
ids = [tid for tid, _ in entries]
|
||||||
|
conflicts.append(Conflict(
|
||||||
|
conflict_id=f"conf-artifact-{path}",
|
||||||
|
type=ConflictType.ARTIFACT_MISMATCH,
|
||||||
|
description=f"Conflicting writes to {path} by {', '.join(ids)}",
|
||||||
|
task_ids=ids,
|
||||||
|
detail={"path": path, "fingerprints": sorted(str(fp) for fp in fingerprints)},
|
||||||
|
))
|
||||||
|
return conflicts
|
||||||
|
|
||||||
|
|
||||||
|
def detect_test_failures(tasks: List[Any]) -> List[Conflict]:
|
||||||
|
"""A task reported a test result that did not pass.
|
||||||
|
|
||||||
|
Reads `tests_passed` (bool) or `test_pass_rate` (0..1 or 0..100) from the
|
||||||
|
task result. No test signal at all is NOT a conflict (rule #9: absence is
|
||||||
|
not failure).
|
||||||
|
"""
|
||||||
|
conflicts: List[Conflict] = []
|
||||||
|
for task in tasks:
|
||||||
|
result = _task_result(task)
|
||||||
|
tid = _task_id(task)
|
||||||
|
passed = result.get("tests_passed")
|
||||||
|
rate = result.get("test_pass_rate")
|
||||||
|
failing = False
|
||||||
|
if passed is False:
|
||||||
|
failing = True
|
||||||
|
elif isinstance(rate, (int, float)):
|
||||||
|
normalized = rate if rate <= 1 else rate / 100.0
|
||||||
|
failing = normalized < 1.0
|
||||||
|
if failing:
|
||||||
|
conflicts.append(Conflict(
|
||||||
|
conflict_id=f"conf-test-{tid}",
|
||||||
|
type=ConflictType.TEST_FAILURE,
|
||||||
|
description=f"Task {tid} reported failing tests",
|
||||||
|
task_ids=[tid],
|
||||||
|
detail={"tests_passed": passed, "test_pass_rate": rate},
|
||||||
|
))
|
||||||
|
return conflicts
|
||||||
|
|
||||||
|
|
||||||
|
def detect_review_disagreement(
|
||||||
|
tasks: List[Any],
|
||||||
|
review_verdict: Optional[Dict[str, Any]],
|
||||||
|
) -> List[Conflict]:
|
||||||
|
"""The master critic rejected work that otherwise looks complete.
|
||||||
|
|
||||||
|
`review_verdict` is the shape produced by master_agent.review_and_decide:
|
||||||
|
{accepted: bool, retry_tasks: [..], summary: str}. A rejection is a
|
||||||
|
review-disagreement conflict over the tasks the critic flagged for redo.
|
||||||
|
"""
|
||||||
|
if not review_verdict or review_verdict.get("accepted", True):
|
||||||
|
return []
|
||||||
|
retry_tasks = list(review_verdict.get("retry_tasks") or [])
|
||||||
|
if not retry_tasks:
|
||||||
|
# Rejected but nothing actionable flagged — still a (run-wide) disagreement.
|
||||||
|
retry_tasks = [_task_id(t) for t in tasks if _task_status(t) in _TASK_TERMINAL_OK]
|
||||||
|
return [Conflict(
|
||||||
|
conflict_id="conf-review",
|
||||||
|
type=ConflictType.REVIEW_DISAGREEMENT,
|
||||||
|
description=review_verdict.get("summary") or "Master critic rejected the work",
|
||||||
|
task_ids=retry_tasks,
|
||||||
|
detail={"summary": review_verdict.get("summary")},
|
||||||
|
)]
|
||||||
|
|
||||||
|
|
||||||
|
def detect_dependency_inconsistency(tasks: List[Any]) -> List[Conflict]:
|
||||||
|
"""A task completed while a declared dependency did not complete.
|
||||||
|
|
||||||
|
Reads `depends_on` from each task and checks the dependency's terminal
|
||||||
|
status within the same run. A completed task whose dependency failed / is
|
||||||
|
still active is an inconsistency.
|
||||||
|
"""
|
||||||
|
by_id = {_task_id(t): t for t in tasks}
|
||||||
|
conflicts: List[Conflict] = []
|
||||||
|
for task in tasks:
|
||||||
|
if _task_status(task) not in _TASK_TERMINAL_OK:
|
||||||
|
continue
|
||||||
|
tid = _task_id(task)
|
||||||
|
depends_on = task.get("depends_on") if isinstance(task, dict) else getattr(task, "depends_on", None)
|
||||||
|
for dep_id in depends_on or []:
|
||||||
|
dep = by_id.get(dep_id)
|
||||||
|
if dep is None:
|
||||||
|
continue # dependency not in this run's task set; not our call to judge
|
||||||
|
if _task_status(dep) not in _TASK_TERMINAL_OK:
|
||||||
|
conflicts.append(Conflict(
|
||||||
|
conflict_id=f"conf-dep-{tid}-{dep_id}",
|
||||||
|
type=ConflictType.DEPENDENCY_INCONSISTENCY,
|
||||||
|
description=f"Task {tid} completed but dependency {dep_id} is {_task_status(dep)}",
|
||||||
|
task_ids=[tid, dep_id],
|
||||||
|
detail={"task_id": tid, "dependency_id": dep_id, "dependency_status": _task_status(dep)},
|
||||||
|
))
|
||||||
|
return conflicts
|
||||||
|
|
||||||
|
|
||||||
|
def detect_conflicts(
|
||||||
|
tasks: List[Any],
|
||||||
|
review_verdict: Optional[Dict[str, Any]] = None,
|
||||||
|
) -> List[Conflict]:
|
||||||
|
"""Run every detector and return the combined conflict list."""
|
||||||
|
conflicts: List[Conflict] = []
|
||||||
|
conflicts.extend(detect_artifact_mismatch(tasks))
|
||||||
|
conflicts.extend(detect_test_failures(tasks))
|
||||||
|
conflicts.extend(detect_review_disagreement(tasks, review_verdict))
|
||||||
|
conflicts.extend(detect_dependency_inconsistency(tasks))
|
||||||
|
return conflicts
|
||||||
|
|
||||||
|
|
||||||
|
# --- conflict resolution --------------------------------------------------
|
||||||
|
|
||||||
|
def resolve_conflicts(
|
||||||
|
conflicts: List[Conflict],
|
||||||
|
review_verdict: Optional[Dict[str, Any]] = None,
|
||||||
|
) -> tuple[List[Conflict], List[Conflict]]:
|
||||||
|
"""Mark each conflict resolved or unresolved with a simple, explainable pass.
|
||||||
|
|
||||||
|
This is a deliberately conservative first pass, not an auto-merge engine:
|
||||||
|
|
||||||
|
* ARTIFACT_MISMATCH: resolved only if the review verdict accepted the work
|
||||||
|
(the critic implicitly picked a winning version); otherwise unresolved.
|
||||||
|
* REVIEW_DISAGREEMENT: resolved if the verdict names actionable
|
||||||
|
`retry_tasks` (the swarm CAN act on it by reopening them); a blanket
|
||||||
|
rejection with nothing actionable stays unresolved.
|
||||||
|
* TEST_FAILURE / DEPENDENCY_INCONSISTENCY: treated as hard blockers — not
|
||||||
|
auto-resolvable here; left unresolved for a human / redo cycle.
|
||||||
|
|
||||||
|
Returns (resolved, unresolved). Each input Conflict is mutated in place to
|
||||||
|
carry its `resolved` flag and `resolution` note.
|
||||||
|
"""
|
||||||
|
resolved: List[Conflict] = []
|
||||||
|
unresolved: List[Conflict] = []
|
||||||
|
accepted = bool(review_verdict and review_verdict.get("accepted"))
|
||||||
|
actionable = bool(review_verdict and review_verdict.get("retry_tasks"))
|
||||||
|
|
||||||
|
for conflict in conflicts:
|
||||||
|
if conflict.type == ConflictType.ARTIFACT_MISMATCH and accepted:
|
||||||
|
conflict.resolved = True
|
||||||
|
conflict.resolution = "Master review accepted a winning version"
|
||||||
|
elif conflict.type == ConflictType.REVIEW_DISAGREEMENT and actionable:
|
||||||
|
conflict.resolved = True
|
||||||
|
conflict.resolution = "Reopened flagged tasks for a redo cycle"
|
||||||
|
else:
|
||||||
|
conflict.resolved = False
|
||||||
|
conflict.resolution = "No automatic resolution; requires redo or human review"
|
||||||
|
|
||||||
|
(resolved if conflict.resolved else unresolved).append(conflict)
|
||||||
|
return resolved, unresolved
|
||||||
|
|
||||||
|
|
||||||
|
# --- budget / quality / risk derivation -----------------------------------
|
||||||
|
|
||||||
|
def derive_budget_state(run_state: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""Summarize budget consumption from the run state inputs (no I/O).
|
||||||
|
|
||||||
|
Recognized inputs under `run_state["budget"]`:
|
||||||
|
max_tokens/token_limit, max_cost_usd, max_duration_seconds/duration_seconds
|
||||||
|
and under `run_state["usage"]`:
|
||||||
|
total_tokens, total_cost_usd, elapsed_seconds.
|
||||||
|
`exhausted` is True if any present limit is met or exceeded.
|
||||||
|
"""
|
||||||
|
budget = run_state.get("budget") or {}
|
||||||
|
usage = run_state.get("usage") or {}
|
||||||
|
|
||||||
|
max_tokens = budget.get("max_tokens") or budget.get("token_limit")
|
||||||
|
max_cost = budget.get("max_cost_usd")
|
||||||
|
max_duration = budget.get("max_duration_seconds") or budget.get("duration_seconds")
|
||||||
|
|
||||||
|
used_tokens = usage.get("total_tokens")
|
||||||
|
used_cost = usage.get("total_cost_usd")
|
||||||
|
elapsed = usage.get("elapsed_seconds")
|
||||||
|
|
||||||
|
def _exhausted(limit, used):
|
||||||
|
return (
|
||||||
|
isinstance(limit, (int, float)) and limit > 0
|
||||||
|
and isinstance(used, (int, float)) and used >= limit
|
||||||
|
)
|
||||||
|
|
||||||
|
exhausted = (
|
||||||
|
_exhausted(max_tokens, used_tokens)
|
||||||
|
or _exhausted(max_cost, used_cost)
|
||||||
|
or _exhausted(max_duration, elapsed)
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"max_tokens": max_tokens,
|
||||||
|
"used_tokens": used_tokens,
|
||||||
|
"max_cost_usd": max_cost,
|
||||||
|
"used_cost_usd": used_cost,
|
||||||
|
"max_duration_seconds": max_duration,
|
||||||
|
"elapsed_seconds": elapsed,
|
||||||
|
"exhausted": exhausted,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def derive_quality_state(run_state: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""Summarize the run quality gate from inputs (no I/O).
|
||||||
|
|
||||||
|
`run_state["quality"]` mirrors SwarmRun.quality (Group B fixture grade):
|
||||||
|
{test_pass_rate, graded, ...}. `reached` is True when a grade is present and
|
||||||
|
meets `acceptance_threshold` (default 1.0 == all fixture tests pass). Absent
|
||||||
|
grade -> reached False, graded False (rule #9: not graded != passed).
|
||||||
|
"""
|
||||||
|
quality = run_state.get("quality") or {}
|
||||||
|
threshold = run_state.get("acceptance_threshold", 1.0)
|
||||||
|
rate = quality.get("test_pass_rate")
|
||||||
|
graded = quality.get("graded", rate is not None)
|
||||||
|
reached = False
|
||||||
|
if isinstance(rate, (int, float)):
|
||||||
|
normalized = rate if rate <= 1 else rate / 100.0
|
||||||
|
reached = normalized >= threshold
|
||||||
|
return {
|
||||||
|
"graded": bool(graded),
|
||||||
|
"test_pass_rate": rate,
|
||||||
|
"acceptance_threshold": threshold,
|
||||||
|
"reached": reached,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def derive_unresolved_risks(
|
||||||
|
run_state: Dict[str, Any],
|
||||||
|
unresolved_conflicts: List[Conflict],
|
||||||
|
) -> List[Dict[str, Any]]:
|
||||||
|
"""Collect blocking risks: explicit input risks + unresolved hard conflicts.
|
||||||
|
|
||||||
|
`run_state["risks"]` is a list of {id, description, blocking: bool}. Any risk
|
||||||
|
with blocking=True, plus every unresolved TEST_FAILURE / DEPENDENCY_INCONSISTENCY
|
||||||
|
/ unresolved REVIEW_DISAGREEMENT, surfaces here. These drive RISK_BLOCKED.
|
||||||
|
"""
|
||||||
|
risks: List[Dict[str, Any]] = []
|
||||||
|
for risk in run_state.get("risks") or []:
|
||||||
|
if isinstance(risk, dict) and risk.get("blocking"):
|
||||||
|
risks.append({
|
||||||
|
"id": risk.get("id"),
|
||||||
|
"description": risk.get("description"),
|
||||||
|
"source": "input_risk",
|
||||||
|
})
|
||||||
|
for conflict in unresolved_conflicts:
|
||||||
|
risks.append({
|
||||||
|
"id": conflict.conflict_id,
|
||||||
|
"description": conflict.description,
|
||||||
|
"source": f"conflict:{conflict.type.value}",
|
||||||
|
})
|
||||||
|
return risks
|
||||||
|
|
||||||
|
|
||||||
|
def compute_consensus_score(tasks: List[Any], conflicts: List[Conflict]) -> float:
|
||||||
|
"""Share of completed work that is NOT entangled in a conflict, as 0..100.
|
||||||
|
|
||||||
|
consensus = 100 * (completed_tasks_without_conflict / completed_tasks).
|
||||||
|
No completed tasks -> 0.0 (nothing has been agreed yet). This is a concrete,
|
||||||
|
explainable proxy for "how much of the swarm's output is in agreement",
|
||||||
|
derived only from real task states and detected conflicts.
|
||||||
|
"""
|
||||||
|
completed = [t for t in tasks if _task_status(t) in _TASK_TERMINAL_OK]
|
||||||
|
if not completed:
|
||||||
|
return 0.0
|
||||||
|
conflicted_ids = set()
|
||||||
|
for conflict in conflicts:
|
||||||
|
conflicted_ids.update(conflict.task_ids)
|
||||||
|
agreeing = [t for t in completed if _task_id(t) not in conflicted_ids]
|
||||||
|
return round(100.0 * len(agreeing) / len(completed), 2)
|
||||||
|
|
||||||
|
|
||||||
|
# --- the termination function ---------------------------------------------
|
||||||
|
|
||||||
|
def evaluate_convergence(run_state: Dict[str, Any]) -> ConvergenceReport:
|
||||||
|
"""Derive a ConvergenceReport from a run-state snapshot (pure, no I/O).
|
||||||
|
|
||||||
|
`run_state` is a plain dict (the caller assembles it from SwarmRun + tasks):
|
||||||
|
tasks: list of tasks (dicts or models) with status/result/depends_on
|
||||||
|
review_verdict: optional master critic verdict
|
||||||
|
budget: optional budget limits
|
||||||
|
usage: optional consumed usage
|
||||||
|
quality: optional Group B fixture grade
|
||||||
|
risks: optional list of blocking risks
|
||||||
|
review_cycles: int cycles already run
|
||||||
|
max_review_cycles: int cap
|
||||||
|
acceptance_threshold:optional quality threshold (default 1.0)
|
||||||
|
|
||||||
|
Decision order (first match wins) for terminal runs:
|
||||||
|
1. Any active/blocked task -> RUNNING (no termination_reason yet)
|
||||||
|
2. Any blocking risk / unresolved hard conflict -> BLOCKED, RISK_BLOCKED
|
||||||
|
3. Any failed task -> FAILED, BUDGET_EXHAUSTED if budget
|
||||||
|
spent else MAX_ROUNDS_REACHED if the
|
||||||
|
redo cap was hit else TASKS_COMPLETED
|
||||||
|
4. Budget exhausted (all tasks done) -> CONVERGED, BUDGET_EXHAUSTED
|
||||||
|
5. Review cap hit -> CONVERGED, MAX_ROUNDS_REACHED
|
||||||
|
6. Quality gate reached -> CONVERGED, QUALITY_REACHED
|
||||||
|
7. Otherwise (all tasks done) -> CONVERGED, TASKS_COMPLETED (the honest
|
||||||
|
fallback == today's only real signal)
|
||||||
|
|
||||||
|
Every terminal report carries exactly one non-None termination_reason.
|
||||||
|
"""
|
||||||
|
tasks = list(run_state.get("tasks") or [])
|
||||||
|
review_verdict = run_state.get("review_verdict")
|
||||||
|
|
||||||
|
conflicts = detect_conflicts(tasks, review_verdict)
|
||||||
|
resolved, unresolved = resolve_conflicts(conflicts, review_verdict)
|
||||||
|
consensus_score = compute_consensus_score(tasks, conflicts)
|
||||||
|
budget_state = derive_budget_state(run_state)
|
||||||
|
quality_state = derive_quality_state(run_state)
|
||||||
|
unresolved_risks = derive_unresolved_risks(run_state, unresolved)
|
||||||
|
|
||||||
|
def _report(status: ConvergenceStatus, reason: Optional[TerminationReason]) -> ConvergenceReport:
|
||||||
|
return ConvergenceReport(
|
||||||
|
status=status,
|
||||||
|
termination_reason=reason,
|
||||||
|
consensus_score=consensus_score,
|
||||||
|
conflicts=conflicts,
|
||||||
|
resolved_conflicts=resolved,
|
||||||
|
unresolved_risks=unresolved_risks,
|
||||||
|
budget_state=budget_state,
|
||||||
|
quality_state=quality_state,
|
||||||
|
)
|
||||||
|
|
||||||
|
statuses = [_task_status(t) for t in tasks]
|
||||||
|
|
||||||
|
# 1. still running — not terminal yet.
|
||||||
|
if not tasks or any(s in _TASK_ACTIVE for s in statuses):
|
||||||
|
return _report(ConvergenceStatus.RUNNING, None)
|
||||||
|
|
||||||
|
# A blocked task with no unresolved risk is still "in flight" (mirrors
|
||||||
|
# refresh_swarm_run_status keeping blocked-with-active-child runs running).
|
||||||
|
# 2. blocking risk / unresolved hard conflict -> blocked.
|
||||||
|
if unresolved_risks:
|
||||||
|
return _report(ConvergenceStatus.BLOCKED, TerminationReason.RISK_BLOCKED)
|
||||||
|
|
||||||
|
if any(s in _TASK_BLOCKED for s in statuses):
|
||||||
|
return _report(ConvergenceStatus.RUNNING, None)
|
||||||
|
|
||||||
|
cycles = int(run_state.get("review_cycles", 0) or 0)
|
||||||
|
max_cycles = int(run_state.get("max_review_cycles", 0) or 0)
|
||||||
|
rounds_hit = max_cycles > 0 and cycles >= max_cycles
|
||||||
|
|
||||||
|
# 3. a failed task -> failed run, with the best available explanation.
|
||||||
|
if any(s in _TASK_TERMINAL_FAIL for s in statuses):
|
||||||
|
if budget_state.get("exhausted"):
|
||||||
|
reason = TerminationReason.BUDGET_EXHAUSTED
|
||||||
|
elif rounds_hit:
|
||||||
|
reason = TerminationReason.MAX_ROUNDS_REACHED
|
||||||
|
else:
|
||||||
|
reason = TerminationReason.TASKS_COMPLETED
|
||||||
|
return _report(ConvergenceStatus.FAILED, reason)
|
||||||
|
|
||||||
|
# All tasks completed. Explain WHY it stopped, most-specific reason first.
|
||||||
|
# 4. budget exhausted.
|
||||||
|
if budget_state.get("exhausted"):
|
||||||
|
return _report(ConvergenceStatus.CONVERGED, TerminationReason.BUDGET_EXHAUSTED)
|
||||||
|
# 5. review/redo cap hit.
|
||||||
|
if rounds_hit:
|
||||||
|
return _report(ConvergenceStatus.CONVERGED, TerminationReason.MAX_ROUNDS_REACHED)
|
||||||
|
# 6. explicit quality gate satisfied.
|
||||||
|
if quality_state.get("reached"):
|
||||||
|
return _report(ConvergenceStatus.CONVERGED, TerminationReason.QUALITY_REACHED)
|
||||||
|
# 7. honest fallback: nothing else to explain beyond "all tasks done".
|
||||||
|
return _report(ConvergenceStatus.CONVERGED, TerminationReason.TASKS_COMPLETED)
|
||||||
|
|
||||||
|
|
||||||
|
# --- event payload builders -----------------------------------------------
|
||||||
|
# Match SwarmRuntime.emit_event(run, event_type, payload=...) convention: each
|
||||||
|
# builder returns (event_type, payload) so a caller can splat it. Payloads are
|
||||||
|
# plain dicts of JSON-safe primitives. These event types are NOT yet in the
|
||||||
|
# Manager event registry — see the doc's integration notes before subscribing.
|
||||||
|
|
||||||
|
def event_convergence_started(run_state: Dict[str, Any]) -> tuple[str, Dict[str, Any]]:
|
||||||
|
"""convergence.started — emitted once when the convergence pass begins."""
|
||||||
|
return "convergence.started", {
|
||||||
|
"summary": "Convergence evaluation started",
|
||||||
|
"task_count": len(run_state.get("tasks") or []),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def event_conflict_detected(conflict: Conflict) -> tuple[str, Dict[str, Any]]:
|
||||||
|
"""conflict.detected — one per detected conflict."""
|
||||||
|
return "conflict.detected", {
|
||||||
|
"summary": conflict.description,
|
||||||
|
**conflict.to_dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def event_conflict_resolved(conflict: Conflict) -> tuple[str, Dict[str, Any]]:
|
||||||
|
"""conflict.resolved — one per conflict that resolve_conflicts closed."""
|
||||||
|
return "conflict.resolved", {
|
||||||
|
"summary": conflict.resolution or "Conflict resolved",
|
||||||
|
**conflict.to_dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def event_consensus_updated(report: ConvergenceReport) -> tuple[str, Dict[str, Any]]:
|
||||||
|
"""consensus.updated — current consensus score and conflict tallies."""
|
||||||
|
return "consensus.updated", {
|
||||||
|
"summary": f"Consensus {report.consensus_score}%",
|
||||||
|
"consensus_score": report.consensus_score,
|
||||||
|
"conflicts_total": len(report.conflicts),
|
||||||
|
"conflicts_resolved": len(report.resolved_conflicts),
|
||||||
|
"conflicts_unresolved": len(report.unresolved_risks),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def event_convergence_reached(report: ConvergenceReport) -> tuple[str, Dict[str, Any]]:
|
||||||
|
"""convergence.reached — terminal success with its termination_reason."""
|
||||||
|
return "convergence.reached", {
|
||||||
|
"summary": "Swarm converged",
|
||||||
|
**report.to_dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def event_convergence_failed(report: ConvergenceReport) -> tuple[str, Dict[str, Any]]:
|
||||||
|
"""convergence.failed — terminal failure/block with its termination_reason."""
|
||||||
|
return "convergence.failed", {
|
||||||
|
"summary": f"Swarm did not converge: {report.status.value}",
|
||||||
|
**report.to_dict(),
|
||||||
|
}
|
||||||
@@ -0,0 +1,448 @@
|
|||||||
|
"""Cross-review protocol — multi-reviewer swarm validation closure (issue #11).
|
||||||
|
|
||||||
|
WHY THIS EXISTS
|
||||||
|
---------------
|
||||||
|
Today's "review loop" (``orchestrator/main.py:maybe_run_review_cycle`` +
|
||||||
|
``master_agent.review_and_decide`` + ``planner.review``) is a *single*-critic gate: the
|
||||||
|
Master judges the combined result pass/fail and reopens whatever ``retry_tasks`` it names.
|
||||||
|
Functionally that is equivalent to a Supervisor Retry — one authority decides, the workers
|
||||||
|
redo. There is no second, independent opinion; no recorded disagreement; no structured
|
||||||
|
attribution of *why* the rework was needed and *who* introduced the fault. So the loop never
|
||||||
|
forms a swarm-style cross-validation closure, and it cannot feed a faithful ``P_rework`` input
|
||||||
|
to the benchmark (``docs/benchmark/swarm-metrics-schema.md`` §2).
|
||||||
|
|
||||||
|
WHAT THIS MODULE ADDS
|
||||||
|
---------------------
|
||||||
|
A pure, side-effect-free protocol layer (no Redis, no WebSocket, no model calls):
|
||||||
|
|
||||||
|
* ``ReviewDecision`` — one reviewer's structured verdict with evidence + rework target.
|
||||||
|
* ``aggregate_reviews`` — combine >= 2 independent reviewers, detect disagreement, and
|
||||||
|
arbitrate (majority, then reviewer-weight tie-break), recording
|
||||||
|
the disagreement rather than hiding it.
|
||||||
|
* ``ReworkAttribution`` + ``classify_rework`` — attribute a rework to its root cause and
|
||||||
|
source task/agent, and classify it into a ``ReworkCategory``
|
||||||
|
usable as a ``P_rework`` input.
|
||||||
|
* event payload builders — ``review.started`` / ``review.decision_made`` /
|
||||||
|
``rework.requested`` / ``rework.completed`` shaped like the
|
||||||
|
existing ``swarm_runtime.emit_event`` payloads.
|
||||||
|
|
||||||
|
This module is INTENTIONALLY NOT WIRED into the live finalize path. Integration (how it would
|
||||||
|
replace/augment the Master review, behind ``ENABLE_CROSS_REVIEW``) is documented in
|
||||||
|
``docs/swarm/review-loop-protocol.md`` under "集成说明(Integration notes)". See that doc for
|
||||||
|
the honest non-integration statement (org honesty rule #9).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from collections import Counter
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
# Rework taxonomy (P_rework input — benchmark schema §2)
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
class ReworkCategory(str, Enum):
|
||||||
|
"""Why a piece of work had to be redone.
|
||||||
|
|
||||||
|
These categories partition the ``ReworkCount`` that feeds ``P_rework =
|
||||||
|
ReworkCount/TotalTasks*100`` (``docs/benchmark/swarm-metrics-schema.md`` §2). Splitting by
|
||||||
|
cause lets the collector attribute rework to the phase that introduced it instead of
|
||||||
|
treating every redo as an undifferentiated retry.
|
||||||
|
"""
|
||||||
|
|
||||||
|
REQUIREMENT = "requirement" # objective / spec misunderstanding
|
||||||
|
IMPLEMENTATION = "implementation" # impl defect: wrong/missing behavior
|
||||||
|
TEST = "test" # test defect: wrong assertions / framework mismatch
|
||||||
|
DOC = "doc" # documentation drift vs. implementation
|
||||||
|
COLLABORATION = "collaboration" # cross-specialist inconsistency / handoff gap
|
||||||
|
UNKNOWN = "unknown" # no signal — do NOT fabricate a cause (honesty rule #9)
|
||||||
|
|
||||||
|
|
||||||
|
# Keyword signals per category. Deterministic and explainable: the same heuristic family the
|
||||||
|
# existing planner._heuristic_consistency_check uses (e.g. pytest/unittest, ValueError clash).
|
||||||
|
_CATEGORY_SIGNALS: Dict[ReworkCategory, tuple] = {
|
||||||
|
ReworkCategory.REQUIREMENT: ("requirement", "objective", "scope", "misunderstood", "spec"),
|
||||||
|
ReworkCategory.IMPLEMENTATION: ("implementation", "impl", "bug", "logic", "incorrect behavior",
|
||||||
|
"wrong output", "raise", "exception"),
|
||||||
|
ReworkCategory.TEST: ("test", "pytest", "unittest", "assert", "coverage", "fixture"),
|
||||||
|
ReworkCategory.DOC: ("doc", "documentation", "readme", "docstring", "example"),
|
||||||
|
ReworkCategory.COLLABORATION: ("conflict", "inconsistent", "mismatch", "disagree",
|
||||||
|
"handoff", "between specialists", "across specialists"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
# A single reviewer's verdict
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
@dataclass
|
||||||
|
class ReviewDecision:
|
||||||
|
"""One independent reviewer's structured verdict on a swarm artifact.
|
||||||
|
|
||||||
|
Unlike today's Master verdict (a bare ``{accepted, summary, retry_tasks}`` dict), this
|
||||||
|
carries the *evidence* the verdict rests on, which acceptance *criteria* failed, which tasks
|
||||||
|
are *affected*, the *recommended rework*, and the *reviewer identity* — so >= 2 reviewers can
|
||||||
|
be cross-checked and any disagreement can be located and recorded.
|
||||||
|
"""
|
||||||
|
|
||||||
|
verdict: str # "pass" | "fail"
|
||||||
|
reviewer_agent_id: str
|
||||||
|
evidence: List[str] = field(default_factory=list)
|
||||||
|
failed_criteria: List[str] = field(default_factory=list)
|
||||||
|
affected_tasks: List[str] = field(default_factory=list)
|
||||||
|
recommended_rework: List[str] = field(default_factory=list)
|
||||||
|
confidence: float = 1.0 # reviewer self-confidence in [0,1]
|
||||||
|
weight: float = 1.0 # arbitration weight (e.g. seniority/role trust)
|
||||||
|
summary: str = ""
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
normalized = (self.verdict or "").strip().lower()
|
||||||
|
if normalized not in {"pass", "fail"}:
|
||||||
|
raise ValueError(f"verdict must be 'pass' or 'fail', got {self.verdict!r}")
|
||||||
|
self.verdict = normalized
|
||||||
|
if not self.reviewer_agent_id:
|
||||||
|
raise ValueError("reviewer_agent_id is required")
|
||||||
|
try:
|
||||||
|
self.confidence = max(0.0, min(1.0, float(self.confidence)))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
self.confidence = 1.0
|
||||||
|
try:
|
||||||
|
self.weight = max(0.0, float(self.weight))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
self.weight = 1.0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def passed(self) -> bool:
|
||||||
|
return self.verdict == "pass"
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"verdict": self.verdict,
|
||||||
|
"reviewer_agent_id": self.reviewer_agent_id,
|
||||||
|
"evidence": list(self.evidence),
|
||||||
|
"failed_criteria": list(self.failed_criteria),
|
||||||
|
"affected_tasks": list(self.affected_tasks),
|
||||||
|
"recommended_rework": list(self.recommended_rework),
|
||||||
|
"confidence": self.confidence,
|
||||||
|
"weight": self.weight,
|
||||||
|
"summary": self.summary,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
# Rework attribution (who/what caused the redo)
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
@dataclass
|
||||||
|
class ReworkAttribution:
|
||||||
|
"""Attribution record for one requested rework.
|
||||||
|
|
||||||
|
Answers *why* (``rework_reason``), *what kind* (``root_cause`` -> ``ReworkCategory``), and
|
||||||
|
*whose work introduced it* (``source_task_id`` / ``introduced_by_agent_id``). This is the
|
||||||
|
structured input the benchmark collector needs to compute a meaningful ``P_rework`` and to
|
||||||
|
split rework by phase instead of by blind retry count.
|
||||||
|
"""
|
||||||
|
|
||||||
|
target_task_id: str # task being reopened / redone
|
||||||
|
rework_reason: str # human-readable cause
|
||||||
|
root_cause: ReworkCategory = ReworkCategory.UNKNOWN
|
||||||
|
source_task_id: Optional[str] = None # upstream task that introduced the fault
|
||||||
|
introduced_by_agent_id: Optional[str] = None # agent whose output introduced the fault
|
||||||
|
detected_by_agent_id: Optional[str] = None # reviewer who flagged it
|
||||||
|
evidence: List[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"target_task_id": self.target_task_id,
|
||||||
|
"rework_reason": self.rework_reason,
|
||||||
|
"root_cause": self.root_cause.value,
|
||||||
|
"source_task_id": self.source_task_id,
|
||||||
|
"introduced_by_agent_id": self.introduced_by_agent_id,
|
||||||
|
"detected_by_agent_id": self.detected_by_agent_id,
|
||||||
|
"evidence": list(self.evidence),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def classify_rework(
|
||||||
|
reason: str,
|
||||||
|
failed_criteria: Optional[List[str]] = None,
|
||||||
|
*,
|
||||||
|
disagreement: bool = False,
|
||||||
|
) -> ReworkCategory:
|
||||||
|
"""Classify a rework into a :class:`ReworkCategory` from its textual signals.
|
||||||
|
|
||||||
|
Deterministic keyword scoring (no model). When two reviewers disagreed on the same
|
||||||
|
artifact, the rework is biased toward ``COLLABORATION`` because a cross-specialist
|
||||||
|
inconsistency is the most common root of reviewer disagreement. Returns ``UNKNOWN`` when no
|
||||||
|
signal matches — it never fabricates a cause (honesty rule #9).
|
||||||
|
"""
|
||||||
|
text = " ".join([reason or ""] + list(failed_criteria or [])).lower()
|
||||||
|
if not text.strip():
|
||||||
|
return ReworkCategory.COLLABORATION if disagreement else ReworkCategory.UNKNOWN
|
||||||
|
|
||||||
|
scores: Counter = Counter()
|
||||||
|
for category, signals in _CATEGORY_SIGNALS.items():
|
||||||
|
for signal in signals:
|
||||||
|
if signal in text:
|
||||||
|
scores[category] += 1
|
||||||
|
|
||||||
|
if disagreement:
|
||||||
|
# A recorded disagreement is itself evidence of a collaboration/consistency gap, so it
|
||||||
|
# both adds a vote AND wins ties: when reviewers split, the cross-specialist
|
||||||
|
# inconsistency is the root we want surfaced over any single-phase signal.
|
||||||
|
scores[ReworkCategory.COLLABORATION] += 1
|
||||||
|
|
||||||
|
if not scores:
|
||||||
|
return ReworkCategory.UNKNOWN
|
||||||
|
|
||||||
|
def _rank(kv):
|
||||||
|
category, score = kv
|
||||||
|
# Highest score wins. On a tie, prefer COLLABORATION when reviewers disagreed; otherwise
|
||||||
|
# fall back to a stable category order for reproducibility.
|
||||||
|
collab_tiebreak = 1 if (disagreement and category is ReworkCategory.COLLABORATION) else 0
|
||||||
|
return (score, collab_tiebreak, -list(ReworkCategory).index(category))
|
||||||
|
|
||||||
|
best = max(scores.items(), key=_rank)
|
||||||
|
return best[0]
|
||||||
|
|
||||||
|
|
||||||
|
def build_rework_attributions(
|
||||||
|
verdict: "AggregatedVerdict",
|
||||||
|
*,
|
||||||
|
task_owner: Optional[Dict[str, str]] = None,
|
||||||
|
) -> List[ReworkAttribution]:
|
||||||
|
"""Build one :class:`ReworkAttribution` per task the arbitrated verdict wants redone.
|
||||||
|
|
||||||
|
``task_owner`` maps task_id -> agent_id so the introducing agent can be attributed; when a
|
||||||
|
task's owner is unknown the field stays ``None`` rather than being guessed.
|
||||||
|
"""
|
||||||
|
task_owner = task_owner or {}
|
||||||
|
attributions: List[ReworkAttribution] = []
|
||||||
|
# Prefer the failing reviewers' evidence/criteria as the reason source.
|
||||||
|
failing = [d for d in verdict.decisions if not d.passed]
|
||||||
|
reason_bits = []
|
||||||
|
failed_criteria: List[str] = []
|
||||||
|
detected_by: Optional[str] = None
|
||||||
|
evidence: List[str] = []
|
||||||
|
for d in failing:
|
||||||
|
if d.summary:
|
||||||
|
reason_bits.append(d.summary)
|
||||||
|
failed_criteria.extend(d.failed_criteria)
|
||||||
|
evidence.extend(d.evidence)
|
||||||
|
detected_by = detected_by or d.reviewer_agent_id
|
||||||
|
reason = "; ".join(reason_bits) or verdict.summary or "rework requested by cross-review"
|
||||||
|
|
||||||
|
for task_id in verdict.rework_targets:
|
||||||
|
category = classify_rework(reason, failed_criteria, disagreement=verdict.disagreement)
|
||||||
|
attributions.append(
|
||||||
|
ReworkAttribution(
|
||||||
|
target_task_id=task_id,
|
||||||
|
rework_reason=reason,
|
||||||
|
root_cause=category,
|
||||||
|
source_task_id=task_id,
|
||||||
|
introduced_by_agent_id=task_owner.get(task_id),
|
||||||
|
detected_by_agent_id=detected_by,
|
||||||
|
evidence=list(dict.fromkeys(evidence)), # de-dup, keep order
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return attributions
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
# Aggregated, arbitrated verdict across >= 2 reviewers
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
@dataclass
|
||||||
|
class AggregatedVerdict:
|
||||||
|
"""The cross-review outcome after combining and arbitrating independent reviewers."""
|
||||||
|
|
||||||
|
accepted: bool
|
||||||
|
method: str # arbitration method actually used
|
||||||
|
disagreement: bool # reviewers did not unanimously agree
|
||||||
|
pass_votes: int
|
||||||
|
fail_votes: int
|
||||||
|
rework_targets: List[str] = field(default_factory=list)
|
||||||
|
decisions: List[ReviewDecision] = field(default_factory=list)
|
||||||
|
summary: str = ""
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"accepted": self.accepted,
|
||||||
|
"method": self.method,
|
||||||
|
"disagreement": self.disagreement,
|
||||||
|
"pass_votes": self.pass_votes,
|
||||||
|
"fail_votes": self.fail_votes,
|
||||||
|
"rework_targets": list(self.rework_targets),
|
||||||
|
"summary": self.summary,
|
||||||
|
"decisions": [d.to_dict() for d in self.decisions],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def aggregate_reviews(
|
||||||
|
reviews: List[ReviewDecision],
|
||||||
|
*,
|
||||||
|
method: str = "majority",
|
||||||
|
) -> AggregatedVerdict:
|
||||||
|
"""Combine >= 2 independent reviewers into one arbitrated verdict.
|
||||||
|
|
||||||
|
This is the core difference from today's single-critic gate. It:
|
||||||
|
1. Requires at least two reviewers (a single reviewer is a Supervisor Retry, not a
|
||||||
|
cross-review) — raises ``ValueError`` otherwise.
|
||||||
|
2. Detects *disagreement* (reviewers split on pass/fail) and records it explicitly.
|
||||||
|
3. Arbitrates:
|
||||||
|
- ``method="majority"``: more fails than passes -> reject. A pass/fail tie is
|
||||||
|
resolved conservatively as a rejection (safety-biased: never silently accept a
|
||||||
|
split decision).
|
||||||
|
- ``method="weighted"``: compares Σ(weight·confidence) of pass vs. fail reviewers; the
|
||||||
|
heavier side wins, ties -> reject.
|
||||||
|
4. Unions the ``recommended_rework``/``affected_tasks`` of the *failing* reviewers into the
|
||||||
|
rework target set, so every concern raised is acted on.
|
||||||
|
|
||||||
|
Acceptance requires no rework targets AND a non-reject arbitration outcome.
|
||||||
|
"""
|
||||||
|
if len(reviews) < 2:
|
||||||
|
raise ValueError("cross-review requires at least 2 independent reviewers")
|
||||||
|
|
||||||
|
pass_votes = sum(1 for d in reviews if d.passed)
|
||||||
|
fail_votes = len(reviews) - pass_votes
|
||||||
|
disagreement = pass_votes > 0 and fail_votes > 0
|
||||||
|
|
||||||
|
if method == "weighted":
|
||||||
|
pass_w = sum(d.weight * d.confidence for d in reviews if d.passed)
|
||||||
|
fail_w = sum(d.weight * d.confidence for d in reviews if not d.passed)
|
||||||
|
# Tie or fail-heavy -> reject (safety bias).
|
||||||
|
accepted = pass_w > fail_w
|
||||||
|
used_method = "weighted"
|
||||||
|
else:
|
||||||
|
# Majority; tie -> reject (safety bias).
|
||||||
|
accepted = pass_votes > fail_votes
|
||||||
|
used_method = "majority"
|
||||||
|
|
||||||
|
rework_targets: List[str] = []
|
||||||
|
for d in reviews:
|
||||||
|
if d.passed:
|
||||||
|
continue
|
||||||
|
for tid in list(d.recommended_rework) + list(d.affected_tasks):
|
||||||
|
if tid and tid not in rework_targets:
|
||||||
|
rework_targets.append(tid)
|
||||||
|
|
||||||
|
# A reject with no nameable target still must not be silently accepted; surface it so the
|
||||||
|
# caller can decide (the live loop would, e.g., reopen all completed tasks or stop).
|
||||||
|
if not accepted and not rework_targets:
|
||||||
|
rework_targets = [] # explicit: empty target set, accepted stays False
|
||||||
|
|
||||||
|
accepted = accepted and not rework_targets
|
||||||
|
|
||||||
|
if disagreement:
|
||||||
|
summary = (
|
||||||
|
f"reviewers disagreed ({pass_votes} pass / {fail_votes} fail); "
|
||||||
|
f"arbitrated by {used_method} -> {'accept' if accepted else 'reject'}"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
summary = (
|
||||||
|
f"reviewers unanimous ({pass_votes} pass / {fail_votes} fail) -> "
|
||||||
|
f"{'accept' if accepted else 'reject'}"
|
||||||
|
)
|
||||||
|
|
||||||
|
logger.info("[cross_review] %s; rework_targets=%s", summary, rework_targets)
|
||||||
|
return AggregatedVerdict(
|
||||||
|
accepted=accepted,
|
||||||
|
method=used_method,
|
||||||
|
disagreement=disagreement,
|
||||||
|
pass_votes=pass_votes,
|
||||||
|
fail_votes=fail_votes,
|
||||||
|
rework_targets=rework_targets,
|
||||||
|
decisions=list(reviews),
|
||||||
|
summary=summary,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
# Event payload builders (shape matches swarm_runtime.emit_event payloads)
|
||||||
|
# --------------------------------------------------------------------------------------
|
||||||
|
def review_started_payload(
|
||||||
|
run_id: str,
|
||||||
|
*,
|
||||||
|
reviewer_agent_ids: List[str],
|
||||||
|
artifact_task_ids: List[str],
|
||||||
|
cycle: int = 0,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Payload for a ``review.started`` event (cross-review round begins)."""
|
||||||
|
return {
|
||||||
|
"swarm_id": run_id,
|
||||||
|
"phase": "Review",
|
||||||
|
"review_kind": "cross_review",
|
||||||
|
"cycle": cycle,
|
||||||
|
"reviewer_agent_ids": list(reviewer_agent_ids),
|
||||||
|
"artifact_task_ids": list(artifact_task_ids),
|
||||||
|
"reviewer_count": len(reviewer_agent_ids),
|
||||||
|
"summary": f"Cross-review round {cycle} started with {len(reviewer_agent_ids)} reviewers",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def review_decision_made_payload(
|
||||||
|
run_id: str,
|
||||||
|
verdict: AggregatedVerdict,
|
||||||
|
*,
|
||||||
|
cycle: int = 0,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Payload for a ``review.decision_made`` event (arbitrated verdict produced)."""
|
||||||
|
return {
|
||||||
|
"swarm_id": run_id,
|
||||||
|
"phase": "Review",
|
||||||
|
"review_kind": "cross_review",
|
||||||
|
"cycle": cycle,
|
||||||
|
"accepted": verdict.accepted,
|
||||||
|
"arbitration_method": verdict.method,
|
||||||
|
"disagreement": verdict.disagreement,
|
||||||
|
"pass_votes": verdict.pass_votes,
|
||||||
|
"fail_votes": verdict.fail_votes,
|
||||||
|
"rework_targets": list(verdict.rework_targets),
|
||||||
|
"reviews": [d.to_dict() for d in verdict.decisions],
|
||||||
|
"summary": verdict.summary,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def rework_requested_payload(
|
||||||
|
run_id: str,
|
||||||
|
attribution: ReworkAttribution,
|
||||||
|
*,
|
||||||
|
cycle: int = 0,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Payload for a ``rework.requested`` event (one task sent back for redo)."""
|
||||||
|
return {
|
||||||
|
"swarm_id": run_id,
|
||||||
|
"phase": "Review",
|
||||||
|
"cycle": cycle,
|
||||||
|
"task_id": attribution.target_task_id,
|
||||||
|
"rework_reason": attribution.rework_reason,
|
||||||
|
"root_cause": attribution.root_cause.value,
|
||||||
|
"source_task_id": attribution.source_task_id,
|
||||||
|
"introduced_by_agent_id": attribution.introduced_by_agent_id,
|
||||||
|
"detected_by_agent_id": attribution.detected_by_agent_id,
|
||||||
|
"evidence": list(attribution.evidence),
|
||||||
|
"summary": f"Rework requested for {attribution.target_task_id} "
|
||||||
|
f"({attribution.root_cause.value})",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def rework_completed_payload(
|
||||||
|
run_id: str,
|
||||||
|
attribution: ReworkAttribution,
|
||||||
|
*,
|
||||||
|
cycle: int = 0,
|
||||||
|
succeeded: bool = True,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Payload for a ``rework.completed`` event (a redone task reached a terminal state)."""
|
||||||
|
return {
|
||||||
|
"swarm_id": run_id,
|
||||||
|
"phase": "Review",
|
||||||
|
"cycle": cycle,
|
||||||
|
"task_id": attribution.target_task_id,
|
||||||
|
"root_cause": attribution.root_cause.value,
|
||||||
|
"introduced_by_agent_id": attribution.introduced_by_agent_id,
|
||||||
|
"status": "completed" if succeeded else "failed",
|
||||||
|
"summary": f"Rework {'completed' if succeeded else 'failed'} for "
|
||||||
|
f"{attribution.target_task_id}",
|
||||||
|
}
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
"""Swarm health guard — detect when the swarm CANNOT function, and say why.
|
||||||
|
|
||||||
|
The decentralized swarm has no central controller to fall back on, so the one piece of
|
||||||
|
non-happy-path handling it keeps is a GUARD: a pure diagnostic that inspects a run snapshot and
|
||||||
|
reports concrete blockers when the swarm cannot make progress. It does NOT steer the run (it is
|
||||||
|
advisory) — the orchestrator stores the report and surfaces it so an operator/Manager can see
|
||||||
|
*why* a run is stuck instead of watching it hang.
|
||||||
|
|
||||||
|
Blockers detected (each with a human-readable detail; never a fabricated cause — rule #9):
|
||||||
|
- NO_AGENTS_CONNECTED — pending work but no connected agent at all.
|
||||||
|
- NO_CAPABLE_AGENT — a ready task whose required capabilities no connected agent covers.
|
||||||
|
- DEPENDENCY_DEADLOCK — a pending task can never run: a dependency FAILED, is missing, or
|
||||||
|
forms a cycle.
|
||||||
|
- BUDGET_EXHAUSTED — the run's budget is spent while work remains.
|
||||||
|
- SEED_UNDECOMPOSED — only the seed exists and it is terminal, yet no subtasks were
|
||||||
|
proposed (the swarm produced nothing to do).
|
||||||
|
|
||||||
|
Pure: no Redis/WebSocket/model. The orchestrator assembles the snapshot and calls `diagnose`.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, Dict, List, Sequence, Set
|
||||||
|
|
||||||
|
|
||||||
|
class Blocker(str, Enum):
|
||||||
|
NO_AGENTS_CONNECTED = "no_agents_connected"
|
||||||
|
NO_CAPABLE_AGENT = "no_capable_agent"
|
||||||
|
DEPENDENCY_DEADLOCK = "dependency_deadlock"
|
||||||
|
BUDGET_EXHAUSTED = "budget_exhausted"
|
||||||
|
SEED_UNDECOMPOSED = "seed_undecomposed"
|
||||||
|
|
||||||
|
|
||||||
|
_ACTIVE = {"pending", "assigned", "in_progress", "blocked"}
|
||||||
|
_PENDING = {"pending"}
|
||||||
|
_FAILED = {"failed"}
|
||||||
|
_OK = {"completed"}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class HealthReport:
|
||||||
|
healthy: bool
|
||||||
|
blockers: List[Dict[str, Any]] = field(default_factory=list)
|
||||||
|
summary: str = ""
|
||||||
|
|
||||||
|
def to_dict(self) -> Dict[str, Any]:
|
||||||
|
return {"healthy": self.healthy, "blockers": list(self.blockers), "summary": self.summary}
|
||||||
|
|
||||||
|
|
||||||
|
def _status(task: Any) -> str:
|
||||||
|
s = task.get("status") if isinstance(task, dict) else getattr(task, "status", None)
|
||||||
|
return str(getattr(s, "value", s) or "").lower()
|
||||||
|
|
||||||
|
|
||||||
|
def _field(task: Any, name: str, default):
|
||||||
|
return task.get(name, default) if isinstance(task, dict) else getattr(task, name, default)
|
||||||
|
|
||||||
|
|
||||||
|
def diagnose(run_state: Dict[str, Any]) -> HealthReport:
|
||||||
|
"""Inspect a run snapshot and report why the swarm cannot make progress (pure).
|
||||||
|
|
||||||
|
run_state:
|
||||||
|
tasks: list of {task_id, status, required_capabilities, depends_on, source}
|
||||||
|
connected_agent_caps: list of capability lists, one per connected agent
|
||||||
|
budget_state: optional {exhausted: bool, ...} (from convergence.derive_budget_state)
|
||||||
|
"""
|
||||||
|
tasks = list(run_state.get("tasks") or [])
|
||||||
|
agent_caps: List[Set[str]] = [set(c or []) for c in (run_state.get("connected_agent_caps") or [])]
|
||||||
|
budget_state = run_state.get("budget_state") or {}
|
||||||
|
|
||||||
|
by_id = {_field(t, "task_id", ""): t for t in tasks}
|
||||||
|
active = [t for t in tasks if _status(t) in _ACTIVE]
|
||||||
|
blockers: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
# If nothing is active, the run is either done or empty — not "stuck". No blockers.
|
||||||
|
if not active:
|
||||||
|
# SEED_UNDECOMPOSED: the only task is a terminal seed and nothing else was produced.
|
||||||
|
if len(tasks) == 1 and _field(tasks[0], "source", "") == "seed" and _status(tasks[0]) in _OK:
|
||||||
|
# A completed seed with no proposed subtasks means the swarm did no real work.
|
||||||
|
blockers.append({
|
||||||
|
"reason": Blocker.SEED_UNDECOMPOSED.value,
|
||||||
|
"detail": "seed completed but no subtasks were proposed — the swarm produced no work",
|
||||||
|
})
|
||||||
|
return _finalize(blockers)
|
||||||
|
|
||||||
|
# 1. No agents at all.
|
||||||
|
if not agent_caps:
|
||||||
|
blockers.append({
|
||||||
|
"reason": Blocker.NO_AGENTS_CONNECTED.value,
|
||||||
|
"detail": f"{len(active)} task(s) need work but no agent is connected",
|
||||||
|
})
|
||||||
|
|
||||||
|
# 2. Per ready pending task: is any connected agent capable? + dependency deadlock.
|
||||||
|
for task in tasks:
|
||||||
|
if _status(task) not in _PENDING:
|
||||||
|
continue
|
||||||
|
required = set(_field(task, "required_capabilities", []) or [])
|
||||||
|
tid = _field(task, "task_id", "")
|
||||||
|
|
||||||
|
# dependency deadlock: a dep that failed / is missing / cycles → never satisfiable.
|
||||||
|
for dep_id in _field(task, "depends_on", []) or []:
|
||||||
|
dep = by_id.get(dep_id)
|
||||||
|
if dep is None:
|
||||||
|
blockers.append({"reason": Blocker.DEPENDENCY_DEADLOCK.value,
|
||||||
|
"detail": f"task {tid} depends on missing task {dep_id}"})
|
||||||
|
elif _status(dep) in _FAILED:
|
||||||
|
blockers.append({"reason": Blocker.DEPENDENCY_DEADLOCK.value,
|
||||||
|
"detail": f"task {tid} depends on FAILED task {dep_id} (can never complete)"})
|
||||||
|
elif dep_id == tid:
|
||||||
|
blockers.append({"reason": Blocker.DEPENDENCY_DEADLOCK.value,
|
||||||
|
"detail": f"task {tid} depends on itself (cycle)"})
|
||||||
|
|
||||||
|
# capability coverage (only meaningful when agents exist).
|
||||||
|
if agent_caps and required and not any(required <= caps for caps in agent_caps):
|
||||||
|
blockers.append({
|
||||||
|
"reason": Blocker.NO_CAPABLE_AGENT.value,
|
||||||
|
"detail": f"task {tid} requires {sorted(required)}; no connected agent covers it",
|
||||||
|
})
|
||||||
|
|
||||||
|
# 3. Budget exhausted while work remains.
|
||||||
|
if budget_state.get("exhausted"):
|
||||||
|
blockers.append({
|
||||||
|
"reason": Blocker.BUDGET_EXHAUSTED.value,
|
||||||
|
"detail": "run budget exhausted while tasks are still active",
|
||||||
|
})
|
||||||
|
|
||||||
|
return _finalize(blockers)
|
||||||
|
|
||||||
|
|
||||||
|
def _finalize(blockers: List[Dict[str, Any]]) -> HealthReport:
|
||||||
|
# De-dup identical blockers, keep order.
|
||||||
|
seen, deduped = set(), []
|
||||||
|
for b in blockers:
|
||||||
|
key = (b["reason"], b["detail"])
|
||||||
|
if key not in seen:
|
||||||
|
seen.add(key)
|
||||||
|
deduped.append(b)
|
||||||
|
if not deduped:
|
||||||
|
return HealthReport(healthy=True, summary="swarm healthy: progress is possible")
|
||||||
|
reasons = ", ".join(sorted({b["reason"] for b in deduped}))
|
||||||
|
return HealthReport(healthy=False, blockers=deduped,
|
||||||
|
summary=f"swarm blocked: {reasons}")
|
||||||
+555
-204
@@ -26,11 +26,16 @@ from .swarm_runtime import RuntimeValidationError, swarm_runtime
|
|||||||
from .planner import planner
|
from .planner import planner
|
||||||
from .master_agent import master_agent
|
from .master_agent import master_agent
|
||||||
from .quality import evaluate_run_quality
|
from .quality import evaluate_run_quality
|
||||||
from .decision_engine import decision_engine, aco_dispatch_enabled
|
from .decision_engine import decision_engine
|
||||||
from .dispatch_score import (
|
from .dispatch_score import (
|
||||||
DispatchCandidate, rank_candidates, build_dispatch_decision_event,
|
DispatchCandidate, rank_candidates, build_dispatch_decision_event,
|
||||||
normalize_capability_match, normalize_tau, normalize_load, dispatch_score_event_enabled,
|
normalize_capability_match, normalize_tau, normalize_load,
|
||||||
)
|
)
|
||||||
|
from . import convergence as convergence_mod
|
||||||
|
from . import autonomous_tasks as autonomous_mod
|
||||||
|
from . import task_competition as competition_mod
|
||||||
|
from . import cross_review as cross_review_mod
|
||||||
|
from . import guard as guard_mod
|
||||||
|
|
||||||
# Configure logging
|
# Configure logging
|
||||||
logging.basicConfig(
|
logging.basicConfig(
|
||||||
@@ -175,11 +180,6 @@ async def failure_detection_loop():
|
|||||||
logger.error(f"Error in failure detection loop: {e}")
|
logger.error(f"Error in failure detection loop: {e}")
|
||||||
|
|
||||||
|
|
||||||
def review_loop_enabled() -> bool:
|
|
||||||
"""Whether the master critic runs before a run is declared complete (default off)."""
|
|
||||||
return os.getenv("ENABLE_REVIEW_LOOP", "false").lower() in {"1", "true", "yes"}
|
|
||||||
|
|
||||||
|
|
||||||
def review_max_cycles() -> int:
|
def review_max_cycles() -> int:
|
||||||
try:
|
try:
|
||||||
return int(os.getenv("MAX_REVIEW_CYCLES", "2") or 2)
|
return int(os.getenv("MAX_REVIEW_CYCLES", "2") or 2)
|
||||||
@@ -240,57 +240,6 @@ async def build_dispatch_context(run, task) -> Dict[str, Any]:
|
|||||||
return context
|
return context
|
||||||
|
|
||||||
|
|
||||||
async def maybe_run_review_cycle(run, tasks) -> bool:
|
|
||||||
"""Run the master critic on completed work; reopen rejected tasks if budget remains.
|
|
||||||
|
|
||||||
Returns True when tasks were re-opened (the run stays 'running' and will re-finalize after
|
|
||||||
the redo completes), False when the work is accepted or the cycle budget is exhausted.
|
|
||||||
"""
|
|
||||||
cycles = int(run.metadata.get("review_cycles", 0) or 0)
|
|
||||||
if cycles >= review_max_cycles():
|
|
||||||
return False
|
|
||||||
|
|
||||||
completed = [t for t in tasks if t.status == TaskStatus.COMPLETED]
|
|
||||||
if not completed:
|
|
||||||
return False
|
|
||||||
|
|
||||||
results = {t.task_id: {"result": parse_task_result(t) or {}} for t in completed}
|
|
||||||
verdict = await master_agent.review_and_decide(
|
|
||||||
run.objective,
|
|
||||||
[task_payload(t) for t in completed],
|
|
||||||
results,
|
|
||||||
)
|
|
||||||
if verdict.get("accepted", True):
|
|
||||||
run.metadata["review_summary"] = verdict.get("summary")
|
|
||||||
await swarm_runtime.save_run(run)
|
|
||||||
return False
|
|
||||||
|
|
||||||
retry_tasks = [tid for tid in (verdict.get("retry_tasks") or []) if tid in run.task_ids]
|
|
||||||
reopened = [tid for tid in retry_tasks if await task_queue.reopen_task(tid)]
|
|
||||||
if not reopened:
|
|
||||||
# Rejected but nothing actionable to redo: accept rather than loop forever.
|
|
||||||
run.metadata["review_summary"] = verdict.get("summary")
|
|
||||||
await swarm_runtime.save_run(run)
|
|
||||||
return False
|
|
||||||
|
|
||||||
run.metadata["review_cycles"] = cycles + 1
|
|
||||||
run.metadata["review_summary"] = verdict.get("summary")
|
|
||||||
run.status = "running"
|
|
||||||
await swarm_runtime.save_run(run)
|
|
||||||
await swarm_runtime.emit_event(
|
|
||||||
run,
|
|
||||||
"deployment.status_changed",
|
|
||||||
payload=swarm_runtime.status_payload(run, phase="Review", reason=verdict.get("summary")),
|
|
||||||
)
|
|
||||||
await swarm_runtime.emit_event(run, "timeline.updated", payload={
|
|
||||||
"summary": f"Review cycle {cycles + 1}: {verdict.get('summary')}",
|
|
||||||
"status": "running",
|
|
||||||
"retry_tasks": reopened,
|
|
||||||
})
|
|
||||||
logger.info(f"Review rejected run {run.swarm_id}; reopened {reopened} (cycle {cycles + 1})")
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
async def task_dispatch_loop():
|
async def task_dispatch_loop():
|
||||||
"""Assign pending tasks to idle, connected agents."""
|
"""Assign pending tasks to idle, connected agents."""
|
||||||
while True:
|
while True:
|
||||||
@@ -308,44 +257,25 @@ async def task_dispatch_loop():
|
|||||||
and agent_has_capacity(agent.agent_id)
|
and agent_has_capacity(agent.agent_id)
|
||||||
]
|
]
|
||||||
|
|
||||||
# Issue #9: task-centric scored matchmaking — for each ready task, score the capable
|
# Swarm dispatch (the ONLY dispatch path): each idle agent perceives the shared task
|
||||||
# idle agents on ≥4 non-capability dimensions and assign the best, recording an
|
# pool and SELF-SELECTS the best-fit ready task by capability + pheromone(τ) + load +
|
||||||
# explainable dispatch.decision_made record. Gated (default OFF); when off, the
|
# budget, recording an explainable dispatch.decision_made. There is no central greedy
|
||||||
# agent-centric greedy/ACO paths below are byte-for-byte unchanged.
|
# assignment, no mode toggle — this repo is the swarm runtime (see rework plan §0).
|
||||||
if dispatch_score_enabled():
|
assigned = await swarm_dispatch(connected_idle_agents)
|
||||||
for agent, task, event_payload in await scored_matchmake(connected_idle_agents):
|
|
||||||
await task_queue.remove_pending_task(task.task_id)
|
|
||||||
await finalize_dispatch(agent, task, dispatch_event=event_payload)
|
|
||||||
continue
|
|
||||||
|
|
||||||
for agent in connected_idle_agents:
|
# P-guard: pending work but nothing got dispatched this tick → the swarm may be stuck.
|
||||||
decision = None
|
# Diagnose each affected run and record/emit a health report explaining why (no capable
|
||||||
if aco_dispatch_enabled():
|
# agent, dependency deadlock, budget, etc.). Advisory — does not change the run.
|
||||||
# Group A (Option A, score-at-pull): enumerate ALL eligible ready tasks for
|
if not assigned and await task_queue.get_pending_count() > 0:
|
||||||
# this agent and SAMPLE one with P=τ^α·η^β/Σ instead of taking the first
|
diagnosed = set()
|
||||||
# match. One-sided by design: the agent is fixed by arrival order.
|
for task in await task_queue.get_all_tasks():
|
||||||
candidates = await task_queue.get_ready_pending_tasks(agent.capabilities)
|
if task.status != TaskStatus.PENDING or task.task_id in diagnosed:
|
||||||
if not candidates:
|
continue
|
||||||
break
|
run = await swarm_runtime.get_run_for_task(task.task_id)
|
||||||
dependents_counts: Dict[str, int] = {}
|
if run and run.swarm_id not in diagnosed and run.status not in ("completed", "failed", "stopped"):
|
||||||
for t in await task_queue.get_all_tasks():
|
await assess_swarm_health(run)
|
||||||
for dep in t.depends_on:
|
diagnosed.add(run.swarm_id)
|
||||||
dependents_counts[dep] = dependents_counts.get(dep, 0) + 1
|
diagnosed.update(run.task_ids)
|
||||||
decision = await decision_engine.select(
|
|
||||||
agent.agent_id,
|
|
||||||
agent.capabilities,
|
|
||||||
candidates,
|
|
||||||
free_slots=AGENT_SLOTS.get(agent.agent_id, 1),
|
|
||||||
dependents_counts=dependents_counts,
|
|
||||||
)
|
|
||||||
task = next(t for t in candidates if t.task_id == decision.task_id)
|
|
||||||
await task_queue.remove_pending_task(task.task_id)
|
|
||||||
else:
|
|
||||||
task = await task_queue.get_ready_pending_task(agent.capabilities)
|
|
||||||
if not task:
|
|
||||||
break
|
|
||||||
|
|
||||||
await finalize_dispatch(agent, task, decision=decision)
|
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error in task dispatch loop: {e}")
|
logger.error(f"Error in task dispatch loop: {e}")
|
||||||
@@ -398,8 +328,39 @@ async def finalize_dispatch(agent, task, *, decision=None, dispatch_event=None)
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
def dispatch_score_enabled() -> bool:
|
async def compute_convergence_report(run, tasks, *, terminal: bool):
|
||||||
return os.getenv("ENABLE_DISPATCH_SCORE", "false").lower() in {"1", "true", "yes"}
|
"""Build a run-state snapshot and evaluate the convergence report (#12, shadow by default).
|
||||||
|
|
||||||
|
Pure-input wrapper over convergence.evaluate_convergence: assembles tasks (with parsed
|
||||||
|
results + depends_on), budget/usage, the Group B quality grade, and the review-cycle counters,
|
||||||
|
then returns the ConvergenceReport. The caller stores it on the run and may attach the
|
||||||
|
termination_reason — it does NOT (yet) override run.status (shadow adoption; see the plan doc).
|
||||||
|
"""
|
||||||
|
task_dicts = []
|
||||||
|
used_cost = 0.0
|
||||||
|
for t in tasks:
|
||||||
|
result = parse_task_result(t) or {}
|
||||||
|
try:
|
||||||
|
used_cost += float((result.get("usage") or {}).get("model_cost_usd") or 0.0)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
task_dicts.append({
|
||||||
|
"task_id": t.task_id,
|
||||||
|
"status": t.status.value if hasattr(t.status, "value") else t.status,
|
||||||
|
"depends_on": t.depends_on,
|
||||||
|
"result": result,
|
||||||
|
})
|
||||||
|
plan = (run.request_body or {}).get("orchestration_plan") or {}
|
||||||
|
budget = plan.get("budget") or {}
|
||||||
|
run_state = {
|
||||||
|
"tasks": task_dicts,
|
||||||
|
"budget": budget,
|
||||||
|
"usage": {"total_cost_usd": used_cost},
|
||||||
|
"quality": run.quality or {},
|
||||||
|
"review_cycles": int(run.metadata.get("review_cycles", 0) or 0),
|
||||||
|
"max_review_cycles": review_max_cycles(),
|
||||||
|
}
|
||||||
|
return convergence_mod.evaluate_convergence(run_state)
|
||||||
|
|
||||||
|
|
||||||
async def _run_budget_pressure(task) -> Optional[float]:
|
async def _run_budget_pressure(task) -> Optional[float]:
|
||||||
@@ -428,61 +389,99 @@ async def _run_budget_pressure(task) -> Optional[float]:
|
|||||||
return max(0.0, min(1.0, consumed / float(max_cost)))
|
return max(0.0, min(1.0, consumed / float(max_cost)))
|
||||||
|
|
||||||
|
|
||||||
async def scored_matchmake(idle_agents):
|
async def assess_swarm_health(run, *, connected_agent_ids=None):
|
||||||
"""Issue #9: task-centric explainable scored dispatch.
|
"""P-guard: diagnose whether the swarm can make progress on a run, and why not.
|
||||||
|
|
||||||
For each ready pending task, score every capable, has-capacity idle agent on capability_match
|
Assembles a snapshot (tasks + connected agents' capabilities + budget) and runs the pure
|
||||||
+ historical_success (decision_engine τ) + load (free slots) + permission_fit + budget_pressure
|
guard. Stores the report on run.metadata["health"], and appends unhealthy reports to a bounded
|
||||||
(≥4 non-capability dimensions with real in-repo signals; risk/estimated_cost/estimated_time
|
internal run.metadata["health_log"], so a stall is visible to an operator/Manager pulling the
|
||||||
have no source → None, disclosed), pick the highest-scoring agent, and build the
|
run. INTERNAL only — does NOT emit a Manager event (see the comment below) and does not change
|
||||||
dispatch.decision_made payload (candidate breakdown + exclusion reasons). Deterministic. An
|
the run. Returns the HealthReport.
|
||||||
agent chosen for one task is not reused in the same tick. Returns [(agent, task, event)].
|
|
||||||
"""
|
"""
|
||||||
assignments = []
|
tasks = [await task_queue.get_task(tid) for tid in run.task_ids]
|
||||||
used_agents: set = set()
|
tasks = [t for t in tasks if t]
|
||||||
all_tasks = await task_queue.get_all_tasks()
|
task_dicts = [{
|
||||||
ready = [t for t in all_tasks
|
"task_id": t.task_id,
|
||||||
if t.status == TaskStatus.PENDING and await task_queue.is_task_ready(t)]
|
"status": t.status.value if hasattr(t.status, "value") else t.status,
|
||||||
ready.sort(key=lambda t: t.created_at) # stable, oldest-first
|
"required_capabilities": t.required_capabilities,
|
||||||
|
"depends_on": t.depends_on,
|
||||||
|
"source": t.source,
|
||||||
|
} for t in tasks]
|
||||||
|
# Connected agents' capabilities (those actually reachable over the WS).
|
||||||
|
agent_ids = connected_agent_ids if connected_agent_ids is not None else list(manager.active_connections.keys())
|
||||||
|
caps = []
|
||||||
|
for aid in agent_ids:
|
||||||
|
a = await agent_registry.get_agent(aid)
|
||||||
|
if a:
|
||||||
|
caps.append(list(a.capabilities or []))
|
||||||
|
budget_state = convergence_mod.derive_budget_state({
|
||||||
|
"budget": ((run.request_body or {}).get("orchestration_plan") or {}).get("budget") or {},
|
||||||
|
"usage": {"total_cost_usd": sum(_task_cost_for_health(t) for t in tasks)},
|
||||||
|
})
|
||||||
|
report = guard_mod.diagnose({
|
||||||
|
"tasks": task_dicts,
|
||||||
|
"connected_agent_caps": caps,
|
||||||
|
"budget_state": budget_state,
|
||||||
|
})
|
||||||
|
# INTERNAL state only. We deliberately do NOT route this through swarm_runtime.emit_event:
|
||||||
|
# `swarm.health` is not a registered Manager event, and emit_event can forward to a
|
||||||
|
# subscribe-all Manager callback (agent_callback.go). Stored on the run (+ a bounded log) so an
|
||||||
|
# operator/Manager can pull it; registering a Manager-facing health event is a separate contract
|
||||||
|
# change (event-schema.md).
|
||||||
|
report_dict = report.to_dict()
|
||||||
|
run.metadata["health"] = report_dict
|
||||||
|
if not report.healthy:
|
||||||
|
run.metadata["health_log"] = (run.metadata.get("health_log", [])[-49:] + [report_dict])
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return report
|
||||||
|
|
||||||
for task in ready:
|
|
||||||
required = task.required_capabilities or []
|
def _task_cost_for_health(task) -> float:
|
||||||
budget_pressure = await _run_budget_pressure(task)
|
try:
|
||||||
candidates = [] # DispatchCandidate for permitted agents
|
data = parse_task_result(task) or {}
|
||||||
cand_agent = {} # agent_id -> agent
|
return float((data.get("usage") or {}).get("model_cost_usd") or 0.0)
|
||||||
excluded = {} # agent_id -> reason
|
except Exception:
|
||||||
for agent in idle_agents:
|
return 0.0
|
||||||
if agent.agent_id in used_agents or not agent_has_capacity(agent.agent_id):
|
|
||||||
continue
|
|
||||||
caps = set(agent.capabilities or [])
|
async def swarm_dispatch(idle_agents):
|
||||||
permitted = task_queue.can_agent_run_task(task, caps)
|
"""P6: each idle agent perceives the eligible ready tasks and SELF-SELECTS the best fit.
|
||||||
if not permitted:
|
|
||||||
excluded[agent.agent_id] = "capability_mismatch"
|
Decentralized, agent-centric (the OpenAI-Swarm/stigmergy spirit: no central scheduler assigns;
|
||||||
continue
|
each agent picks from the shared pool). The fit score unifies #9's explainable dimensions with
|
||||||
tau = await decision_engine.get_tau(task.agent_role, agent.agent_id)
|
#10's pheromone τ as `historical_success`: capability_match + τ + load + budget_pressure. The
|
||||||
cand = DispatchCandidate(
|
chosen pairing is recorded as an explainable `dispatch.decision_made` (internal). Returns the
|
||||||
|
list of (agent_id, task_id) assigned this tick.
|
||||||
|
"""
|
||||||
|
assigned = []
|
||||||
|
for agent in idle_agents:
|
||||||
|
if not agent_has_capacity(agent.agent_id):
|
||||||
|
continue
|
||||||
|
ready = await task_queue.get_ready_pending_tasks(agent.capabilities)
|
||||||
|
if not ready:
|
||||||
|
continue
|
||||||
|
candidates = []
|
||||||
|
for t in ready:
|
||||||
|
tau = await decision_engine.get_tau(t.agent_role, agent.agent_id)
|
||||||
|
candidates.append(DispatchCandidate(
|
||||||
agent_id=agent.agent_id,
|
agent_id=agent.agent_id,
|
||||||
task_id=task.task_id,
|
task_id=t.task_id,
|
||||||
agent_role=task.agent_role,
|
agent_role=t.agent_role,
|
||||||
capability_match=normalize_capability_match(required, caps),
|
capability_match=normalize_capability_match(t.required_capabilities, agent.capabilities),
|
||||||
historical_success=normalize_tau(tau),
|
historical_success=normalize_tau(tau),
|
||||||
load=normalize_load(AGENT_SLOTS.get(agent.agent_id, 1)),
|
load=normalize_load(AGENT_SLOTS.get(agent.agent_id, 1)),
|
||||||
permission_fit=1.0,
|
permission_fit=1.0,
|
||||||
budget_pressure=budget_pressure,
|
budget_pressure=await _run_budget_pressure(t),
|
||||||
)
|
))
|
||||||
candidates.append(cand)
|
|
||||||
cand_agent[agent.agent_id] = agent
|
|
||||||
if not candidates:
|
|
||||||
continue
|
|
||||||
ranked = rank_candidates(candidates)
|
ranked = rank_candidates(candidates)
|
||||||
chosen = ranked[0]
|
chosen = ranked[0]
|
||||||
for cand in ranked[1:]:
|
chosen_task = next(t for t in ready if t.task_id == chosen.task_id)
|
||||||
excluded[cand.agent_id] = "lower_score"
|
excluded = {c.task_id: "lower_fit" for c in ranked[1:]} # other tasks this agent passed over
|
||||||
event = build_dispatch_decision_event(candidates, chosen, excluded, task_id=task.task_id)
|
event = build_dispatch_decision_event(candidates, chosen, excluded, task_id=chosen.task_id)
|
||||||
chosen_agent = cand_agent[chosen.agent_id]
|
await task_queue.remove_pending_task(chosen_task.task_id)
|
||||||
used_agents.add(chosen_agent.agent_id)
|
if await finalize_dispatch(agent, chosen_task, dispatch_event=event):
|
||||||
assignments.append((chosen_agent, task, event))
|
assigned.append((agent.agent_id, chosen_task.task_id))
|
||||||
return assignments
|
return assigned
|
||||||
|
|
||||||
|
|
||||||
async def refresh_swarm_run_status(run):
|
async def refresh_swarm_run_status(run):
|
||||||
@@ -535,8 +534,14 @@ async def refresh_swarm_run_status(run):
|
|||||||
# Master review-and-iterate gate: before declaring success, optionally run the critic and
|
# Master review-and-iterate gate: before declaring success, optionally run the critic and
|
||||||
# send rejected work back to the specialists. Off unless ENABLE_REVIEW_LOOP is set, so the
|
# send rejected work back to the specialists. Off unless ENABLE_REVIEW_LOOP is set, so the
|
||||||
# default completion semantics (and Manager contract) are unchanged.
|
# default completion semantics (and Manager contract) are unchanged.
|
||||||
if next_status == "completed" and review_loop_enabled():
|
# P5 (#11): peer cross-review gate — when enabled and >=2 peer reviews were submitted, aggregate
|
||||||
if await maybe_run_review_cycle(run, tasks):
|
# them (replacing the single-critic Master review) and reopen rework targets on rejection. At
|
||||||
|
# P-cutover this becomes the only review path; until then it precedes the Master loop.
|
||||||
|
# Swarm review (the ONLY review path): >=2 peers independently validate; disagreement is
|
||||||
|
# arbitrated and rework targets reopened. No single-critic Master gate. run_cross_review is a
|
||||||
|
# no-op (returns False) when fewer than 2 reviews were submitted.
|
||||||
|
if next_status == "completed":
|
||||||
|
if await run_cross_review(run, tasks):
|
||||||
return
|
return
|
||||||
|
|
||||||
if run.status == next_status:
|
if run.status == next_status:
|
||||||
@@ -545,17 +550,30 @@ async def refresh_swarm_run_status(run):
|
|||||||
run.status = next_status
|
run.status = next_status
|
||||||
await swarm_runtime.save_run(run)
|
await swarm_runtime.save_run(run)
|
||||||
|
|
||||||
|
# Swarm convergence (#12): always compute a report explaining WHY the run stopped
|
||||||
|
# (termination_reason + consensus + conflicts), stored on the run and surfaced in the status
|
||||||
|
# payload. Authoritative status-override is a follow-on; today next_status (task bookkeeping)
|
||||||
|
# and the report agree on completed/failed.
|
||||||
|
termination_reason = None
|
||||||
|
try:
|
||||||
|
report = await compute_convergence_report(run, tasks, terminal=True)
|
||||||
|
run.metadata["convergence"] = report.to_dict()
|
||||||
|
termination_reason = report.to_dict().get("termination_reason")
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("convergence eval failed for run %s: %s", run.swarm_id, exc)
|
||||||
|
|
||||||
deliverable = None
|
deliverable = None
|
||||||
final_summary = None
|
final_summary = None
|
||||||
if next_status == "completed":
|
if next_status == "completed":
|
||||||
deliverable = build_run_deliverable(run, tasks)
|
deliverable = build_run_deliverable(run, tasks)
|
||||||
if review_loop_enabled():
|
# Synthesize the specialist results into one coherent, user-facing answer (a swarm tool,
|
||||||
# Synthesize the specialist results into one coherent, user-facing answer.
|
# not a central controller — it only summarizes the converged outputs).
|
||||||
completed = [t for t in tasks if t.status == TaskStatus.COMPLETED]
|
completed = [t for t in tasks if t.status == TaskStatus.COMPLETED]
|
||||||
results = {t.task_id: {"result": parse_task_result(t) or {}} for t in completed}
|
results = {t.task_id: {"result": parse_task_result(t) or {}} for t in completed}
|
||||||
final_summary = await master_agent.synthesize(run.objective, results)
|
final_summary = await master_agent.synthesize(run.objective, results)
|
||||||
run.metadata["final_summary"] = final_summary
|
run.metadata["final_summary"] = final_summary
|
||||||
await swarm_runtime.save_run(run)
|
await swarm_runtime.save_run(run)
|
||||||
|
|
||||||
# Benchmark Group B: grade the run's generated code against its held-out fixture tests in
|
# Benchmark Group B: grade the run's generated code against its held-out fixture tests in
|
||||||
# the sandbox. Gated (ENABLE_QUALITY_EVAL) + fixture-bound; a no-op otherwise. Never fails
|
# the sandbox. Gated (ENABLE_QUALITY_EVAL) + fixture-bound; a no-op otherwise. Never fails
|
||||||
@@ -583,6 +601,9 @@ async def refresh_swarm_run_status(run):
|
|||||||
}
|
}
|
||||||
if run.metadata.get("review_summary"):
|
if run.metadata.get("review_summary"):
|
||||||
timeline_payload["review_summary"] = run.metadata["review_summary"]
|
timeline_payload["review_summary"] = run.metadata["review_summary"]
|
||||||
|
if termination_reason:
|
||||||
|
# Backward-compatible addition: explains WHY the run ended (#12).
|
||||||
|
timeline_payload["termination_reason"] = termination_reason
|
||||||
await swarm_runtime.emit_event(run, "timeline.updated", payload=timeline_payload)
|
await swarm_runtime.emit_event(run, "timeline.updated", payload=timeline_payload)
|
||||||
await maybe_emit_budget_alert(run)
|
await maybe_emit_budget_alert(run)
|
||||||
|
|
||||||
@@ -701,77 +722,361 @@ def request_context_headers(request: Request) -> Dict[str, Optional[str]]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def planner_fallback_enabled(body: Dict[str, Any]) -> bool:
|
def _manager_provided_agents(body: Dict[str, Any]) -> bool:
|
||||||
"""Return whether the LLM planner should synthesize the task graph.
|
"""Whether the Manager supplied an explicit agent breakdown (Manager-first; never overridden)."""
|
||||||
|
|
||||||
Manager-first: only when the operator opts in via ENABLE_PLANNER_FALLBACK AND the
|
|
||||||
Manager supplied no explicit agent breakdown. A Manager-provided plan is never overridden.
|
|
||||||
"""
|
|
||||||
if os.getenv("ENABLE_PLANNER_FALLBACK", "false").lower() not in {"1", "true", "yes"}:
|
|
||||||
return False
|
|
||||||
normalized = swarm_runtime.normalize_create_request(body)
|
normalized = swarm_runtime.normalize_create_request(body)
|
||||||
plan = normalized.get("orchestration_plan") or {}
|
plan = normalized.get("orchestration_plan") or {}
|
||||||
agents = plan.get("agents") or normalized.get("agents")
|
return bool(plan.get("agents") or normalized.get("agents"))
|
||||||
return not agents
|
|
||||||
|
|
||||||
|
|
||||||
async def build_planner_task_specs(run, body: Dict[str, Any], base_specs: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
def build_seed_task_specs(run, body: Dict[str, Any], base_specs: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||||
"""Map planner subtasks into the build_task_descriptions spec shape.
|
"""Decentralized rework P2: seed the run with ONE objective-carrying task instead of a
|
||||||
|
Master-decomposed plan.
|
||||||
|
|
||||||
Reuses the base spec's context so orchestration_plan/resource_grants/billing plumbing
|
This is the demotion of the Master: the orchestrator only *injects the initial task*; it does
|
||||||
is preserved identically to the Manager-driven path.
|
NOT call master_agent.plan / decompose up front. Agents perceive the seed + shared state and
|
||||||
|
grow the task graph themselves (autonomous task generation = P3). The seed has no required
|
||||||
|
capabilities so any agent may self-select it.
|
||||||
"""
|
"""
|
||||||
objective = run.objective or "Complete swarm objective"
|
objective = run.objective or "Complete swarm objective"
|
||||||
base_context = (base_specs[0].get("context") if base_specs else {}) or {}
|
base_context = (base_specs[0].get("context") if base_specs else {}) or {}
|
||||||
subtasks = await master_agent.plan(run.swarm_id, objective)
|
return [{
|
||||||
|
"task_id": "seed",
|
||||||
|
"title": "swarm seed",
|
||||||
|
"description": objective,
|
||||||
|
"agent_role": "general",
|
||||||
|
"required_capabilities": [], # any agent may claim the seed (self-selection)
|
||||||
|
"depends_on": [],
|
||||||
|
"parent_task_id": None,
|
||||||
|
"root_task_id": "seed",
|
||||||
|
"source": "seed",
|
||||||
|
"workflow_mode": "swarm",
|
||||||
|
"allow_handoff": True,
|
||||||
|
"context": {
|
||||||
|
**base_context,
|
||||||
|
"agent_role": "general",
|
||||||
|
"workflow_mode": "swarm",
|
||||||
|
"is_seed": True,
|
||||||
|
"objective": objective,
|
||||||
|
},
|
||||||
|
}]
|
||||||
|
|
||||||
prefix = f"{run.swarm_id}-"
|
|
||||||
|
|
||||||
def _local_id(value: str) -> str:
|
async def handle_task_proposal(agent_id: str, message: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
# Planner ids are prefixed with the run id; strip it so create_tasks_for_run prepends
|
"""Decentralized rework P3 (#7): an executing agent proposes a NEW task from shared state.
|
||||||
# the swarm id exactly once (avoids doubled swarm-X-swarm-X-... task ids).
|
|
||||||
return value[len(prefix):] if value.startswith(prefix) else value
|
|
||||||
|
|
||||||
specs: List[Dict[str, Any]] = []
|
This is the bottom-up decomposition that replaces the Master's top-down plan: an agent working
|
||||||
for index, sub in enumerate(subtasks, start=1):
|
the seed (or any task) perceives the shared run state and proposes follow-up subtasks. The
|
||||||
role = sub.get("role") or "general"
|
orchestrator reviews each proposal (confidence floor / dedup-merge / per-run budget) and, on
|
||||||
task_id = _local_id(sub.get("subtask_id") or f"task-{index}")
|
ACCEPT, enqueues a real PENDING task carrying full lineage. Proposal lifecycle is stored on the
|
||||||
specs.append({
|
run (internal telemetry, NOT a Manager event); an accepted proposal emits the standard,
|
||||||
"task_id": task_id,
|
registered `task.created`. This is the swarm's only decomposition path (no Master plan).
|
||||||
"title": f"{role} task",
|
|
||||||
"description": sub.get("description") or objective,
|
|
||||||
"agent_role": role,
|
|
||||||
"required_capabilities": sub.get("required_capabilities") or [role],
|
|
||||||
"depends_on": [_local_id(dep) for dep in (sub.get("depends_on") or [])],
|
|
||||||
"parent_task_id": None,
|
|
||||||
"root_task_id": task_id,
|
|
||||||
"source": "planner",
|
|
||||||
"workflow_mode": "multi_agent",
|
|
||||||
"allow_handoff": True,
|
|
||||||
"context": {
|
|
||||||
**base_context,
|
|
||||||
"agent_role": role,
|
|
||||||
"workflow_mode": "multi_agent",
|
|
||||||
},
|
|
||||||
})
|
|
||||||
|
|
||||||
if not specs:
|
Returns a small result dict for the agent ack. Never raises into the WS loop.
|
||||||
return base_specs
|
"""
|
||||||
|
origin_task_id = message.get("origin_task_id") or message.get("task_id")
|
||||||
|
run = await swarm_runtime.get_run_for_task(origin_task_id) if origin_task_id else None
|
||||||
|
if not run:
|
||||||
|
return {"accepted": False, "decision": "no_run"}
|
||||||
|
|
||||||
# Drop any dependency that does not reference a known subtask so a malformed plan
|
try:
|
||||||
# cannot leave tasks permanently blocked on a phantom dependency.
|
proposal = autonomous_mod.TaskProposal(
|
||||||
known_ids = {spec["task_id"] for spec in specs}
|
proposed_by_agent_id=agent_id,
|
||||||
for spec in specs:
|
description=(message.get("description") or "").strip(),
|
||||||
spec["depends_on"] = [dep for dep in spec["depends_on"] if dep in known_ids]
|
proposal_reason=(message.get("reason") or message.get("proposal_reason") or "").strip(),
|
||||||
return specs
|
proposal_confidence=message.get("confidence", message.get("proposal_confidence", 0.0)),
|
||||||
|
title=message.get("title"),
|
||||||
|
agent_role=message.get("agent_role", "general"),
|
||||||
|
required_capabilities=message.get("required_capabilities") or [],
|
||||||
|
depends_on=message.get("depends_on") or [],
|
||||||
|
lineage=autonomous_mod.ProposalLineage(
|
||||||
|
origin_task_id=origin_task_id,
|
||||||
|
trigger_event=message.get("trigger_event"),
|
||||||
|
shared_state_snapshot=message.get("shared_state_snapshot") or {},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
return {"accepted": False, "decision": "invalid", "reason": str(exc)}
|
||||||
|
|
||||||
|
# Per-run proposal budget: a hard cap on how many bottom-up tasks one run may spawn.
|
||||||
|
cap = int(os.getenv("AGENT_PROPOSAL_BUDGET", "5"))
|
||||||
|
accepted_so_far = int(run.metadata.get("proposal_accepted_count", 0) or 0)
|
||||||
|
policy = autonomous_mod.ProposalPolicy(remaining_proposal_budget=max(0, cap - accepted_so_far))
|
||||||
|
|
||||||
|
existing = []
|
||||||
|
for tid in run.task_ids:
|
||||||
|
t = await task_queue.get_task(tid)
|
||||||
|
if t:
|
||||||
|
existing.append(autonomous_mod.ExistingTaskRef(
|
||||||
|
task_id=t.task_id, description=t.description,
|
||||||
|
agent_role=t.agent_role,
|
||||||
|
status=t.status.value if hasattr(t.status, "value") else str(t.status),
|
||||||
|
))
|
||||||
|
|
||||||
|
outcome = autonomous_mod.review_proposal(proposal, policy, existing)
|
||||||
|
|
||||||
|
# Internal proposal-lifecycle telemetry on the run (not a Manager event).
|
||||||
|
run.metadata.setdefault("proposals", [])
|
||||||
|
run.metadata["proposals"] = (run.metadata["proposals"][-49:] +
|
||||||
|
[{"event_type": ev["event_type"], **ev}
|
||||||
|
for ev in autonomous_mod.build_lifecycle_events_for_outcome(proposal, outcome)])
|
||||||
|
|
||||||
|
result: Dict[str, Any] = {"accepted": False, "decision": outcome.decision.value,
|
||||||
|
"reason": outcome.reason, "proposal_id": proposal.proposal_id}
|
||||||
|
if outcome.decision == autonomous_mod.ProposalDecision.MERGE:
|
||||||
|
result["merge_target_task_id"] = outcome.merge_target_task_id
|
||||||
|
if outcome.decision == autonomous_mod.ProposalDecision.ACCEPT:
|
||||||
|
spec = autonomous_mod.ingest_accepted_proposal(proposal, swarm_id=run.swarm_id)
|
||||||
|
new_task_id = f"{run.swarm_id}-{proposal.proposal_id}"
|
||||||
|
task = await task_queue.create_task(
|
||||||
|
task_id=new_task_id,
|
||||||
|
title=spec["title"],
|
||||||
|
description=spec["description"],
|
||||||
|
agent_role=spec["agent_role"],
|
||||||
|
required_capabilities=spec["required_capabilities"],
|
||||||
|
depends_on=spec["depends_on"], # runtime ids (origin task is already runtime-scoped)
|
||||||
|
source=autonomous_mod.PROPOSED_SOURCE,
|
||||||
|
context={**spec["context"], "swarm_id": run.swarm_id,
|
||||||
|
"runtime_deployment_id": run.deployment_id},
|
||||||
|
enqueue=True,
|
||||||
|
)
|
||||||
|
await swarm_runtime.attach_task(run, task.task_id)
|
||||||
|
run.metadata["proposal_accepted_count"] = accepted_so_far + 1
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
await swarm_runtime.emit_event(
|
||||||
|
run, "task.created", task_id=task.task_id,
|
||||||
|
payload={"task_id": task.task_id, "agent_role": task.agent_role,
|
||||||
|
"source": autonomous_mod.PROPOSED_SOURCE,
|
||||||
|
"proposed_by_agent_id": agent_id},
|
||||||
|
)
|
||||||
|
result["accepted"] = True
|
||||||
|
result["task_id"] = task.task_id
|
||||||
|
else:
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_review_decision(agent_id: str, message: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""Decentralized rework P5 (#11): a peer agent submits an independent structured review.
|
||||||
|
|
||||||
|
Reviews accumulate on the run; `run_cross_review` aggregates >=2 of them (the cross-validation
|
||||||
|
closure that is the swarm's only review — no single-critic Master gate).
|
||||||
|
"""
|
||||||
|
task_id = message.get("task_id") or (message.get("affected_tasks") or [None])[0]
|
||||||
|
run = await swarm_runtime.get_run_for_task(task_id) if task_id else None
|
||||||
|
if not run:
|
||||||
|
return {"recorded": False, "reason": "no_run"}
|
||||||
|
try:
|
||||||
|
decision = cross_review_mod.ReviewDecision(
|
||||||
|
verdict=message.get("verdict", "pass"),
|
||||||
|
reviewer_agent_id=agent_id,
|
||||||
|
evidence=message.get("evidence") or [],
|
||||||
|
failed_criteria=message.get("failed_criteria") or [],
|
||||||
|
affected_tasks=message.get("affected_tasks") or [],
|
||||||
|
recommended_rework=message.get("recommended_rework") or [],
|
||||||
|
confidence=message.get("confidence", 1.0),
|
||||||
|
weight=message.get("weight", 1.0),
|
||||||
|
summary=message.get("summary", ""),
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
return {"recorded": False, "reason": str(exc)}
|
||||||
|
reviews = run.metadata.setdefault("reviews", [])
|
||||||
|
# latest review per reviewer wins (an agent may revise its verdict)
|
||||||
|
reviews = [r for r in reviews if r.get("reviewer_agent_id") != agent_id]
|
||||||
|
reviews.append(decision.to_dict())
|
||||||
|
run.metadata["reviews"] = reviews
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return {"recorded": True, "review_count": len(reviews)}
|
||||||
|
|
||||||
|
|
||||||
|
async def run_cross_review(run, tasks) -> bool:
|
||||||
|
"""P5 (#11): aggregate >=2 collected peer reviews; reopen rework targets if rejected.
|
||||||
|
|
||||||
|
Returns True when it reopened tasks (the run stays 'running' for a redo cycle). Returns False
|
||||||
|
when there aren't >=2 reviews, the cycle budget is spent,
|
||||||
|
or the cross-review accepted the work — letting the caller proceed to completion. Records the
|
||||||
|
arbitrated verdict + rework attributions on the run (internal telemetry).
|
||||||
|
"""
|
||||||
|
reviews_raw = run.metadata.get("reviews") or []
|
||||||
|
if len(reviews_raw) < 2:
|
||||||
|
return False
|
||||||
|
cycles = int(run.metadata.get("review_cycles", 0) or 0)
|
||||||
|
if cycles >= review_max_cycles():
|
||||||
|
return False
|
||||||
|
|
||||||
|
decisions = [cross_review_mod.ReviewDecision(**r) for r in reviews_raw]
|
||||||
|
verdict = cross_review_mod.aggregate_reviews(decisions)
|
||||||
|
run.metadata["cross_review"] = verdict.to_dict()
|
||||||
|
|
||||||
|
task_owner = {t.task_id: t.assigned_agent_id for t in tasks if t.assigned_agent_id}
|
||||||
|
valid_targets = [tid for tid in verdict.rework_targets if tid in run.task_ids]
|
||||||
|
if verdict.accepted or not valid_targets:
|
||||||
|
run.metadata["reviews"] = [] # consumed
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return False
|
||||||
|
|
||||||
|
attributions = cross_review_mod.build_rework_attributions(verdict, task_owner=task_owner)
|
||||||
|
reopened = [tid for tid in valid_targets if await task_queue.reopen_task(tid)]
|
||||||
|
run.metadata["rework_attributions"] = (run.metadata.get("rework_attributions", [])[-49:] +
|
||||||
|
[a.to_dict() for a in attributions])
|
||||||
|
run.metadata["reviews"] = [] # consumed; reviewers re-review the redone work next cycle
|
||||||
|
if not reopened:
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return False
|
||||||
|
run.metadata["review_cycles"] = cycles + 1
|
||||||
|
run.metadata["review_summary"] = verdict.summary
|
||||||
|
run.status = "running"
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
await swarm_runtime.emit_event(run, "timeline.updated", payload={
|
||||||
|
"summary": f"Cross-review cycle {cycles + 1}: {verdict.summary}",
|
||||||
|
"status": "running", "retry_tasks": reopened,
|
||||||
|
"disagreement": verdict.disagreement,
|
||||||
|
})
|
||||||
|
logger.info(f"Cross-review reopened {reopened} on run {run.swarm_id} (cycle {cycles + 1})")
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
async def _historical_success_map(agent_role: str, agent_ids) -> Dict[str, float]:
|
||||||
|
"""τ (decision_engine pheromone) per agent, normalized to [0,1] for arbitration."""
|
||||||
|
out: Dict[str, float] = {}
|
||||||
|
for aid in agent_ids:
|
||||||
|
tau = await decision_engine.get_tau(agent_role, aid)
|
||||||
|
out[aid] = normalize_tau(tau) or 0.5
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_task_bid(agent_id: str, message: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""Decentralized rework P4 (#8): record an agent's bid for a task (de-dup by agent, latest wins).
|
||||||
|
|
||||||
|
Bids accumulate on the run; `arbitrate_and_assign` later picks a winner. Internal telemetry
|
||||||
|
only (not a Manager event).
|
||||||
|
"""
|
||||||
|
task_id = message.get("task_id")
|
||||||
|
run = await swarm_runtime.get_run_for_task(task_id) if task_id else None
|
||||||
|
if not run or not task_id:
|
||||||
|
return {"recorded": False, "reason": "no_run"}
|
||||||
|
bid = competition_mod.TaskBid(
|
||||||
|
task_id=task_id, agent_id=agent_id,
|
||||||
|
confidence=message.get("confidence", 0.5),
|
||||||
|
estimated_cost=message.get("estimated_cost", 0.0),
|
||||||
|
estimated_time=message.get("estimated_time", 0.0),
|
||||||
|
risk_score=message.get("risk_score", 0.0),
|
||||||
|
reason=message.get("reason", ""),
|
||||||
|
capabilities=message.get("capabilities") or [],
|
||||||
|
current_load=message.get("current_load", 0),
|
||||||
|
)
|
||||||
|
bids = run.metadata.setdefault("bids", {})
|
||||||
|
lst = [b for b in bids.get(task_id, []) if b.get("agent_id") != agent_id]
|
||||||
|
lst.append(bid.model_dump())
|
||||||
|
bids[task_id] = lst
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return {"recorded": True, "task_id": task_id, "bid_count": len(lst)}
|
||||||
|
|
||||||
|
|
||||||
|
async def arbitrate_and_assign(run, task_id: str) -> Optional[Dict[str, Any]]:
|
||||||
|
"""Arbitrate the bids collected for a PENDING task and assign the winner (#8).
|
||||||
|
|
||||||
|
Deterministic (task_competition.arbitrate) + τ-weighted. Assigns the winner via the shared
|
||||||
|
finalize_dispatch tail, records the audit trail on the run, and clears the task's bids.
|
||||||
|
Returns the arbitration result payload, or None if there was nothing to arbitrate.
|
||||||
|
"""
|
||||||
|
bids_raw = (run.metadata.get("bids") or {}).get(task_id) or []
|
||||||
|
task = await task_queue.get_task(task_id)
|
||||||
|
if not bids_raw or not task:
|
||||||
|
return None
|
||||||
|
bids = [competition_mod.TaskBid(**b) for b in bids_raw]
|
||||||
|
hist = await _historical_success_map(task.agent_role, [b.agent_id for b in bids])
|
||||||
|
result = competition_mod.arbitrate(
|
||||||
|
bids, required_capabilities=task.required_capabilities, historical_success=hist,
|
||||||
|
)
|
||||||
|
if result.winner_agent_id and task.status == TaskStatus.PENDING:
|
||||||
|
winner = await agent_registry.get_agent(result.winner_agent_id)
|
||||||
|
if winner:
|
||||||
|
await task_queue.remove_pending_task(task_id)
|
||||||
|
assigned = await finalize_dispatch(winner, task)
|
||||||
|
if not assigned:
|
||||||
|
await task_queue.requeue_task(task_id)
|
||||||
|
_, payload = competition_mod.arbitrated_event(result)
|
||||||
|
run.metadata.setdefault("arbitrations", [])
|
||||||
|
run.metadata["arbitrations"] = run.metadata["arbitrations"][-49:] + [payload]
|
||||||
|
(run.metadata.get("bids") or {}).pop(task_id, None)
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_task_yield(agent_id: str, message: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""P4 (#8): an agent voluntarily releases a task back for re-competition (reuses release_task)."""
|
||||||
|
task_id = message.get("task_id")
|
||||||
|
run = await swarm_runtime.get_run_for_task(task_id) if task_id else None
|
||||||
|
if not run or not task_id:
|
||||||
|
return {"released": False, "reason": "no_run"}
|
||||||
|
yield_msg = competition_mod.TaskYield(
|
||||||
|
task_id=task_id, agent_id=agent_id,
|
||||||
|
release_with_reason=message.get("reason", message.get("release_with_reason", "")),
|
||||||
|
recommend_agent=message.get("recommend_agent"),
|
||||||
|
)
|
||||||
|
released = await task_queue.release_task(task_id, agent_id)
|
||||||
|
_, payload = competition_mod.yielded_event(yield_msg)
|
||||||
|
run.metadata.setdefault("yields", [])
|
||||||
|
run.metadata["yields"] = run.metadata["yields"][-49:] + [payload]
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return {"released": bool(released), "task_id": task_id, "recommend_agent": yield_msg.recommend_agent}
|
||||||
|
|
||||||
|
|
||||||
|
async def handle_task_takeover(agent_id: str, message: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""P4 (#8): an agent requests takeover of a held task; arbitrate requester vs incumbent.
|
||||||
|
|
||||||
|
Requester wins → release from incumbent and assign requester (only when decisive). The
|
||||||
|
incumbent is weighed on identical terms (a neutral bid carrying its capabilities), so takeover
|
||||||
|
is never a free steal — it must out-score the holder.
|
||||||
|
"""
|
||||||
|
task_id = message.get("task_id")
|
||||||
|
task = await task_queue.get_task(task_id) if task_id else None
|
||||||
|
run = await swarm_runtime.get_run_for_task(task_id) if task_id else None
|
||||||
|
if not task or not run:
|
||||||
|
return {"taken_over": False, "reason": "no_task"}
|
||||||
|
incumbent_id = task.assigned_agent_id
|
||||||
|
requester_bid = competition_mod.TaskBid(
|
||||||
|
task_id=task_id, agent_id=agent_id,
|
||||||
|
confidence=message.get("confidence", 0.7),
|
||||||
|
capabilities=message.get("capabilities") or [],
|
||||||
|
reason=message.get("reason", "takeover request"),
|
||||||
|
)
|
||||||
|
bids = [requester_bid]
|
||||||
|
if incumbent_id:
|
||||||
|
incumbent = await agent_registry.get_agent(incumbent_id)
|
||||||
|
bids.append(competition_mod.TaskBid(
|
||||||
|
task_id=task_id, agent_id=incumbent_id, confidence=0.5,
|
||||||
|
capabilities=(incumbent.capabilities if incumbent else []),
|
||||||
|
))
|
||||||
|
hist = await _historical_success_map(task.agent_role, [b.agent_id for b in bids])
|
||||||
|
result = competition_mod.arbitrate(bids, required_capabilities=task.required_capabilities,
|
||||||
|
historical_success=hist)
|
||||||
|
taken = False
|
||||||
|
if result.winner_agent_id == agent_id and result.decisive and incumbent_id != agent_id:
|
||||||
|
await task_queue.release_task(task_id, incumbent_id)
|
||||||
|
winner = await agent_registry.get_agent(agent_id)
|
||||||
|
if winner:
|
||||||
|
await task_queue.remove_pending_task(task_id)
|
||||||
|
taken = await finalize_dispatch(winner, task)
|
||||||
|
_, payload = competition_mod.arbitrated_event(result)
|
||||||
|
run.metadata.setdefault("arbitrations", [])
|
||||||
|
run.metadata["arbitrations"] = run.metadata["arbitrations"][-49:] + [payload]
|
||||||
|
await swarm_runtime.save_run(run)
|
||||||
|
return {"taken_over": bool(taken), "winner_agent_id": result.winner_agent_id,
|
||||||
|
"decisive": result.decisive, "task_id": task_id}
|
||||||
|
|
||||||
|
|
||||||
async def create_tasks_for_run(run, body: Dict[str, Any]) -> int:
|
async def create_tasks_for_run(run, body: Dict[str, Any]) -> int:
|
||||||
"""Create the runtime task graph and emit task.created events."""
|
"""Create the runtime task graph and emit task.created events."""
|
||||||
created_count = 0
|
created_count = 0
|
||||||
task_specs = swarm_runtime.build_task_descriptions(body)
|
task_specs = swarm_runtime.build_task_descriptions(body)
|
||||||
if planner_fallback_enabled(body):
|
# Swarm task creation (the ONLY path): when the Manager supplied an explicit agent breakdown we
|
||||||
task_specs = await build_planner_task_specs(run, body, task_specs)
|
# honor it (Manager-first contract); otherwise we SEED a single objective-carrying task and the
|
||||||
|
# agents grow the graph bottom-up via proposals (handle_task_proposal). No up-front Master
|
||||||
|
# decomposition — this repo is the swarm runtime (see rework plan §0).
|
||||||
|
if not _manager_provided_agents(body):
|
||||||
|
task_specs = build_seed_task_specs(run, body, task_specs)
|
||||||
task_id_map = {
|
task_id_map = {
|
||||||
task_spec["task_id"]: f"{run.swarm_id}-{task_spec['task_id']}"
|
task_spec["task_id"]: f"{run.swarm_id}-{task_spec['task_id']}"
|
||||||
for task_spec in task_specs
|
for task_spec in task_specs
|
||||||
@@ -2371,6 +2676,52 @@ async def websocket_endpoint(websocket: WebSocket, agent_id: str):
|
|||||||
delivered=delivered,
|
delivered=delivered,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
elif message_type == "task_proposal":
|
||||||
|
# Decentralized rework P3 (#7): an agent proposes a new task from shared
|
||||||
|
# state (bottom-up decomposition). Reviewed + (maybe) enqueued; ack back.
|
||||||
|
try:
|
||||||
|
proposal_result = await handle_task_proposal(agent_id, message)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"task_proposal from {agent_id} failed: {exc}")
|
||||||
|
proposal_result = {"accepted": False, "decision": "error", "reason": str(exc)}
|
||||||
|
await websocket.send_json({"type": "task_proposal_ack", **proposal_result})
|
||||||
|
|
||||||
|
elif message_type == "review_decision":
|
||||||
|
# P5 (#11): a peer agent submits an independent structured review.
|
||||||
|
try:
|
||||||
|
review_result = await handle_review_decision(agent_id, message)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"review_decision from {agent_id} failed: {exc}")
|
||||||
|
review_result = {"recorded": False, "reason": str(exc)}
|
||||||
|
await websocket.send_json({"type": "review_decision_ack", **review_result})
|
||||||
|
|
||||||
|
elif message_type == "task_bid":
|
||||||
|
# P4 (#8): record an agent's bid for a task (arbitration picks a winner).
|
||||||
|
try:
|
||||||
|
bid_result = await handle_task_bid(agent_id, message)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"task_bid from {agent_id} failed: {exc}")
|
||||||
|
bid_result = {"recorded": False, "reason": str(exc)}
|
||||||
|
await websocket.send_json({"type": "task_bid_ack", **bid_result})
|
||||||
|
|
||||||
|
elif message_type == "task_yield":
|
||||||
|
# P4 (#8): an agent releases a task back for re-competition.
|
||||||
|
try:
|
||||||
|
yield_result = await handle_task_yield(agent_id, message)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"task_yield from {agent_id} failed: {exc}")
|
||||||
|
yield_result = {"released": False, "reason": str(exc)}
|
||||||
|
await websocket.send_json({"type": "task_yield_ack", **yield_result})
|
||||||
|
|
||||||
|
elif message_type == "task_takeover_request":
|
||||||
|
# P4 (#8): an agent requests takeover; arbitrate requester vs incumbent.
|
||||||
|
try:
|
||||||
|
takeover_result = await handle_task_takeover(agent_id, message)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(f"task_takeover from {agent_id} failed: {exc}")
|
||||||
|
takeover_result = {"taken_over": False, "reason": str(exc)}
|
||||||
|
await websocket.send_json({"type": "task_takeover_ack", **takeover_result})
|
||||||
|
|
||||||
else:
|
else:
|
||||||
logger.warning(f"Unknown message type from {agent_id}: {message_type}")
|
logger.warning(f"Unknown message type from {agent_id}: {message_type}")
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,354 @@
|
|||||||
|
"""Agent task-competition protocol: bidding, yielding, takeover and arbitration.
|
||||||
|
|
||||||
|
Resolves issue #8 — "Agent 自主竞争机制缺失" — by giving the swarm a *real, tested*
|
||||||
|
mechanism for agents to (1) bid for a task, (2) yield a task back with a reason and an
|
||||||
|
optional recommendation, (3) request takeover of a task another agent holds, and (4)
|
||||||
|
have a deterministic arbitrator pick a winner with a full, auditable explanation.
|
||||||
|
|
||||||
|
This module is INTENTIONALLY self-contained and side-effect free:
|
||||||
|
- It defines the message/data models (`TaskBid`, `TaskYield`, `TaskTakeoverRequest`,
|
||||||
|
`TaskArbitrationResult`), the event-payload builders (`task.bid_submitted`,
|
||||||
|
`task.yielded`, `task.takeover_requested`, `task.arbitrated`), and a pure
|
||||||
|
`arbitrate(bids, policy) -> TaskArbitrationResult`.
|
||||||
|
- It does NOT touch Redis, the WebSocket loop, dispatch, the Manager callback stream,
|
||||||
|
billing, or the approval chain. Wiring those is documented (not done) in
|
||||||
|
`docs/swarm/task-competition-protocol.md` §「集成说明」. This honours org rule #9:
|
||||||
|
the arbitration math is genuinely implemented and tested; the integration is
|
||||||
|
explicitly declared as not wired.
|
||||||
|
|
||||||
|
Determinism (DoD): `arbitrate` is a pure function of its inputs. Given the same bids and
|
||||||
|
the same policy it always returns the same winner, the same per-bid scores, and the same
|
||||||
|
ordered loser list. Ties break on a stable, documented key (score desc, then agent_id
|
||||||
|
asc) so there is never RNG or dict-ordering ambiguity. This mirrors the auditability
|
||||||
|
contract the ACO `DecisionEngine` already established (`decision_engine.py`): the result
|
||||||
|
carries enough to re-derive and explain the choice without any live state.
|
||||||
|
|
||||||
|
τ reuse: `arbitrate` accepts an optional `historical_success` map (per-agent τ in
|
||||||
|
[0, 1]) — the same earned-reputation signal the ACO pheromone trail produces
|
||||||
|
(`pheromone:{agent_role}:{agent_id}`). The arbitrator treats it as ONE weighted input
|
||||||
|
among capability fit, budget headroom, risk and current load; it is never the sole
|
||||||
|
decider, and when absent it contributes a documented neutral 0.5 (same for every bid →
|
||||||
|
no ranking distortion), exactly as `decision_engine.compute_eta` handles missing
|
||||||
|
confidence.
|
||||||
|
|
||||||
|
Code English; companion doc Simplified Chinese, per PROJECT_STANDARD.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Dict, List, Optional, Tuple
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# UNCONDITIONAL: the competition protocol is the swarm's only contention mechanism (no enable
|
||||||
|
# flag — this repo is the swarm runtime). The WS task_bid/task_yield/task_takeover_request branches
|
||||||
|
# + their handlers in main.py are always active.
|
||||||
|
|
||||||
|
|
||||||
|
# --- arbitration policy ------------------------------------------------------
|
||||||
|
# Neutral constant for a missing τ signal: identical for every bid, so it cannot distort
|
||||||
|
# ranking — it only shifts every score by the same amount. Mirrors decision_engine's
|
||||||
|
# CONFIDENCE_NEUTRAL=0.5 treatment of absent confidence (honesty rule #9: a missing
|
||||||
|
# signal is a documented neutral, never a fabricated value).
|
||||||
|
HISTORICAL_SUCCESS_NEUTRAL = 0.5
|
||||||
|
|
||||||
|
# Normalisation anchors so heterogeneous bid fields collapse to [0, 1] before weighting.
|
||||||
|
# estimated_cost/time are "lower is better"; they are scored as headroom against a cap.
|
||||||
|
COST_NORM = 1.0 # estimated_cost expressed as a fraction of run budget; >=1 ⇒ no headroom
|
||||||
|
TIME_NORM_SECONDS = 600.0 # estimated_time at which the time score saturates to 0 (10 min)
|
||||||
|
LOAD_NORM_SLOTS = 4.0 # current_load (# in-flight tasks) at which the load score hits 0
|
||||||
|
|
||||||
|
|
||||||
|
class ArbitrationPolicy(BaseModel):
|
||||||
|
"""Weights for the arbitration scoring function. All non-negative; need not sum to 1
|
||||||
|
(the score is a weighted sum, comparisons are relative). Deterministic given inputs.
|
||||||
|
|
||||||
|
The default mirrors the issue's stated criteria order of importance:
|
||||||
|
capability + historical_success(τ) dominate, budget/risk/load are correctors.
|
||||||
|
"""
|
||||||
|
w_confidence: float = 0.20 # bidder's self-reported confidence in TaskBid
|
||||||
|
w_capability: float = 0.25 # capability fit vs. the task's required capabilities
|
||||||
|
w_historical_success: float = 0.25 # τ — earned reputation (ACO pheromone trail)
|
||||||
|
w_budget: float = 0.10 # cost headroom (cheaper bid scores higher)
|
||||||
|
w_time: float = 0.05 # speed (faster estimate scores higher)
|
||||||
|
w_risk: float = 0.10 # lower risk_score scores higher
|
||||||
|
w_load: float = 0.05 # less-loaded agent scores higher
|
||||||
|
|
||||||
|
# Minimum score gap for a *clear* winner; below it the result is flagged contested
|
||||||
|
# (still deterministic — the winner is the stable-sort head — but callers can choose
|
||||||
|
# to escalate to a human/Manager approval step rather than auto-assign).
|
||||||
|
decisive_margin: float = 0.02
|
||||||
|
|
||||||
|
|
||||||
|
# --- message / data models ---------------------------------------------------
|
||||||
|
class TaskBid(BaseModel):
|
||||||
|
"""An agent's bid to take (or keep) a task.
|
||||||
|
|
||||||
|
confidence/risk_score in [0, 1]; estimated_cost is a fraction of the run budget
|
||||||
|
(0 = free, >=1 = at/over budget); estimated_time in seconds; current_load = number of
|
||||||
|
tasks the bidding agent currently has in flight.
|
||||||
|
"""
|
||||||
|
type: str = "task_bid"
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
confidence: float = 0.5
|
||||||
|
estimated_cost: float = 0.0
|
||||||
|
estimated_time: float = 0.0
|
||||||
|
risk_score: float = 0.0
|
||||||
|
reason: str = ""
|
||||||
|
capabilities: List[str] = Field(default_factory=list)
|
||||||
|
current_load: int = 0
|
||||||
|
|
||||||
|
|
||||||
|
class TaskYield(BaseModel):
|
||||||
|
"""An agent voluntarily releasing a task back for re-competition, with rationale and
|
||||||
|
an optional recommendation of who should pick it up next."""
|
||||||
|
type: str = "task_yield"
|
||||||
|
task_id: str
|
||||||
|
agent_id: str
|
||||||
|
release_with_reason: str
|
||||||
|
recommend_agent: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class TaskTakeoverRequest(BaseModel):
|
||||||
|
"""An agent asking to take over a task currently held by another agent (e.g. it is
|
||||||
|
stalled, or the requester is a better fit). Carries a bid so the same arbitrator can
|
||||||
|
weigh requester vs. incumbent on identical terms."""
|
||||||
|
type: str = "task_takeover_request"
|
||||||
|
task_id: str
|
||||||
|
requesting_agent_id: str
|
||||||
|
current_agent_id: Optional[str] = None
|
||||||
|
reason: str = ""
|
||||||
|
bid: Optional[TaskBid] = None
|
||||||
|
|
||||||
|
|
||||||
|
class ArbitrationScore(BaseModel):
|
||||||
|
"""Per-bid, fully broken-down score — the audit trail for one competitor."""
|
||||||
|
agent_id: str
|
||||||
|
total: float
|
||||||
|
components: Dict[str, float]
|
||||||
|
|
||||||
|
|
||||||
|
class TaskArbitrationResult(BaseModel):
|
||||||
|
"""The deterministic, explainable outcome of arbitrating a set of bids."""
|
||||||
|
type: str = "task_arbitration_result"
|
||||||
|
task_id: str
|
||||||
|
winner_agent_id: Optional[str]
|
||||||
|
reason: str
|
||||||
|
decisive: bool
|
||||||
|
scores: List[ArbitrationScore]
|
||||||
|
losers: List[str]
|
||||||
|
policy: ArbitrationPolicy
|
||||||
|
arbitrated_at: float = Field(default_factory=time.time)
|
||||||
|
|
||||||
|
|
||||||
|
# --- scoring -----------------------------------------------------------------
|
||||||
|
def _capability_fit(bid_caps: List[str], required: List[str]) -> float:
|
||||||
|
"""Fraction of required capabilities the bidder covers. No requirement ⇒ perfect fit.
|
||||||
|
Pure set math — same shape as task_queue.can_agent_run_task / decision_engine match."""
|
||||||
|
req = set(required or [])
|
||||||
|
if not req:
|
||||||
|
return 1.0
|
||||||
|
caps = set(bid_caps or [])
|
||||||
|
return len(req & caps) / len(req)
|
||||||
|
|
||||||
|
|
||||||
|
def _clamp01(x: float) -> float:
|
||||||
|
return max(0.0, min(1.0, x))
|
||||||
|
|
||||||
|
|
||||||
|
def _score_bid(
|
||||||
|
bid: TaskBid,
|
||||||
|
*,
|
||||||
|
required_capabilities: List[str],
|
||||||
|
historical_success: Dict[str, float],
|
||||||
|
policy: ArbitrationPolicy,
|
||||||
|
) -> ArbitrationScore:
|
||||||
|
"""Map one bid to a weighted scalar with every component recorded for audit.
|
||||||
|
|
||||||
|
Each component is normalised to [0, 1] first (so weights are comparable), then
|
||||||
|
multiplied by its policy weight. "Lower is better" fields (cost/time/risk/load) are
|
||||||
|
converted to headroom (1 - normalised) so that higher always means better.
|
||||||
|
"""
|
||||||
|
confidence = _clamp01(bid.confidence)
|
||||||
|
capability = _capability_fit(bid.capabilities, required_capabilities)
|
||||||
|
# τ: documented neutral when this agent has no recorded reputation yet.
|
||||||
|
tau = historical_success.get(bid.agent_id, HISTORICAL_SUCCESS_NEUTRAL)
|
||||||
|
tau = _clamp01(tau)
|
||||||
|
budget = _clamp01(1.0 - bid.estimated_cost / COST_NORM)
|
||||||
|
speed = _clamp01(1.0 - bid.estimated_time / TIME_NORM_SECONDS)
|
||||||
|
risk = _clamp01(1.0 - bid.risk_score)
|
||||||
|
load = _clamp01(1.0 - max(0, bid.current_load) / LOAD_NORM_SLOTS)
|
||||||
|
|
||||||
|
components = {
|
||||||
|
"confidence": round(policy.w_confidence * confidence, 6),
|
||||||
|
"capability": round(policy.w_capability * capability, 6),
|
||||||
|
"historical_success": round(policy.w_historical_success * tau, 6),
|
||||||
|
"budget": round(policy.w_budget * budget, 6),
|
||||||
|
"time": round(policy.w_time * speed, 6),
|
||||||
|
"risk": round(policy.w_risk * risk, 6),
|
||||||
|
"load": round(policy.w_load * load, 6),
|
||||||
|
}
|
||||||
|
total = round(sum(components.values()), 6)
|
||||||
|
return ArbitrationScore(agent_id=bid.agent_id, total=total, components=components)
|
||||||
|
|
||||||
|
|
||||||
|
def arbitrate(
|
||||||
|
bids: List[TaskBid],
|
||||||
|
policy: Optional[ArbitrationPolicy] = None,
|
||||||
|
*,
|
||||||
|
required_capabilities: Optional[List[str]] = None,
|
||||||
|
historical_success: Optional[Dict[str, float]] = None,
|
||||||
|
) -> TaskArbitrationResult:
|
||||||
|
"""Pick a winner from competing bids — deterministic and fully explainable.
|
||||||
|
|
||||||
|
Scoring: weighted sum of capability fit, historical_success (τ), self-confidence,
|
||||||
|
budget headroom, speed, inverse risk and inverse load (see `_score_bid`).
|
||||||
|
|
||||||
|
Determinism: bids are scored independently (no shared mutable state), then ordered by
|
||||||
|
(total DESC, agent_id ASC). The leading tuple is the winner. The agent_id tiebreak
|
||||||
|
removes any dependence on input order or dict iteration order, so the same inputs
|
||||||
|
always yield the same winner, the same scores and the same ordered losers.
|
||||||
|
|
||||||
|
The result records the human-readable reason (winning agent, its margin, dominant
|
||||||
|
component) and flags `decisive=False` when the top-two gap is below
|
||||||
|
`policy.decisive_margin`, so callers can route contested ties to Manager approval
|
||||||
|
instead of auto-assigning.
|
||||||
|
"""
|
||||||
|
policy = policy or ArbitrationPolicy()
|
||||||
|
required = required_capabilities or []
|
||||||
|
hist = historical_success or {}
|
||||||
|
|
||||||
|
task_id = bids[0].task_id if bids else ""
|
||||||
|
|
||||||
|
if not bids:
|
||||||
|
return TaskArbitrationResult(
|
||||||
|
task_id=task_id,
|
||||||
|
winner_agent_id=None,
|
||||||
|
reason="No bids submitted; nothing to arbitrate.",
|
||||||
|
decisive=False,
|
||||||
|
scores=[],
|
||||||
|
losers=[],
|
||||||
|
policy=policy,
|
||||||
|
)
|
||||||
|
|
||||||
|
scored: List[ArbitrationScore] = [
|
||||||
|
_score_bid(
|
||||||
|
bid,
|
||||||
|
required_capabilities=required,
|
||||||
|
historical_success=hist,
|
||||||
|
policy=policy,
|
||||||
|
)
|
||||||
|
for bid in bids
|
||||||
|
]
|
||||||
|
|
||||||
|
# Stable, documented ordering: higher total first; agent_id ascending breaks ties.
|
||||||
|
ranked: List[ArbitrationScore] = sorted(
|
||||||
|
scored, key=lambda s: (-s.total, s.agent_id)
|
||||||
|
)
|
||||||
|
|
||||||
|
winner = ranked[0]
|
||||||
|
runner_up_total = ranked[1].total if len(ranked) > 1 else None
|
||||||
|
margin = (winner.total - runner_up_total) if runner_up_total is not None else winner.total
|
||||||
|
decisive = margin >= policy.decisive_margin or len(ranked) == 1
|
||||||
|
|
||||||
|
dominant = max(winner.components.items(), key=lambda kv: (kv[1], kv[0]))[0]
|
||||||
|
if len(ranked) == 1:
|
||||||
|
reason = (
|
||||||
|
f"Sole bidder {winner.agent_id} wins task {task_id} uncontested "
|
||||||
|
f"(score={winner.total}, dominant factor={dominant})."
|
||||||
|
)
|
||||||
|
elif decisive:
|
||||||
|
reason = (
|
||||||
|
f"Agent {winner.agent_id} wins task {task_id} with score {winner.total} "
|
||||||
|
f"(margin {round(margin, 6)} over {ranked[1].agent_id}@{ranked[1].total}); "
|
||||||
|
f"dominant factor={dominant}."
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
reason = (
|
||||||
|
f"Agent {winner.agent_id} narrowly leads task {task_id} "
|
||||||
|
f"(score {winner.total} vs {ranked[1].agent_id}@{ranked[1].total}, "
|
||||||
|
f"margin {round(margin, 6)} < decisive_margin {policy.decisive_margin}); "
|
||||||
|
f"contested — recommend Manager review before assignment."
|
||||||
|
)
|
||||||
|
|
||||||
|
losers = [s.agent_id for s in ranked[1:]]
|
||||||
|
|
||||||
|
logger.debug(
|
||||||
|
"arbitrate task=%s winner=%s decisive=%s losers=%s",
|
||||||
|
task_id, winner.agent_id, decisive, losers,
|
||||||
|
)
|
||||||
|
return TaskArbitrationResult(
|
||||||
|
task_id=task_id,
|
||||||
|
winner_agent_id=winner.agent_id,
|
||||||
|
reason=reason,
|
||||||
|
decisive=decisive,
|
||||||
|
scores=ranked,
|
||||||
|
losers=losers,
|
||||||
|
policy=policy,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# --- event payload builders --------------------------------------------------
|
||||||
|
# These build the PAYLOAD dicts only; they do NOT emit. Wiring them onto
|
||||||
|
# swarm_runtime.emit_event(run, event_type, ..., payload=...) is documented (not done)
|
||||||
|
# in the companion doc's integration notes. Shapes mirror existing HM event payloads
|
||||||
|
# (task_id + agent role/id + a human summary) so they slot into the contract cleanly.
|
||||||
|
def bid_submitted_event(bid: TaskBid) -> Tuple[str, Dict]:
|
||||||
|
"""task.bid_submitted — one agent has entered a bid."""
|
||||||
|
return "task.bid_submitted", {
|
||||||
|
"task_id": bid.task_id,
|
||||||
|
"agent_id": bid.agent_id,
|
||||||
|
"confidence": bid.confidence,
|
||||||
|
"estimated_cost": bid.estimated_cost,
|
||||||
|
"estimated_time": bid.estimated_time,
|
||||||
|
"risk_score": bid.risk_score,
|
||||||
|
"current_load": bid.current_load,
|
||||||
|
"capabilities": bid.capabilities,
|
||||||
|
"summary": bid.reason or f"Agent {bid.agent_id} bid for task {bid.task_id}",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def yielded_event(yield_msg: TaskYield) -> Tuple[str, Dict]:
|
||||||
|
"""task.yielded — an agent has released a task with a reason."""
|
||||||
|
return "task.yielded", {
|
||||||
|
"task_id": yield_msg.task_id,
|
||||||
|
"agent_id": yield_msg.agent_id,
|
||||||
|
"reason": yield_msg.release_with_reason,
|
||||||
|
"recommend_agent": yield_msg.recommend_agent,
|
||||||
|
"summary": (
|
||||||
|
f"Agent {yield_msg.agent_id} yielded task {yield_msg.task_id}: "
|
||||||
|
f"{yield_msg.release_with_reason}"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def takeover_requested_event(req: TaskTakeoverRequest) -> Tuple[str, Dict]:
|
||||||
|
"""task.takeover_requested — an agent asks to take a task from the current holder."""
|
||||||
|
return "task.takeover_requested", {
|
||||||
|
"task_id": req.task_id,
|
||||||
|
"requesting_agent_id": req.requesting_agent_id,
|
||||||
|
"current_agent_id": req.current_agent_id,
|
||||||
|
"reason": req.reason,
|
||||||
|
"summary": (
|
||||||
|
f"Agent {req.requesting_agent_id} requests takeover of task {req.task_id}"
|
||||||
|
+ (f" from {req.current_agent_id}" if req.current_agent_id else "")
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def arbitrated_event(result: TaskArbitrationResult) -> Tuple[str, Dict]:
|
||||||
|
"""task.arbitrated — the arbitrator has chosen a winner; carries the full audit trail."""
|
||||||
|
return "task.arbitrated", {
|
||||||
|
"task_id": result.task_id,
|
||||||
|
"winner_agent_id": result.winner_agent_id,
|
||||||
|
"decisive": result.decisive,
|
||||||
|
"reason": result.reason,
|
||||||
|
"losers": result.losers,
|
||||||
|
"scores": [s.model_dump() for s in result.scores],
|
||||||
|
"summary": result.reason,
|
||||||
|
}
|
||||||
+35
-22
@@ -1,9 +1,10 @@
|
|||||||
"""Keyless stub agent for workflow testing.
|
"""Keyless stub agent for the SWARM workflow test (decentralized flow).
|
||||||
|
|
||||||
Connects to the orchestrator over WebSocket, registers, and returns canned results WITHOUT
|
Connects over WebSocket, registers, and drives the swarm flow WITHOUT any LLM (no key needed):
|
||||||
calling any LLM (no OPENAI_API_KEY needed). To make the master review loop deterministic, the
|
when it self-selects the SEED task it PROPOSES the subtasks (bottom-up decomposition, #7) and then
|
||||||
"testing" task returns a conflicting framework (unittest) on its FIRST execution and an aligned
|
completes the seed; for every subsequently self-selected subtask it returns a canned result. This
|
||||||
one (pytest) on retry — so the heuristic critic rejects exactly once and then accepts.
|
exercises seed → agent-proposed decomposition → self-selected execution → convergence — with no
|
||||||
|
Master plan and no central dispatch.
|
||||||
|
|
||||||
Run standalone against a manually started orchestrator:
|
Run standalone against a manually started orchestrator:
|
||||||
set "ORCHESTRATOR_URL=ws://localhost:8000"
|
set "ORCHESTRATOR_URL=ws://localhost:8000"
|
||||||
@@ -18,6 +19,16 @@ import time
|
|||||||
|
|
||||||
import websockets
|
import websockets
|
||||||
|
|
||||||
|
# Subtasks the agent proposes when it perceives the seed (bottom-up decomposition).
|
||||||
|
SEED_SUBTASKS = [
|
||||||
|
{"description": "Implement the core add(a, b) function in calc.py",
|
||||||
|
"agent_role": "implementation", "required_capabilities": ["python", "code_generation"]},
|
||||||
|
{"description": "Write pytest unit tests covering add(a, b)",
|
||||||
|
"agent_role": "testing", "required_capabilities": ["testing", "pytest"]},
|
||||||
|
{"description": "Document usage of add(a, b) in the README",
|
||||||
|
"agent_role": "documentation", "required_capabilities": ["technical-writing", "general"]},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
class StubAgent:
|
class StubAgent:
|
||||||
def __init__(self, orchestrator_url: str, agent_id: str, capabilities: list[str]):
|
def __init__(self, orchestrator_url: str, agent_id: str, capabilities: list[str]):
|
||||||
@@ -25,26 +36,15 @@ class StubAgent:
|
|||||||
self.agent_id = agent_id
|
self.agent_id = agent_id
|
||||||
self.capabilities = capabilities
|
self.capabilities = capabilities
|
||||||
self.ws = None
|
self.ws = None
|
||||||
self.runs: dict[str, int] = {} # task_id -> execution count
|
self.proposed = False # only decompose the seed once
|
||||||
self.running = True
|
self.running = True
|
||||||
|
|
||||||
def _result_for(self, task_id: str, description: str) -> dict:
|
def _result_for(self, task_id: str, description: str) -> dict:
|
||||||
count = self.runs.get(task_id, 0) + 1
|
|
||||||
self.runs[task_id] = count
|
|
||||||
tid = task_id.lower()
|
|
||||||
if "implementation" in tid:
|
|
||||||
summary = "implemented add(a, b); tests should use pytest"
|
|
||||||
elif "testing" in tid:
|
|
||||||
# First attempt conflicts (unittest); retry aligns with implementation (pytest).
|
|
||||||
summary = "wrote tests with unittest" if count == 1 else "wrote tests with pytest"
|
|
||||||
elif "documentation" in tid:
|
|
||||||
summary = "documented usage of add(a, b)"
|
|
||||||
else:
|
|
||||||
summary = f"completed: {description[:40]}"
|
|
||||||
return {
|
return {
|
||||||
"success": True,
|
"success": True,
|
||||||
"task_id": task_id,
|
"task_id": task_id,
|
||||||
"subtasks": [{"status": "completed", "summary": summary, "changes": summary, "files": []}],
|
"subtasks": [{"status": "completed", "summary": f"completed: {description[:60]}",
|
||||||
|
"changes": description[:60], "files": []}],
|
||||||
"awaiting_handoff": False,
|
"awaiting_handoff": False,
|
||||||
"agent_id": self.agent_id,
|
"agent_id": self.agent_id,
|
||||||
"usage": {
|
"usage": {
|
||||||
@@ -60,16 +60,29 @@ class StubAgent:
|
|||||||
msg_type = msg.get("type")
|
msg_type = msg.get("type")
|
||||||
if msg_type == "task_assignment":
|
if msg_type == "task_assignment":
|
||||||
task_id = msg["task_id"]
|
task_id = msg["task_id"]
|
||||||
|
context = msg.get("context") or {}
|
||||||
await self.send({"type": "task_accepted", "task_id": task_id, "available_slots": 4})
|
await self.send({"type": "task_accepted", "task_id": task_id, "available_slots": 4})
|
||||||
await self.send({"type": "task_start", "agent_id": self.agent_id, "task_id": task_id, "timestamp": time.time()})
|
await self.send({"type": "task_start", "agent_id": self.agent_id, "task_id": task_id, "timestamp": time.time()})
|
||||||
await asyncio.sleep(0.2) # simulate work
|
await asyncio.sleep(0.2) # simulate work
|
||||||
result = self._result_for(task_id, msg.get("description", ""))
|
|
||||||
|
# Perceived the seed → propose the subtasks (decentralized decomposition), then finish it.
|
||||||
|
if context.get("is_seed") and not self.proposed:
|
||||||
|
self.proposed = True
|
||||||
|
for sub in SEED_SUBTASKS:
|
||||||
|
await self.send({
|
||||||
|
"type": "task_proposal", "agent_id": self.agent_id,
|
||||||
|
"origin_task_id": task_id, "confidence": 0.9,
|
||||||
|
"reason": "decomposing the seed objective into specialist subtasks",
|
||||||
|
"trigger_event": "task.assignment", **sub,
|
||||||
|
})
|
||||||
|
await asyncio.sleep(0.4) # let the orchestrator review + enqueue the proposals
|
||||||
|
|
||||||
await self.send({
|
await self.send({
|
||||||
"type": "task_complete", "agent_id": self.agent_id,
|
"type": "task_complete", "agent_id": self.agent_id,
|
||||||
"task_id": task_id, "result": result, "timestamp": time.time(),
|
"task_id": task_id, "result": self._result_for(task_id, msg.get("description", "")),
|
||||||
|
"timestamp": time.time(),
|
||||||
})
|
})
|
||||||
elif msg_type == "peer_message" and not msg.get("is_reply"):
|
elif msg_type == "peer_message" and not msg.get("is_reply"):
|
||||||
# Answer an inbound peer query so collaboration round-trips complete.
|
|
||||||
await self.send({
|
await self.send({
|
||||||
"type": "peer_message", "agent_id": self.agent_id,
|
"type": "peer_message", "agent_id": self.agent_id,
|
||||||
"target_agent_id": msg.get("from_agent_id"), "task_id": msg.get("task_id"),
|
"target_agent_id": msg.get("from_agent_id"), "task_id": msg.get("task_id"),
|
||||||
|
|||||||
@@ -0,0 +1,239 @@
|
|||||||
|
r"""Module-level tests for agent-proposed autonomous task generation (issue #7).
|
||||||
|
|
||||||
|
Hermetic: exercises orchestrator/autonomous_tasks.py directly. The module is pure (no Redis, no
|
||||||
|
WebSocket, no FastAPI), so this needs no running orchestrator, no model key, and no Redis — it
|
||||||
|
simulates an agent submitting a proposal from a shared-state snapshot, the policy
|
||||||
|
accepting/rejecting/merging it, and the accepted proposal producing a new pending-task spec with
|
||||||
|
full lineage and status transitions.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — the module itself has no third-party deps, but the
|
||||||
|
repo's standard env is assumed):
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
..\.venv\Scripts\python.exe scripts/test-autonomous-tasks.py
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.autonomous_tasks import (
|
||||||
|
TaskProposal,
|
||||||
|
ProposalLineage,
|
||||||
|
ProposalPolicy,
|
||||||
|
ProposalStatus,
|
||||||
|
ProposalDecision,
|
||||||
|
ExistingTaskRef,
|
||||||
|
PROPOSED_SOURCE,
|
||||||
|
EVENT_SUBMITTED,
|
||||||
|
EVENT_ACCEPTED,
|
||||||
|
EVENT_REJECTED,
|
||||||
|
EVENT_MERGED,
|
||||||
|
review_proposal,
|
||||||
|
ingest_accepted_proposal,
|
||||||
|
build_lifecycle_events_for_outcome,
|
||||||
|
description_similarity,
|
||||||
|
assert_not_master_origin,
|
||||||
|
)
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
def make_proposal(**overrides):
|
||||||
|
"""An agent proposing a follow-up task it discovered from the shared run state."""
|
||||||
|
snapshot = {
|
||||||
|
"completed_tasks": ["swarm-1-impl"],
|
||||||
|
"open_gaps": ["no unit tests for the new module"],
|
||||||
|
"pending_task_ids": [],
|
||||||
|
}
|
||||||
|
kwargs = dict(
|
||||||
|
proposed_by_agent_id="agent-7a3f",
|
||||||
|
title="Add unit tests for payment module",
|
||||||
|
description="Write unit tests covering the new payment module added by the impl task",
|
||||||
|
proposal_reason="Implementation landed with no test coverage; gap found in shared state",
|
||||||
|
proposal_confidence=0.82,
|
||||||
|
agent_role="testing",
|
||||||
|
required_capabilities=["testing"],
|
||||||
|
lineage=ProposalLineage(
|
||||||
|
origin_task_id="swarm-1-impl",
|
||||||
|
trigger_event="task.completed",
|
||||||
|
shared_state_snapshot=snapshot,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
kwargs.update(overrides)
|
||||||
|
return TaskProposal(**kwargs)
|
||||||
|
|
||||||
|
|
||||||
|
def test_proposal_invariants():
|
||||||
|
p = make_proposal()
|
||||||
|
check("source pinned to agent_proposed", p.source == PROPOSED_SOURCE)
|
||||||
|
check("initial status PROPOSED", p.status == ProposalStatus.PROPOSED)
|
||||||
|
check("proposal_id generated", bool(p.proposal_id) and p.proposal_id.startswith("prop-"))
|
||||||
|
check("confidence preserved", abs(p.proposal_confidence - 0.82) < 1e-9)
|
||||||
|
|
||||||
|
# source cannot be overridden away from agent_proposed
|
||||||
|
p2 = make_proposal(source="planner")
|
||||||
|
check("source override ignored", p2.source == PROPOSED_SOURCE)
|
||||||
|
|
||||||
|
# confidence clamped into [0,1]
|
||||||
|
p3 = make_proposal(proposal_confidence=5.0)
|
||||||
|
check("confidence clamped high", p3.proposal_confidence == 1.0)
|
||||||
|
p4 = make_proposal(proposal_confidence=-1.0)
|
||||||
|
check("confidence clamped low", p4.proposal_confidence == 0.0)
|
||||||
|
|
||||||
|
# requires an agent id
|
||||||
|
try:
|
||||||
|
make_proposal(proposed_by_agent_id="")
|
||||||
|
check("empty proposer rejected", False)
|
||||||
|
except ValueError:
|
||||||
|
check("empty proposer rejected", True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_not_master_origin():
|
||||||
|
# A Master/planner must not launder generated tasks through this agent path.
|
||||||
|
bad = make_proposal(proposed_by_agent_id="master-agent")
|
||||||
|
try:
|
||||||
|
assert_not_master_origin(bad)
|
||||||
|
check("master proposer rejected", False)
|
||||||
|
except ValueError:
|
||||||
|
check("master proposer rejected", True)
|
||||||
|
|
||||||
|
bad2 = make_proposal(
|
||||||
|
lineage=ProposalLineage(origin_task_id="x", trigger_event="planner.decomposed")
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
assert_not_master_origin(bad2)
|
||||||
|
check("planner trigger rejected", False)
|
||||||
|
except ValueError:
|
||||||
|
check("planner trigger rejected", True)
|
||||||
|
|
||||||
|
check("agent proposer allowed", assert_not_master_origin(make_proposal()) is None)
|
||||||
|
|
||||||
|
|
||||||
|
def test_accept_path():
|
||||||
|
p = make_proposal()
|
||||||
|
outcome = review_proposal(p, ProposalPolicy(), existing_tasks=[])
|
||||||
|
check("accept decision", outcome.decision == ProposalDecision.ACCEPT)
|
||||||
|
check("accept status transition", p.status == ProposalStatus.ACCEPTED)
|
||||||
|
check("accept reason set", bool(p.decision_reason))
|
||||||
|
|
||||||
|
spec = ingest_accepted_proposal(p, swarm_id="swarm-1")
|
||||||
|
check("spec source agent_proposed", spec["source"] == PROPOSED_SOURCE)
|
||||||
|
check("spec description carried", spec["description"] == p.description)
|
||||||
|
check("spec role carried", spec["agent_role"] == "testing")
|
||||||
|
# lineage fully threaded into context for audit
|
||||||
|
prov = spec["context"]["proposal"]
|
||||||
|
check("lineage proposer", prov["proposed_by_agent_id"] == "agent-7a3f")
|
||||||
|
check("lineage reason", "test coverage" in prov["proposal_reason"].lower())
|
||||||
|
check("lineage origin task", prov["origin_task_id"] == "swarm-1-impl")
|
||||||
|
check("lineage trigger", prov["trigger_event"] == "task.completed")
|
||||||
|
check("lineage snapshot preserved", prov["shared_state_snapshot"]["open_gaps"][0].startswith("no unit tests"))
|
||||||
|
# origin task becomes parent so the new task threads into the run
|
||||||
|
check("spec parent = origin", spec["parent_task_id"] == "swarm-1-impl")
|
||||||
|
|
||||||
|
# cannot ingest a non-accepted proposal
|
||||||
|
rej = make_proposal(proposal_confidence=0.1)
|
||||||
|
review_proposal(rej, ProposalPolicy(), existing_tasks=[])
|
||||||
|
try:
|
||||||
|
ingest_accepted_proposal(rej)
|
||||||
|
check("ingest blocks non-accepted", False)
|
||||||
|
except ValueError:
|
||||||
|
check("ingest blocks non-accepted", True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_reject_low_confidence():
|
||||||
|
p = make_proposal(proposal_confidence=0.3)
|
||||||
|
outcome = review_proposal(p, ProposalPolicy(min_confidence=0.6), existing_tasks=[])
|
||||||
|
check("reject low confidence", outcome.decision == ProposalDecision.REJECT)
|
||||||
|
check("reject status", p.status == ProposalStatus.REJECTED)
|
||||||
|
check("reject reason mentions confidence", "confidence" in (p.decision_reason or ""))
|
||||||
|
|
||||||
|
|
||||||
|
def test_reject_budget_exhausted():
|
||||||
|
p = make_proposal()
|
||||||
|
policy = ProposalPolicy(remaining_proposal_budget=0)
|
||||||
|
outcome = review_proposal(p, policy, existing_tasks=[])
|
||||||
|
check("reject when budget exhausted", outcome.decision == ProposalDecision.REJECT)
|
||||||
|
check("budget reject status", p.status == ProposalStatus.REJECTED)
|
||||||
|
|
||||||
|
|
||||||
|
def test_merge_duplicate():
|
||||||
|
existing = [
|
||||||
|
ExistingTaskRef(
|
||||||
|
task_id="swarm-1-existing-tests",
|
||||||
|
description="Write unit tests covering the new payment module added by the impl task",
|
||||||
|
agent_role="testing",
|
||||||
|
status="pending",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
p = make_proposal()
|
||||||
|
outcome = review_proposal(p, ProposalPolicy(), existing_tasks=existing)
|
||||||
|
check("merge decision on duplicate", outcome.decision == ProposalDecision.MERGE)
|
||||||
|
check("merge status", p.status == ProposalStatus.MERGED)
|
||||||
|
check("merge target recorded", p.merged_into_task_id == "swarm-1-existing-tests")
|
||||||
|
check("merge outcome target", outcome.merge_target_task_id == "swarm-1-existing-tests")
|
||||||
|
|
||||||
|
# a completed task is NOT a dedup target (follow-up rounds allowed)
|
||||||
|
done = [ExistingTaskRef(task_id="t", description=p.description, status="completed")]
|
||||||
|
p2 = make_proposal()
|
||||||
|
outcome2 = review_proposal(p2, ProposalPolicy(), existing_tasks=done)
|
||||||
|
check("completed task not deduped", outcome2.decision == ProposalDecision.ACCEPT)
|
||||||
|
|
||||||
|
|
||||||
|
def test_similarity_metric():
|
||||||
|
check("identical similarity 1.0", description_similarity("add tests now", "add tests now") == 1.0)
|
||||||
|
check("disjoint similarity 0.0", description_similarity("alpha beta", "gamma delta") == 0.0)
|
||||||
|
check("empty similarity 0.0", description_similarity("", "anything") == 0.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_lifecycle_events():
|
||||||
|
# accept -> submitted + accepted
|
||||||
|
p = make_proposal()
|
||||||
|
out = review_proposal(p, ProposalPolicy(), existing_tasks=[])
|
||||||
|
events = build_lifecycle_events_for_outcome(p, out)
|
||||||
|
check("accept emits 2 events", len(events) == 2)
|
||||||
|
check("first event submitted", events[0]["event_type"] == EVENT_SUBMITTED)
|
||||||
|
check("second event accepted", events[1]["event_type"] == EVENT_ACCEPTED)
|
||||||
|
check("event carries proposer", events[0]["proposed_by_agent_id"] == "agent-7a3f")
|
||||||
|
check("event carries source", events[0]["source"] == PROPOSED_SOURCE)
|
||||||
|
|
||||||
|
# reject -> submitted + rejected
|
||||||
|
r = make_proposal(proposal_confidence=0.1)
|
||||||
|
out_r = review_proposal(r, ProposalPolicy(), existing_tasks=[])
|
||||||
|
ev_r = build_lifecycle_events_for_outcome(r, out_r)
|
||||||
|
check("reject second event", ev_r[1]["event_type"] == EVENT_REJECTED)
|
||||||
|
|
||||||
|
# merge -> submitted + merged with target
|
||||||
|
existing = [ExistingTaskRef(task_id="dupe", description=make_proposal().description, status="pending")]
|
||||||
|
m = make_proposal()
|
||||||
|
out_m = review_proposal(m, ProposalPolicy(), existing_tasks=existing)
|
||||||
|
ev_m = build_lifecycle_events_for_outcome(m, out_m)
|
||||||
|
check("merge second event", ev_m[1]["event_type"] == EVENT_MERGED)
|
||||||
|
check("merge event has target", ev_m[1]["merge_target_task_id"] == "dupe")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
test_proposal_invariants()
|
||||||
|
test_not_master_origin()
|
||||||
|
test_accept_path()
|
||||||
|
test_reject_low_confidence()
|
||||||
|
test_reject_budget_exhausted()
|
||||||
|
test_merge_duplicate()
|
||||||
|
test_similarity_metric()
|
||||||
|
test_lifecycle_events()
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("ALL AUTONOMOUS-TASK TESTS PASSED")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,233 @@
|
|||||||
|
"""Hermetic unit tests for the swarm convergence protocol (Issue #12).
|
||||||
|
|
||||||
|
Module-level only: no WebSocket, no Redis, no model, no FastAPI. Exercises the
|
||||||
|
pure functions in orchestrator/convergence.py directly.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (Windows, repo .venv):
|
||||||
|
|
||||||
|
..\\.venv\\Scripts\\python.exe scripts\\test-convergence.py
|
||||||
|
|
||||||
|
If the .venv is missing, it only needs the stdlib + the repo on sys.path:
|
||||||
|
|
||||||
|
py -m venv ..\\.venv
|
||||||
|
..\\.venv\\Scripts\\python.exe -m pip install -r orchestrator\\requirements.txt
|
||||||
|
..\\.venv\\Scripts\\python.exe scripts\\test-convergence.py
|
||||||
|
|
||||||
|
(No third-party import is actually required by this test — convergence.py uses
|
||||||
|
only the stdlib — but the command above matches the repo convention.)
|
||||||
|
"""
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.convergence import ( # noqa: E402
|
||||||
|
ConflictType,
|
||||||
|
ConvergenceStatus,
|
||||||
|
TerminationReason,
|
||||||
|
detect_conflicts,
|
||||||
|
evaluate_convergence,
|
||||||
|
event_conflict_detected,
|
||||||
|
event_conflict_resolved,
|
||||||
|
event_consensus_updated,
|
||||||
|
event_convergence_failed,
|
||||||
|
event_convergence_reached,
|
||||||
|
event_convergence_started,
|
||||||
|
resolve_conflicts,
|
||||||
|
)
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
# --- scenario: two conflicting artifacts; one conflict resolvable ---------
|
||||||
|
# task-a and task-b both wrote different content to src/app.py (artifact
|
||||||
|
# mismatch). The master critic REJECTED the work and flagged task-b for redo,
|
||||||
|
# which makes the review-disagreement conflict actionable (resolvable), while
|
||||||
|
# the artifact mismatch stays unresolved (review did not accept a winner).
|
||||||
|
|
||||||
|
run_state = {
|
||||||
|
"tasks": [
|
||||||
|
{
|
||||||
|
"task_id": "task-a",
|
||||||
|
"status": "completed",
|
||||||
|
"depends_on": [],
|
||||||
|
"result": {
|
||||||
|
"files_modified": ["src/app.py"],
|
||||||
|
"file_hashes": {"src/app.py": "hashAAA"},
|
||||||
|
"tests_passed": True,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_id": "task-b",
|
||||||
|
"status": "completed",
|
||||||
|
"depends_on": [],
|
||||||
|
"result": {
|
||||||
|
"files_modified": ["src/app.py"],
|
||||||
|
"file_hashes": {"src/app.py": "hashBBB"},
|
||||||
|
"tests_passed": True,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"review_verdict": {
|
||||||
|
"accepted": False,
|
||||||
|
"retry_tasks": ["task-b"],
|
||||||
|
"summary": "Two versions of src/app.py disagree; redo task-b",
|
||||||
|
},
|
||||||
|
"budget": {"max_cost_usd": 10.0},
|
||||||
|
"usage": {"total_cost_usd": 1.0},
|
||||||
|
}
|
||||||
|
|
||||||
|
conflicts = detect_conflicts(run_state["tasks"], run_state["review_verdict"])
|
||||||
|
types = {c.type for c in conflicts}
|
||||||
|
check("artifact mismatch detected", ConflictType.ARTIFACT_MISMATCH in types)
|
||||||
|
check("review disagreement detected", ConflictType.REVIEW_DISAGREEMENT in types)
|
||||||
|
|
||||||
|
resolved, unresolved = resolve_conflicts(conflicts, run_state["review_verdict"])
|
||||||
|
check("at least one conflict resolved (review disagreement closes)", len(resolved) >= 1)
|
||||||
|
check("review disagreement is among the resolved",
|
||||||
|
any(c.type == ConflictType.REVIEW_DISAGREEMENT and c.resolved for c in resolved))
|
||||||
|
check("artifact mismatch stays unresolved (no accepted winner)",
|
||||||
|
any(c.type == ConflictType.ARTIFACT_MISMATCH and not c.resolved for c in unresolved))
|
||||||
|
check("every resolved conflict carries a resolution note",
|
||||||
|
all(c.resolution for c in resolved))
|
||||||
|
|
||||||
|
report = evaluate_convergence(run_state)
|
||||||
|
check("conflicting run is not silently CONVERGED (blocked by unresolved artifact)",
|
||||||
|
report.status == ConvergenceStatus.BLOCKED)
|
||||||
|
check("blocked run carries a concrete termination_reason",
|
||||||
|
report.termination_reason == TerminationReason.RISK_BLOCKED)
|
||||||
|
check("consensus < 100 when work is in conflict", report.consensus_score < 100.0)
|
||||||
|
check("report.to_dict() round-trips conflicts",
|
||||||
|
len(report.to_dict()["conflicts"]) == len(conflicts))
|
||||||
|
|
||||||
|
|
||||||
|
# --- assert: every TERMINAL state carries a termination_reason ------------
|
||||||
|
# Build one run-state per intended terminal outcome and assert the reason.
|
||||||
|
|
||||||
|
def completed_task(tid, **result):
|
||||||
|
return {"task_id": tid, "status": "completed", "depends_on": [], "result": result}
|
||||||
|
|
||||||
|
|
||||||
|
terminal_cases = {
|
||||||
|
# clean completion -> tasks_completed fallback
|
||||||
|
"tasks_completed": {
|
||||||
|
"tasks": [completed_task("t1", tests_passed=True)],
|
||||||
|
"expect_status": ConvergenceStatus.CONVERGED,
|
||||||
|
"expect_reason": TerminationReason.TASKS_COMPLETED,
|
||||||
|
},
|
||||||
|
# quality gate reached
|
||||||
|
"quality_reached": {
|
||||||
|
"tasks": [completed_task("t1", tests_passed=True)],
|
||||||
|
"quality": {"test_pass_rate": 1.0, "graded": True},
|
||||||
|
"expect_status": ConvergenceStatus.CONVERGED,
|
||||||
|
"expect_reason": TerminationReason.QUALITY_REACHED,
|
||||||
|
},
|
||||||
|
# budget exhausted
|
||||||
|
"budget_exhausted": {
|
||||||
|
"tasks": [completed_task("t1", tests_passed=True)],
|
||||||
|
"budget": {"max_cost_usd": 5.0},
|
||||||
|
"usage": {"total_cost_usd": 5.0},
|
||||||
|
"expect_status": ConvergenceStatus.CONVERGED,
|
||||||
|
"expect_reason": TerminationReason.BUDGET_EXHAUSTED,
|
||||||
|
},
|
||||||
|
# max rounds reached
|
||||||
|
"max_rounds_reached": {
|
||||||
|
"tasks": [completed_task("t1", tests_passed=True)],
|
||||||
|
"review_cycles": 2,
|
||||||
|
"max_review_cycles": 2,
|
||||||
|
"expect_status": ConvergenceStatus.CONVERGED,
|
||||||
|
"expect_reason": TerminationReason.MAX_ROUNDS_REACHED,
|
||||||
|
},
|
||||||
|
# risk blocked (explicit blocking input risk)
|
||||||
|
"risk_blocked": {
|
||||||
|
"tasks": [completed_task("t1", tests_passed=True)],
|
||||||
|
"risks": [{"id": "r1", "description": "secret leak suspected", "blocking": True}],
|
||||||
|
"expect_status": ConvergenceStatus.BLOCKED,
|
||||||
|
"expect_reason": TerminationReason.RISK_BLOCKED,
|
||||||
|
},
|
||||||
|
# a failed task -> FAILED, fallback reason when nothing else explains it
|
||||||
|
"failed_fallback": {
|
||||||
|
"tasks": [
|
||||||
|
completed_task("t1", tests_passed=True),
|
||||||
|
{"task_id": "t2", "status": "failed", "depends_on": [], "result": {}},
|
||||||
|
],
|
||||||
|
"expect_status": ConvergenceStatus.FAILED,
|
||||||
|
"expect_reason": TerminationReason.TASKS_COMPLETED,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for label, case in terminal_cases.items():
|
||||||
|
rep = evaluate_convergence(case)
|
||||||
|
check(f"[{label}] terminal status == {case['expect_status'].value}",
|
||||||
|
rep.status == case["expect_status"])
|
||||||
|
check(f"[{label}] is_terminal()", rep.is_terminal())
|
||||||
|
check(f"[{label}] carries a non-None termination_reason",
|
||||||
|
rep.termination_reason is not None)
|
||||||
|
check(f"[{label}] termination_reason == {case['expect_reason'].value}",
|
||||||
|
rep.termination_reason == case["expect_reason"])
|
||||||
|
|
||||||
|
# Invariant: EVERY terminal report has a reason; non-terminal has none.
|
||||||
|
for label, case in terminal_cases.items():
|
||||||
|
rep = evaluate_convergence(case)
|
||||||
|
if rep.is_terminal():
|
||||||
|
check(f"[{label}] invariant: terminal -> reason set",
|
||||||
|
rep.termination_reason is not None)
|
||||||
|
|
||||||
|
# A still-running run must NOT carry a termination_reason.
|
||||||
|
running = evaluate_convergence({"tasks": [{"task_id": "x", "status": "in_progress", "depends_on": []}]})
|
||||||
|
check("running run is RUNNING", running.status == ConvergenceStatus.RUNNING)
|
||||||
|
check("running run has no termination_reason", running.termination_reason is None)
|
||||||
|
check("running run is not terminal", not running.is_terminal())
|
||||||
|
|
||||||
|
|
||||||
|
# --- dependency inconsistency detector ------------------------------------
|
||||||
|
dep_state = {
|
||||||
|
"tasks": [
|
||||||
|
{"task_id": "parent", "status": "completed", "depends_on": ["child"], "result": {}},
|
||||||
|
{"task_id": "child", "status": "failed", "depends_on": [], "result": {}},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
dep_conflicts = detect_conflicts(dep_state["tasks"])
|
||||||
|
check("dependency inconsistency detected",
|
||||||
|
any(c.type == ConflictType.DEPENDENCY_INCONSISTENCY for c in dep_conflicts))
|
||||||
|
|
||||||
|
|
||||||
|
# --- test failure detector ------------------------------------------------
|
||||||
|
tf_conflicts = detect_conflicts([completed_task("t1", tests_passed=False)])
|
||||||
|
check("test failure detected",
|
||||||
|
any(c.type == ConflictType.TEST_FAILURE for c in tf_conflicts))
|
||||||
|
check("no test signal is not a failure",
|
||||||
|
not any(c.type == ConflictType.TEST_FAILURE
|
||||||
|
for c in detect_conflicts([completed_task("t1")])))
|
||||||
|
|
||||||
|
|
||||||
|
# --- event builders -------------------------------------------------------
|
||||||
|
et, _ = event_convergence_started(run_state)
|
||||||
|
check("convergence.started event type", et == "convergence.started")
|
||||||
|
et, _ = event_conflict_detected(conflicts[0])
|
||||||
|
check("conflict.detected event type", et == "conflict.detected")
|
||||||
|
et, _ = event_conflict_resolved(resolved[0])
|
||||||
|
check("conflict.resolved event type", et == "conflict.resolved")
|
||||||
|
et, payload = event_consensus_updated(report)
|
||||||
|
check("consensus.updated event type + score", et == "consensus.updated" and "consensus_score" in payload)
|
||||||
|
et, payload = event_convergence_failed(report)
|
||||||
|
check("convergence.failed carries termination_reason",
|
||||||
|
et == "convergence.failed" and payload["termination_reason"] == "risk_blocked")
|
||||||
|
|
||||||
|
clean = evaluate_convergence(terminal_cases["tasks_completed"])
|
||||||
|
et, payload = event_convergence_reached(clean)
|
||||||
|
check("convergence.reached carries termination_reason",
|
||||||
|
et == "convergence.reached" and payload["termination_reason"] == "tasks_completed")
|
||||||
|
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} convergence check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all convergence protocol checks passed")
|
||||||
@@ -0,0 +1,198 @@
|
|||||||
|
r"""Hermetic module-level tests for the cross-review protocol (issue #11).
|
||||||
|
|
||||||
|
No WebSocket, no Redis, no model calls — exercises orchestrator/cross_review.py purely. The
|
||||||
|
core scenario: two reviewers give DIFFERENT verdicts on the same artifact, aggregation detects
|
||||||
|
the disagreement and arbitrates, structured evidence + a rework target are produced, and the
|
||||||
|
rework is attributed and classified for the benchmark P_rework input.
|
||||||
|
|
||||||
|
Run (from the repo root, agent_swarm_v6):
|
||||||
|
|
||||||
|
pip install -r orchestrator/requirements.txt # cross_review has no extra deps; stdlib only
|
||||||
|
..\.venv\Scripts\python.exe scripts\test-cross-review.py
|
||||||
|
|
||||||
|
(The pip step is only needed for a fresh env; cross_review.py itself imports only the stdlib.)
|
||||||
|
"""
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.cross_review import (
|
||||||
|
ReviewDecision,
|
||||||
|
ReworkCategory,
|
||||||
|
aggregate_reviews,
|
||||||
|
build_rework_attributions,
|
||||||
|
classify_rework,
|
||||||
|
review_started_payload,
|
||||||
|
review_decision_made_payload,
|
||||||
|
rework_requested_payload,
|
||||||
|
rework_completed_payload,
|
||||||
|
)
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
def test_single_reviewer_rejected():
|
||||||
|
"""A single reviewer is a Supervisor Retry, not a cross-review — must be refused."""
|
||||||
|
only = ReviewDecision(verdict="pass", reviewer_agent_id="r1")
|
||||||
|
try:
|
||||||
|
aggregate_reviews([only])
|
||||||
|
check("single reviewer rejected", False)
|
||||||
|
except ValueError:
|
||||||
|
check("single reviewer rejected", True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_disagreement_detected_and_arbitrated():
|
||||||
|
"""Two reviewers, DIFFERENT verdicts on the same artifact -> disagreement + arbitration."""
|
||||||
|
passer = ReviewDecision(
|
||||||
|
verdict="pass",
|
||||||
|
reviewer_agent_id="reviewer-impl",
|
||||||
|
evidence=["add() returns sum for valid input"],
|
||||||
|
summary="implementation looks correct",
|
||||||
|
confidence=0.6,
|
||||||
|
)
|
||||||
|
failer = ReviewDecision(
|
||||||
|
verdict="fail",
|
||||||
|
reviewer_agent_id="reviewer-test",
|
||||||
|
evidence=["tests assert ValueError but impl raises ZeroDivisionError"],
|
||||||
|
failed_criteria=["error semantics consistent across specialists"],
|
||||||
|
affected_tasks=["swarm-x-testing"],
|
||||||
|
recommended_rework=["swarm-x-testing"],
|
||||||
|
summary="conflicting error semantics between impl and tests",
|
||||||
|
confidence=0.9,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Majority of 2 with a 1/1 split -> tie -> safety-biased reject.
|
||||||
|
verdict = aggregate_reviews([passer, failer], method="majority")
|
||||||
|
check("disagreement detected", verdict.disagreement is True)
|
||||||
|
check("split vote counted (1 pass / 1 fail)", verdict.pass_votes == 1 and verdict.fail_votes == 1)
|
||||||
|
check("majority tie arbitrates to reject", verdict.accepted is False)
|
||||||
|
check("rework target produced from failing reviewer", verdict.rework_targets == ["swarm-x-testing"])
|
||||||
|
check("structured evidence preserved on decisions",
|
||||||
|
any("ZeroDivisionError" in e for d in verdict.decisions for e in d.evidence))
|
||||||
|
check("arbitration method recorded", verdict.method == "majority")
|
||||||
|
|
||||||
|
# Weighted arbitration: the failer is more confident/weighty -> still reject.
|
||||||
|
weighted = aggregate_reviews([passer, failer], method="weighted")
|
||||||
|
check("weighted arbitration rejects when fail side heavier", weighted.accepted is False)
|
||||||
|
check("weighted method recorded", weighted.method == "weighted")
|
||||||
|
|
||||||
|
|
||||||
|
def test_unanimous_pass_accepts():
|
||||||
|
a = ReviewDecision(verdict="pass", reviewer_agent_id="r1", summary="ok")
|
||||||
|
b = ReviewDecision(verdict="pass", reviewer_agent_id="r2", summary="ok")
|
||||||
|
verdict = aggregate_reviews([a, b])
|
||||||
|
check("unanimous pass accepted", verdict.accepted is True)
|
||||||
|
check("unanimous has no disagreement", verdict.disagreement is False)
|
||||||
|
check("accepted verdict has no rework targets", verdict.rework_targets == [])
|
||||||
|
|
||||||
|
|
||||||
|
def test_majority_fail_three_reviewers():
|
||||||
|
a = ReviewDecision(verdict="fail", reviewer_agent_id="r1", recommended_rework=["t-impl"],
|
||||||
|
summary="impl wrong")
|
||||||
|
b = ReviewDecision(verdict="fail", reviewer_agent_id="r2", recommended_rework=["t-impl"],
|
||||||
|
summary="impl wrong")
|
||||||
|
c = ReviewDecision(verdict="pass", reviewer_agent_id="r3", summary="fine")
|
||||||
|
verdict = aggregate_reviews([a, b, c])
|
||||||
|
check("3-reviewer majority fail rejects", verdict.accepted is False)
|
||||||
|
check("majority disagreement detected", verdict.disagreement is True)
|
||||||
|
check("union of rework targets de-duped", verdict.rework_targets == ["t-impl"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_rework_attribution_and_classification():
|
||||||
|
"""Disagreement-driven rework is attributed to a source task/agent and classified."""
|
||||||
|
failer = ReviewDecision(
|
||||||
|
verdict="fail",
|
||||||
|
reviewer_agent_id="reviewer-test",
|
||||||
|
failed_criteria=["test framework consistency"],
|
||||||
|
recommended_rework=["swarm-x-testing"],
|
||||||
|
summary="docs say unittest but tests use pytest — inconsistent across specialists",
|
||||||
|
confidence=0.8,
|
||||||
|
)
|
||||||
|
passer = ReviewDecision(verdict="pass", reviewer_agent_id="reviewer-impl", summary="impl ok")
|
||||||
|
verdict = aggregate_reviews([failer, passer], method="majority")
|
||||||
|
|
||||||
|
owners = {"swarm-x-testing": "agent-tester-7"}
|
||||||
|
attributions = build_rework_attributions(verdict, task_owner=owners)
|
||||||
|
check("one attribution per rework target", len(attributions) == 1)
|
||||||
|
att = attributions[0]
|
||||||
|
check("attribution targets the reopened task", att.target_task_id == "swarm-x-testing")
|
||||||
|
check("attribution records introducing agent", att.introduced_by_agent_id == "agent-tester-7")
|
||||||
|
check("attribution records detecting reviewer", att.detected_by_agent_id == "reviewer-test")
|
||||||
|
check("attribution carries a reason", bool(att.rework_reason))
|
||||||
|
# Disagreement + framework/consistency wording -> COLLABORATION root cause.
|
||||||
|
check("rework classified as collaboration", att.root_cause == ReworkCategory.COLLABORATION)
|
||||||
|
|
||||||
|
# Direct classifier checks across categories.
|
||||||
|
check("impl defect classified", classify_rework("implementation logic bug") == ReworkCategory.IMPLEMENTATION)
|
||||||
|
check("test defect classified", classify_rework("pytest assertion wrong") == ReworkCategory.TEST)
|
||||||
|
check("doc drift classified", classify_rework("readme documentation outdated") == ReworkCategory.DOC)
|
||||||
|
check("requirement miss classified", classify_rework("misunderstood the objective scope") == ReworkCategory.REQUIREMENT)
|
||||||
|
check("no signal -> unknown (not fabricated)", classify_rework("") == ReworkCategory.UNKNOWN)
|
||||||
|
check("empty reason + disagreement -> collaboration",
|
||||||
|
classify_rework("", disagreement=True) == ReworkCategory.COLLABORATION)
|
||||||
|
|
||||||
|
|
||||||
|
def test_event_payload_builders():
|
||||||
|
failer = ReviewDecision(verdict="fail", reviewer_agent_id="r-test",
|
||||||
|
recommended_rework=["t1"], summary="bad")
|
||||||
|
passer = ReviewDecision(verdict="pass", reviewer_agent_id="r-impl", summary="ok")
|
||||||
|
verdict = aggregate_reviews([failer, passer])
|
||||||
|
att = build_rework_attributions(verdict, task_owner={"t1": "agent-1"})[0]
|
||||||
|
|
||||||
|
started = review_started_payload("swarm-x", reviewer_agent_ids=["r-test", "r-impl"],
|
||||||
|
artifact_task_ids=["t1"], cycle=1)
|
||||||
|
check("review.started carries reviewer count", started["reviewer_count"] == 2)
|
||||||
|
check("review.started marks cross_review kind", started["review_kind"] == "cross_review")
|
||||||
|
|
||||||
|
decided = review_decision_made_payload("swarm-x", verdict, cycle=1)
|
||||||
|
check("review.decision_made carries disagreement flag", decided["disagreement"] is True)
|
||||||
|
check("review.decision_made carries rework targets", decided["rework_targets"] == ["t1"])
|
||||||
|
check("review.decision_made embeds per-reviewer reviews", len(decided["reviews"]) == 2)
|
||||||
|
|
||||||
|
requested = rework_requested_payload("swarm-x", att, cycle=1)
|
||||||
|
check("rework.requested carries task_id", requested["task_id"] == "t1")
|
||||||
|
check("rework.requested carries root_cause", "root_cause" in requested)
|
||||||
|
check("rework.requested carries introducing agent", requested["introduced_by_agent_id"] == "agent-1")
|
||||||
|
|
||||||
|
completed = rework_completed_payload("swarm-x", att, cycle=1, succeeded=True)
|
||||||
|
check("rework.completed marks status", completed["status"] == "completed")
|
||||||
|
check("rework.completed carries task_id", completed["task_id"] == "t1")
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_verdict_rejected():
|
||||||
|
try:
|
||||||
|
ReviewDecision(verdict="maybe", reviewer_agent_id="r1")
|
||||||
|
check("invalid verdict rejected", False)
|
||||||
|
except ValueError:
|
||||||
|
check("invalid verdict rejected", True)
|
||||||
|
try:
|
||||||
|
ReviewDecision(verdict="pass", reviewer_agent_id="")
|
||||||
|
check("missing reviewer id rejected", False)
|
||||||
|
except ValueError:
|
||||||
|
check("missing reviewer id rejected", True)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
test_single_reviewer_rejected()
|
||||||
|
test_disagreement_detected_and_arbitrated()
|
||||||
|
test_unanimous_pass_accepts()
|
||||||
|
test_majority_fail_three_reviewers()
|
||||||
|
test_rework_attribution_and_classification()
|
||||||
|
test_event_payload_builders()
|
||||||
|
test_invalid_verdict_rejected()
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all cross-review checks passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -1,117 +0,0 @@
|
|||||||
"""Integration test for issue #9: task-centric SCORED dispatch picks among AGENTS.
|
|
||||||
|
|
||||||
Exercises orchestrator.main.scored_matchmake against the real task_queue + agent_registry +
|
|
||||||
decision_engine (REDIS_FAKE, no WS/model): two capable agents differ in historical success (τ)
|
|
||||||
and load; the higher-scored agent is chosen; an incapable agent is excluded; the explainable
|
|
||||||
dispatch.decision_made record (candidate breakdown + exclusion reasons + ≥4 non-capability
|
|
||||||
dimensions with real signals) is produced and stored on the run.
|
|
||||||
|
|
||||||
Run from agent_swarm_v6 (install deps first — needs fakeredis):
|
|
||||||
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
|
||||||
REDIS_FAKE=1 python scripts/test-dispatch-scored.py
|
|
||||||
"""
|
|
||||||
import asyncio
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
os.environ["REDIS_FAKE"] = "1"
|
|
||||||
os.environ["ENABLE_DISPATCH_SCORE"] = "1"
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
|
||||||
|
|
||||||
from orchestrator.redis_client import redis_client
|
|
||||||
from orchestrator import swarm_runtime as sr_mod
|
|
||||||
from orchestrator.swarm_runtime import swarm_runtime
|
|
||||||
from orchestrator.task_queue import task_queue, TaskStatus
|
|
||||||
from orchestrator.agent_registry import agent_registry
|
|
||||||
from orchestrator.decision_engine import decision_engine
|
|
||||||
from orchestrator import main as orch
|
|
||||||
|
|
||||||
failures = []
|
|
||||||
|
|
||||||
|
|
||||||
def check(name, cond):
|
|
||||||
print(("PASS" if cond else "FAIL"), "-", name)
|
|
||||||
if not cond:
|
|
||||||
failures.append(name)
|
|
||||||
|
|
||||||
|
|
||||||
async def _noop(self, *a, **k):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
async def main():
|
|
||||||
await redis_client.connect()
|
|
||||||
sr_mod.SwarmRuntime._post_callback = _noop
|
|
||||||
|
|
||||||
body = {
|
|
||||||
"mode": "swarm",
|
|
||||||
"requirement": {"objective": "scored dispatch test"},
|
|
||||||
"orchestration_plan": {"budget": {"max_cost_usd": 10}},
|
|
||||||
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
|
||||||
"metadata": {"manager_deployment_id": "m-disp"},
|
|
||||||
}
|
|
||||||
run, _ = await swarm_runtime.get_or_create_run(body=body, idempotency_key=None, correlation_id="cd")
|
|
||||||
|
|
||||||
# One ready task requiring python.
|
|
||||||
task = await task_queue.create_task(task_id="t-implementation", description="impl",
|
|
||||||
agent_role="implementation",
|
|
||||||
required_capabilities=["python"], enqueue=True)
|
|
||||||
await swarm_runtime.attach_task(run, task.task_id)
|
|
||||||
|
|
||||||
# Three agents: A & B both capable (python); C incapable (java only).
|
|
||||||
await agent_registry.register_agent("agent-A", ["python"])
|
|
||||||
await agent_registry.register_agent("agent-B", ["python"])
|
|
||||||
await agent_registry.register_agent("agent-C", ["java"])
|
|
||||||
orch.AGENT_SLOTS.update({"agent-A": 1, "agent-B": 1, "agent-C": 1})
|
|
||||||
|
|
||||||
# Historical success (τ) differs: A strong, B weak — same capability & load otherwise.
|
|
||||||
for _ in range(8):
|
|
||||||
await decision_engine.deposit(agent_role="implementation", agent_id="agent-A", success=True)
|
|
||||||
await decision_engine.deposit(agent_role="implementation", agent_id="agent-B", success=False)
|
|
||||||
|
|
||||||
idle = [a for a in await agent_registry.get_idle_agents()
|
|
||||||
if a.agent_id in {"agent-A", "agent-B", "agent-C"}]
|
|
||||||
assignments = await orch.scored_matchmake(idle)
|
|
||||||
|
|
||||||
check("exactly one assignment produced", len(assignments) == 1)
|
|
||||||
agent, chosen_task, event = assignments[0]
|
|
||||||
check("higher-τ agent A chosen over B (historical_success decides)", agent.agent_id == "agent-A")
|
|
||||||
check("chosen task is the python task", chosen_task.task_id == "t-implementation")
|
|
||||||
|
|
||||||
# Explainable event payload
|
|
||||||
check("event names chosen agent/task",
|
|
||||||
event["chosen_agent_id"] == "agent-A" and event["chosen_task_id"] == "t-implementation")
|
|
||||||
check("incapable agent C excluded as capability_mismatch",
|
|
||||||
event["excluded"].get("agent-C") == "capability_mismatch")
|
|
||||||
check("loser B excluded as lower_score", event["excluded"].get("agent-B") == "lower_score")
|
|
||||||
a_cand = next(c for c in event["candidates"] if c["agent_id"] == "agent-A")
|
|
||||||
present = {k for k, v in a_cand["score"]["breakdown"].items() if v is not None}
|
|
||||||
noncap_present = present - {"capability_match"}
|
|
||||||
check("≥4 non-capability dimensions have real signals",
|
|
||||||
{"historical_success", "load", "permission_fit", "budget_pressure"} <= noncap_present
|
|
||||||
and len(noncap_present) >= 4)
|
|
||||||
check("uncollected dims disclosed (no fabrication)",
|
|
||||||
set(event["uncollected_dimensions"]) >= {"estimated_cost", "estimated_time", "risk_score"})
|
|
||||||
check("A's historical_success > B's in breakdown",
|
|
||||||
a_cand["score"]["breakdown"]["historical_success"]
|
|
||||||
> next(c for c in event["candidates"] if c["agent_id"] == "agent-B")["score"]["breakdown"]["historical_success"])
|
|
||||||
|
|
||||||
# finalize_dispatch stores the decision on the run (audit/benchmark-replayable, internal)
|
|
||||||
await task_queue.remove_pending_task(chosen_task.task_id)
|
|
||||||
ok = await orch.finalize_dispatch(agent, chosen_task, dispatch_event=event)
|
|
||||||
check("finalize_dispatch assigned the task", ok is True)
|
|
||||||
refreshed = await swarm_runtime.get_run(run.swarm_id)
|
|
||||||
check("dispatch decision recorded on run (replayable)",
|
|
||||||
len(refreshed.dispatch_decisions) == 1
|
|
||||||
and refreshed.dispatch_decisions[0]["chosen_agent_id"] == "agent-A")
|
|
||||||
|
|
||||||
print()
|
|
||||||
if failures:
|
|
||||||
print(f"{len(failures)} scored-dispatch check(s) FAILED: {failures}")
|
|
||||||
sys.exit(1)
|
|
||||||
print("all scored-dispatch (#9) checks passed")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
asyncio.run(main())
|
|
||||||
+26
-48
@@ -63,37 +63,21 @@ async def test_release_task():
|
|||||||
check("release requeues task", task.task_id in pending)
|
check("release requeues task", task.task_id in pending)
|
||||||
|
|
||||||
|
|
||||||
async def test_planner_fallback():
|
async def test_seeder():
|
||||||
subtasks = await planner.build_plan("swarm-test", "Build a calculator")
|
# Swarm task-creation: an objective is SEEDED as one task (no Master decomposition). Agents
|
||||||
roles = [s.get("role") for s in subtasks]
|
# grow the graph bottom-up at runtime (handle_task_proposal). Tested e2e in test-workflow-e2e.
|
||||||
check("planner static fallback (3 specialists)", roles == ["implementation", "testing", "documentation"])
|
|
||||||
|
|
||||||
run = types.SimpleNamespace(swarm_id="swarm-test", objective="Build a calculator")
|
run = types.SimpleNamespace(swarm_id="swarm-test", objective="Build a calculator")
|
||||||
base_specs = [{"context": {"orchestration_plan": {"x": 1}}}]
|
base_specs = [{"context": {"orchestration_plan": {"x": 1}}}]
|
||||||
specs = await orch.build_planner_task_specs(run, {}, base_specs)
|
specs = orch.build_seed_task_specs(run, {}, base_specs)
|
||||||
check("planner specs preserve base context", specs[0]["context"].get("orchestration_plan") == {"x": 1})
|
check("seeder injects exactly one seed task", len(specs) == 1)
|
||||||
check("planner specs carry deps", any(s["depends_on"] for s in specs))
|
check("seed source is 'seed' (no planner)", specs[0]["source"] == "seed")
|
||||||
# Inject a phantom dependency and confirm it is filtered out.
|
check("seed has no required capabilities (any agent self-selects)", specs[0]["required_capabilities"] == [])
|
||||||
specs2 = list(specs)
|
check("seed preserves base context", specs[0]["context"].get("orchestration_plan") == {"x": 1})
|
||||||
specs2[0]["depends_on"] = ["does-not-exist"]
|
check("seed marks is_seed + objective", specs[0]["context"].get("is_seed") is True
|
||||||
run2 = types.SimpleNamespace(swarm_id="swarm-test", objective="x")
|
and specs[0]["context"].get("objective") == "Build a calculator")
|
||||||
|
# Manager-provided agents are honored (Manager-first; seeder does not override).
|
||||||
async def fake_plan(swarm_id, objective):
|
check("Manager-provided agent breakdown bypasses the seeder",
|
||||||
return [
|
orch._manager_provided_agents({"orchestration_plan": {"agents": [{"role": "impl"}]}}) is True)
|
||||||
{"subtask_id": "a", "description": "A", "role": "implementation", "required_capabilities": ["python"], "depends_on": ["ghost"]},
|
|
||||||
{"subtask_id": "b", "description": "B", "role": "testing", "required_capabilities": ["testing"], "depends_on": ["a"]},
|
|
||||||
]
|
|
||||||
|
|
||||||
orig = planner.build_plan
|
|
||||||
planner.build_plan = fake_plan
|
|
||||||
try:
|
|
||||||
filtered = await orch.build_planner_task_specs(run2, {}, base_specs)
|
|
||||||
finally:
|
|
||||||
planner.build_plan = orig
|
|
||||||
deps_a = next(s for s in filtered if s["task_id"] == "a")["depends_on"]
|
|
||||||
deps_b = next(s for s in filtered if s["task_id"] == "b")["depends_on"]
|
|
||||||
check("phantom dependency filtered", deps_a == [])
|
|
||||||
check("valid dependency kept", deps_b == ["a"])
|
|
||||||
|
|
||||||
|
|
||||||
async def test_agent_peer_routing():
|
async def test_agent_peer_routing():
|
||||||
@@ -161,8 +145,9 @@ async def test_review_and_synthesis():
|
|||||||
|
|
||||||
|
|
||||||
async def test_review_cycle():
|
async def test_review_cycle():
|
||||||
# Patch out callback/persistence side effects and force a rejection from the critic.
|
# Swarm review is peer cross-review (no single-critic Master): >=2 reviewers, one rejects →
|
||||||
orig_save, orig_emit, orig_review = orch.swarm_runtime.save_run, orch.swarm_runtime.emit_event, orch.planner.review
|
# reopen the flagged task. (Full coverage in test-swarm-cross-review.py.)
|
||||||
|
orig_save, orig_emit = orch.swarm_runtime.save_run, orch.swarm_runtime.emit_event
|
||||||
|
|
||||||
async def noop(*a, **k):
|
async def noop(*a, **k):
|
||||||
return None
|
return None
|
||||||
@@ -175,31 +160,24 @@ async def test_review_cycle():
|
|||||||
completed = await task_queue.get_task(task.task_id)
|
completed = await task_queue.get_task(task.task_id)
|
||||||
run = types.SimpleNamespace(
|
run = types.SimpleNamespace(
|
||||||
swarm_id="swarm-rev", deployment_id="dep-rev", manager_deployment_id="mgr-rev",
|
swarm_id="swarm-rev", deployment_id="dep-rev", manager_deployment_id="mgr-rev",
|
||||||
objective="obj", task_ids=["t-rev-1"], status="completed", metadata={}
|
objective="obj", task_ids=["t-rev-1"], status="completed",
|
||||||
|
metadata={"reviews": [
|
||||||
|
{"verdict": "pass", "reviewer_agent_id": "r1"},
|
||||||
|
{"verdict": "fail", "reviewer_agent_id": "r2",
|
||||||
|
"recommended_rework": ["t-rev-1"], "summary": "implementation incorrect"},
|
||||||
|
]},
|
||||||
)
|
)
|
||||||
|
|
||||||
async def reject(objective, tasks, results):
|
|
||||||
return {"accepted": False, "summary": "needs work", "retry_tasks": ["t-rev-1"]}
|
|
||||||
|
|
||||||
orch.planner.review = reject
|
|
||||||
try:
|
try:
|
||||||
reopened = await orch.maybe_run_review_cycle(run, [completed])
|
reopened = await orch.run_cross_review(run, [completed])
|
||||||
check("review cycle reopens rejected task", reopened is True)
|
check("cross-review reopens on reviewer split (safety-biased reject)", reopened is True)
|
||||||
reloaded = await task_queue.get_task("t-rev-1")
|
reloaded = await task_queue.get_task("t-rev-1")
|
||||||
check("reopened task is PENDING again", reloaded.status == TaskStatus.PENDING)
|
check("reopened task is PENDING again", reloaded.status == TaskStatus.PENDING)
|
||||||
check("review cycle counter incremented", run.metadata.get("review_cycles") == 1)
|
check("review cycle counter incremented", run.metadata.get("review_cycles") == 1)
|
||||||
check("run set back to running", run.status == "running")
|
check("run set back to running", run.status == "running")
|
||||||
|
check("disagreement recorded", (run.metadata.get("cross_review") or {}).get("disagreement") is True)
|
||||||
# Exhaust the cycle budget -> no further reopen.
|
|
||||||
run.metadata["review_cycles"] = orch.review_max_cycles()
|
|
||||||
await task_queue.complete_task("t-rev-1", '{"summary": "did impl again"}')
|
|
||||||
completed2 = await task_queue.get_task("t-rev-1")
|
|
||||||
reopened2 = await orch.maybe_run_review_cycle(run, [completed2])
|
|
||||||
check("review cycle respects budget", reopened2 is False)
|
|
||||||
finally:
|
finally:
|
||||||
orch.swarm_runtime.save_run = orig_save
|
orch.swarm_runtime.save_run = orig_save
|
||||||
orch.swarm_runtime.emit_event = orig_emit
|
orch.swarm_runtime.emit_event = orig_emit
|
||||||
orch.planner.review = orig_review
|
|
||||||
|
|
||||||
|
|
||||||
async def test_dispatch_context():
|
async def test_dispatch_context():
|
||||||
@@ -265,7 +243,7 @@ async def test_agent_peer_shares_summary():
|
|||||||
async def main():
|
async def main():
|
||||||
await test_redis_fallback()
|
await test_redis_fallback()
|
||||||
await test_release_task()
|
await test_release_task()
|
||||||
await test_planner_fallback()
|
await test_seeder()
|
||||||
await test_agent_peer_routing()
|
await test_agent_peer_routing()
|
||||||
await test_review_and_synthesis()
|
await test_review_and_synthesis()
|
||||||
await test_review_cycle()
|
await test_review_cycle()
|
||||||
|
|||||||
@@ -0,0 +1,114 @@
|
|||||||
|
"""Integration test for decentralized-rework P3: agent-driven decomposition (#7).
|
||||||
|
|
||||||
|
An executing agent proposes follow-up tasks from the shared seed/run state; the orchestrator
|
||||||
|
reviews each (confidence floor / dedup-merge / per-run budget) and enqueues accepted ones as real
|
||||||
|
PENDING tasks with full lineage. This is the bottom-up decomposition that replaces the Master plan.
|
||||||
|
Hermetic, no model key.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — needs fakeredis):
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
REDIS_FAKE=1 ENABLE_AGENT_TASK_PROPOSALS=1 AGENT_PROPOSAL_BUDGET=3 python scripts/test-swarm-autonomous.py
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
os.environ["REDIS_FAKE"] = "1"
|
||||||
|
os.environ["ENABLE_AGENT_TASK_PROPOSALS"] = "1"
|
||||||
|
os.environ["AGENT_PROPOSAL_BUDGET"] = "3"
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.redis_client import redis_client
|
||||||
|
from orchestrator import swarm_runtime as sr_mod
|
||||||
|
from orchestrator.swarm_runtime import swarm_runtime
|
||||||
|
from orchestrator.task_queue import task_queue
|
||||||
|
from orchestrator import main as orch
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
async def _noop(self, *a, **k):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def propose(seed_id, desc, *, agent="agent-A", conf=0.9, role="general", caps=None):
|
||||||
|
return await orch.handle_task_proposal(agent, {
|
||||||
|
"origin_task_id": seed_id, "description": desc, "reason": "discovered gap in shared state",
|
||||||
|
"confidence": conf, "agent_role": role, "required_capabilities": caps or [],
|
||||||
|
"trigger_event": "task.completed",
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
await redis_client.connect()
|
||||||
|
sr_mod.SwarmRuntime._post_callback = _noop
|
||||||
|
|
||||||
|
body = {"mode": "swarm", "requirement": {"objective": "Build a CSV parser"},
|
||||||
|
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
||||||
|
"metadata": {"manager_deployment_id": "m-auto"}}
|
||||||
|
run, _ = await swarm_runtime.get_or_create_run(body=body, idempotency_key=None, correlation_id="ca")
|
||||||
|
seed = await task_queue.create_task(task_id=f"{run.swarm_id}-seed", description="Build a CSV parser",
|
||||||
|
agent_role="general", required_capabilities=[], enqueue=False)
|
||||||
|
await swarm_runtime.attach_task(run, seed.task_id)
|
||||||
|
|
||||||
|
# 1) two distinct high-confidence proposals → ACCEPT → real tasks with lineage
|
||||||
|
r1 = await propose(seed.task_id, "Write unit tests for the CSV parser module", role="testing", caps=["testing"])
|
||||||
|
r2 = await propose(seed.task_id, "Write API documentation for the parser", role="documentation")
|
||||||
|
check("proposal 1 accepted → task created", r1.get("accepted") and r1.get("task_id"))
|
||||||
|
check("proposal 2 accepted → task created", r2.get("accepted") and r2.get("task_id"))
|
||||||
|
|
||||||
|
t1 = await task_queue.get_task(r1["task_id"])
|
||||||
|
check("created task is agent_proposed (bottom-up, not Master)", t1 and t1.source == "agent_proposed")
|
||||||
|
lineage = (t1.context or {}).get("proposal") or {}
|
||||||
|
check("created task carries lineage (proposer + origin + reason)",
|
||||||
|
lineage.get("proposed_by_agent_id") == "agent-A"
|
||||||
|
and lineage.get("origin_task_id") == seed.task_id
|
||||||
|
and lineage.get("proposal_reason"))
|
||||||
|
|
||||||
|
# 2) near-duplicate of proposal 1 → MERGE (not a new task)
|
||||||
|
rdup = await propose(seed.task_id, "Write unit tests for the CSV parser module")
|
||||||
|
check("duplicate proposal → MERGE", rdup.get("decision") == "merge" and not rdup.get("accepted"))
|
||||||
|
check("merge names the target task", rdup.get("merge_target_task_id") == r1["task_id"])
|
||||||
|
|
||||||
|
# 3) third distinct proposal → ACCEPT (budget 3: A,B + this = 3)
|
||||||
|
r3 = await propose(seed.task_id, "Add a benchmark harness measuring parser throughput", role="general")
|
||||||
|
check("proposal 3 accepted (within budget)", r3.get("accepted"))
|
||||||
|
|
||||||
|
# 4) fourth distinct proposal → budget exhausted → REJECT
|
||||||
|
r4 = await propose(seed.task_id, "Containerize the parser service with a Dockerfile")
|
||||||
|
check("proposal 4 rejected (budget exhausted)",
|
||||||
|
not r4.get("accepted") and r4.get("decision") == "reject" and "budget" in (r4.get("reason") or ""))
|
||||||
|
|
||||||
|
# 5) low-confidence proposal → REJECT
|
||||||
|
r5 = await propose(seed.task_id, "Maybe refactor something unspecified", conf=0.2)
|
||||||
|
check("low-confidence proposal rejected",
|
||||||
|
not r5.get("accepted") and r5.get("decision") == "reject")
|
||||||
|
|
||||||
|
# run grew from 1 seed → 1 seed + 3 accepted = 4 tasks; proposals telemetry recorded
|
||||||
|
refreshed = await swarm_runtime.get_run(run.swarm_id)
|
||||||
|
check("run task graph grew bottom-up to 4 tasks (seed + 3)", len(refreshed.task_ids) == 4)
|
||||||
|
check("proposal lifecycle telemetry recorded on run",
|
||||||
|
bool(refreshed.metadata.get("proposals")))
|
||||||
|
check("accepted-proposal counter = 3", refreshed.metadata.get("proposal_accepted_count") == 3)
|
||||||
|
|
||||||
|
# Post-cutover: proposals are unconditional (swarm is the runtime; no enable flag).
|
||||||
|
os.environ.pop("ENABLE_AGENT_TASK_PROPOSALS", None)
|
||||||
|
roff = await propose(seed.task_id, "Add a CLI entrypoint for the parser")
|
||||||
|
check("proposals processed without any flag (swarm default)", roff.get("decision") != "disabled")
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} autonomous-task (P3) check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all swarm autonomous-task (P3) checks passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
"""Integration test for decentralized-rework P4: task competition (#8).
|
||||||
|
|
||||||
|
Agents bid for a task; a deterministic arbitrator picks the winner and assigns it. Agents can
|
||||||
|
yield a task back, and request takeover of a held task (only winning decisively reassigns it).
|
||||||
|
Hermetic, no model key.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — needs fakeredis):
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
REDIS_FAKE=1 ENABLE_TASK_COMPETITION=1 python scripts/test-swarm-competition.py
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
os.environ["REDIS_FAKE"] = "1"
|
||||||
|
os.environ["ENABLE_TASK_COMPETITION"] = "1"
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.redis_client import redis_client
|
||||||
|
from orchestrator import swarm_runtime as sr_mod
|
||||||
|
from orchestrator.swarm_runtime import swarm_runtime
|
||||||
|
from orchestrator.task_queue import task_queue, TaskStatus
|
||||||
|
from orchestrator.agent_registry import agent_registry
|
||||||
|
from orchestrator import main as orch
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
async def _noop(self, *a, **k):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def mk_task(run, tid, caps=None):
|
||||||
|
t = await task_queue.create_task(task_id=f"{run.swarm_id}-{tid}", description=tid,
|
||||||
|
agent_role="implementation",
|
||||||
|
required_capabilities=caps or ["python"], enqueue=True)
|
||||||
|
await swarm_runtime.attach_task(run, t.task_id)
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
await redis_client.connect()
|
||||||
|
sr_mod.SwarmRuntime._post_callback = _noop
|
||||||
|
|
||||||
|
body = {"mode": "swarm", "requirement": {"objective": "competition test"},
|
||||||
|
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
||||||
|
"metadata": {"manager_deployment_id": "m-comp"}}
|
||||||
|
run, _ = await swarm_runtime.get_or_create_run(body=body, idempotency_key=None, correlation_id="cc")
|
||||||
|
|
||||||
|
for aid in ("bid-A", "bid-B", "yielder", "incumbent", "requester"):
|
||||||
|
await agent_registry.register_agent(aid, ["python"])
|
||||||
|
|
||||||
|
# --- 1. two agents bid → arbitrate → stronger wins + gets assigned ---
|
||||||
|
t1 = await mk_task(run, "t1")
|
||||||
|
await orch.handle_task_bid("bid-A", {"task_id": t1.task_id, "confidence": 0.9, "capabilities": ["python"]})
|
||||||
|
r = await orch.handle_task_bid("bid-B", {"task_id": t1.task_id, "confidence": 0.5, "capabilities": ["python"]})
|
||||||
|
check("two bids recorded", r.get("bid_count") == 2)
|
||||||
|
# arbitrate_and_assign is called by the loop with a freshly-fetched run (bids live in Redis).
|
||||||
|
run = await swarm_runtime.get_run(run.swarm_id)
|
||||||
|
arb = await orch.arbitrate_and_assign(run, t1.task_id)
|
||||||
|
check("arbitration picked higher-confidence bidder A", arb and arb["winner_agent_id"] == "bid-A")
|
||||||
|
check("arbitration decisive + losers recorded", arb and arb["decisive"] and arb["losers"] == ["bid-B"])
|
||||||
|
t1r = await task_queue.get_task(t1.task_id)
|
||||||
|
check("winner assigned the task", t1r.assigned_agent_id == "bid-A" and t1r.status != TaskStatus.PENDING)
|
||||||
|
run_r = await swarm_runtime.get_run(run.swarm_id)
|
||||||
|
check("bids cleared after arbitration", not (run_r.metadata.get("bids") or {}).get(t1.task_id))
|
||||||
|
check("arbitration audit recorded on run", bool(run_r.metadata.get("arbitrations")))
|
||||||
|
|
||||||
|
# --- 2. yield: an assigned task is released back to pending ---
|
||||||
|
t2 = await mk_task(run, "t2")
|
||||||
|
await task_queue.remove_pending_task(t2.task_id)
|
||||||
|
await task_queue.assign_task(t2.task_id, "yielder")
|
||||||
|
y = await orch.handle_task_yield("yielder", {"task_id": t2.task_id, "reason": "context too large", "recommend_agent": "bid-A"})
|
||||||
|
check("yield released the task", y.get("released") is True)
|
||||||
|
t2r = await task_queue.get_task(t2.task_id)
|
||||||
|
check("yielded task back to PENDING + agent cleared",
|
||||||
|
t2r.status == TaskStatus.PENDING and t2r.assigned_agent_id is None)
|
||||||
|
|
||||||
|
# --- 3. takeover: strong requester wins held task from incumbent ---
|
||||||
|
t3 = await mk_task(run, "t3")
|
||||||
|
await task_queue.remove_pending_task(t3.task_id)
|
||||||
|
await task_queue.assign_task(t3.task_id, "incumbent")
|
||||||
|
tk = await orch.handle_task_takeover("requester", {"task_id": t3.task_id, "confidence": 0.95, "capabilities": ["python"]})
|
||||||
|
check("decisive requester took over", tk.get("taken_over") is True and tk.get("winner_agent_id") == "requester")
|
||||||
|
t3r = await task_queue.get_task(t3.task_id)
|
||||||
|
check("task reassigned to requester", t3r.assigned_agent_id == "requester")
|
||||||
|
|
||||||
|
# --- 4. post-cutover: competition is unconditional (swarm default; no enable flag) ---
|
||||||
|
os.environ.pop("ENABLE_TASK_COMPETITION", None)
|
||||||
|
off = await orch.handle_task_bid("bid-A", {"task_id": t1.task_id, "confidence": 0.9, "capabilities": ["python"]})
|
||||||
|
check("bids processed without any flag (swarm default)", off.get("recorded") is True)
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} competition (P4) check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all swarm competition (P4) checks passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
"""Integration test for the decentralized-rework P1: convergence wired into run completion (#12).
|
||||||
|
|
||||||
|
Boots the in-memory store, drives a run to a terminal state via refresh_swarm_run_status with the
|
||||||
|
convergence report enabled, and asserts a ConvergenceReport with a termination_reason is stored on
|
||||||
|
the run (shadow adoption — it does NOT override run.status). Hermetic, no model key.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — needs fakeredis):
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
REDIS_FAKE=1 ENABLE_CONVERGENCE_REPORT=1 python scripts/test-swarm-convergence.py
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
os.environ["REDIS_FAKE"] = "1"
|
||||||
|
os.environ["ENABLE_CONVERGENCE_REPORT"] = "1"
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.redis_client import redis_client
|
||||||
|
from orchestrator import swarm_runtime as sr_mod
|
||||||
|
from orchestrator.swarm_runtime import swarm_runtime
|
||||||
|
from orchestrator.task_queue import task_queue, TaskStatus
|
||||||
|
from orchestrator import main as orch
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
async def _noop(self, *a, **k):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def add_task(run, task_id, status, *, cost=0.0, depends_on=None):
|
||||||
|
t = await task_queue.create_task(task_id=task_id, description=task_id,
|
||||||
|
agent_role=task_id.split("-")[-1],
|
||||||
|
depends_on=depends_on or [], enqueue=False)
|
||||||
|
t.status = status
|
||||||
|
t.result = json.dumps({"usage": {"model_cost_usd": cost}})
|
||||||
|
await task_queue._save_task(t)
|
||||||
|
await swarm_runtime.attach_task(run, t.task_id)
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
await redis_client.connect()
|
||||||
|
sr_mod.SwarmRuntime._post_callback = _noop
|
||||||
|
|
||||||
|
# --- 1. all tasks completed → CONVERGED / tasks_completed ---
|
||||||
|
body = {"mode": "swarm", "requirement": {"objective": "converge test"},
|
||||||
|
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
||||||
|
"metadata": {"manager_deployment_id": "m-conv"}}
|
||||||
|
run, _ = await swarm_runtime.get_or_create_run(body=body, idempotency_key=None, correlation_id="c1")
|
||||||
|
await add_task(run, "t-implementation", TaskStatus.COMPLETED)
|
||||||
|
await add_task(run, "t-testing", TaskStatus.COMPLETED, depends_on=["t-implementation"])
|
||||||
|
|
||||||
|
await orch.refresh_swarm_run_status(run)
|
||||||
|
refreshed = await swarm_runtime.get_run(run.swarm_id)
|
||||||
|
conv = refreshed.metadata.get("convergence")
|
||||||
|
check("convergence report stored on run", isinstance(conv, dict) and bool(conv))
|
||||||
|
check("run is completed (shadow: status unchanged by report)", refreshed.status == "completed")
|
||||||
|
check("status CONVERGED", conv and conv["status"] == "converged")
|
||||||
|
check("termination_reason = tasks_completed", conv and conv["termination_reason"] == "tasks_completed")
|
||||||
|
check("consensus 100 (no conflicts)", conv and conv["consensus_score"] == 100.0)
|
||||||
|
|
||||||
|
# --- 2. a dependency inconsistency surfaces as a conflict + unresolved risk → BLOCKED ---
|
||||||
|
body2 = {**body, "metadata": {"manager_deployment_id": "m-conv2"}}
|
||||||
|
run2, _ = await swarm_runtime.get_or_create_run(body=body2, idempotency_key=None, correlation_id="c2")
|
||||||
|
# child COMPLETED but its dependency FAILED → dependency_inconsistency
|
||||||
|
await add_task(run2, "dep-implementation", TaskStatus.FAILED)
|
||||||
|
await add_task(run2, "child-testing", TaskStatus.COMPLETED, depends_on=["dep-implementation"])
|
||||||
|
await orch.refresh_swarm_run_status(run2)
|
||||||
|
r2 = await swarm_runtime.get_run(run2.swarm_id)
|
||||||
|
conv2 = r2.metadata.get("convergence")
|
||||||
|
check("conflict detected (dependency inconsistency)",
|
||||||
|
conv2 and any(c["type"] == "dependency_inconsistency" for c in conv2["conflicts"]))
|
||||||
|
check("unresolved risk → termination_reason risk_blocked",
|
||||||
|
conv2 and conv2["termination_reason"] == "risk_blocked")
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} convergence-wiring check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all swarm convergence-wiring (P1) checks passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
"""Integration test for decentralized-rework P5: peer cross-review (#11).
|
||||||
|
|
||||||
|
>=2 independent peer reviewers replace the single-critic Master gate: disagreement is recorded,
|
||||||
|
the arbitrated verdict reopens rework targets, and each rework is attributed (root cause + who
|
||||||
|
introduced it). Hermetic, no model key.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — needs fakeredis):
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
REDIS_FAKE=1 ENABLE_CROSS_REVIEW=1 python scripts/test-swarm-cross-review.py
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
os.environ["REDIS_FAKE"] = "1"
|
||||||
|
os.environ["ENABLE_CROSS_REVIEW"] = "1"
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.redis_client import redis_client
|
||||||
|
from orchestrator import swarm_runtime as sr_mod
|
||||||
|
from orchestrator.swarm_runtime import swarm_runtime
|
||||||
|
from orchestrator.task_queue import task_queue, TaskStatus
|
||||||
|
from orchestrator import main as orch
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
async def _noop(self, *a, **k):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def completed_task(run, tid, agent):
|
||||||
|
t = await task_queue.create_task(task_id=f"{run.swarm_id}-{tid}", description=tid,
|
||||||
|
agent_role=tid.split("-")[-1], enqueue=False)
|
||||||
|
t.status = TaskStatus.COMPLETED
|
||||||
|
t.assigned_agent_id = agent
|
||||||
|
await task_queue._save_task(t)
|
||||||
|
await swarm_runtime.attach_task(run, t.task_id)
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
await redis_client.connect()
|
||||||
|
sr_mod.SwarmRuntime._post_callback = _noop
|
||||||
|
|
||||||
|
body = {"mode": "swarm", "requirement": {"objective": "cross review test"},
|
||||||
|
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
||||||
|
"metadata": {"manager_deployment_id": "m-cr"}}
|
||||||
|
run, _ = await swarm_runtime.get_or_create_run(body=body, idempotency_key=None, correlation_id="cr")
|
||||||
|
impl = await completed_task(run, "t-implementation", "agent-impl")
|
||||||
|
await completed_task(run, "t-documentation", "agent-doc")
|
||||||
|
|
||||||
|
# two peers submit INDEPENDENT reviews of the implementation; they DISAGREE
|
||||||
|
await orch.handle_review_decision("reviewer-1", {
|
||||||
|
"task_id": impl.task_id, "verdict": "pass", "summary": "looks correct"})
|
||||||
|
r = await orch.handle_review_decision("reviewer-2", {
|
||||||
|
"task_id": impl.task_id, "verdict": "fail",
|
||||||
|
"failed_criteria": ["implementation raises wrong exception"],
|
||||||
|
"recommended_rework": [impl.task_id], "summary": "wrong exception on bad input"})
|
||||||
|
check("two independent reviews recorded", r.get("review_count") == 2)
|
||||||
|
|
||||||
|
# aggregate (called by the run lifecycle with a fresh run)
|
||||||
|
fresh = await swarm_runtime.get_run(run.swarm_id)
|
||||||
|
tasks = [await task_queue.get_task(tid) for tid in fresh.task_ids]
|
||||||
|
reopened = await orch.run_cross_review(fresh, tasks)
|
||||||
|
check("cross-review reopened rework target (rejected on split)", reopened is True)
|
||||||
|
|
||||||
|
after = await swarm_runtime.get_run(run.swarm_id)
|
||||||
|
cr = after.metadata.get("cross_review") or {}
|
||||||
|
check("verdict recorded with disagreement", cr.get("disagreement") is True and cr.get("accepted") is False)
|
||||||
|
check("split arbitrated by majority → reject (safety bias)",
|
||||||
|
cr.get("method") == "majority" and cr.get("pass_votes") == 1 and cr.get("fail_votes") == 1)
|
||||||
|
impl_after = await task_queue.get_task(impl.task_id)
|
||||||
|
check("flagged task reopened to PENDING", impl_after.status == TaskStatus.PENDING)
|
||||||
|
attrs = after.metadata.get("rework_attributions") or []
|
||||||
|
check("rework attributed (root cause + detector)",
|
||||||
|
attrs and attrs[0]["target_task_id"] == impl.task_id
|
||||||
|
and attrs[0]["root_cause"] in {"implementation", "collaboration"}
|
||||||
|
and attrs[0]["detected_by_agent_id"] == "reviewer-2")
|
||||||
|
check("reviews consumed after the cycle", not after.metadata.get("reviews"))
|
||||||
|
check("review cycle counter advanced", after.metadata.get("review_cycles") == 1)
|
||||||
|
|
||||||
|
# --- unanimous pass → accept, no reopen ---
|
||||||
|
run2, _ = await swarm_runtime.get_or_create_run(
|
||||||
|
body={**body, "metadata": {"manager_deployment_id": "m-cr2"}}, idempotency_key=None, correlation_id="cr2")
|
||||||
|
a2 = await completed_task(run2, "t-implementation", "agent-impl")
|
||||||
|
for rid in ("reviewer-1", "reviewer-2"):
|
||||||
|
await orch.handle_review_decision(rid, {"task_id": a2.task_id, "verdict": "pass", "summary": "ok"})
|
||||||
|
fresh2 = await swarm_runtime.get_run(run2.swarm_id)
|
||||||
|
reopened2 = await orch.run_cross_review(fresh2, [await task_queue.get_task(t) for t in fresh2.task_ids])
|
||||||
|
check("unanimous pass → not reopened", reopened2 is False)
|
||||||
|
a2_after = await task_queue.get_task(a2.task_id)
|
||||||
|
check("accepted task stays COMPLETED", a2_after.status == TaskStatus.COMPLETED)
|
||||||
|
|
||||||
|
# --- single reviewer is NOT a cross-review (needs >=2) ---
|
||||||
|
run3, _ = await swarm_runtime.get_or_create_run(
|
||||||
|
body={**body, "metadata": {"manager_deployment_id": "m-cr3"}}, idempotency_key=None, correlation_id="cr3")
|
||||||
|
a3 = await completed_task(run3, "t-implementation", "agent-impl")
|
||||||
|
await orch.handle_review_decision("reviewer-1", {"task_id": a3.task_id, "verdict": "fail",
|
||||||
|
"recommended_rework": [a3.task_id]})
|
||||||
|
fresh3 = await swarm_runtime.get_run(run3.swarm_id)
|
||||||
|
reopened3 = await orch.run_cross_review(fresh3, [await task_queue.get_task(t) for t in fresh3.task_ids])
|
||||||
|
check("single reviewer → no cross-review (needs >=2)", reopened3 is False)
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} cross-review (P5) check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all swarm cross-review (P5) checks passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
"""Integration test for decentralized-rework P6: pheromone-driven agent self-selection.
|
||||||
|
|
||||||
|
Each idle agent perceives the eligible ready tasks and self-selects the best fit. With capability
|
||||||
|
and load equal, the differentiator is the pheromone trail (τ): an agent self-selects the role it
|
||||||
|
has historically succeeded at. Hermetic, no model key.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — needs fakeredis):
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
REDIS_FAKE=1 ENABLE_SWARM_DISPATCH=1 python scripts/test-swarm-dispatch.py
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
os.environ["REDIS_FAKE"] = "1"
|
||||||
|
os.environ["ENABLE_SWARM_DISPATCH"] = "1"
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.redis_client import redis_client
|
||||||
|
from orchestrator import swarm_runtime as sr_mod
|
||||||
|
from orchestrator.swarm_runtime import swarm_runtime
|
||||||
|
from orchestrator.task_queue import task_queue
|
||||||
|
from orchestrator.agent_registry import agent_registry
|
||||||
|
from orchestrator.decision_engine import decision_engine
|
||||||
|
from orchestrator import main as orch
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
async def _noop(self, *a, **k):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def ready_task(run, role):
|
||||||
|
t = await task_queue.create_task(task_id=f"{run.swarm_id}-t-{role}", description=role,
|
||||||
|
agent_role=role, required_capabilities=["python"], enqueue=True)
|
||||||
|
await swarm_runtime.attach_task(run, t.task_id)
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
|
async def fresh_run(label):
|
||||||
|
body = {"mode": "swarm", "requirement": {"objective": "self-select test"},
|
||||||
|
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
||||||
|
"metadata": {"manager_deployment_id": label}}
|
||||||
|
run, _ = await swarm_runtime.get_or_create_run(body=body, idempotency_key=None, correlation_id=label)
|
||||||
|
return run
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
await redis_client.connect()
|
||||||
|
sr_mod.SwarmRuntime._post_callback = _noop
|
||||||
|
|
||||||
|
# Build pheromone profiles: X strong at implementation, Y strong at testing.
|
||||||
|
for _ in range(10):
|
||||||
|
await decision_engine.deposit(agent_role="implementation", agent_id="agent-X", success=True)
|
||||||
|
await decision_engine.deposit(agent_role="testing", agent_id="agent-X", success=False)
|
||||||
|
await decision_engine.deposit(agent_role="testing", agent_id="agent-Y", success=True)
|
||||||
|
await decision_engine.deposit(agent_role="implementation", agent_id="agent-Y", success=False)
|
||||||
|
|
||||||
|
# --- Run A: agent-X with BOTH tasks available self-selects its high-τ role (implementation) ---
|
||||||
|
await agent_registry.register_agent("agent-X", ["python"])
|
||||||
|
runA = await fresh_run("m-selA")
|
||||||
|
await ready_task(runA, "implementation")
|
||||||
|
await ready_task(runA, "testing")
|
||||||
|
x = next(a for a in await agent_registry.get_idle_agents() if a.agent_id == "agent-X")
|
||||||
|
assigned_a = await orch.swarm_dispatch([x])
|
||||||
|
check("agent-X self-selected its high-τ role (implementation)",
|
||||||
|
assigned_a == [("agent-X", f"{runA.swarm_id}-t-implementation")])
|
||||||
|
runA_r = await swarm_runtime.get_run(runA.swarm_id)
|
||||||
|
check("self-selection recorded an explainable dispatch decision",
|
||||||
|
bool(runA_r.dispatch_decisions)
|
||||||
|
and runA_r.dispatch_decisions[-1]["chosen_task_id"] == f"{runA.swarm_id}-t-implementation")
|
||||||
|
|
||||||
|
# --- Run B: agent-Y with BOTH tasks available self-selects its high-τ role (testing) ---
|
||||||
|
await agent_registry.register_agent("agent-Y", ["python"])
|
||||||
|
runB = await fresh_run("m-selB")
|
||||||
|
await ready_task(runB, "implementation")
|
||||||
|
await ready_task(runB, "testing")
|
||||||
|
y = next(a for a in await agent_registry.get_idle_agents() if a.agent_id == "agent-Y")
|
||||||
|
assigned_b = await orch.swarm_dispatch([y])
|
||||||
|
check("agent-Y self-selected its high-τ role (testing)",
|
||||||
|
assigned_b == [("agent-Y", f"{runB.swarm_id}-t-testing")])
|
||||||
|
|
||||||
|
# --- capability gate still holds: an agent lacking caps self-selects nothing ---
|
||||||
|
await agent_registry.register_agent("agent-Z", ["rust"])
|
||||||
|
runC = await fresh_run("m-selC")
|
||||||
|
await ready_task(runC, "implementation") # requires python
|
||||||
|
z = next(a for a in await agent_registry.get_idle_agents() if a.agent_id == "agent-Z")
|
||||||
|
assigned_c = await orch.swarm_dispatch([z])
|
||||||
|
check("incapable agent self-selects nothing", assigned_c == [])
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} self-selection (P6) check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all swarm self-selection (P6) checks passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
"""Test the swarm health guard (P-guard): detect 'swarm can't run' + reasons.
|
||||||
|
|
||||||
|
Covers the pure diagnostic (orchestrator/guard.diagnose) across blocker types, and the
|
||||||
|
orchestrator wrapper (assess_swarm_health) that records/emits the report. Hermetic, no model key.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — needs fakeredis):
|
||||||
|
pip install -r orchestrator/requirements.txt -r agent/requirements.txt
|
||||||
|
REDIS_FAKE=1 python scripts/test-swarm-guard.py
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
os.environ["REDIS_FAKE"] = "1"
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator import guard
|
||||||
|
from orchestrator.redis_client import redis_client
|
||||||
|
from orchestrator import swarm_runtime as sr_mod
|
||||||
|
from orchestrator.swarm_runtime import swarm_runtime
|
||||||
|
from orchestrator.task_queue import task_queue, TaskStatus
|
||||||
|
from orchestrator.agent_registry import agent_registry
|
||||||
|
from orchestrator import main as orch
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
def reasons(report):
|
||||||
|
return {b["reason"] for b in report.blockers}
|
||||||
|
|
||||||
|
|
||||||
|
# ---- pure diagnose ----
|
||||||
|
def test_pure():
|
||||||
|
# healthy: a ready task and a capable agent
|
||||||
|
r = guard.diagnose({
|
||||||
|
"tasks": [{"task_id": "t1", "status": "pending", "required_capabilities": ["python"], "depends_on": []}],
|
||||||
|
"connected_agent_caps": [["python", "testing"]],
|
||||||
|
})
|
||||||
|
check("healthy when a capable agent exists", r.healthy and not r.blockers)
|
||||||
|
|
||||||
|
# no agents connected
|
||||||
|
r = guard.diagnose({
|
||||||
|
"tasks": [{"task_id": "t1", "status": "pending", "required_capabilities": [], "depends_on": []}],
|
||||||
|
"connected_agent_caps": [],
|
||||||
|
})
|
||||||
|
check("NO_AGENTS_CONNECTED when nobody is connected",
|
||||||
|
not r.healthy and guard.Blocker.NO_AGENTS_CONNECTED.value in reasons(r))
|
||||||
|
|
||||||
|
# no capable agent
|
||||||
|
r = guard.diagnose({
|
||||||
|
"tasks": [{"task_id": "t1", "status": "pending", "required_capabilities": ["rust"], "depends_on": []}],
|
||||||
|
"connected_agent_caps": [["python"]],
|
||||||
|
})
|
||||||
|
check("NO_CAPABLE_AGENT when caps uncovered",
|
||||||
|
not r.healthy and guard.Blocker.NO_CAPABLE_AGENT.value in reasons(r))
|
||||||
|
|
||||||
|
# dependency deadlock (dep failed)
|
||||||
|
r = guard.diagnose({
|
||||||
|
"tasks": [
|
||||||
|
{"task_id": "dep", "status": "failed", "required_capabilities": [], "depends_on": []},
|
||||||
|
{"task_id": "t1", "status": "pending", "required_capabilities": ["python"], "depends_on": ["dep"]},
|
||||||
|
],
|
||||||
|
"connected_agent_caps": [["python"]],
|
||||||
|
})
|
||||||
|
check("DEPENDENCY_DEADLOCK when a dependency FAILED",
|
||||||
|
not r.healthy and guard.Blocker.DEPENDENCY_DEADLOCK.value in reasons(r))
|
||||||
|
|
||||||
|
# budget exhausted while active
|
||||||
|
r = guard.diagnose({
|
||||||
|
"tasks": [{"task_id": "t1", "status": "pending", "required_capabilities": ["python"], "depends_on": []}],
|
||||||
|
"connected_agent_caps": [["python"]],
|
||||||
|
"budget_state": {"exhausted": True},
|
||||||
|
})
|
||||||
|
check("BUDGET_EXHAUSTED while work remains",
|
||||||
|
not r.healthy and guard.Blocker.BUDGET_EXHAUSTED.value in reasons(r))
|
||||||
|
|
||||||
|
# seed completed but nothing proposed
|
||||||
|
r = guard.diagnose({
|
||||||
|
"tasks": [{"task_id": "seed", "status": "completed", "required_capabilities": [], "depends_on": [], "source": "seed"}],
|
||||||
|
"connected_agent_caps": [["python"]],
|
||||||
|
})
|
||||||
|
check("SEED_UNDECOMPOSED when only a terminal seed exists",
|
||||||
|
not r.healthy and guard.Blocker.SEED_UNDECOMPOSED.value in reasons(r))
|
||||||
|
|
||||||
|
# blockers carry a human-readable detail
|
||||||
|
check("blockers include a detail string", all(b.get("detail") for b in r.blockers))
|
||||||
|
|
||||||
|
|
||||||
|
# ---- orchestrator wrapper records the report INTERNALLY (not a Manager event) ----
|
||||||
|
async def test_wrapper():
|
||||||
|
await redis_client.connect()
|
||||||
|
|
||||||
|
async def _noop_cb(self, *a, **k):
|
||||||
|
return None
|
||||||
|
sr_mod.SwarmRuntime._post_callback = _noop_cb
|
||||||
|
emitted = []
|
||||||
|
orig_emit = sr_mod.SwarmRuntime.emit_event
|
||||||
|
|
||||||
|
async def spy_emit(self, run, event_type, **k):
|
||||||
|
emitted.append(event_type)
|
||||||
|
return await orig_emit(self, run, event_type, **k)
|
||||||
|
sr_mod.SwarmRuntime.emit_event = spy_emit
|
||||||
|
|
||||||
|
body = {"mode": "swarm", "requirement": {"objective": "guard test"},
|
||||||
|
"callback": {"url": "http://127.0.0.1:9/cb", "subscribed_events": []},
|
||||||
|
"metadata": {"manager_deployment_id": "m-guard"}}
|
||||||
|
run, _ = await swarm_runtime.get_or_create_run(body=body, idempotency_key=None, correlation_id="cg")
|
||||||
|
# a pending task requiring caps nobody connected has
|
||||||
|
t = await task_queue.create_task(task_id=f"{run.swarm_id}-t", description="x",
|
||||||
|
agent_role="impl", required_capabilities=["rust"], enqueue=True)
|
||||||
|
await swarm_runtime.attach_task(run, t.task_id)
|
||||||
|
# no connected agents → NO_AGENTS_CONNECTED (+ would be NO_CAPABLE if any)
|
||||||
|
report = await orch.assess_swarm_health(run, connected_agent_ids=[])
|
||||||
|
check("wrapper reports unhealthy", report.healthy is False)
|
||||||
|
refreshed = await swarm_runtime.get_run(run.swarm_id)
|
||||||
|
check("health report stored on run", bool(refreshed.metadata.get("health"))
|
||||||
|
and refreshed.metadata["health"]["healthy"] is False)
|
||||||
|
check("unhealthy report appended to internal health_log", bool(refreshed.metadata.get("health_log")))
|
||||||
|
# Contract hygiene: health is INTERNAL — no unregistered Manager event is emitted.
|
||||||
|
check("no swarm.health Manager event emitted", "swarm.health" not in emitted)
|
||||||
|
|
||||||
|
# now register a capable agent → healthy
|
||||||
|
await agent_registry.register_agent("rust-agent", ["rust"])
|
||||||
|
report2 = await orch.assess_swarm_health(run, connected_agent_ids=["rust-agent"])
|
||||||
|
check("healthy once a capable agent is connected", report2.healthy is True)
|
||||||
|
|
||||||
|
sr_mod.SwarmRuntime.emit_event = orig_emit
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
test_pure()
|
||||||
|
await test_wrapper()
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} guard check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all swarm guard (P-guard) checks passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
"""Unit test for the decentralized-rework seeder (build_seed_task_specs, #6 Path B / P2).
|
||||||
|
|
||||||
|
The seeder turns an objective into ONE seed task with NO up-front Master decomposition. This tests
|
||||||
|
the pure function; wiring it as the swarm-only task-creation path (deleting the planner fallback)
|
||||||
|
happens at P-cutover once agent-driven decomposition (#7) exists — see
|
||||||
|
docs/swarm/decentralized-rework-plan.md. No flag, no Redis, no model.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6:
|
||||||
|
python scripts/test-swarm-seed.py
|
||||||
|
"""
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.main import build_seed_task_specs
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
class _StubRun:
|
||||||
|
"""Minimal stand-in: build_seed_task_specs only reads run.objective."""
|
||||||
|
objective = "Build a CSV parser with tests and docs"
|
||||||
|
|
||||||
|
|
||||||
|
specs = build_seed_task_specs(_StubRun(), body={}, base_specs=[])
|
||||||
|
|
||||||
|
check("seeds exactly ONE task", len(specs) == 1)
|
||||||
|
seed = specs[0] if specs else {}
|
||||||
|
check("source = 'seed'", seed.get("source") == "seed")
|
||||||
|
check("carries the objective", "CSV parser" in (seed.get("description") or ""))
|
||||||
|
check("no required capabilities (any agent may self-select)", seed.get("required_capabilities") == [])
|
||||||
|
check("workflow_mode = swarm", seed.get("workflow_mode") == "swarm")
|
||||||
|
check("context marks is_seed + objective",
|
||||||
|
(seed.get("context") or {}).get("is_seed") is True and (seed.get("context") or {}).get("objective"))
|
||||||
|
check("no depends_on (it's the root seed)", seed.get("depends_on") == [])
|
||||||
|
# The seeder is a PURE spec builder — it must not itself decompose / call any planner.
|
||||||
|
check("seeder does not pre-decompose (single root task only)",
|
||||||
|
len(specs) == 1 and seed.get("root_task_id") == "seed")
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} seeder (P2) check(s) FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all swarm seeder (P2) unit checks passed")
|
||||||
@@ -0,0 +1,205 @@
|
|||||||
|
r"""Hermetic tests for the agent task-competition protocol (issue #8).
|
||||||
|
|
||||||
|
Module-level only: NO Redis, NO WebSocket, NO model calls. `orchestrator.task_competition`
|
||||||
|
is side-effect free (pure data models + a pure `arbitrate`), so this exercises the real
|
||||||
|
arbitration math directly and asserts it is explainable and deterministic.
|
||||||
|
|
||||||
|
Run from agent_swarm_v6 (install deps first — pydantic is the only requirement here):
|
||||||
|
pip install -r orchestrator/requirements.txt
|
||||||
|
..\.venv\Scripts\python.exe scripts/test-task-competition.py
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Belt-and-braces: this test never reaches Redis, but mirror the other scripts' hermetic
|
||||||
|
# guard so an accidental import that DOES touch redis_client stays in-memory.
|
||||||
|
os.environ.setdefault("REDIS_FAKE", "1")
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||||
|
|
||||||
|
from orchestrator.task_competition import (
|
||||||
|
ArbitrationPolicy,
|
||||||
|
TaskBid,
|
||||||
|
TaskTakeoverRequest,
|
||||||
|
TaskYield,
|
||||||
|
arbitrate,
|
||||||
|
arbitrated_event,
|
||||||
|
bid_submitted_event,
|
||||||
|
takeover_requested_event,
|
||||||
|
yielded_event,
|
||||||
|
)
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond):
|
||||||
|
print(("PASS" if cond else "FAIL"), "-", name)
|
||||||
|
if not cond:
|
||||||
|
failures.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
task_id = "task-42"
|
||||||
|
required = ["python", "testing"]
|
||||||
|
# τ map: the strong agent has earned reputation, the weak one has not (→ neutral 0.5).
|
||||||
|
historical = {"agent-strong": 0.9, "agent-weak": 0.3}
|
||||||
|
|
||||||
|
# --- two agents bid for the SAME task ------------------------------------
|
||||||
|
strong = TaskBid(
|
||||||
|
task_id=task_id,
|
||||||
|
agent_id="agent-strong",
|
||||||
|
confidence=0.9,
|
||||||
|
estimated_cost=0.2,
|
||||||
|
estimated_time=120.0,
|
||||||
|
risk_score=0.1,
|
||||||
|
reason="Specialist; has done this many times.",
|
||||||
|
capabilities=["python", "testing", "docs"],
|
||||||
|
current_load=0,
|
||||||
|
)
|
||||||
|
weak = TaskBid(
|
||||||
|
task_id=task_id,
|
||||||
|
agent_id="agent-weak",
|
||||||
|
confidence=0.6,
|
||||||
|
estimated_cost=0.7,
|
||||||
|
estimated_time=400.0,
|
||||||
|
risk_score=0.5,
|
||||||
|
reason="Available but unproven on this kind of task.",
|
||||||
|
capabilities=["python"], # missing 'testing' → lower capability fit
|
||||||
|
current_load=2,
|
||||||
|
)
|
||||||
|
|
||||||
|
result = arbitrate(
|
||||||
|
[strong, weak],
|
||||||
|
required_capabilities=required,
|
||||||
|
historical_success=historical,
|
||||||
|
)
|
||||||
|
|
||||||
|
check("better agent wins", result.winner_agent_id == "agent-strong")
|
||||||
|
check("loser recorded", result.losers == ["agent-weak"])
|
||||||
|
check("decision is decisive", result.decisive is True)
|
||||||
|
# Auditable: a non-empty reason naming the winner, plus a full per-bid score breakdown.
|
||||||
|
check("reason names winner", "agent-strong" in result.reason)
|
||||||
|
check("reason is explainable", "dominant factor" in result.reason)
|
||||||
|
check("scores cover every bid", {s.agent_id for s in result.scores} == {"agent-strong", "agent-weak"})
|
||||||
|
winner_score = next(s for s in result.scores if s.agent_id == "agent-strong")
|
||||||
|
check("winner score broken down into components", set(winner_score.components.keys()) == {
|
||||||
|
"confidence", "capability", "historical_success", "budget", "time", "risk", "load",
|
||||||
|
})
|
||||||
|
check("component sum equals total", round(sum(winner_score.components.values()), 6) == winner_score.total)
|
||||||
|
|
||||||
|
# --- determinism: input order must not change the outcome ----------------
|
||||||
|
reversed_result = arbitrate(
|
||||||
|
[weak, strong],
|
||||||
|
required_capabilities=required,
|
||||||
|
historical_success=historical,
|
||||||
|
)
|
||||||
|
check("winner deterministic under input reorder", reversed_result.winner_agent_id == "agent-strong")
|
||||||
|
check("scores deterministic under input reorder",
|
||||||
|
[(s.agent_id, s.total) for s in reversed_result.scores]
|
||||||
|
== [(s.agent_id, s.total) for s in result.scores])
|
||||||
|
|
||||||
|
# --- determinism: repeated calls are byte-identical ----------------------
|
||||||
|
again = arbitrate([strong, weak], required_capabilities=required, historical_success=historical)
|
||||||
|
check("repeat call identical winner", again.winner_agent_id == result.winner_agent_id)
|
||||||
|
check("repeat call identical scores",
|
||||||
|
[(s.agent_id, s.total) for s in again.scores] == [(s.agent_id, s.total) for s in result.scores])
|
||||||
|
|
||||||
|
# --- τ as a real arbitration input ---------------------------------------
|
||||||
|
# Make the two bids identical EXCEPT reputation; the higher-τ agent must win, proving
|
||||||
|
# historical_success actually moves the result (not a decorative field).
|
||||||
|
twin_a = TaskBid(task_id=task_id, agent_id="agent-a", confidence=0.7,
|
||||||
|
capabilities=required, estimated_cost=0.3, risk_score=0.2)
|
||||||
|
twin_b = TaskBid(task_id=task_id, agent_id="agent-b", confidence=0.7,
|
||||||
|
capabilities=required, estimated_cost=0.3, risk_score=0.2)
|
||||||
|
tau_result = arbitrate(
|
||||||
|
[twin_a, twin_b],
|
||||||
|
required_capabilities=required,
|
||||||
|
historical_success={"agent-a": 0.95, "agent-b": 0.10},
|
||||||
|
)
|
||||||
|
check("higher historical_success (tau) wins all-else-equal", tau_result.winner_agent_id == "agent-a")
|
||||||
|
|
||||||
|
# --- contested (narrow) ties are flagged, not silently auto-assigned -----
|
||||||
|
# Fully identical inputs incl. τ → zero score gap → not decisive; winner falls back to
|
||||||
|
# the stable agent_id tiebreak so the call is still deterministic.
|
||||||
|
tie = arbitrate(
|
||||||
|
[TaskBid(task_id=task_id, agent_id="agent-z", capabilities=required),
|
||||||
|
TaskBid(task_id=task_id, agent_id="agent-a", capabilities=required)],
|
||||||
|
required_capabilities=required,
|
||||||
|
historical_success={},
|
||||||
|
)
|
||||||
|
check("dead tie is not decisive", tie.decisive is False)
|
||||||
|
check("dead tie still deterministic (agent_id tiebreak)", tie.winner_agent_id == "agent-a")
|
||||||
|
check("contested reason recommends Manager review", "Manager review" in tie.reason)
|
||||||
|
|
||||||
|
# --- empty bid set is handled ---------------------------------------------
|
||||||
|
empty = arbitrate([], required_capabilities=required)
|
||||||
|
check("no bids → no winner", empty.winner_agent_id is None)
|
||||||
|
check("no bids → not decisive", empty.decisive is False)
|
||||||
|
|
||||||
|
# --- yield-with-reason ----------------------------------------------------
|
||||||
|
yield_msg = TaskYield(
|
||||||
|
task_id=task_id,
|
||||||
|
agent_id="agent-weak",
|
||||||
|
release_with_reason="Blocked on missing 'testing' capability.",
|
||||||
|
recommend_agent="agent-strong",
|
||||||
|
)
|
||||||
|
y_type, y_payload = yielded_event(yield_msg)
|
||||||
|
check("yield event type", y_type == "task.yielded")
|
||||||
|
check("yield carries reason", y_payload["reason"] == "Blocked on missing 'testing' capability.")
|
||||||
|
check("yield carries recommendation", y_payload["recommend_agent"] == "agent-strong")
|
||||||
|
check("yield summary explainable", "yielded task" in y_payload["summary"])
|
||||||
|
|
||||||
|
# --- takeover request -----------------------------------------------------
|
||||||
|
takeover = TaskTakeoverRequest(
|
||||||
|
task_id=task_id,
|
||||||
|
requesting_agent_id="agent-strong",
|
||||||
|
current_agent_id="agent-weak",
|
||||||
|
reason="Incumbent stalled; I am a better fit.",
|
||||||
|
bid=strong,
|
||||||
|
)
|
||||||
|
t_type, t_payload = takeover_requested_event(takeover)
|
||||||
|
check("takeover event type", t_type == "task.takeover_requested")
|
||||||
|
check("takeover names requester", t_payload["requesting_agent_id"] == "agent-strong")
|
||||||
|
check("takeover names incumbent", t_payload["current_agent_id"] == "agent-weak")
|
||||||
|
# The takeover bid can be arbitrated against the incumbent's bid on identical terms.
|
||||||
|
contest = arbitrate(
|
||||||
|
[takeover.bid, weak],
|
||||||
|
required_capabilities=required,
|
||||||
|
historical_success=historical,
|
||||||
|
)
|
||||||
|
check("takeover bid wins arbitration vs incumbent", contest.winner_agent_id == "agent-strong")
|
||||||
|
|
||||||
|
# --- remaining event builders ---------------------------------------------
|
||||||
|
b_type, b_payload = bid_submitted_event(strong)
|
||||||
|
check("bid_submitted event type", b_type == "task.bid_submitted")
|
||||||
|
check("bid_submitted carries confidence", b_payload["confidence"] == 0.9)
|
||||||
|
a_type, a_payload = arbitrated_event(result)
|
||||||
|
check("arbitrated event type", a_type == "task.arbitrated")
|
||||||
|
check("arbitrated event carries winner", a_payload["winner_agent_id"] == "agent-strong")
|
||||||
|
check("arbitrated event carries full scores", len(a_payload["scores"]) == 2)
|
||||||
|
check("arbitrated event carries losers", a_payload["losers"] == ["agent-weak"])
|
||||||
|
|
||||||
|
# --- custom policy changes the weighting deterministically ----------------
|
||||||
|
# Zero out everything except load; the LESS-loaded agent must then win regardless of τ.
|
||||||
|
load_policy = ArbitrationPolicy(
|
||||||
|
w_confidence=0.0, w_capability=0.0, w_historical_success=0.0,
|
||||||
|
w_budget=0.0, w_time=0.0, w_risk=0.0, w_load=1.0,
|
||||||
|
)
|
||||||
|
loaded = TaskBid(task_id=task_id, agent_id="agent-busy", current_load=4, capabilities=required)
|
||||||
|
free = TaskBid(task_id=task_id, agent_id="agent-free", current_load=0, capabilities=required)
|
||||||
|
load_result = arbitrate(
|
||||||
|
[loaded, free], policy=load_policy,
|
||||||
|
required_capabilities=required, historical_success={"agent-busy": 0.99},
|
||||||
|
)
|
||||||
|
check("policy override: least-loaded wins under load-only policy",
|
||||||
|
load_result.winner_agent_id == "agent-free")
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} FAILED: {failures}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("ALL PASSED")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -21,9 +21,8 @@ import threading
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
# Configure the runtime for a deterministic, hermetic run BEFORE importing the app.
|
# Configure the runtime for a deterministic, hermetic run BEFORE importing the app.
|
||||||
|
# No mode flags: this repo IS the swarm runtime — seed + self-organize is the only flow.
|
||||||
os.environ["REDIS_FAKE"] = "1"
|
os.environ["REDIS_FAKE"] = "1"
|
||||||
os.environ["ENABLE_PLANNER_FALLBACK"] = "1"
|
|
||||||
os.environ["ENABLE_REVIEW_LOOP"] = "1"
|
|
||||||
os.environ["MAX_REVIEW_CYCLES"] = "2"
|
os.environ["MAX_REVIEW_CYCLES"] = "2"
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[1]
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
@@ -104,17 +103,22 @@ async def main():
|
|||||||
|
|
||||||
tasks = (await client.get(f"{BASE}/api/swarms/{dep}/tasks")).json()["data"]["tasks"]
|
tasks = (await client.get(f"{BASE}/api/swarms/{dep}/tasks")).json()["data"]["tasks"]
|
||||||
events = (await client.get(f"{BASE}/api/swarms/{dep}/logs")).json()["data"]["events"]
|
events = (await client.get(f"{BASE}/api/swarms/{dep}/logs")).json()["data"]["events"]
|
||||||
roles = sorted(t["agent_role"] for t in tasks)
|
roles = sorted(t.get("agent_role") for t in tasks)
|
||||||
summaries = [(e.get("payload") or {}).get("summary", "") or "" for e in events]
|
sources = [t.get("source") for t in tasks]
|
||||||
review_seen = any("Review cycle" in s for s in summaries)
|
payloads = [(e.get("payload") or {}) for e in events]
|
||||||
|
termination_seen = any(p.get("termination_reason") for p in payloads)
|
||||||
|
|
||||||
# ---- workflow assertions ----
|
# ---- swarm-flow assertions (seed → agent-decompose → self-select → converge) ----
|
||||||
check("decompose: 3 specialist tasks created", len(tasks) == 3)
|
check("seed: a single objective seed task was injected (no Master plan)",
|
||||||
check("decompose: roles are implementation/testing/documentation",
|
sources.count("seed") == 1)
|
||||||
roles == ["documentation", "implementation", "testing"])
|
check("decompose: agents grew the graph bottom-up (>=2 agent-proposed subtasks)",
|
||||||
check("execute: every task completed", bool(tasks) and all(t["status"] == "completed" for t in tasks))
|
sources.count("agent_proposed") >= 2)
|
||||||
check("iterate: at least one master review cycle occurred", review_seen)
|
check("decompose: proposed roles include specialist roles",
|
||||||
check("deliver: run reached completed", status == "completed")
|
{"implementation", "testing", "documentation"} & set(roles))
|
||||||
|
check("execute: every task (seed + proposed) completed",
|
||||||
|
bool(tasks) and all(t["status"] == "completed" for t in tasks))
|
||||||
|
check("converge: run reached completed", status == "completed")
|
||||||
|
check("converge: a termination_reason was emitted (convergence report)", termination_seen)
|
||||||
check("synthesize: a final unified summary is present", bool(wf.get("summary")))
|
check("synthesize: a final unified summary is present", bool(wf.get("summary")))
|
||||||
finally:
|
finally:
|
||||||
agent.running = False
|
agent.running = False
|
||||||
@@ -127,7 +131,7 @@ async def main():
|
|||||||
if failures:
|
if failures:
|
||||||
print(f"{len(failures)} workflow check(s) FAILED: {failures}")
|
print(f"{len(failures)} workflow check(s) FAILED: {failures}")
|
||||||
return 1
|
return 1
|
||||||
print("workflow follows the expected sequence: decompose -> dispatch -> execute -> review/iterate -> synthesize")
|
print("swarm flow verified: seed -> agent self-select -> bottom-up decompose -> execute -> converge")
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user