docs(README): benchmark collector has landed — only scoring standard pending
The capability table claimed "🔴 规划中(采集器尚未落地)", but the benchmark collector is in fact implemented under benchmark/: metric formulas (metrics.py: S_swarm/G_E/G_E,c/Benchmark_Agent + governance/collaboration/communication/ robustness), live-run collection (collectors/), the 4 baseline runners (runners/), G_E/G_E,c comparison (baselines.compare), self-cert merge (selfcert_collector), and telemetry export to Cosmos/Blob (export/). Updated the row to 🟡: collector landed; the remaining piece is the scoring standard — the O (observability) formula, BASE_COEFFICIENTS, and S_gain thresholds must be empirically calibrated from accumulated real user usage data (long-term ticket). Keeps the honesty rule: metrics lacking real inputs return NaN, never fabricated scores. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
a978244cdd
commit
4aaf63cea5
@@ -10,7 +10,7 @@
|
||||
> | 任务分解(规划回退)、能力路由派发、专家执行(OpenAI 兼容)、评审/重做循环、结果汇总、peer 协作消息路由、WebSocket Agent 协议、Redis 持久化、Prometheus 指标 | ✅ 已实现(仓内可运行) |
|
||||
> | Manager↔Runtime 生命周期契约(对齐 `heicode-am-contract`)、HMAC 签名回调事件 envelope、稳定的 `deployment_id`/`workflow_id`/`trace_id` | 🟡 待接入(见 `docs/integration/runtime-contract.md`;事件 envelope 与签名待 Manager 对齐) |
|
||||
> | 统一 usage/计费聚合(归属 NewAPI)、审计/lineage trace、前端事件 API、统一 Agent registry/scheduling、统一 secret/workspace/tool/MCP/tenant 安全边界 | 🟡 待接入(需与 Billing / Audit / Frontend / Infra / Security Team 对齐) |
|
||||
> | Benchmark 自证(`Benchmark_Agent`、`S_swarm`、`G_E`、`G_E,c`、治理/协作/通信/鲁棒性指标、baseline 对比、telemetry 架构) | 🔴 规划中(标准见 `docs/benchmark/`,采集器尚未落地) |
|
||||
> | Benchmark 自证(`Benchmark_Agent`、`S_swarm`、`G_E`、`G_E,c`、治理/协作/通信/鲁棒性指标、baseline 对比、telemetry 架构) | 🟡 采集器已落地(`benchmark/`:指标公式 `metrics.py`、活体 run 采集 `collectors/`、4 基线 runner `runners/`、`baselines.compare` 算 `G_E`/`G_E,c`、自证合流 `selfcert_collector`、telemetry 导出 Cosmos/Blob `export/`);**评分标准待定**——`O`(可观测性)公式、`BASE_COEFFICIENTS`、`S_gain` 阈值需累积真实用户使用数据后经验标定(长期工单)。诚实原则:缺真实输入的量返回 `NaN`、不伪造分值(标准见 `docs/benchmark/`) |
|
||||
>
|
||||
> 在以上「待接入 / 规划中」项目完成并经对应 Team 验收前,本文与各子文档**不得宣称**「已接入主链路」或「已具备完整 Agent Swarm 工程能力」。
|
||||
>
|
||||
|
||||
Reference in New Issue
Block a user