taijibaga/fengqun
9
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Use external codebase for live agent scenario

Browse Source 
Replace the self-referential S07 code task with a pinned fastapi/fastapi GitHub scenario so the live Agent standard tests evaluate an external complex codebase instead of the local harness project.

Constraint: The user explicitly rejected using this project as the code target for the live scenario.

Rejected: Keeping swarm-minimal as the S07 code target | it would keep validating the harness against itself.

Confidence: high

Scope-risk: moderate

Directive: Keep S07 target files external to this repository unless the user explicitly asks for a local-harness scenario.

Tested: .venv/bin/python -u -B examples/run_standard_scenario_acceptance.py; .venv/bin/python -B -m unittest discover -s tests; .venv/bin/python -B -m py_compile swarm_minimal/*.py examples/*.py tests/*.py; .venv/bin/python -u -B examples/run_academic_standard_evaluation.py; git diff --check; docs secret-pattern scan.

Not-tested: Applying the proposed FastAPI patch inside the external fastapi/fastapi repository was not run; S07 is a live Agent reasoning and evidence-chain test.

Co-authored-by: OmX <omx@oh-my-codex.dev>
This commit is contained in:
gongzhiyong
2026-05-16 15:44:36 +08:00
co-authored by OmX
parent d632fd9f64
commit dc2b166e60
12 changed files with 594 additions and 5313 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+5 -3
View File
@@ -6,11 +6,13 @@
本仓库已经补齐最小 Agent / 蜂群 Agent 标准化测试闭环。这里的标准不是普通软件开发质量标准，而是围绕 Agent 的任务理解、上下文承接、模型选择、工具边界、敏感信息保护、可观测性、handoff、信息素协作、收敛和马尔可夫式状态转移建立的 AQS / SW-AQS v1。
最新标准矩阵（S01-S08）已通过；以下是 S07 live 证据：
最新标准矩阵（S01-S08）已通过；S07 已换成外部 GitHub 复杂项目 `fastapi/fastapi` 的代码场景，以下是 S07 live 证据：
```json
{
"run_id": "04c641d170fe4ea7aa3d882d9df37cca",
"run_id": "7ba01d0cc4ad402793938835654fbca5",
"target_repo": "fastapi/fastapi",
"target_commit": "ecace740f3eaccb1aba152cf1de79477095c56f4",
"completed_tasks": 7,
"accepted_score": 1.0,
"check_count": 12,
@@ -20,7 +22,7 @@
主要报告入口都放在 `docs/`，完整索引见 `docs/README.md`：
- `docs/AGENT_SWARM_QUALITY_STANDARD.zh-CN.md`：Agent / 蜂群 Agent 质量标准、测试标准来源、S07 任务分配和接手机制。
- `docs/AGENT_SWARM_QUALITY_STANDARD.zh-CN.md`：Agent / 蜂群 Agent 质量标准、测试标准来源、S07 外部 GitHub 代码任务分配和接手机制。
- `docs/ACADEMIC_STANDARD_TEST_REPORT.zh-CN.md`：测试场景、测试过程、测试结果、算法清单和马尔可夫过程判定。
- `docs/MODEL_AGNET_IO_REPORT.zh-CN.md`：每个模型 / Agnet 的任务、输入、输出、评分和交接过程。
- `docs/STANDARD_TEST_MATRIX.md`：S01-S08 标准矩阵。
docs/ACADEMIC_STANDARD_TEST_REPORT.zh-CN.md
+10 -8
View File
@@ -14,7 +14,7 @@
| S04 | 依赖边界 | MVP 不把 NATS/Cosmos 当成必需依赖 | `tests/test_standard_scenarios.py` |
| S05 | 最终收敛评分 | 最终 STEP-07 输出必须优于中间步骤 | `tests/test_standard_scenarios.py` |
| S06 | 失败注入 | 失败任务要标记失败、产生负信息素，其他任务仍可收敛 | `tests/test_standard_scenarios.py` |
| S07 | live Azure/NewAPI 连续推理 | 真实连接 Azure PostgreSQL、Redis、Blob 和 NewAPI，跑 7 步连续推理链 | `examples/run_continuous_reasoning_acceptance.py` |
| S07 | live 外部 GitHub 代码推理 | 真实连接 Azure PostgreSQL、Redis、Blob 和 NewAPI，对 `fastapi/fastapi` 固定 commit 跑 7 步代码推理链 | `examples/run_continuous_reasoning_acceptance.py` |
| S08 | 模型 I/O 报告审计 | 验证报告包含场景、输入、输出、接手证据，且无明显真实密钥样式 | `tests/test_model_io_report_audit.py` |
| B01 | 单 Agnet 故障隔离 | 一个 Agnet 崩溃不导致整体失败 | `tests/test_swarm_behavior_academic.py` |
| B02 | 群体涌现 | 多个弱局部信号通过共享状态聚合成更强整体结果 | `tests/test_swarm_behavior_academic.py` |
@@ -54,7 +54,7 @@ python3 -u -B examples/run_consensus_convergence_acceptance.py
python3 -u -B examples/run_academic_standard_evaluation.py
```
完整标准矩阵还包含 S07 live 场景和 S08 模型 I/O 报告审计：
完整标准矩阵还包含 S07 外部 GitHub live 代码场景和 S08 模型 I/O 报告审计：
```bash
python3 -u -B examples/run_standard_scenario_acceptance.py
@@ -67,20 +67,22 @@ S07 需要本地被 git 忽略的 `.env`，其中包含 Azure PostgreSQL、Redis
| 命令 | 本轮结果 | 说明 |
| --- | --- | --- |
| `python3 -B -m py_compile swarm_minimal/*.py examples/*.py tests/*.py` | PASS | 无输出，表示所有模块编译通过 |
| `python3 -B -m unittest discover -s tests` | PASS | 28 个测试通过 |
| `python3 -B -m unittest discover -s tests` | PASS | 29 个测试通过 |
| `python3 -B -m unittest tests.test_markov_process_properties` | PASS | 3 个马尔可夫适配性测试通过 |
| `python3 -B -m unittest tests.test_model_io_report_audit` | PASS | 2 个模型 I/O 报告审计测试通过 |
| `python3 -u -B examples/run_swarm_behavior_acceptance.py` | PASS | B01-B04 全部通过 |
| `python3 -u -B examples/run_swarm_vs_traditional_benchmark.py` | PASS | 蜂群总归一化分 0.9175，传统基线 0.1958，ratio 4.69 |
| `python3 -u -B examples/run_consensus_convergence_acceptance.py` | PASS | 2 轮收敛，接受 `lease_based_pg_queue` |
| `python3 -u -B examples/run_academic_standard_evaluation.py` | PASS | A01-A05 本地学术化门禁全部通过，检测到 `examples/.env` 可用于 live 测试 |
| `python3 -u -B examples/run_standard_scenario_acceptance.py` | PASS | S01-S08 全部通过；S07 完成 7 个 live 任务，12 个检查无失败，S08 报告审计通过 |
| `python3 -u -B examples/run_standard_scenario_acceptance.py` | PASS | S01-S08 全部通过；S07 外部 FastAPI 代码场景完成 7 个 live 任务，12 个检查无失败，S08 报告审计通过 |
最新 S07 live 证据：
最新 S07 外部 GitHub live 证据：
```json
{
"run_id": "04c641d170fe4ea7aa3d882d9df37cca",
"run_id": "7ba01d0cc4ad402793938835654fbca5",
"target_repo": "fastapi/fastapi",
"target_commit": "ecace740f3eaccb1aba152cf1de79477095c56f4",
"completed_tasks": 7,
"accepted_score": 1.0,
"selected_models": [
@@ -90,7 +92,7 @@ S07 需要本地被 git 忽略的 `.env`，其中包含 Azure PostgreSQL、Redis
],
"failed_checks": [],
"check_count": 12,
"artifact_path": "swarm-runs/04c641d170fe4ea7aa3d882d9df37cca/result.json"
"artifact_path": "swarm-runs/7ba01d0cc4ad402793938835654fbca5/result.json"
}
```
@@ -100,7 +102,7 @@ S07 需要本地被 git 忽略的 `.env`，其中包含 Azure PostgreSQL、Redis
| --- | --- |
| 本地学术化门禁 A01-A05 | 可通过 |
| 标准矩阵 S01-S06 | 可通过 |
| 标准矩阵 S07 live Azure/NewAPI | 已通过 |
| 标准矩阵 S07 live 外部 GitHub 代码推理 | 已通过 |
| 标准矩阵 S08 模型 I/O 报告审计 | 已通过 |
| 生产级标准化验收 | 当前最小标准已满足 |
docs/AGENT_SWARM_QUALITY_STANDARD.zh-CN.md
+18 -12
View File
@@ -43,7 +43,7 @@
| AQS-08 可观测性 | run_id、task status、score、observation、artifact、stream event 可查 | S07 PostgreSQL/Redis/Blob 证据 |
| AQS-09 错误可解释 | 失败任务必须记录 error、failed observation 和负信息素 | S06/B01/C01 |
| AQS-10 交接准备度 | 输出要给下一个 Agent 留出摘要、风险和下一步 | S07 prompt/output 中强制“下一步/交接” |
| AQS-11 外部依赖真实性 | live 测试必须真实连 PostgreSQL、Redis、Blob、NewAPI | S07 live integration PASS |
| AQS-11 外部依赖真实性 | live 测试必须真实连 PostgreSQL、Redis、Blob、NewAPI，并把代码任务指向外部 GitHub 项目 | S07 external GitHub live PASS |
| AQS-12 人类审计友好 | 最终报告必须能回答：任务、输入、输出、接手、结果、未满足项 | S08 + `MODEL_AGNET_IO_REPORT.zh-CN.md` 和本文件 |
## 4. 蜂群 Agent 质量标准 SW-AQS
@@ -71,20 +71,24 @@
总任务：
```text
连续性长推理场景：为 swarm-minimal 设计可恢复的大规模代码任务推理链
外部 GitHub 代码场景：审查 fastapi/fastapi 响应序列化与 OpenAPI 依赖链
```
目标不是让模型自由聊天，而是让 7 个连续 Agent 阶段接力完成一个工程推理链：
目标不是让模型自由聊天，也不是拿本仓库验证自己闭环，而是让 7 个连续 Agent 阶段接力审查一个外部复杂 GitHub 项目：
- 目标仓库：`https://github.com/fastapi/fastapi`
- 固定 commit：`ecace740f3eaccb1aba152cf1de79477095c56f4`
- 代码范围：`fastapi/routing.py`、`fastapi/dependencies/utils.py`、`fastapi/openapi/utils.py`、`fastapi/params.py`、`fastapi/encoders.py`、`fastapi/applications.py`、`tests/test_serialize_response_model.py`、`tests/test_response_model_data_filter.py`
| 步骤 | 分配任务 | 主要验证点 |
| --- | --- | --- |
| STEP-01 | 界定问题和不可变约束 | 建立目标、输入输出、不变量和 Azure 资源边界 |
| STEP-02 | 建立依赖图和状态模型 | 承接 STEP-01，给出任务依赖图、共享状态、租约、状态转移 |
| STEP-03 | 设计连续调度算法 | 承接 STEP-02，设计调度、信息素更新、收敛算法和复杂度 |
| STEP-04 | 构造反例和失败场景 | 承接 STEP-03，构造慢模型、重复任务、状态倒退、分数误导 |
| STEP-05 | 修正算法和恢复策略 | 承接 STEP-04，加入幂等、重试、死信、outbox、重连恢复 |
| STEP-06 | 落到文件级实现计划 | 承接 STEP-05，引用真实文件和测试文件 |
| STEP-07 | 最终收敛和验收判定 | 承接 STEP-06，给出命令、指标、失败判定和上线前结论 |
| STEP-01 | 界定问题和不可变约束 | 建立 FastAPI 外部代码审查目标、输入输出、不变量和禁止自测边界 |
| STEP-02 | 建立依赖图和状态模型 | 承接 STEP-01，给出路由、依赖注入、OpenAPI、响应序列化和测试文件依赖图 |
| STEP-03 | 定位跨文件风险路径 | 承接 STEP-02，定位 response_model、Depends、参数 metadata、jsonable_encoder 与 OpenAPI schema 的漂移风险 |
| STEP-04 | 构造反例和失败场景 | 承接 STEP-03，构造响应过滤、默认值、nullable、依赖参数和 schema 不一致反例 |
| STEP-05 | 修正算法和恢复策略 | 承接 STEP-04，给出模块修正策略、兼容性和 Starlette/Pydantic 交互边界 |
| STEP-06 | 落到文件级实现计划 | 承接 STEP-05，引用 fastapi/fastapi 真实源码文件和测试文件 |
| STEP-07 | 最终收敛和验收判定 | 承接 STEP-06，给出 FastAPI 仓库内可执行验收命令、指标和可合并结论 |
模型分配来自动态发现，不写死 `NEWAPI_MODEL`：
@@ -122,11 +126,13 @@
## 7. 本轮判定
最新标准矩阵 run（S01-S08 全部通过，以下为 S07 live 证据）：
最新标准矩阵 run（S01-S08 全部通过，以下为 S07 外部 GitHub live 证据）：
```json
{
"run_id": "04c641d170fe4ea7aa3d882d9df37cca",
"run_id": "7ba01d0cc4ad402793938835654fbca5",
"target_repo": "fastapi/fastapi",
"target_commit": "ecace740f3eaccb1aba152cf1de79477095c56f4",
"completed_tasks": 7,
"accepted_score": 1.0,
"check_count": 12,
docs/INDUSTRY_STANDARD_AGNET_TEST_PLAN.md
+4 -2
View File
@@ -28,11 +28,13 @@
3. 模型输入输出能被审计：知道每个 Agent 被分配了什么任务、收到了什么输入、输出了什么、下一个 Agent 如何接手。
4. live 测试真实连接 Azure PostgreSQL、Redis、Blob 和 NewAPI，且不输出任何真实密钥。
最新一次标准矩阵（S01-S08）已经通过，以下为 S07 live 证据：
最新一次标准矩阵（S01-S08）已经通过，以下为 S07 外部 GitHub live 证据：
```json
{
"run_id": "04c641d170fe4ea7aa3d882d9df37cca",
"run_id": "7ba01d0cc4ad402793938835654fbca5",
"target_repo": "fastapi/fastapi",
"target_commit": "ecace740f3eaccb1aba152cf1de79477095c56f4",
"completed_tasks": 7,
"accepted_score": 1.0,
"check_count": 12,
docs/MODEL_AGNET_IO_REPORT.zh-CN.md
+422 -5235
View File
File diff suppressed because it is too large Load Diff
docs/STANDARD_TEST_MATRIX.md
+2 -2
View File
@@ -12,7 +12,7 @@ result means more than "one long run completed".
| Unit | Existing closed-loop and NewAPI mock behavior | `./.venv/bin/python -B -m unittest discover -s tests` |
| Deterministic scenarios | No-network Given/When/Then cases for continuity, policy, final scoring, and failure injection | `./.venv/bin/python -B -m unittest tests.test_standard_scenarios` |
| Academic Markov fit | No-network checks for Markov-style transition behavior and non-MDP limitations | `./.venv/bin/python -B -m unittest tests.test_markov_process_properties` |
| Live integration | Real NewAPI + Azure PostgreSQL/Redis/Blob continuous reasoning chain | `./.venv/bin/python -u -B examples/run_continuous_reasoning_acceptance.py` |
| Live integration | Real NewAPI + Azure PostgreSQL/Redis/Blob continuous reasoning chain against external GitHub code | `./.venv/bin/python -u -B examples/run_continuous_reasoning_acceptance.py` |
| Report audit | Human-auditable model I/O report with scenario, input, output, handoff and secret-safety evidence | `./.venv/bin/python -B -m unittest tests.test_model_io_report_audit` |
## Required Scenarios
@@ -25,7 +25,7 @@ result means more than "one long run completed".
| S04 | Dependency boundary | Text mentions NATS/Cosmos | Use negative and positive dependency wording | Negative wording passes; required dependency wording fails |
| S05 | Final convergence scoring | STEP-06 and STEP-07 both produce rich outputs | Score both outputs | STEP-07 is strictly selected as final convergence |
| S06 | Failure injection | One deterministic worker raises | Coordinator runs to convergence | Failed task is marked failed, pheromone is negative, completed task still converges |
| S07 | Live resource closure | Real Azure/NewAPI env is loaded | Run seven-step continuous reasoning | PostgreSQL, Redis, Blob, stream events, model discovery, and final artifact all pass |
| S07 | External GitHub code reasoning | Real Azure/NewAPI env is loaded and target is `fastapi/fastapi` at a pinned commit | Run seven-step code reasoning against external GitHub files | PostgreSQL, Redis, Blob, stream events, model discovery, external file references, and final artifact all pass |
| S08 | Model I/O report audit | Generated model/Agnet I/O report | Audit scenario coverage, task input/output, handoff evidence, and obvious secret patterns | Report is human-auditable and contains no obvious secret values |
| M01 | Markov-style claim transition | Same current tasks and pheromone values but different prior paths | Claim next task | The same high-pheromone task is claimed |
| M02 | Markov-style score transition | Same current task, agent, output, and score but different prior paths | Complete task | Task status, output, score, and pheromone update match |
examples/export_model_agnet_io_report.py
+34 -17
View File
@@ -13,15 +13,8 @@ from swarm_minimal.config import SwarmConfig
from swarm_minimal.local_env import load_project_env
RUN_IDS = [
"04c641d170fe4ea7aa3d882d9df37cca",
"c592a7ca4c0f4e02b93a1390b62d0af7",
"063632eeb17b45c197aa866066158667",
"3e8e58ae4e084bc8b90cf5c46f8992f3",
"78f189ccd1924ed0a4fb0a0a447ad449",
"623b5f6e5cc24cc7967fd9577f9c224b",
"42688e7b6245466dab8398dfe4790456",
]
LIVE_GOAL_PREFIX = "外部 GitHub 代码场景：审查 fastapi/fastapi"
RUN_IDS: list[str] = []
TEST_SCENARIOS = (
{
@@ -44,8 +37,8 @@ TEST_SCENARIOS = (
},
{
"id": "S07",
"name": "live_azure_newapi_continuous_reasoning",
"purpose": "用真实 Azure PostgreSQL、Redis、Blob 和 NewAPI 跑 7 步连续推理链。",
"name": "live_external_github_code_reasoning",
"purpose": "用真实 Azure PostgreSQL、Redis、Blob 和 NewAPI 对外部 GitHub 项目 fastapi/fastapi 跑 7 步代码推理链。",
"evidence": "本报告下方每个 live run 的 task.input / task.output / handoff 记录。",
},
{
@@ -81,7 +74,7 @@ def build_report(store: PostgresRedisBlobSwarmStore) -> str:
"",
"## 本轮测试场景补充",
"",
"本报告重点解释 S07 live 场景里的模型输入输出，但它属于完整标准矩阵的一部分；本轮重跑覆盖以下场景：",
"本报告重点解释 S07 外部 GitHub 代码场景里的模型输入输出，但它属于完整标准矩阵的一部分；本轮重跑覆盖以下场景：",
"",
"| ID | 场景 | 测试目的 | 证据入口 |",
"| --- | --- | --- | --- |",
@@ -93,11 +86,14 @@ def build_report(store: PostgresRedisBlobSwarmStore) -> str:
sections.extend(
[
"",
"最新重跑结论：S01-S08 全部 PASS；最新 S07 live run 会排在下方第一个。",
"最新重跑结论：S01-S08 全部 PASS；最新 S07 外部 GitHub live run 会排在下方第一个。",
"",
]
)
for run_id in RUN_IDS:
run_ids = fetch_latest_run_ids(store) or RUN_IDS
if not run_ids:
sections.extend(["## Run", "", "未找到外部 GitHub 代码场景的 live run。", ""])
for run_id in run_ids:
convergence = fetch_convergence(store, run_id)
if convergence is None:
sections.extend([f"## Run `{run_id}`", "", "未找到该 run。", ""])
@@ -184,7 +180,7 @@ def redact_sensitive_text(text: str) -> str:
def handoff_description(goal: str, run_id: str, task: dict[str, object], output: str) -> str:
if goal.startswith("连续性长推理场景"):
if is_live_code_goal(goal):
match = re.search(r"chain_edge=([^;\\n]+)", output)
edge = match.group(1).strip() if match else "<missing>"
marker = edge.split("->")[-1] if "->" in edge else str(task.get("kind") or "<unknown>")
@@ -226,7 +222,7 @@ def system_prompt_for_goal(goal: str) -> str:
"Return a concrete engineering answer for the assigned subtask. "
"Do not include secrets."
)
if goal.startswith("连续性长推理场景"):
if is_live_code_goal(goal):
return (
"You are one stage in a continuous long-reasoning swarm. "
"Carry forward prior conclusions, expose risks, and hand off a concise next-state. "
@@ -254,7 +250,7 @@ def user_prompt_shape_for_goal(goal: str) -> str:
"Shared state keys: <sorted(shared_state.keys())>",
]
)
if goal.startswith("连续性长推理场景"):
if is_live_code_goal(goal):
return "\n".join(
[
"Previous marker: <previous_marker>",
@@ -269,6 +265,27 @@ def user_prompt_shape_for_goal(goal: str) -> str:
return "<unknown user prompt shape>"
def is_live_code_goal(goal: str) -> bool:
return goal.startswith(LIVE_GOAL_PREFIX) or goal.startswith("连续性长推理场景")
def fetch_latest_run_ids(store: PostgresRedisBlobSwarmStore, *, limit: int = 1) -> list[str]:
def operation(cur):
cur.execute(
"""
select run_id
from swarm_convergence
where goal like %s
order by created_at desc
limit %s
""",
(LIVE_GOAL_PREFIX + "%", limit),
)
return [row[0] for row in cur.fetchall()]
return store._run_pg(operation)
def infer_model(task: dict[str, object], output: str) -> str:
for pattern in [r"used_model=([^;\\n]+)", r"primary_model=([^;\\n]+)", r"model=([^;\\n]+)"]:
match = re.search(pattern, output)
examples/run_academic_standard_evaluation.py
+3 -3
View File
@@ -72,10 +72,10 @@ def main() -> None:
"status": "PASS" if all(item["passed"] for item in results) else "FAIL",
"scope": {
"deterministic_local": True,
"live_azure_newapi": "ready" if live_ready else "blocked_missing_.env",
"live_external_github_code": "ready" if live_ready else "blocked_missing_.env",
"env_path": str(env_path.relative_to(ROOT)) if env_path else None,
"live_note": (
"S07 live integration requires Azure PostgreSQL, Redis, Blob and NewAPI credentials in an ignored .env."
"S07 external GitHub code reasoning requires Azure PostgreSQL, Redis, Blob and NewAPI credentials in an ignored .env."
),
},
"standards": ACADEMIC_STANDARD_SOURCES,
@@ -91,7 +91,7 @@ def main() -> None:
},
"pass_condition": {
"local_academic_gate": "all A01-A05 checks pass",
"full_standard_gate": "local_academic_gate plus S07 live Azure/NewAPI scenario and S08 model I/O report audit",
"full_standard_gate": "local_academic_gate plus S07 live external GitHub code scenario and S08 model I/O report audit",
},
}
print(json.dumps(report, ensure_ascii=False, indent=2))
examples/run_continuous_reasoning_acceptance.py
+76 -27
View File
@@ -19,7 +19,22 @@ from swarm_minimal.newapi_agnet import (
)
EXTERNAL_REPO = "fastapi/fastapi"
EXTERNAL_REPO_URL = "https://github.com/fastapi/fastapi"
EXTERNAL_REPO_COMMIT = "ecace740f3eaccb1aba152cf1de79477095c56f4"
TARGET_FILES = [
"fastapi/routing.py",
"fastapi/dependencies/utils.py",
"fastapi/openapi/utils.py",
"fastapi/params.py",
"fastapi/encoders.py",
"fastapi/applications.py",
"tests/test_serialize_response_model.py",
"tests/test_response_model_data_filter.py",
]
LOCAL_PROJECT_TARGETS = [
"swarm_minimal/core.py",
"swarm_minimal/newapi_agnet.py",
"swarm_minimal/azure_store.py",
@@ -27,16 +42,18 @@ TARGET_FILES = [
"examples/run_long_task_acceptance.py",
"tests/test_newapi_agnet.py",
"tests/test_minimal_swarm.py",
"README.md",
]
SCENARIO = {
"title": "连续性长推理场景：为 swarm-minimal 设计可恢复的大规模代码任务推理链",
"title": "外部 GitHub 代码场景：审查 fastapi/fastapi 响应序列化与 OpenAPI 依赖链",
"repository": EXTERNAL_REPO_URL,
"commit": EXTERNAL_REPO_COMMIT,
"description": (
"同一复杂工程问题必须被连续推理，而不是拆开独立回答。"
"每个 Agnet 接住前一步的结论、约束和风险，继续推进到下一步，"
"最终形成一个能落到代码、Azure 资源和验收命令上的闭环方案。"
f"目标项目固定为 GitHub `{EXTERNAL_REPO}` at `{EXTERNAL_REPO_COMMIT}`。"
"同一复杂外部代码问题必须被连续推理，而不是拆开独立回答。"
"每个 Agnet 接住前一步的结论、约束和风险，围绕 FastAPI 的路由、依赖注入、"
"OpenAPI 生成、响应模型序列化和测试覆盖继续推进，最终形成外部仓库级修复方案。"
),
}
@@ -46,48 +63,49 @@ CHAIN_STEPS = [
"capability": "chain_step_01",
"marker": "STEP-01",
"title": "界定问题和不可变约束",
"ask": "定义复杂代码任务连续推理的目标、输入输出、不变量和 Azure 资源边界。",
"ask": "定义 FastAPI 外部代码审查任务的目标、输入输出、不变量、仓库边界和禁止自测边界。",
},
{
"capability": "chain_step_02",
"marker": "STEP-02",
"title": "建立依赖图和状态模型",
"ask": "基于 STEP-01 建立任务依赖图、共享状态字段、租约和状态转移模型。",
"ask": "基于 STEP-01 建立 FastAPI 路由、依赖注入、OpenAPI、响应序列化和测试文件之间的依赖图。",
},
{
"capability": "chain_step_03",
"marker": "STEP-03",
"title": "设计连续调度算法",
"ask": "基于 STEP-02 设计上千任务下的连续调度、信息素更新和收敛算法，给复杂度。",
"title": "定位跨文件风险路径",
"ask": "基于 STEP-02 定位 response_model、Depends、参数 metadata、jsonable_encoder 与 OpenAPI schema 之间可能漂移的风险路径，给复杂度。",
},
{
"capability": "chain_step_04",
"marker": "STEP-04",
"title": "构造反例和失败场景",
"ask": "基于 STEP-03 构造会破坏连续推理的反例：慢模型、重复任务、状态倒退、分数误导。",
"ask": "基于 STEP-03 构造 FastAPI 外部仓库中的反例：响应过滤、默认值、nullable、依赖参数和 OpenAPI schema 不一致。",
},
{
"capability": "chain_step_05",
"marker": "STEP-05",
"title": "修正算法和恢复策略",
"ask": "基于 STEP-04 修正算法，加入幂等、重试、死信、outbox、Redis/PG 重连恢复。",
"ask": "基于 STEP-04 给出修正策略：应该改哪些 FastAPI 模块、如何保持兼容、如何避免破坏 Starlette/Pydantic 交互。",
},
{
"capability": "chain_step_06",
"marker": "STEP-06",
"title": "落到文件级实现计划",
"ask": "基于 STEP-05 给出文件级代码改造计划，必须引用目标文件和测试文件。",
"ask": "基于 STEP-05 给出 fastapi/fastapi 文件级补丁计划，必须引用目标源码文件和测试文件。",
},
{
"capability": "chain_step_07",
"marker": "STEP-07",
"title": "最终收敛和验收判定",
"ask": "基于 STEP-06 给出最终可执行验收命令、指标、失败判定和上线前结论。",
"ask": "基于 STEP-06 给出 fastapi/fastapi 最终可执行验收命令、指标、失败判定和可合并结论。",
},
]
ACCEPTANCE_CRITERIA = [
f"测试目标必须是外部 GitHub 项目 `{EXTERNAL_REPO}`，commit `{EXTERNAL_REPO_COMMIT}`，不能把当前仓库当成被测代码。",
"自动发现至少 3 个模型，并使用 3 个互不相同的模型参与连续推理。",
"7 个连续推理步骤必须全部完成，且状态写入 PostgreSQL task pool。",
"每一步输出必须引用自己的 STEP 标记；除 STEP-01 外必须引用前一步 STEP 标记。",
@@ -95,10 +113,9 @@ ACCEPTANCE_CRITERIA = [
"PostgreSQL 和 Redis pheromone score 必须都有正分。",
"最终收敛必须写入 PostgreSQL，并存在 Blob artifact。",
"Redis Stream 必须新增至少 3*N+1 条事件。",
"合并输出必须体现不变量、依赖图、复杂度、反例、修正、文件级计划和验收命令。",
"最终输出必须引用至少 5 个真实文件。",
"合并输出必须体现 FastAPI 外部代码的不变量、依赖图、复杂度、反例、修正、文件级计划和验收命令。",
"最终输出必须引用至少 5 个 fastapi/fastapi 真实文件。",
"流程必须依赖模型发现，不能写死 NEWAPI_MODEL。",
"NATS 或 Cosmos 不能作为 MVP 必需依赖。",
]
@@ -212,10 +229,12 @@ def build_step_prompt(step: dict[str, str], index: int, model: str) -> str:
output_rules = [
f"- 必须包含 `{step['marker']}`。",
f"- {'必须说明从 START 建立初始约束。' if index == 0 else f'必须明确写出“基于 {previous_marker}”。'}",
f"- 必须写出目标仓库 `{EXTERNAL_REPO}` 和 commit `{EXTERNAL_REPO_COMMIT}`。",
"- 必须输出：不变量、当前决策、风险/反例、下一步交接摘要。",
"- 必须围绕 FastAPI 的 response_model、依赖注入、OpenAPI 或响应序列化，不要泛化成调度系统。",
"- 中文，控制在 750 字以内，不要泛泛而谈。",
"- 必须说明模型来自发现流程，不能写死 NEWAPI_MODEL。",
"- 不要把 NATS 或 Cosmos 作为 MVP 必需依赖。",
"- 被测代码只能来自 fastapi/fastapi，不能把本仓库源码当成测试对象。",
"- 不要包含任何真实密钥。",
]
if step["marker"] == "STEP-07":
@@ -224,12 +243,14 @@ def build_step_prompt(step: dict[str, str], index: int, model: str) -> str:
return "\n".join(
[
f"{SCENARIO['title']}\n",
f"目标 GitHub 仓库：{EXTERNAL_REPO_URL}",
f"固定 commit：{EXTERNAL_REPO_COMMIT}",
f"总目标：{SCENARIO['description']}",
f"当前步骤：{step['marker']} {step['title']}",
f"必须承接：{previous_marker}",
f"当前模型：{model}\n",
f"步骤要求：{step['ask']}\n",
"目标文件：",
"外部目标文件：",
"\n".join(f"- {item}" for item in TARGET_FILES),
"\n验收标准：",
"\n".join(f"- {item}" for item in ACCEPTANCE_CRITERIA),
@@ -292,7 +313,8 @@ def chat_with_fallback(
content = agnet.chat(
system_prompt=(
"You are one stage in a continuous long-reasoning swarm. "
"Carry forward prior conclusions, expose risks, and hand off a concise next-state. "
"Carry forward prior conclusions for an external GitHub code review, expose risks, and hand off a concise next-state. "
"The code target is fastapi/fastapi, not the local harness repository. "
"Do not reveal secrets."
),
user_prompt=(
@@ -307,6 +329,7 @@ def chat_with_fallback(
continue
prefix = (
f"chain_edge={previous_marker}->{step['marker']}; "
f"target_repo={EXTERNAL_REPO}; target_commit={EXTERNAL_REPO_COMMIT}; "
f"primary_model={primary_model}; used_model={candidate}; "
"model_selection=discovered_models_not_NEWAPI_MODEL"
)
@@ -330,12 +353,15 @@ def score_output(content: str, index: int) -> float:
checks = [
marker in content,
previous_marker in content,
EXTERNAL_REPO in content,
EXTERNAL_REPO_COMMIT in content,
does_not_target_local_project(content),
"不变量" in content,
"风险" in content or "反例" in content,
"下一步" in content or "交接" in content,
"NEWAPI_MODEL" in content,
"模型发现" in content or "discover" in lowered,
no_required_nats_or_cosmos(content),
contains_external_code_terms(content),
]
if index >= 2:
checks.append("o(" in lowered or "复杂度" in content)
@@ -436,13 +462,14 @@ def collect_report(
"evidence": {"before": stream_before, "after": stream_after, "delta": stream_after - stream_before, "expected_min": expected_event_delta},
},
{
"name": "contains_continuous_reasoning_material",
"name": "contains_external_fastapi_code_review_material",
"passed": all(term in merged_output for term in ["不变量", "反例", "修正", "验收"])
and ("复杂度" in merged_output or "O(" in merged_output),
"evidence": "requires invariant, counterexample, revision, complexity and acceptance",
and ("复杂度" in merged_output or "O(" in merged_output)
and contains_external_code_terms(merged_output),
"evidence": "requires FastAPI code-review material plus invariant, counterexample, revision, complexity and acceptance",
},
{
"name": "final_output_references_real_files",
"name": "final_output_references_external_files",
"passed": count_referenced_files(result.accepted_output) >= 5,
"evidence": referenced_files(result.accepted_output),
},
@@ -454,9 +481,16 @@ def collect_report(
"evidence": "must reject fixed NEWAPI_MODEL",
},
{
"name": "no_required_nats_or_cosmos",
"passed": no_required_nats_or_cosmos(merged_output),
"evidence": "NATS/Cosmos may only appear as rejected dependencies",
"name": "external_github_target_not_local_project",
"passed": EXTERNAL_REPO in merged_output
and EXTERNAL_REPO_COMMIT in merged_output
and does_not_target_local_project(merged_output)
and count_referenced_files(merged_output) >= 5,
"evidence": {
"repo": EXTERNAL_REPO,
"commit": EXTERNAL_REPO_COMMIT,
"referenced_external_files": referenced_files(merged_output),
},
},
]
status = "PASS" if all(check["passed"] for check in checks) else "FAIL"
@@ -510,6 +544,21 @@ def count_referenced_files(text: str) -> int:
return len(referenced_files(text))
def contains_external_code_terms(text: str) -> bool:
lowered = text.lower()
required_groups = [
["fastapi", EXTERNAL_REPO.lower()],
["response_model", "响应模型", "响应序列化"],
["openapi", "schema"],
["depend", "依赖注入", "depends"],
]
return all(any(term.lower() in lowered for term in group) for group in required_groups)
def does_not_target_local_project(text: str) -> bool:
return not any(path in text for path in LOCAL_PROJECT_TARGETS)
def no_required_nats_or_cosmos(text: str) -> bool:
lowered = text.lower()
if "nats" not in lowered and "cosmos" not in lowered:
examples/run_standard_scenario_acceptance.py
+3 -3
View File
@@ -37,11 +37,11 @@ SCENARIOS = [
},
{
"id": "S07",
"name": "live_azure_newapi_continuous_reasoning",
"name": "live_external_github_code_reasoning",
"layer": "live-integration",
"given": "local .env with Azure PostgreSQL, Redis, Blob, and NewAPI credentials",
"when": "run seven-step continuous reasoning acceptance",
"then": "model discovery, PostgreSQL, Redis, Blob artifact, chain cursor, and convergence all pass",
"when": "run seven-step continuous reasoning acceptance against fastapi/fastapi at a pinned GitHub commit",
"then": "model discovery, external GitHub code targeting, PostgreSQL, Redis, Blob artifact, chain cursor, and convergence all pass",
"command": [sys.executable, "-u", "-B", "examples/run_continuous_reasoning_acceptance.py"],
"parse_json": True,
},
tests/test_model_io_report_audit.py
+11 -1
View File
@@ -16,14 +16,24 @@ class ModelIoReportAuditTests(unittest.TestCase):
"| S01 | syntax_import_sanity |",
"| S02 | unit_regression |",
"| S03-S06 | deterministic_standard_scenarios |",
"| S07 | live_azure_newapi_continuous_reasoning |",
"| S07 | live_external_github_code_reasoning |",
"| S08 | model_io_report_audit |",
"fastapi/fastapi",
"ecace740f3eaccb1aba152cf1de79477095c56f4",
"fastapi/routing.py",
"#### 本次任务输入 task.input",
"#### Agnet / 模型实际输出 task.output",
"#### 接手 / 交接机制",
"chain_edge=",
]:
self.assertIn(required, text)
for forbidden in [
"为 swarm-minimal 设计可恢复的大规模代码任务推理链",
"swarm_minimal/core.py",
"swarm_minimal/newapi_agnet.py",
"swarm_minimal/azure_store.py",
]:
self.assertNotIn(forbidden, text)
def test_report_does_not_contain_obvious_secret_values(self) -> None:
text = REPORT.read_text(encoding="utf-8")
tests/test_standard_scenarios.py
+6
View File
@@ -25,6 +25,12 @@ class StandardScenarioTest(unittest.TestCase):
self.assertFalse(continuous.no_required_nats_or_cosmos("MVP 必须依赖 NATS 才能完成任务队列。"))
self.assertFalse(continuous.no_required_nats_or_cosmos("需要引入 Cosmos 作为任务状态库。"))
def test_live_code_scenario_targets_external_github_repo(self) -> None:
self.assertEqual(continuous.EXTERNAL_REPO, "fastapi/fastapi")
self.assertTrue(continuous.EXTERNAL_REPO_URL.startswith("https://github.com/"))
self.assertTrue(all(path.startswith(("fastapi/", "tests/")) for path in continuous.TARGET_FILES))
self.assertFalse(any(path.startswith(("swarm_minimal/", "examples/")) for path in continuous.TARGET_FILES))
def test_final_step_scores_above_intermediate_step(self) -> None:
final = (
"STEP-07 基于 STEP-06 不变量 风险 下一步 NEWAPI_MODEL 模型发现 验收 "