feat(benchmark): SC-10 P_rework 计入评审重开(#10)
修正 P_rework 已知低估:此前 ReworkCount 仅由 retry_count>0 派生,漏了 cross_review/queen 质量门的重开(reopen_task 不增 retry_count——是质量决策非失败)。 - cross_review + queen_quality_gate 重开时累计 run.metadata["rework_reopens"] - run_collector:rework_count = retry 派生 + rework_reopens 测试 test-benchmark-collector/metrics 通过。影响:仅 benchmark 度量(P_rework 更准),不改运行路径。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
2672a3d4b6
commit
28a6c618ab
@@ -140,7 +140,11 @@ class SwarmRunMetricsCollector(SwarmMetricsCollector):
|
||||
have_quality = isinstance(q_quality, (int, float)) and not math.isnan(q_quality)
|
||||
have_speed = isinstance(target_time, (int, float)) and target_time and duration > 0
|
||||
if have_quality and have_speed and self.coverage["s_cost"]:
|
||||
rework_count = sum(1 for t in tasks if getattr(t, "retry_count", 0) > 0)
|
||||
# P_rework counts BOTH transient retries AND quality-driven review/queen reopens
|
||||
# (reopen_task doesn't bump retry_count — it's a quality decision, not a failure;
|
||||
# SC-10 fixes the prior under-count). Reopen tally accumulated on run.metadata.
|
||||
rework_count = (sum(1 for t in tasks if getattr(t, "retry_count", 0) > 0)
|
||||
+ int((run.metadata or {}).get("rework_reopens", 0) or 0))
|
||||
risky = sum(1 for a in approvals
|
||||
if str(a.get("risk_level", "")).lower() in {"high", "critical"})
|
||||
p_risk = (100.0 * risky / len(approvals)) if approvals else 0.0
|
||||
|
||||
Reference in New Issue
Block a user