Files
fengqun/STANDARD_TEST_MATRIX.md
T
gongzhiyong 111be3e435 Promote the minimal swarm prototype to the repository root
The standalone prototype should be the root-level project shape for fengqun while preserving the existing planning documents already at the root. This keeps README, examples, tests, and the Python package directly discoverable without deleting the prior docs.

Constraint: User clarified that swarm-minimal is the repository root, but other existing root files must remain.
Rejected: Deleting existing root docs | They are part of the fengqun repository context and were explicitly protected.
Confidence: high
Scope-risk: narrow
Directive: Keep secrets in ignored .env only; do not commit live credentials.
Tested: python3 -B -m unittest discover -s tests; git diff --check; secret-pattern scan showed only placeholders/test values/task-id false positives.
Not-tested: Remote web UI rendering after push.
2026-05-16 13:32:11 +08:00

2.2 KiB

Standard Test Matrix

This file defines the stricter acceptance shape for the standalone minimal swarm. It separates deterministic checks from live Azure/NewAPI checks so a passing result means more than "one long run completed".

Layers

Layer Purpose Command
Static Python syntax and importability ./.venv/bin/python -B -m py_compile swarm_minimal/*.py examples/*.py tests/*.py
Unit Existing closed-loop and NewAPI mock behavior ./.venv/bin/python -B -m unittest discover -s tests
Deterministic scenarios No-network Given/When/Then cases for continuity, policy, final scoring, and failure injection ./.venv/bin/python -B -m unittest tests.test_standard_scenarios
Live integration Real NewAPI + Azure PostgreSQL/Redis/Blob continuous reasoning chain ./.venv/bin/python -u -B examples/run_continuous_reasoning_acceptance.py

Required Scenarios

ID Scenario Given When Then
S01 Syntax/import sanity All prototype Python files Compile every module No syntax/import compile errors
S02 Unit regression Mock NewAPI and in-memory store Run all unit tests All tests pass
S03 Chain continuity Seven deterministic step outputs Remove one previous-step link Continuity checker fails the broken chain
S04 Dependency boundary Text mentions NATS/Cosmos Use negative and positive dependency wording Negative wording passes; required dependency wording fails
S05 Final convergence scoring STEP-06 and STEP-07 both produce rich outputs Score both outputs STEP-07 is strictly selected as final convergence
S06 Failure injection One deterministic worker raises Coordinator runs to convergence Failed task is marked failed, pheromone is negative, completed task still converges
S07 Live resource closure Real Azure/NewAPI env is loaded Run seven-step continuous reasoning PostgreSQL, Redis, Blob, stream events, model discovery, and final artifact all pass

Stop Rule

The standard acceptance result is PASS only if every scenario passes. Any FAIL must include the scenario ID, the failed assertion, and enough evidence to reproduce it without printing secrets.