Generate and interpret a pytest report¶

Generate the report¶

From the product repository root, run the project's pytest suite with its usual environment:

uv run pytest --junitxml=report.xml

Keep pytest's normal JUnit names; do not add --junit-prefix. Both pytest XML families are supported. The report may be Git-ignored; an explicit report path is still read.

Read the results, including after pytest fails¶

Run this as a separate command so a failing pytest exit code does not skip it:

qmstrace results report.xml

For another directory, use qmstrace -C path/to/product results report.xml. The report path is relative to that directory. Repository discovery does not move the report path.

Several reports — say, one pytest run per test level — are read as one: qmstrace results unit.xml integration.xml. A test reported in both keeps its worst verdict, and the header lists each report with its entry count.

The miniature service illustrates a partial report with failed and missing results:

report: report.xml
  6 XML entries · 5 reported runs (distinct full identities)
  matched: 4 runs of 3 automated tests · scanned: 4 automated tests · unmatched: 1 run identities
  matched against the current working tree — report commit NOT VERIFIED
  a partial parameter set does not prove every possible variant ran
  baselines: judged from reachable local tags — remote-tag completeness NOT VERIFIED; qmstrace never fetches
  baselines: no matching reachable local tag — nothing frozen locally
  change requests: NOT VERIFIED — ../ledger/change-requests is unavailable; fresh citations are verified at the ledger's import
  1 duplicate XML entries combined by full identity; retry attempts are not inferred

requirements — direct verifies links only:
  REQ-SVC-01  pass      1 automated tests · 2 runs — A call carries a key · requirement · docs/req/service-srs.md:12
  REQ-SVC-02  fail      2 automated tests · 1 runs — Records survive a restart · requirement · docs/req/service-srs.md:21
    fail      tests/integration/test_store.py::test_a_record_survives_a_restart · 1 runs · shared: also verifies DSN-SVC-02
      fail: test_a_record_survives_a_restart (error, failure)
        error: teardown could not close store
        failure: record missing after restart
    no-result tests/integration/test_store.py::test_a_restart_loses_no_accepted_record · 0 runs

inspection / analysis — no verification or approval is established by this report:
  REQ-SVC-03 — The record format is written down · Verify-by: inspection · requirement · docs/req/service-srs.md:28
    no linked automated tests

tests without a direct requirement target — no requirement verdict inferred:
    skip      tests/unit/test_keys.py::TestTheKeyCheck::test_it_runs_before_any_handler · 1 runs · verifies DSN-SVC-01
      skip: test_it_runs_before_any_handler (skipped)
        skipped: handler fixture unavailable

unmatched report entries:
  pass      ('tests.removed.test_old', 'test_old') · 1 XML entries · no scanned automated test matches this identity
  Fix: check report provenance and scan patterns; use default pytest JUnit names.

result: 3 matched automated tests · 4 matched runs · 1 unmatched run identities · exit 0 (informational) — report commit NOT VERIFIED; this repo's tests only

Follow up on incomplete evidence¶

1. Check the matched automated-test and run counts. A parameterized function counts as one automated test and can contribute several runs. Repeated XML entries are combined, not counted as retry attempts.
2. Inspect fail and skip details. Shared tests name other affected items. Where the project keeps a Test Specification, a requirement's row names the test cases its verdict came through, each with its own verdict. A test or a test case that reaches only a design element does not supply a requirement verdict; it is listed apart, so its failure still shows.
3. For no-result, run the missing test or explain why this report selected only part of the suite. For untested, add the intended test link — on the automated test, or on the test case it runs — or review the requirement's verification method. A few reported parameter variants do not establish a complete set.
4. Resolve unmatched entries by checking report provenance, scan patterns and default pytest names. Zero matched tests is an error; a partly matched report still shows recognized results.
5. Review inspection and analysis requirements through their own verification process. Linked test results do not approve them.

The output always states that the report's commit is not verified. Generate a fresh report from the intended working tree and retain the run's provenance through the project's evidence process. This command interprets one file; it does not certify freshness or store results.

For complete matching, aggregation and exit semantics, see results reference.