Skip to content

The chain

TL;DR

Everything qmstrace traces is one chain: an agreement clause, the requirements that derive from it, the design elements that satisfy them, the code that implements them, and the tests that verify them — directly, or through the test cases they run. Each link is written at its downstream end — the later item names the earlier one — so nothing is recorded twice. qmstrace thread walks the chain either way.

The chain

flowchart RL
  req["requirement"] -- Origin --> agr["origin clause<br/>(in the agreement)"]
  dsn["design element"] -- Satisfies --> req
  mod["component<br/>(a module)"] -- Implements --> dsn
  tc["test case<br/>(in a Test Specification)"] -- verifies --> req
  tc -- verifies --> dsn
  auto["automated test<br/>(a test function)"] -- verifies --> tc
  auto -. "or directly" .-> req

Each arrow is a line in something already written: a field under a requirement's heading, a line under a design section, a sentence in a module's docstring, a marker on a test.

Written at the downstream end

A link is declared by the item that comes later in the chain: the test names the requirement it verifies; the requirement names the clause it comes from. The later item is written with the earlier one in mind, so that is when the link is known — and there is one place to change it.

A table of links kept beside the documents was the alternative, and it drifts: it is a second copy of what the test or the design section already says.

So the backward trace — from a test to what it serves — is what people write. The forward trace — from a clause to everything that derives from it — is derived by the tool.

Many to many

A design element may satisfy several requirements, a requirement may need several design elements, and a test may verify a requirement and its design element at once. Nothing forces one to one, and the layout of the code should not bend towards it.

A walk therefore meets some items by two paths. thread shows each item once, under the link that reaches it first, and notes the other link on the same line:

AGR-3
  origin clause, in the agreement

backward — what it derives from:
  declared in the agreement, outside any product repo

forward — what hangs off it, nearest first:
  Origin      REQ-SVC-02 — Records survive a restart · requirement · docs/req/service-srs.md:21
    Satisfies   DSN-SVC-02 — Record store · design element · docs/design/service-design.md:13
      Implements  src/service/store.py · component
    verifies    tests/integration/test_store.py::test_a_record_survives_a_restart · automated test, integration · also verifies DSN-SVC-02
    verifies    tests/integration/test_store.py::test_a_restart_loses_no_accepted_record · automated test, integration
  Origin      REQ-SVC-03 — The record format is written down · requirement · docs/req/service-srs.md:28

forward: 2 requirements, 1 design element, 1 component, 2 automated tests · backward: nothing — this repo only

Test cases and automated tests

A test case is planned: a heading in a Test Specification, a Markdown file of test cases kept with the other documents, and a verifies: line naming what it checks. An automated test is the code that runs it — a pytest function — and names the test case it runs. Several automated tests may run one test case, which passes only when all of them pass. This is the fixture project's Test Specification:

# Service — test cases

The planned test cases for the service. `TC-SVC-NN.m` is test case *m* of design element
`DSN-SVC-NN`; what each verifies is the line under it. Several automated tests may run one
test case, which passes only when all of them pass.

## TC-SVC-01.1 — A call without a valid key is refused
verifies: REQ-SVC-01

A call with no key, and a call with a wrong one, are each answered 401.

## TC-SVC-01.2 — The key check runs before any handler
verifies: DSN-SVC-01

No handler sees a call the key check has refused.

## TC-SVC-02.1 — A restart keeps every field of a record
verifies: DSN-SVC-02

A record read after a restart equals the record written before it.

## TC-SVC-02.2 — A record written during a restart is not lost
verifies: REQ-SVC-02

Planned; no automated test runs it yet.

A project that keeps no Test Specification lets its automated tests verify requirements and design elements directly, as in the diagram's dotted arrow; both may sit in one project. Its schema says which links it allows, and its coverage rules say what must be met — here, that every test case is run:

read — each repo's working tree, not pinned:
  agent    ../agent    1 component · 2 automated tests
  ledger   .           2 requirements · 2 design elements · 4 test cases
  service  ../service  2 components · 5 automated tests
  external AGR-*: verified against agreement.md (3 declared)
  controlled document items: 0 frozen · 0 changed · 0 new · 0 removed · 0 withdrawn
  baselines: judged from each repo's reachable local tags — remote-tag completeness NOT VERIFIED; qmstrace never fetches
  baselines in ledger: no matching reachable local tag — nothing frozen
  change requests: change-requests (1 readable; 0 malformed) — working tree, not pinned

gaps — coverage not met yet; only the gate (qmstrace ledger check --strict) fails on them:
  G09  docs/test/service-test-cases.md:22  TC-SVC-02.2: no automated test runs it yet. Fix: write one of these — on the automated test in service, `@pytest.mark.verifies("TC-SVC-02.2")` on the test; on the automated test in agent, `[Fact(DisplayName = "TC-SVC-02.2 …")]` on the test method — only if it is true: the link states that the automated test verifies TC-SVC-02.2.

result: 0 violations, 1 gap in 18 items across 3 repos · 0 suspects (informational) · exit 1 · gate: gaps fail · working trees, not pinned · controlled document items: 0 frozen · 0 changed · 0 new · 0 removed · 0 withdrawn · remote-tag completeness NOT VERIFIED

Assumes: a dependency, not coverage

Sometimes an item relies on something another item provides — often in another repo: that registering twice is harmless, that the other side checks a pattern. Assumes: says so.

  • It counts toward no coverage rule. Nothing is covered by being assumed.
  • It carries suspicion forward. When the assumed item changes, the assuming one may break, so walking forward crosses an Assumes link and goes on past it.
  • Walking backward, it ends the walk. What the assumed item derives from is not what the assuming item derives from. Following it would name clauses that have nothing to do with the item you asked about.

Where the walk stops

At this repo's edge. A sibling's item is shown with its path and not pinned, but its own links — what it derives from, what hangs off it over there — are not read. For the same reason, a sibling item that assumes something here cannot be seen from here.

In the ledger, qmstrace ledger thread walks the whole chain in one place, across every repo of the project — from their working trees, not pinned. Reading each repo at a pin is the ledger's import, later. See Around the repo.