pytest as the primary runner #

Litmus is a hardware test platform; the bundled pytest plugin is its primary runner integration. OpenHTF and the LabVIEW / TestStand results API are the alternatives — see Integrations. Tests under the pytest path are plain pytest — no decorator, no base class. The plugin contributes a set of fixtures (context, verify, measure are the three a test hits every time), seven markers, and a sidecar YAML; everything else is stock pytest.

The choice carries the rest of the pytest ecosystem with it. The sections below cover what pytest already gives you, what Litmus adds on top, and why that split helps when an AI assistant writes the tests.

Shape of a Litmus test #

class TestPowerBoard:
    def test_voltage(self, context, dmm, verify):
        verify("output_voltage", dmm.measure_dc_voltage())

Plain pytest collection — no proprietary IDE, no test DSL. Runs with the pytest command, shows up in the IDE test explorer, works alongside every pytest plugin.

What stock pytest provides #

  • Test discovery and selection — pytest -k, -m, node IDs, --lf/--ff for last-failed / failures-first
  • Markers for classification — @pytest.mark.smoke, @pytest.mark.slow, etc.; Litmus adds hardware-specific markers on top
  • Fixtures — yield-based setup/teardown, scope resolution, automatic composition
  • Parametrize — @pytest.mark.parametrize first-class (context.get_param(...) works on it directly)
  • Rich failure output — assertion rewriting, stack traces, --tb control
  • Plugin ecosystem — pytest-xdist (parallel), pytest-timeout, pytest-rerunfailures (retries), pytest-dependency, pytest-html
  • IDE integration — click-to-run, breakpoints, inline results in PyCharm / VS Code
  • CI/CD — pytest --junitxml=..., exit codes, pytest-html

What Litmus adds on top #

ConcernLitmus addition
Measurement/event persistencemeasure(name, v, ...) → parquet, traceable
Part-spec-driven limits + pinsverify(name, v) resolves from part YAML
Vector parameters + change detectcontext.get_param(k), context.changed(k)
Operator-editable sweepsSidecar test_<module>.yaml sweeps: overrides
Instrument role fixturesStation config → dmm, psu, scope auto-fixtures
Mock mode--mock-instruments, sidecar mocks:, pytest-mock
Session flags--station, --part, --operator, --uut-serial, --test-phase
Per-test-imposed limits@pytest.mark.litmus_limits(name={...})

Retries and explicit test dependencies are ecosystem plugins, not Litmus additions — use @pytest.mark.flaky(reruns=N) (pytest-rerunfailures) and @pytest.mark.dependency(depends=[...]) (pytest-dependency).

Implication for AI-assisted authoring #

AI assistants already know pytest well, so an assistant only has to learn Litmus's added vocabulary on top — the fixtures (most often context, verify, measure, pins, instruments, plus the per-instrument-role fixtures from the active station) and the seven markers (litmus_limits, litmus_sweeps, litmus_mocks, litmus_characteristics, litmus_connections, litmus_retry, litmus_prompts). A custom test runner would have to be taught from scratch.

See also #