pytest Plugin

TraceCov ships a pytest plugin that prints a coverage summary after your test run and can save HTML and Markdown reports.

Activation

Activation takes two fixtures in your conftest.py: tracecov_schema supplies the OpenAPI schema, and your HTTP client fixture sends its requests through the tracecov_map fixture.

DjangoDjango REST Frameworkrequestshttpx

Requires pytest-django, which provides the client fixture.

# conftest.py
import json
from pathlib import Path

import pytest


@pytest.fixture(scope="session")
def tracecov_schema():
    return json.loads(Path("openapi.json").read_text())


@pytest.fixture
def client(client, tracecov_map):
    if tracecov_map is not None:
        tracecov_map.django.track_client(client)
    return client

# conftest.py
import json
from pathlib import Path

import pytest
from rest_framework.test import APIClient


@pytest.fixture(scope="session")
def tracecov_schema():
    return json.loads(Path("openapi.json").read_text())


@pytest.fixture
def api_client(tracecov_map):
    client = APIClient()
    if tracecov_map is not None:
        tracecov_map.django.track_client(client)
    return client

# conftest.py
import json
from pathlib import Path

import pytest
import requests


@pytest.fixture(scope="session")
def tracecov_schema():
    return json.loads(Path("openapi.json").read_text())


@pytest.fixture(scope="session")
def api_client(tracecov_map):
    session = requests.Session()
    if tracecov_map is not None:
        tracecov_map.requests.track_session(session)
    return session

# conftest.py
import json
from pathlib import Path

import httpx
import pytest


@pytest.fixture(scope="session")
def tracecov_schema():
    return json.loads(Path("openapi.json").read_text())


@pytest.fixture(scope="session")
def api_client(tracecov_map):
    client = httpx.Client()
    if tracecov_map is not None:
        tracecov_map.httpx.track_client(client)
    return client

Tests that use the wrapped client are recorded. Requests sent through any other client are not.

Run pytest as usual. After the tests, TraceCov prints a summary:

=============================== tracecov summary ===============================
Operation                                    Params   Missed   Partial     Cover
--------------------------------------------------------------------------------
GET /users                                        1        0         1     0.00%
POST /users                                       1        1         0     0.00%
GET /users/{id}                                   1        0         0   100.00%
--------------------------------------------------------------------------------
TOTAL                                             3        1         1    33.33%

If tracecov_schema is not overridden (returns None), the plugin stays dormant and adds no output.

Fixture Hierarchy

The plugin exposes two overridable fixtures:

Fixture	Scope	Purpose
`tracecov_schema`	session	Supply the raw OpenAPI dict. Override this for the common case.
`tracecov_map`	session	Supply a fully-constructed `CoverageMap`. Override this when you need a non-dict source.

Override tracecov_map directly when you need capabilities beyond a plain dict — for example, loading from a file path or a live URL, or using base_path to prepend a path prefix the spec doesn't include, such as /api/v1. An overriding tracecov_map activates the plugin on its own, without tracecov_schema:

import pytest
import tracecov


@pytest.fixture(scope="session")
def tracecov_map():
    return tracecov.CoverageMap.from_url("https://api.example.com/openapi.json")

import pytest
import tracecov


@pytest.fixture(scope="session")
def tracecov_map():
    # All spec paths like /users become /api/v1/users
    return tracecov.CoverageMap.from_path("openapi.yaml", base_path="/api/v1")

Output Formats

By default TraceCov prints a text summary to the terminal. Use --tracecov-format to choose one or more formats:

# Text only (default)
$ pytest

# HTML report only
$ pytest --tracecov-format=html

# Markdown report only
$ pytest --tracecov-format=markdown

# JSON report only
$ pytest --tracecov-format=json

# Terminal summary + HTML report
$ pytest --tracecov-format=text,html

Default report paths

Format	Default path
HTML	`tracecov-report.html`
Markdown	`tracecov-report.md`
JSON	`tracecov-report.json`

Override with --tracecov-report-html-path, --tracecov-report-markdown-path or --tracecov-report-json-path.

CLI Options

Option	Type	Description
`--tracecov-format`	`text,html,markdown,json`	Comma-separated output formats (default: `text`)
`--tracecov-report-html-path`	path	Path for the HTML report
`--tracecov-report-markdown-path`	path	Path for the Markdown report
`--tracecov-report-json-path`	path	Path for the JSON report
`--tracecov-no-report`	flag	Suppress TraceCov report output (thresholds are still enforced)
`--tracecov-skip-covered`	flag	Hide fully-covered operations from the text report
`--tracecov-skip-empty`	flag	Hide operations with no recorded traffic
`--tracecov-show-missing`	`parameters`	List parameter names with no coverage recorded
`--tracecov-width`	int	Terminal width override (default: auto-detect)
`--tracecov-sub-text`	string	Subtitle shown after the terminal summary and in the Markdown footer
`--tracecov-markdown-report-url`	URL	Link to the full HTML report from the Markdown footer
`--tracecov-markdown-weak-threshold`	0–100	Flag operations below this parameter coverage percentage
`--tracecov-markdown-is-pull-request`	flag	Use pull-request framing in the Markdown headline
`--tracecov-markdown-columns`	`parameters,keywords,responses,examples`	Comma-separated columns to show in the Markdown report (default: all)
`--tracecov-fail-under`	0–100	Fail the session if any of the five dimensions below is under this percentage
`--tracecov-fail-under-operations`	0–100	Fail if operation coverage is below this percentage
`--tracecov-fail-under-parameters`	0–100	Fail if parameter coverage is below this percentage
`--tracecov-fail-under-keywords`	0–100	Fail if keyword coverage is below this percentage
`--tracecov-fail-under-examples`	0–100	Fail if example coverage is below this percentage
`--tracecov-fail-under-responses`	0–100	Fail if response coverage is below this percentage
`--tracecov-fail-under-label`	`LABEL:DIMENSION=N`	Fail if a label's coverage of `DIMENSION` is below `N` percent; repeatable. See Which Tests Cover What
`--tracecov-no-labels`	flag	Do not label coverage by test node id and `tracecov` markers

`pytest.ini` / `pyproject.toml` Configuration

All options are also available as [tool.pytest.ini_options] keys (underscores, no leading dashes):

[tool.pytest.ini_options]
tracecov_format = "text,html"
tracecov_report_html_path = "reports/coverage.html"
tracecov_skip_covered = true
tracecov_markdown_weak_threshold = "80"
tracecov_markdown_columns = "parameters,responses"
tracecov_sub_text = "Full report: https://ci.example.com/coverage"
tracecov_fail_under = "80"
tracecov_fail_under_operations = "100"

CLI options take precedence over ini values when both are set.

Threshold Enforcement

Use --tracecov-fail-under to fail the session when any coverage metric falls below a threshold:

$ pytest --tracecov-fail-under=80

Per-dimension flags override the global value. In the example below, operations must reach 100% but all other metrics only need 80%:

$ pytest --tracecov-fail-under=80 --tracecov-fail-under-operations=100

Dimensions with no items in the specification are skipped. Response keyword coverage has no threshold option.

When thresholds are violated, TraceCov prints a failure table and exits with a non-zero status even if all tests passed. File reports are still saved.

------------------------- tracecov threshold failures --------------------------
Operations       66.7% / 100% required
Parameters       33.3% / 80% required
Keywords         25.0% / 80% required
Responses        50.0% / 80% required