Skip to content

JSON Report Format

JSON reports are for CI integration, custom tooling, and programmatic analysis.

For simple threshold checks, use --fail-under flags instead (see CLI docs).

Structure Overview

{
  "$schema": "https://tracecov.sh/schemas/coverage-report.v1.json",
  "version": "1.0",
  "tracecov_version": "0.24.2",
  "schema_location": "openapi.json",
  "generated_at": "2026-09-26T15:57:01.365525827Z",
  "summary": {
    "operations": { "covered": 1, "total": 2, "percent": 50.0 },
    "parameters": { "covered": 0, "partial": 1, "total": 2, "percent": 0.0 },
    "keywords": { "covered": 1, "partial": 1, "total": 3, "percent": 33.333333333333336 },
    "examples": { "covered": 0, "total": 0, "percent": null },
    "responses": { "covered": 1, "total": 3, "percent": 33.333333333333336 },
    "response_keywords": { "reached": 5, "violated": 2, "total": 5, "percent": 100.0 }
  },
  "operations": [
    {
      "method": "GET",
      "path": "/pets",
      "operation_id": "listPets",
      "hits": 3,
      "coverage": {
        "parameters": { "covered": 0, "partial": 1, "total": 1, "percent": 0.0 },
        "keywords": { "covered": 1, "partial": 1, "total": 2, "percent": 50.0 },
        "examples": { "covered": 0, "total": 0, "percent": null },
        "responses": { "covered": 1, "total": 2, "percent": 50.0 },
        "response_keywords": { "reached": 5, "violated": 2, "total": 5, "percent": 100.0 }
      },
      "parameters": [
        {
          "location": "query",
          "name": "limit",
          "hits": 3,
          "coverage": { "covered": 1, "partial": 1, "total": 2, "percent": 50.0 },
          "keywords": [
            { "schema_path": "/maximum", "state": "partial", "valid": 2, "invalid": 0 }
          ]
        }
      ],
      "responses": [
        {
          "status": "200",
          "hits": 2,
          "avg_elapsed_ms": 40.0,
          "bodies": [
            {
              "media_type": "application/json",
              "hits": 2,
              "coverage": { "reached": 5, "violated": 2, "total": 5, "percent": 100.0 },
              "keywords": [
                { "schema_path": "/items/properties", "state": "mixed", "valid": 1, "invalid": 1 },
                { "schema_path": "/items/properties/name/maxLength", "state": "mixed", "valid": 1, "invalid": 1 }
              ]
            }
          ]
        },
        { "status": "400", "hits": 0 }
      ],
      "unexpected_responses": [
        { "status": "500", "hits": 1, "avg_elapsed_ms": 20.0 }
      ],
      "errors": []
    },
    {
      "method": "DELETE",
      "path": "/pets/{petId}",
      "hits": 0,
      "coverage": {
        "parameters": { "covered": 0, "partial": 0, "total": 1, "percent": 0.0 },
        "keywords": { "covered": 0, "partial": 0, "total": 1, "percent": 0.0 },
        "examples": { "covered": 0, "total": 0, "percent": null },
        "responses": { "covered": 0, "total": 1, "percent": 0.0 },
        "response_keywords": { "reached": 0, "violated": 0, "total": 0, "percent": null }
      },
      "parameters": [
        {
          "location": "path",
          "name": "petId",
          "hits": 0,
          "coverage": { "covered": 0, "partial": 0, "total": 1, "percent": 0.0 },
          "keywords": [
            { "schema_path": "/type", "state": "miss" }
          ]
        }
      ],
      "responses": [
        { "status": "204", "hits": 0 }
      ],
      "errors": []
    }
  ]
}

Key fields:

  • $schema - URL of the report's JSON Schema
  • version - report format version; tracecov_version - the TraceCov release that wrote it
  • schema_location - the specification path or URL, omitted when unknown
  • percent - null when total is 0
  • hits - number of requests recorded for the operation, parameter, response, or response body
  • operation_id - from the specification, omitted when not defined
  • warnings - omitted when empty; {"type": "shadowed", "by": "/pets/{id}"} marks an operation with no hits whose requests route to the conflicting path in by
  • location - parameter location: query, querystring, path, header, cookie, body, or form_data
  • Query/path/header/cookie parameters have name, body parameters have media_type
  • schema_path - JSON pointer to the schema keyword (/properties/id/type points to the type keyword of the id property)
  • valid/invalid - count of requests that passed/failed validation for this keyword; present on partial parameter keywords and on every response keyword
  • status - response status code as a string ("200", "2XX", "default")
  • avg_elapsed_ms - mean response time in milliseconds, omitted when the response has no hits
  • unexpected_responses - status codes received but not documented for the operation, omitted when empty
  • errors - error messages recorded against the operation, such as Schemathesis run errors; always present, [] when none
  • parameters, responses, bodies, and keywords arrays are omitted when empty

Diagnostics

Specification problems that affect coverage are listed in a top-level diagnostics array, omitted when there are none:

{
  "diagnostics": [
    {
      "severity": "warning",
      "kind": "unsatisfiable",
      "causes": [
        { "pointer": "", "keywords": ["type"] },
        { "pointer": "", "keywords": ["minLength", "maxLength"] }
      ],
      "at": {
        "path": "/pets/{petId}",
        "in": "parameter",
        "method": "DELETE",
        "location": "header",
        "name": "X-Code"
      }
    }
  ]
}
  • severity - error when coverage is missing and the specification needs a fix; warning when coverage is still recorded
  • kind - the problem, with kind-specific fields beside it (causes above)
  • at - the position: path, and in set to path_item, operation, parameter, or response_body with the method, location/name, or status/media_type that level needs

Unmatched Requests

Requests that hit no documented operation are aggregated into a top-level unmatched array, omitted entirely when every request matched:

{
  "unmatched": [
    { "method": "GET", "host": "api.example.com", "path": "/admin/debug", "reason": "no_matching_path", "count": 412, "first_seen_at": 1733308200.0 },
    { "method": "PATCH", "host": "api.example.com", "path": "/pets/{petId}", "reason": "method_not_documented", "count": 3, "first_seen_at": 1733308260.0 }
  ]
}
reason Meaning path holds
no_matching_path No documented path template matches the observed request path
method_not_documented A path matches, but the spec documents no such method on it the path template

Entries are grouped by (method, host, path) and sorted by count descending. host is the request authority including the port, so third-party traffic - OAuth providers, CDNs, telemetry - and separate local services stay distinguishable.

These requests never affect coverage percentages or --fail-under thresholds.

Two sibling counters cover what the array cannot, each omitted when zero:

  • unmatched_overflow - requests dropped after the 1000-distinct-group cap. Its presence means unmatched is truncated.
  • dropped_requests - requests discarded before matching was attempted: an unparsable URL (an unresolved {{baseUrl}} in a Postman collection, say) or an HTTP method TraceCov does not model. Non-zero here with an empty unmatched explains zero coverage.

Schemathesis undocumented-method probes never appear in unmatched.

Available programmatically as cov.unmatched_requests() in Python and cov.unmatchedRequests() in JavaScript, alongside unmatched_overflow() / dropped_requests(). Rendered in the HTML, Markdown, and text reports, which each show the 10 most frequent groups.

Observed paths are truncated to 120 characters. They are environment-controlled and reach PR comments and CI artifacts, so treat them as untrusted input.

Labels

With coverage labels recorded, a top-level labels object maps each label to its coverage, in the shape of summary. It is omitted when no label was recorded:

{
  "labels": {
    "suite:smoke": {
      "operations": { "covered": 1, "total": 2, "percent": 50.0 },
      "parameters": { "covered": 0, "partial": 1, "total": 1, "percent": 0.0 },
      "keywords": { "covered": 1, "partial": 1, "total": 2, "percent": 50.0 },
      "examples": { "covered": 0, "total": 0, "percent": null },
      "responses": { "covered": 1, "total": 2, "percent": 50.0 },
      "response_keywords": {
        "reached": 0, "violated": 0, "total": 0, "percent": null
      }
    }
  }
}

Key Concepts

Coverage Types

Binary coverage (operations, examples, responses): covered / total

Ternary coverage (parameters, keywords): tracks three states, but percent = covered / total * 100

Response coverage (response_keywords): reached / violated / total, with percent = reached / total * 100

  • covered - fully tested (both valid and invalid inputs)
  • partial - partially tested (only valid OR only invalid, doesn't count toward percent)

A keyword that can only pass, or only fail, counts as covered after one request on that side.

Keyword States

State Meaning
partial Only valid or only invalid tested (see valid/invalid counts)
miss Not tested at all
unsupported Cannot be tested; reason says why: {"schema_path": "/properties/content/contentEncoding", "state": "unsupported", "reason": "annotation"}

Fully covered keywords are counted in covered and never listed.

Response Keyword Coverage

response_keywords scores response bodies against their declared schema. percent is reached / total, where a keyword is reached once any response body exercised it; violated counts the keywords some response broke.

State Meaning
not_reached No response body exercised the keyword
conforming Every response satisfied it
violating Every response that exercised it broke it
mixed Both conforming and violating responses seen

The per-body keywords list carries violating and mixed entries only.

Common jq Recipes

# Get overall operation coverage percentage
tracecov report openapi.json traffic.json --format json | jq '.summary.operations.percent'

# List uncovered operations
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | select(.hits == 0) | "\(.method) \(.path)"'

# List operations below 80% keyword coverage
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | select(.coverage.keywords.percent != null and .coverage.keywords.percent < 80) | "\(.method) \(.path): \(.coverage.keywords.percent)%"'

# Find all untested keywords (state: miss)
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | . as $op | .parameters[]? | . as $param | .keywords[]? | select(.state == "miss") | "\($op.method) \($op.path) - \($param.name // $param.media_type)\(.schema_path)"'

# Find keywords needing invalid/negative tests (state: partial with valid > 0)
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | . as $op | .parameters[]? | . as $param | .keywords[]? | select(.state == "partial" and .valid > 0) | "\($op.method) \($op.path) - \($param.name // $param.media_type)\(.schema_path)"'

# Get uncovered response codes per operation
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | {op: "\(.method) \(.path)", missing: [.responses[]? | select(.hits == 0) | .status]} | select(.missing | length > 0) | "\(.op): \(.missing | join(", "))"'

# CI threshold check (exit 1 if below 80%)
tracecov report openapi.json traffic.json --format json | \
  jq -e '.summary.operations.percent >= 80' > /dev/null || exit 1

CI Example

Bash step for GitHub Actions with custom error reporting:

#!/bin/bash
set -e

REPORT=$(tracecov report openapi.json traffic.json --format json)

# Print coverage summary
echo "Operations: $(echo "$REPORT" | jq '.summary.operations.percent')%"
echo "Keywords: $(echo "$REPORT" | jq '.summary.keywords.percent')%"

# Fail if operations below 80%
if ! echo "$REPORT" | jq -e '.summary.operations.percent >= 80' > /dev/null; then
  echo "::error::Operation coverage below 80%"  # GitHub Actions annotation
  echo "$REPORT" | jq -r '.operations[] | select(.hits == 0) | "  - \(.method) \(.path)"'
  exit 1
fi