JSON Report Format

JSON reports are for CI integration, custom tooling, and programmatic analysis.

For simple threshold checks, use --fail-under flags instead (see CLI docs).

Structure Overview

{
  "version": "1.0",
  "schema_location": "openapi.json",
  "generated_at": "2025-12-04T10:30:00Z",
  "summary": {
    "operations": { "covered": 1, "total": 1, "percent": 100.0 },
    "parameters": { "covered": 0, "partial": 2, "total": 2, "percent": 0.0 },
    "keywords": { "covered": 1, "partial": 2, "total": 4, "percent": 25.0 },
    "examples": { "covered": 0, "total": 0, "percent": 0.0 },
    "responses": { "covered": 1, "total": 2, "percent": 50.0 }
  },
  "operations": [
    {
      "method": "GET",
      "path": "/pets",
      "operation_id": "listPets",
      "hits": 2,
      "coverage": {
        "parameters": { "covered": 0, "partial": 2, "total": 2, "percent": 0.0 },
        "keywords": { "covered": 1, "partial": 2, "total": 4, "percent": 25.0 },
        "examples": { "covered": 0, "total": 0, "percent": 0.0 },
        "responses": { "covered": 1, "total": 2, "percent": 50.0 }
      },
      "parameters": [
        {
          "location": "query",
          "name": "limit",
          "hits": 2,
          "coverage": { "covered": 1, "partial": 1, "total": 2, "percent": 50.0 },
          "keywords": [
            { "schema_path": "/type", "state": "full", "valid": 2, "invalid": 1 },
            { "schema_path": "/maximum", "state": "partial", "valid": 2, "invalid": 0 }
          ]
        },
        {
          "location": "body",
          "media_type": "application/json",
          "hits": 2,
          "coverage": { "covered": 0, "partial": 1, "total": 2, "percent": 0.0 },
          "keywords": [
            { "schema_path": "/properties/id/type", "state": "miss" },
            { "schema_path": "/properties/name/type", "state": "partial", "valid": 2, "invalid": 0 }
          ]
        }
      ],
      "responses": [
        { "status": "200", "hits": 2 },
        { "status": "400", "hits": 0 }
      ]
    }
  ]
}

Key fields:

• hits - number of times the operation/parameter was called
• status - response status code (string, not integer)
• location - parameter location: query, path, header, cookie, body, or form_data
• Query/path/header/cookie parameters have name, body parameters have media_type
• schema_path - JSON pointer to the schema keyword (e.g., /properties/id/type points to the type keyword of the id property)
• valid/invalid - count of requests that passed/failed validation for this keyword
• operation_id - from OpenAPI spec, omitted if not defined

Key Concepts

Coverage Types

Binary coverage (operations, examples, responses): covered / total

Ternary coverage (parameters, keywords): tracks three states, but percent = covered / total * 100

• covered - fully tested (both valid and invalid inputs)
• partial - partially tested (only valid OR only invalid, doesn't count toward percent)

Keyword States

State	Meaning
full	Both valid and invalid inputs tested
partial	Only valid or only invalid tested (see valid/invalid counts)
miss	Not tested at all
unsupported	Cannot be tested (e.g., {"schema_path": "/format", "state": "unsupported", "reason": "unknown format 'custom-id'"})

Common jq Recipes

# Get overall operation coverage percentage
tracecov report openapi.json traffic.json --format json | jq '.summary.operations.percent'

# List uncovered operations
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | select(.hits == 0) | "\(.method) \(.path)"'

# List operations below 80% keyword coverage
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | select(.coverage.keywords.percent < 80) | "\(.method) \(.path): \(.coverage.keywords.percent)%"'

# Find all untested keywords (state: miss)
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | . as $op | .parameters[]? | . as $param | .keywords[]? | select(.state == "miss") | "\($op.method) \($op.path) - \($param.name // $param.media_type)\(.schema_path)"'

# Find keywords needing invalid/negative tests (state: partial with valid > 0)
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | . as $op | .parameters[]? | . as $param | .keywords[]? | select(.state == "partial" and .valid > 0) | "\($op.method) \($op.path) - \($param.name // $param.media_type)\(.schema_path)"'

# Get uncovered response codes per operation
tracecov report openapi.json traffic.json --format json | \
  jq -r '.operations[] | {op: "\(.method) \(.path)", missing: [.responses[] | select(.hits == 0) | .status]} | select(.missing | length > 0) | "\(.op): \(.missing | join(", "))"'

# CI threshold check (exit 1 if below 80%)
tracecov report openapi.json traffic.json --format json | \
  jq -e '.summary.operations.percent >= 80' > /dev/null || exit 1

CI Example

GitHub Actions example with custom error reporting:

#!/bin/bash
set -e

REPORT=$(tracecov report openapi.json traffic.json --format json)

# Print coverage summary
echo "Operations: $(echo "$REPORT" | jq '.summary.operations.percent')%"
echo "Keywords: $(echo "$REPORT" | jq '.summary.keywords.percent')%"

# Fail if operations below 80%
if ! echo "$REPORT" | jq -e '.summary.operations.percent >= 80' > /dev/null; then
  echo "::error::Operation coverage below 80%"  # GitHub Actions annotation
  echo "$REPORT" | jq -r '.operations[] | select(.hits == 0) | "  - \(.method) \(.path)"'
  exit 1
fi