Skip to content

TraceCov vs Other API Coverage Tools

Most OpenAPI coverage tools record that a specification element was used, not the values sent. Take a limit query parameter declared as {"type": "integer", "minimum": 1, "maximum": 100} and a suite that only ever sends limit=10. Endpoint and parameter coverage tools report limit as covered. TraceCov reports type, minimum and maximum as exercised with valid input only: the suite never sent 0, 101 or abc. See Keywords.

Code coverage answers a different question. It measures which lines of the implementation ran and needs its source; specification coverage works black-box and shows which inputs the contract allows that the tests never tried. The two complement each other.

Checked 2026-09-26 against each tool's source code or documentation: swagger-coverage 1.5.0, swagger-coverage-py 3.5.2, pw-radar 0.2.2, and Restats as of its November 2025 commit. Later versions may differ.

At a glance

TraceCov swagger-coverage pw-radar Restats Others
Operations, statuses
Parameters
Enum values
Request body fields
Keywords, valid/invalid
Response body
Multi-spec reports
History between runs
CI thresholds

supported   partial or in some tools, see each section below

When another tool fits better

  • RestAssured or Karate suites. swagger-coverage hooks the client in-process. TraceCov has no JVM integration; a Java suite feeds it through the proxy (Professional) or a traffic file.
  • Several services in one run. pw-radar routes each call to the right specification and renders an aggregate report. Planned for TraceCov.
  • Response fields the specification does not declare. pw-radar flags them even when the schema allows extra properties; TraceCov only reports them when the schema forbids them.
  • A coverage trend across runs. swagger-coverage-tool and api-coverage-tracker keep a history of percentages. Planned for TraceCov.
  • Postman without running it. swagger-coverage-cli reads a collection statically and also covers gRPC and GraphQL.

Only in TraceCov

None of the tools above does any of the following:

  • Unsatisfiable schemas explained. A schema no value can satisfy, such as an allOf of conflicting types, is reported with the keywords that empty it and where they sit. See Diagnostics.
  • Specification problems. Unresolvable $ref, invalid schemas, paths no request can reach, and contentEncoding that validation does not enforce, listed in every report.
  • Impossible checks left out. minLength: 0 has no invalid case, so it is not counted against you. See Satisfiability.
  • Unmatched requests. Traffic that hit no documented operation, so a wrong base path shows up instead of silently losing coverage. See Unmatched Requests.
  • Example and default coverage. Whether the specification's example and default values were ever sent.
  • Suggested requests. Ready-to-send requests that cover each operation's missing keywords, through CoverageMap.suggestions() in Python and the MCP server (Professional).
  • MCP server. Coding agents query coverage gaps while they write tests, through tracecov mcp (Professional).

swagger-coverage

swagger-coverage is a Java tool used with RestAssured and Karate.

  • Measures: status codes, parameter presence, every enum value and a value outside the enum, through ten switchable rules.
  • Partial: request body properties, top-level only, and only for form and multipart bodies; JSON bodies go unmeasured.
  • Not measured: response bodies, beyond status and content type.
  • Runs on: a JVM. swagger-coverage-py bundles its Java command-line tool and needs JDK 11 or later.

pw-radar

pw-radar is a TypeScript port of swagger-coverage for Playwright. Like @tracecov/playwright, it records Playwright API calls.

  • Measures: the swagger-coverage rules, plus request body properties up to five levels deep.
  • Partial: response bodies, as top-level field names; parameters, where cookies are never counted.
  • Runs on: Node.js, by wrapping an APIRequestContext in your fixture.

Restats

Restats is the research implementation of REST API test coverage criteria (Corradini et al., ICSME 2021), described on the coverage depth page.

  • Measures: paths, operations, status codes, content types, and the values of boolean and enum parameters.
  • Partial: parameters, where path parameters are left out; request bodies, by top-level key name.
  • Not measured: response bodies.
  • Input: raw request and response pairs recorded through Burp, and JSON specifications without $ref. TraceCov reads YAML and JSON, resolves $ref, and supports OpenAPI 3.1 and 3.2.

Other tools

Tool Traffic source Measures
swagger-coverage-tool httpx, requests operations, status codes, query parameter names
swagger-coverage-cli Postman, Newman operation and status code pairs
api-coverage-tracker Playwright, axios, fetch, Postman operations, status codes, query parameters
Specmatic its own contract tests operation, status code and content type
RESTler its own fuzzing operations that returned a 2xx
EvoMaster its own fuzzing status codes per operation

Commercial platforms go one level further: ReadyAPI and Keploy report which body fields were covered. No tool in this survey tracks JSON Schema keywords beyond enum, or counts valid and invalid input separately.

Try it

Run your existing suite through TraceCov with the Quick Start and compare the keyword percentage with the operation percentage. Expect the keyword number to be much lower on a first run: the uncovered keywords in the report are the list of cases to add.