Skip to content

Concepts

TraceCov measures how thoroughly your tests cover your API specification, tracking validation logic at the schema level.

Coverage Dimensions

Dimension What it tracks Example
Operations HTTP method + path combinations called GET /users, POST /orders
Parameters Path, query, header, cookie, body values userId in /users/{userId}
Keywords JSON Schema validation rules exercised minLength, pattern, enum
Examples Schema examples and defaults used example: "john@example.com"
Responses HTTP status codes returned 200, 404, 5XX

Keywords

Consider this schema:

{
  "type": "object",
  "properties": {
    "name": { "type": "string", "minLength": 1, "maxLength": 100 },
    "age": { "type": "integer", "minimum": 0, "maximum": 150 }
  }
}

TraceCov tracks each validation keyword separately: type, minLength, maxLength, minimum, maximum.

Why this matters: Hitting POST /users once with {"name": "Alice", "age": 30} gives you 100% endpoint coverage but poor keyword coverage - you haven't tested:

  • Empty name (fails minLength)
  • Name too long (fails maxLength)
  • Negative age (fails minimum)
  • Age over 150 (fails maximum)

Keywords like title, description, default, and x-* extensions are excluded since they don't affect validation.

Coverage States

Each keyword tracks two counters:

State Meaning Example
Valid Input passed validation "hello" passes minLength: 3
Invalid Input failed validation "hi" fails minLength: 3

Full coverage means testing both valid and invalid inputs for each keyword.

Satisfiability

Some schema constraints can only be tested one way:

Type Can Pass? Can Fail? Example
Normal Yes Yes minLength: 5
Positive-only Yes No minLength: 0 (can't have negative length)
Negative-only No Yes {"not": {}} (rejects everything)

TraceCov detects these cases automatically and excludes them from coverage calculations.

Coverage Percentage

The report shows separate percentages for each dimension:

  • Operations - called / total operations
  • Parameters - tested / total parameters
  • Keywords - (partial + full) / total
  • Examples - used / total schema examples and defaults
  • Responses - seen / documented status codes