Concepts
TraceCov measures how thoroughly your tests cover your API specification, tracking validation logic at the schema level.
Coverage Dimensions
| Dimension | What it tracks | Example |
|---|---|---|
| Operations | HTTP method + path combinations called | GET /users, POST /orders |
| Parameters | Path, query, header, cookie, body values | userId in /users/{userId} |
| Keywords | JSON Schema validation rules exercised | minLength, pattern, enum |
| Examples | Schema examples and defaults used | example: "john@example.com" |
| Responses | HTTP status codes returned | 200, 404, 5XX |
Keywords
Consider this schema:
{
"type": "object",
"properties": {
"name": { "type": "string", "minLength": 1, "maxLength": 100 },
"age": { "type": "integer", "minimum": 0, "maximum": 150 }
}
}
TraceCov tracks each validation keyword separately: type, minLength, maxLength, minimum, maximum.
Why this matters: Hitting POST /users once with {"name": "Alice", "age": 30} gives you 100% endpoint coverage but poor keyword coverage - you haven't tested:
- Empty name (fails
minLength) - Name too long (fails
maxLength) - Negative age (fails
minimum) - Age over 150 (fails
maximum)
Keywords like title, description, default, and x-* extensions are excluded since they don't affect validation.
Coverage States
Each keyword tracks two counters:
| State | Meaning | Example |
|---|---|---|
| Valid | Input passed validation | "hello" passes minLength: 3 |
| Invalid | Input failed validation | "hi" fails minLength: 3 |
Full coverage means testing both valid and invalid inputs for each keyword.
Satisfiability
Some schema constraints can only be tested one way:
| Type | Can Pass? | Can Fail? | Example |
|---|---|---|---|
| Normal | Yes | Yes | minLength: 5 |
| Positive-only | Yes | No | minLength: 0 (can't have negative length) |
| Negative-only | No | Yes | {"not": {}} (rejects everything) |
TraceCov detects these cases automatically and excludes them from coverage calculations.
Coverage Percentage
The report shows separate percentages for each dimension:
- Operations -
called / totaloperations - Parameters -
tested / totalparameters - Keywords -
(partial + full) / total - Examples -
used / totalschema examples and defaults - Responses -
seen / documentedstatus codes