Validation & QA

Validation & QA Workflow

Before shipping an extension, you need to verify it actually works — code matches requirements, metrics have data, screens display correctly, and the YAML passes lint. This is the workflow we use for every extension delivery.

The 5-Step Validation Process

1. SNMP Lint — Run the validator tool for DED001-DED021 diagnostics
2. Code Validation — Map every requirement to implementation with line numbers
3. Datapoint Validation — Query Dynatrace API to verify metrics have actual data
4. Screens Validation — Verify every metric appears in a screen or dashboard
5. Alert Verification — Confirm alerts fire correctly on test data

Step 1: SNMP Lint (MANDATORY)

Run the validator before every build. Zero errors required, warnings acceptable:

python3 tools/validate_extension.py extensions/my-ext/ext/

# Output:
# Validating: extensions/my-ext/ext/extension.yaml
# ERROR DED016: Scalar OID 1.3.6.1.2.1.2.2.1.10.0 in table subgroup "Interfaces"
# WARNING DED006: Count metric "my-ext.if.in.octets" should end with ".count"
# 1 errors, 1 warnings

Critical DED Codes

Code	Severity	Meaning
DED016	ERROR	Scalar OID (.0) in table subgroup — remove .0 or set table: false
DED017	ERROR	Table OID missing .0 in non-table subgroup — add .0
DED018	ERROR	Cross-table OID mismatch — OIDs from different SNMP tables in same subgroup
DED015	ERROR	OID syntax error — malformed OID string
DED010	WARNING	OID not in bundled MIBs — may need to add MIB file
DED006	WARNING	Count metric key should end with .count
DED007	WARNING	Gauge metric key should not end with .count

Step 2: Code Validation

For each requirement in the requirements document:

1. Find the metric key in extension.yaml
2. Verify the OID or API endpoint matches the requirement
3. Check the metric type (gauge vs count) is correct
4. Note the line number for the report
5. Mark as VALID, BUG, or GAP

Step 3: Datapoint Validation

Query the Dynatrace API to verify metrics have actual data:

# Search for metrics by prefix
curl "$BASE/api/v2/metrics?text=my-ext" -H "Authorization: Api-Token $TOKEN"

# Query specific metric for data
curl "$BASE/api/v2/metrics/query?metricSelector=com.dynatrace.extension.my-ext.cpu&from=now-24h" \
  -H "Authorization: Api-Token $TOKEN"

Possible results:

• VALID — metric exists and has datapoints
• NO DATA — metric registered but no values (device config issue, not a code bug)
• BUG — metric not registered at all (code error)

Step 4: Screens Validation

Every metric collected should be visible somewhere in the UI:

• Check screens: section — is the metric in a chartsCards entry?
• Check dashboards: — is it referenced in a dashboard JSON?
• If a metric is collected but never displayed, flag it as a screens bug

Example: MSSQL v2.10.6 collected 4 metrics that weren't displayed anywhere — worker.threadsPercent, worker.activeWorkers, worker.maxWorkers, locks.elapsedTimeRequestsPercent.

Step 5: Alert Verification

After deploying alerts:

1. Query Settings API to confirm they exist: GET /api/v2/settings/objects?schemaIds=builtin:anomaly-detection.metric-events
2. Verify the metric key in each alert matches an actual metric
3. Check entity dimension key points to the correct parent entity
4. For func: metrics, verify METRIC_SELECTOR query type is used (not METRIC_KEY)

Combined Validation Report

Every extension gets a combined report (.docx + .md) with:

Extension Name - Combined Validation Report

Header: version, env URL, source files, result summary (X/Y VALID)

Per requirement:
  Requirement Name
    Description, OIDs/endpoints
    Code Validation: implementation details, metric keys, VALID/BUG
    Datapoint Validation: VALID/BUG/NO DATA

Summary table: | # | Requirement | Code | Data |

Scorecard

From our 13 extension validations:

Metric	Value
Total requirements validated	200+
Bugs found and fixed	12
Gaps identified	15
Alerts deployed	81
Extensions delivered	8