Security
Security
The scrape endpoint
- Restrict with
TELEMETRY_ALLOWED_IPS(CIDR supported) or replace the middleware stack per endpoint with your own auth. - Use
onlyfilters to publish a minimal metric set on any endpoint that is reachable from less-trusted networks. - Set
TELEMETRY_PROMETHEUS_ENABLED=falseif you export via OTLP only — the routes are never registered.
What leaves the app
- Metric names/labels describe your system's shape. Keep label values bounded and non-personal (no emails, user ids, tokens).
- Query spans include SQL text truncated to 500 characters — bindings are not included, so values stay out of traces by default.
- Exception events on spans carry the message and stack trace. If your exception messages can contain user data, sanitize at the source.
- Events contain exactly the attributes you pass. Treat
Telemetry::event()payloads like log lines: no secrets. - Request spans capture only allowlisted headers
(
instrument.request_headers/response_headers); credential and session headers (Authorization,Cookie,X-Api-Key, …) are denylisted and never captured, even when allowlisted.url.queryredacts common secret parameters (token,signature,code, …).
The redaction engine
Everything above is capture-side hygiene. On top of it, every span attribute, span event (exception messages!) and telemetry event passes through the redaction engine at flush time — the last hands before any exporter:
- Key-based: attribute keys whose dot/underscore segments match a
sensitive word (
password,api_key,authorization, …) have their value replaced entirely. Segment matching, not substring —cache.keyis safe,stripe.api_keyis caught. - Pattern-based: regexes scrub secrets embedded in any string
value — JWTs,
Bearer/Basiccredentials and url userinfo (redis://user:pass@host) by default — wherever they appear: exception messages, event payloads, log records. - Session correlation uses a truncated SHA-256 of the session id — the raw id (an authentication credential) never leaves the app.
- The engine covers log records too — the message itself (key
log.message) and its context attributes are scrubbed like everything else. - Custom hook, run last:
Telemetry::redactUsing(function (string $key, string $value): ?string {
return preg_match('/\d{6}-\d{4}/', $value) ? '[CPR]' : null; // null = keep
});
Configure under telemetry.redaction (keys, patterns,
replacement, enabled) — extend the defaults via
Redactor::defaultKeys() / defaultPatterns(). A broken pattern or
hook never breaks telemetry; it is skipped. Metric labels are not run
through the engine — they are bounded dimensions by design and must
never contain user data in the first place.
Incoming trace headers
traces.continue_incoming trusts traceparent from clients, which lets
callers set your trace ids and sampling decision. That's standard W3C
behaviour and safe for internal meshes. On public edges you have two
knobs:
# Keep trace-id correlation but decide sampling locally —
# clients can no longer force sampling on (bypassing your rate) or off:
TELEMETRY_TRACES_TRUST_INCOMING_SAMPLING=false
# Or ignore incoming trace context entirely:
TELEMETRY_TRACES_CONTINUE_INCOMING=false
Transport
OTLP export honours HTTPS endpoints and custom auth headers. Credentials live in env/config — never in metric names or attributes.
Metric label cardinality
Metric labels are NOT run through the redaction engine — they are bounded dimensions by contract and must never carry user data or unbounded values. The bundled instrumentation keeps labels bounded (route patterns, class basenames, ability names, outcomes). Two labels are bounded-but-churning and worth knowing about on large fleets:
worker.memory.*{pid}mints a new series per worker process; workers recycled often (Horizon--max-jobs, deploys) leave stale gauges in the store until swept. Aggregate per queue if that matters.queue.jobs.dispatched{job.name}uses the queued job's display name. Standard job classes and closures are bounded; a customdisplayName()that embeds an id would not be. Keep display names static.
When you add your own labels (labelRequestsUsing, cache classifier,
custom counters), the same rule applies: bounded values only.