ADR 0003 — Observability telemetry transport: Sentry (frontend) + OTel/LGTM (backend)
Purpose: settle the telemetry-transport question that Wave 2 left as an open PO decision (A09, M6 gate) and that blocks ~10
todos.tasksitems across web and mobile. Track G observability tickets cite this ADR.
- Status: Accepted
- Date: 2026-07-11
- Deciders: Product Owner + terminales engineering (Wave 3 planning session)
- Repos: Multi —
terminales-web(Next.js),terminales(Android/Kotlin),terminales-api(NestJS), infra (terminales-api/observability/) - Related:
wave2-mvp-analysis§3.1 A09 + rows 29/83/84,observability-plan,0002-sentry-sdk-wiring
Context
Section titled “Context”Observability is asymmetric across the stack (audit 2026-07-11):
- API — mature. OpenTelemetry SDK (
src/instrumentation.ts: OTLP traces →:4318, Prometheus metrics →:8081, pino log correlation), nestjs-pino + nestjs-otel, and a full LGTM stack provisioned atterminales-api/observability/(Grafana, Loki, Tempo, Mimir, Prometheus, Promtail, OpenTelemetry Collector). Agrafana-stagedeployment is live onionos(#215). - Frontend — absent. Web declares
NEXT_PUBLIC_SENTRY_DSNbut installs no Sentry SDK; every metric is console-only or a no-op sink (~9 emission points blocked on@blocker(needs-metrics-pipeline)). Mobile analytics are log-stubs (no crash/error sink); worse, mobile over-logs sensitive traffic at BODY level (row 15). - Appwrite — unshipped. Appwrite Function and database logs go nowhere; there is no dashboard or alert on Appwrite-side health.
Wave 2 flagged “pick the web/mobile telemetry transport (Sentry vs OTel collector)” as the blocker and drafted the wiring as rows 83 (web) / 84 (mobile). The PO decided in Wave 3.
Decision — Sentry for frontend errors, OTel/LGTM for backend + Appwrite logs
Section titled “Decision — Sentry for frontend errors, OTel/LGTM for backend + Appwrite logs”Two transports, split by strength:
- Sentry owns frontend error/crash reporting and session tracing on web
(
@sentry/nextjs) and mobile (sentry-android/ Kotlin). It is the production visibility surface for the web admin and the Android POS terminal — uncaught errors, crashes, failed auth/tender flows, release health. - OpenTelemetry → the self-hosted LGTM stack stays the backend transport: API traces (Tempo), metrics (Mimir/Prometheus), and logs (Loki). It is extended to ingest Appwrite Function and database logs into Loki and to add POS/Appwrite dashboards + alerts in Grafana.
- The two are correlated by shared ids (request-id / trace-id propagated from the client Sentry event into the backend OTel trace where feasible) so an incident can be followed across the boundary.
Rationale: Sentry gives managed, low-effort, best-in-class frontend error triage and release health that the raw OTel collector does not — fastest production value for the two surfaces that have zero visibility today. Keeping OTel/LGTM for the backend preserves the mature, already-provisioned pipeline and full data ownership of traces/metrics, and Loki is the natural sink for Appwrite logs. Splitting by strength beats forcing one pipeline to do both jobs poorly.
Alternatives rejected
Section titled “Alternatives rejected”- OTel collector everywhere (route web/mobile through the LGTM stack, no Sentry) → rejected; more wiring on the frontend, and no managed error-grouping/release-health UX — the frontend surfaces would get raw spans/logs but not the triage workflow that makes them actionable, delaying the value we need most.
- Sentry everywhere (retire self-hosted LGTM) → rejected; throws away the mature, provisioned API pipeline and full trace/metric data ownership, and adds cost for backend telemetry that already works well.
Consequences
Section titled “Consequences”- Unblocks Wave 2 rows 83 (web telemetry wiring) and 84 (mobile telemetry wiring) and
the ~10
@blocker(needs-metrics-pipeline)todos — those emission points now have a transport (TDR-0002specifies the wiring). - New MVP prerequisites: a Sentry project/DSN per environment (web + mobile), PII scrubbing configured (no PIN/PAN/JWT in events — reconciles with the row 15 mobile logging fix), and Appwrite→Loki log shipping.
- The consolidated wiring, dashboards, alerts, and Appwrite ingestion are specified in
observability-planand drafted as Track G. - Metric-name/event taxonomy must be shared web↔mobile↔api so dashboards line up; the plan owns the canonical event names.
References
Section titled “References”wave2-mvp-analysis§3.1 A09, rows 29/83/84, §4.6 telemetry decision.0002-sentry-sdk-wiring— the SDK wiring approach.observability-plan— the full plan.terminales-api/observability/— the provisioned LGTM stack.