Skip to content

ADR 0003 — Observability telemetry transport: Sentry (frontend) + OTel/LGTM (backend)

Purpose: settle the telemetry-transport question that Wave 2 left as an open PO decision (A09, M6 gate) and that blocks ~10 todos.tasks items across web and mobile. Track G observability tickets cite this ADR.

  • Status: Accepted
  • Date: 2026-07-11
  • Deciders: Product Owner + terminales engineering (Wave 3 planning session)
  • Repos: Multi — terminales-web (Next.js), terminales (Android/Kotlin), terminales-api (NestJS), infra (terminales-api/observability/)
  • Related: wave2-mvp-analysis §3.1 A09 + rows 29/83/84, observability-plan, 0002-sentry-sdk-wiring

Observability is asymmetric across the stack (audit 2026-07-11):

  • API — mature. OpenTelemetry SDK (src/instrumentation.ts: OTLP traces → :4318, Prometheus metrics → :8081, pino log correlation), nestjs-pino + nestjs-otel, and a full LGTM stack provisioned at terminales-api/observability/ (Grafana, Loki, Tempo, Mimir, Prometheus, Promtail, OpenTelemetry Collector). A grafana-stage deployment is live on ionos (#215).
  • Frontend — absent. Web declares NEXT_PUBLIC_SENTRY_DSN but installs no Sentry SDK; every metric is console-only or a no-op sink (~9 emission points blocked on @blocker(needs-metrics-pipeline)). Mobile analytics are log-stubs (no crash/error sink); worse, mobile over-logs sensitive traffic at BODY level (row 15).
  • Appwrite — unshipped. Appwrite Function and database logs go nowhere; there is no dashboard or alert on Appwrite-side health.

Wave 2 flagged “pick the web/mobile telemetry transport (Sentry vs OTel collector)” as the blocker and drafted the wiring as rows 83 (web) / 84 (mobile). The PO decided in Wave 3.

Decision — Sentry for frontend errors, OTel/LGTM for backend + Appwrite logs

Section titled “Decision — Sentry for frontend errors, OTel/LGTM for backend + Appwrite logs”

Two transports, split by strength:

  • Sentry owns frontend error/crash reporting and session tracing on web (@sentry/nextjs) and mobile (sentry-android / Kotlin). It is the production visibility surface for the web admin and the Android POS terminal — uncaught errors, crashes, failed auth/tender flows, release health.
  • OpenTelemetry → the self-hosted LGTM stack stays the backend transport: API traces (Tempo), metrics (Mimir/Prometheus), and logs (Loki). It is extended to ingest Appwrite Function and database logs into Loki and to add POS/Appwrite dashboards + alerts in Grafana.
  • The two are correlated by shared ids (request-id / trace-id propagated from the client Sentry event into the backend OTel trace where feasible) so an incident can be followed across the boundary.

Rationale: Sentry gives managed, low-effort, best-in-class frontend error triage and release health that the raw OTel collector does not — fastest production value for the two surfaces that have zero visibility today. Keeping OTel/LGTM for the backend preserves the mature, already-provisioned pipeline and full data ownership of traces/metrics, and Loki is the natural sink for Appwrite logs. Splitting by strength beats forcing one pipeline to do both jobs poorly.

  • OTel collector everywhere (route web/mobile through the LGTM stack, no Sentry) → rejected; more wiring on the frontend, and no managed error-grouping/release-health UX — the frontend surfaces would get raw spans/logs but not the triage workflow that makes them actionable, delaying the value we need most.
  • Sentry everywhere (retire self-hosted LGTM) → rejected; throws away the mature, provisioned API pipeline and full trace/metric data ownership, and adds cost for backend telemetry that already works well.
  • Unblocks Wave 2 rows 83 (web telemetry wiring) and 84 (mobile telemetry wiring) and the ~10 @blocker(needs-metrics-pipeline) todos — those emission points now have a transport (TDR-0002 specifies the wiring).
  • New MVP prerequisites: a Sentry project/DSN per environment (web + mobile), PII scrubbing configured (no PIN/PAN/JWT in events — reconciles with the row 15 mobile logging fix), and Appwrite→Loki log shipping.
  • The consolidated wiring, dashboards, alerts, and Appwrite ingestion are specified in observability-plan and drafted as Track G.
  • Metric-name/event taxonomy must be shared web↔mobile↔api so dashboards line up; the plan owns the canonical event names.
  • wave2-mvp-analysis §3.1 A09, rows 29/83/84, §4.6 telemetry decision.
  • 0002-sentry-sdk-wiring — the SDK wiring approach.
  • observability-plan — the full plan.
  • terminales-api/observability/ — the provisioned LGTM stack.