Choosing an Observability Stack

Sentry, Datadog, Grafana, Prometheus, Jaeger, ELK, OpenTelemetry — interviewers will ask “which would you pick?” The strong answer is “depends on these factors” with specific trade-offs, not a single brand recommendation.

The tools at a glance

Sentry is a specialist (errors); the rest split into “general-purpose SaaS” or “self-hosted OSS stacks.”

The two big choices

1. Buy SaaS or self-host?
2. Vendor-locked SDK or OpenTelemetry?

Most other choices follow from these.

SaaS vs self-host trade-off

Rough heuristic: SaaS up to 50-100 engineers / single-region / fast-growing. Self-hosted past that, or where cost / compliance / control matter.

A common-stack tour

“Just shipped MVP” stack

• Sentry for errors.
• Vendor-provided cloud metrics (e.g., AWS CloudWatch, Cloud Run logs) for infra.
• Application logs to stdout → cloud log service.

Cheap, fast, low engineering investment. Misses distributed tracing; that’s fine until you have multiple services.

“Growing startup” stack (~10-50 eng, multi-service)

Two patterns:

A — SaaS unified:

• Datadog (or New Relic) for everything: metrics, logs, traces, RUM.
• Sentry alongside for errors (Datadog’s error tracking is OK but Sentry’s is purpose-built).

B — OSS stack:

• Prometheus for metrics + Alertmanager for alerts.
• Grafana for dashboards.
• Loki for logs (Grafana ecosystem).
• Tempo or Jaeger for traces.
• OpenTelemetry SDK in apps.
• Sentry for errors.

A is faster to set up; B is cheaper at scale and avoids lock-in.

“Mature enterprise” stack (100+ eng)

Heterogeneous: legacy services on what was there, new services on the new standard. Common shape:

• OpenTelemetry SDK everywhere (or migrating to).
• OpenTelemetry Collector as a routing layer.
• Multi-backend: Datadog primary + self-hosted Prometheus + Grafana for cost-sensitive workloads.
• Splunk or Elasticsearch for compliance-driven log retention.
• Sentry for application errors.

The Collector + OTel is the lock-in hedge.

When to pick each

Sentry

• You need detailed error tracking with stack traces, breadcrumbs, release tracking, deduplication.
• Want first-class developer UX for fixing bugs (Linear / Jira integration).
• Sentry covers errors well; its APM is OK but not a full Datadog replacement.

Almost every team uses Sentry alongside whatever else. The error-tracking domain is specialized enough that it earns a dedicated tool.

Datadog

• Small / medium team, multiple data types, need correlation, budget allows.
• Multi-environment / multi-region / Kubernetes setup where built-in integrations save time.
• Compliance-sensitive (SOC 2, HIPAA-eligible).

Avoid if: cost-sensitive at scale, strict data residency, simple use case.

Prometheus + Grafana

• Metrics-focused observability.
• Kubernetes-native (Prometheus is the standard).
• Cost matters; in-house SRE capacity exists.
• Want vendor independence.

Pair with Alertmanager for alerts, Loki for logs, Tempo / Jaeger for traces if you need them.

Jaeger (vs Tempo, vs Datadog APM)

• Want a pure OSS tracing backend.
• Need a polished search UI with service maps.
• Storage: Cassandra or Elasticsearch (existing infra).

Tempo wins if you have object storage and Grafana (cheap traces). Datadog wins if you’ll pay for the polish and need integrated tagging across pillars.

ELK Stack (Elasticsearch + Logstash + Kibana)

• Log-heavy workload.
• Need full-text search over logs.
• Have Elasticsearch ops capacity.
• Compliance retention requirements.

Loki is the lower-cost alternative for “logs as time-series” — cheaper storage, less flexible search. ELK for “we need real search.”

OpenTelemetry

• Always. For new code, instrument with OTel and export to whatever backend.
• Lets you switch backends without recoding apps.
• Use vendor SDKs (ddtrace) only when you need vendor-specific features the OTel ecosystem doesn’t cover.

Sentry vs Datadog APM for errors

Most teams use Sentry for errors because it’s purpose-built. Datadog’s APM also captures errors but as a side effect of trace data; less polished for the bug-fixing workflow.

A pragmatic recommendation

For a new team starting today:

1. Sentry from day 1. Free tier or cheap plan; install in 5 minutes; saves enormous time.
2. OpenTelemetry from day 1. Don’t lock into a vendor SDK; instrument once.
3. Pick a primary backend based on team size and budget:
   • Tiny (<5 eng): cloud-native logging/metrics + Sentry. Add Datadog or self-hosted when growth demands.
   • Growing (5-50 eng): Datadog if budget allows, self-hosted OSS otherwise.
   • Larger / cost-sensitive: self-hosted OSS with strong SRE investment.

Avoid:

• Vendor lock-in via vendor SDKs (use OTel instead).
• DIY error tracking (use Sentry).
• Running ELK at small scale (overkill; Loki is simpler).
• Splunk at startup scale (license cost; complexity).

Migration realities

Switching observability vendors is expensive. Issues:

• All dashboards rebuilt.
• All alerts rebuilt.
• Runbooks reference old tools.
• Muscle memory of where to look.
• Historical data left in old vendor.

Lock-in is real. Mitigations:

• OpenTelemetry instrumentation (so switching backend is just an exporter config).
• Dashboards as code (Grafana JSON, Datadog Terraform).
• Alerts as code (Terraform-managed).

Common interview confusions

• “OSS is always cheaper.” — at scale, yes. At small scale, OSS infrastructure + ops time often exceeds SaaS cost.
• “Datadog can replace Sentry.” — has error tracking but Sentry is more polished for the developer bug-fixing workflow. Most teams use both.
• “OpenTelemetry is a vendor.” — standard / CNCF project, not a vendor. The vendors are backends (Jaeger, Datadog, Tempo, …).

Interview angle

• “What observability stack would you pick for [team size]?” — depend-on-context answer. Small / fast-growing: SaaS (Datadog or NewRelic) + Sentry. Growing with SRE capacity: OSS (Prometheus + Grafana + Loki + Jaeger/Tempo + OpenTelemetry SDK + Sentry). Large enterprise: hybrid via OTel Collector multi-export.
• “When would you choose Datadog over Prometheus + Grafana?” — small / medium team needing fast time-to-value, multiple data types correlated in one UI, budget allows, don’t want to staff SRE for the OSS stack. Prom + Grafana at large scale where cost matters and SRE capacity exists.
• “Sentry vs Datadog for errors?” — Sentry is purpose-built for the developer bug-fix workflow (rich stack traces, source maps, release tracking, issue grouping, Linear/Jira integration). Datadog has error tracking but it’s secondary to APM. Most teams use both.
• “Why use OpenTelemetry instead of a vendor SDK?” — vendor neutrality. OTel instrumentation works with any backend (Jaeger, Tempo, Datadog, Honeycomb, NewRelic, …). Switching vendors is an exporter config change, not a rewrite.
• “What’s the cost trade-off between SaaS and self-hosted?” — SaaS scales linearly with usage (often non-linearly with high-cardinality); self-hosted has fixed infrastructure cost plus SRE time. Crossover is roughly $50-100k annual SaaS = 1 senior SRE running OSS.
• “How do you avoid observability vendor lock-in?” — OpenTelemetry SDK for instrumentation (vendor-neutral data); dashboards as code (Grafana JSON, Datadog Terraform); alerts as code; the OTel Collector for multi-backend routing.