How would you design observability — metrics, logs, and traces — for a Go microservice in production?

0-2 years experience

1. 1Suppose you need to add basic request latency metrics to a Go microservice. Which library would you choose and how would you expose the metrics for Prometheus?
2. 2If your service is logging JSON lines, what fields would you include to make logs useful for debugging in production?
3. 3How would you instrument a simple HTTP handler to emit a trace span using OpenTelemetry?

2-5 years experience

1. 1Your recent deployment caused a spike in error rates, but the logs don’t show any stack traces. Walk me through how you would use metrics, logs, and traces to pinpoint the issue.
2. 2When adding observability to a new feature, you notice increased CPU usage due to metric collection. How would you balance metric granularity versus overhead?
3. 3Explain how you would correlate a request’s trace ID across services when the downstream service is written in a different language.

5-8 years experience

1. 1Design an end‑to‑end observability pipeline for a fleet of Go microservices handling 10k RPS, covering metric aggregation, log storage, and trace sampling. Discuss trade‑offs.
2. 2Your tracing backend is hitting storage limits during peak traffic. What strategies would you employ to reduce trace volume without losing critical debugging information?
3. 3How would you ensure that observability instrumentation does not become a source of latency or panic in a high‑throughput Go service?

8+ years experience

1. 1At a large organization, you need to standardize observability across dozens of teams with heterogeneous tech stacks. How would you define a unified approach for metrics, logs, and traces, and drive adoption?
2. 2A legacy Go service uses custom logging and no tracing. Outline a migration plan to bring it into the modern observability platform while minimizing risk.
3. 3Discuss the long‑term maintenance challenges of observability data (retention, schema evolution, cost) and how you’d address them at the architectural level.