Observability
Sage exposes two integration points, for two different jobs:
SageListeneris an asynchronous observer ofSageEvents (command completions, connection transitions, cache outcomes, topology changes), called off the command path. Use it for metrics and operational logging.CommandTracerproduces distributed-tracing spans synchronously on the command path, so each Redis command appears as a client span nested under the surrounding request in an APM such as Datadog or Jaeger. Use it for distributed tracing: see Distributed tracing.
Events
Register one or more SageListener on SageConfig, and each receives every SageEvent: command completions, connection transitions, cache outcomes, and topology changes. This is how you wire Sage into your metrics or logging.
| Event | Reported when |
|---|---|
CommandCompleted(name, node, duration, outcome) | One logical command settled. duration is client-observed (including any cluster redirects/retries); outcome is Succeeded or Failed(error). A cached read served locally yields no CommandCompleted. |
Connection.Connected(node) | The multiplexed connection connected, on the initial connect and on every reconnect. |
Connection.Disconnected(node) | A live connection was lost and the runtime began reconnecting. Graceful close is not reported. |
Connection.ConnectFailed(node, error) | A connection could not be established, or the node could not be qualified during topology discovery. Names the address and cause, whether the failure is handled internally or also returned to the caller. |
Connection.ReconnectFailed(node, error) | A reconnect attempt failed to establish, carrying its cause. Backoff and retries continue. Not reported for the initial connect. |
Cache.Hit(command) / Cache.Miss(command) | A cached read was served locally, or had to fetch from the server. |
TopologyChanged(masters) | The cluster's slot-owning master set changed (a failover, or scaling a shard in or out). |
Events carry no command arguments or payloads, so secrets such as AUTH credentials and user values never reach a listener. Where an event carries node, it is Some for cluster and master-replica clients and None for a standalone client. A node that keeps failing to connect produces one ConnectFailed, not one per attempt.
Registering a listener
A SageListener has one synchronous, Unit-returning method. Match on the event you care about and forward it to your metrics system:
import sage.{SageEvent, SageListener}
import sage.SageEvent.*
val metrics = new SageListener {
def onEvent(event: SageEvent): Unit = event match {
case CommandCompleted(name, _, duration, _) => // record latency
case Connection.Disconnected(_) => // bump a gauge
case Connection.ConnectFailed(node, error) => // log the unreachable address
case Connection.ReconnectFailed(_, error) => // log the cause
case Cache.Hit(_) => // count a hit
case Cache.Miss(_) => // count a miss
case _ => ()
}
}
val config = SageConfig(
topology = Topology.Standalone(Endpoint("localhost", 6379)),
listeners = Vector(metrics)
)SageListener lives in the core and is the same on every backend, so this snippet is backend-independent.
Delivery guarantees
Listeners are invoked off the command path, so a slow or throwing listener cannot block or break command execution. It only delays or loses events.
WARNING
Delivery is best-effort: a thrown exception is swallowed, and events are dropped once the internal dispatch queue fills. Listeners suit metrics, sampling, and operational logging, not anything that must be a complete record.
Distributed tracing
Use a CommandTracer rather than a listener here: it runs synchronously on the command path, so each Redis span is a child of whatever span is active when the command is issued. A listener runs too late, off that path, and its spans would be orphaned.
Set one on SageConfig.tracer. The sage-opentelemetry module provides an OpenTelemetry implementation:
"com.github.ghostdogpr" %% "sage-opentelemetry" % "0.3.1"import sage.opentelemetry.OpenTelemetryCommandTracer
val config = SageConfig(
topology = Topology.Standalone(Endpoint("localhost", 6379)),
// reads the globally-registered OpenTelemetry
tracer = Some(OpenTelemetryCommandTracer.global())
)It emits one CLIENT span per command, named for the command (GET, SET, ...), carrying db.system, db.operation.name, peer.service (default redis, configurable), component, and the server address; a failure sets an error status with the exception. Only the command name is recorded, never arguments or keys, so secrets and user values stay out of your traces. Spans follow the ambient sampling decision.
One span is emitted per command that reaches the server, including each command in a pipeline (cluster redirects fold into the command's own span). A cached read served locally reaches no server and produces no span; one that misses is traced like any other command. In a transaction, the watch-phase reads are traced individually and the MULTI/EXEC body gets a single span named MULTI, which reflects the round trip rather than whether the transaction committed.
The tracer reads the active span from OpenTelemetry's thread-local current context (Context.current()) on the fiber that submits the command, and the module depends only on the OpenTelemetry API. So under an APM agent that instruments your runtime and propagates context across its threads, the Redis span nests under the active request span with no further wiring. That is the case for ZIO under the Datadog Java agent; configuring the agent itself is covered by its own documentation.
Context on a fiber runtime without an agent
Running a bare OpenTelemetry SDK with no agent is the case that needs attention: on a fiber runtime the active span lives in fiber-local state (a ZIO FiberRef, a Cats Effect IOLocal), which is not the current context the tracer reads, so spans would be orphaned. Configure context storage so that Context.current() sees the active span:
- ZIO: with
zio-telemetry, wire OpenTelemetry through theOpenTelemetry.contextJVMandOpenTelemetry.globallayers (rather thanOpenTelemetry.contextZIOandOpenTelemetry.custom), which back tracing with OpenTelemetry's native context so the SDK reads the active span. See zio-telemetry's auto-instrumentation interop documentation. - cats-effect: with
otel4son Cats Effect 3.6+, add theotel4s-oteljava-context-storagedependency, enable thecats.effect.trackFiberContextsystem property, and provideIOLocalContextStorage.localProvider[IO]. This keeps the JavaContextand the otel4s fiber context aligned so the SDK reads the active span. Note that the stock OpenTelemetry Java agent does not keep Cats Effect context in sync; otel4s ships a dedicated agent distribution for the agent case.
OpenTelemetryCommandTracer.withContextProvider lets you supply a custom () => Context for a context source that is thread-local but non-default.