Observability
Sage exposes two integration points, for two different jobs:
SageListeneris an asynchronous observer ofSageEvents (command completions, connection transitions, cache outcomes, topology changes), called off the command path. Use it for metrics and operational logging.CommandTracerproduces distributed-tracing spans synchronously on the command path, so each Redis command appears as a client span nested under the surrounding request in an APM such as Datadog or Jaeger. Use it for distributed tracing: see Distributed tracing.
Events
Register one or more SageListener instances on SageConfig. Each listener receives every SageEvent: command completions, connection transitions, cache outcomes, and topology changes. You can use these events for metrics or logging.
| Event | Reported when |
|---|---|
CommandCompleted(name, node, duration, outcome) | One command completed. duration is measured by the client and includes any cluster redirects or retries. outcome is Succeeded or Failed(error). A cached read served locally does not produce this event. |
Connection.Connected(node) | The multiplexed connection connected, on the initial connect and on every reconnect. |
Connection.Disconnected(node) | A live connection was lost and the runtime began reconnecting. Graceful close is not reported. |
Connection.ConnectFailed(node, error) | A connection could not be established, or the node could not be qualified during topology discovery. Names the address and cause, whether the failure is handled internally or also returned to the caller. |
Connection.ReconnectFailed(node, error) | A reconnect attempt failed to establish, carrying its cause. Backoff and retries continue. Not reported for the initial connect. |
Cache.Hit(command) / Cache.Miss(command) | A cached read was served locally, or had to fetch from the server. |
TopologyChanged(masters) | The cluster's slot-owning master set changed (a failover, or scaling a shard in or out). |
Events omit command arguments and payloads. This keeps secrets such as AUTH credentials and user values out of listeners. Where an event carries node, it is Some for cluster and master-replica clients and None for a standalone client. A node that keeps failing to connect produces one ConnectFailed rather than one event per attempt.
Registering a listener
A SageListener has one synchronous, Unit-returning method. Match on the event you care about and forward it to your metrics system:
import sage.{SageEvent, SageListener}
import sage.SageEvent.*
val metrics = new SageListener {
def onEvent(event: SageEvent): Unit = event match {
case CommandCompleted(name, _, duration, _) => // record latency
case Connection.Disconnected(_) => // bump a gauge
case Connection.ConnectFailed(node, error) => // log the unreachable address
case Connection.ReconnectFailed(_, error) => // log the cause
case Cache.Hit(_) => // count a hit
case Cache.Miss(_) => // count a miss
case _ => ()
}
}
val config = SageConfig(
topology = Topology.Standalone(Endpoint("localhost", 6379)),
listeners = Vector(metrics)
)This listener example works on every backend.
Delivery guarantees
Listeners run separately from command execution. A slow or throwing listener may delay or lose events, but it does not affect commands.
WARNING
Delivery is best-effort: a thrown exception is swallowed, and events are dropped once the internal dispatch queue fills. Listeners suit metrics, sampling, and operational logging, not anything that must be a complete record.
Distributed tracing
Use a CommandTracer rather than a listener for distributed tracing. It runs while Sage processes the command, which makes each Redis span a child of the active span. A listener runs later and cannot preserve this parent-child relationship.
Set one on SageConfig.tracer. The sage-opentelemetry module provides an OpenTelemetry implementation:
"com.github.ghostdogpr" %% "sage-opentelemetry" % "0.3.1"import sage.opentelemetry.OpenTelemetryCommandTracer
val config = SageConfig(
topology = Topology.Standalone(Endpoint("localhost", 6379)),
// reads the globally-registered OpenTelemetry
tracer = Some(OpenTelemetryCommandTracer.global())
)It emits one CLIENT span per command, named for the command (GET, SET, ...), carrying db.system, db.operation.name, peer.service (default redis, configurable), component, and the server address; a failure sets an error status with the exception. The tracer records the command name but omits arguments and keys, keeping secrets and user values out of traces. Spans follow the ambient sampling decision.
Sage emits one span for each command sent to the server, including each command in a pipeline (cluster redirects remain part of the command's span). A cached read served locally does not produce a span. A cache miss is traced like any other command. In a transaction, Sage traces the watch-phase reads individually and creates one span named MULTI for the MULTI/EXEC body. This span represents the round trip, not whether the transaction committed.
The tracer reads the active span from Context.current() when a fiber submits a command. If an APM agent propagates that context, the Redis span becomes a child of the active request span without additional configuration. This works for ZIO under the Datadog Java agent; see the agent's documentation for its own setup.
Context on a fiber runtime without an agent
When you use the OpenTelemetry SDK without an agent, you need to configure context storage. On a fiber runtime, the active span lives in fiber-local state (a ZIO FiberRef or Cats Effect IOLocal), but the tracer reads the current OpenTelemetry context. Configure context storage so that Context.current() can see the active span:
- ZIO: with
zio-telemetry, wire OpenTelemetry through theOpenTelemetry.contextJVMandOpenTelemetry.globallayers (rather thanOpenTelemetry.contextZIOandOpenTelemetry.custom), which back tracing with OpenTelemetry's native context so the SDK reads the active span. See zio-telemetry's auto-instrumentation interop documentation. - cats-effect: with
otel4son Cats Effect 3.6+, add theotel4s-oteljava-context-storagedependency, enable thecats.effect.trackFiberContextsystem property, and provideIOLocalContextStorage.localProvider[IO]. This keeps the JavaContextand the otel4s fiber context aligned so the SDK reads the active span. Note that the stock OpenTelemetry Java agent does not keep Cats Effect context in sync; otel4s ships a dedicated agent distribution for the agent case.
OpenTelemetryCommandTracer.withContextProvider lets you supply a custom () => Context for a context source that is thread-local but non-default.