Configuration

Sharding Configuration

Here's the list of things you can configure on the pod side:

• role: the role of the current pod (defaults to Role.default)

Pod Roles

A Role is a tag (a simple wrapper around a String name) attached to each pod. Pods with the same role share a pool of shards, while pods with different roles are managed independently by the Shard Manager (each role has its own set of shards, its own rebalancing, and its own assignment state).

This is useful when you want to dedicate certain pods to certain entity types (e.g. a role for "game" pods and a role for "matchmaking" pods), or to scale parts of your system independently.

If you don't need this, just leave the default value: all pods will share the default role.

Note: a pod can only have one role, and a pod can't change its role while it's registered. Trying to re-register an existing pod with a different role will be rejected by the Shard Manager.

• numberOfShards: number of shards (for the pod's role)

How to choose the number of shards?

numberOfShards is used to calculate the shard ID from the entity ID. For that reason, it should be the same on all pods of a given role, and can not be changed while the app is running.

If this value is lower than the number of pods, some pods will have no shards (and host no entities), which is inefficient. If this value is too low, a difference of 1 shard between 2 pods will be quite significant and introduce some unbalance between the number of entities hosted on each pod. On the other hand, if there are too many shards, each pod will have a lot of shards, which will produce an unnecessary overhead.

A good rule of thumb is to set the number of shards to 10x the maximum number of pods that you expect to have.

• selfHost: hostname or IP address of the current pod
• shardingPort: port used for pods to communicate together
• shardManagerHost: hostname or IP address of the Shard Manager
• shardManagerPort: port of the Shard Manager gRPC API
• shardManagerTimeout: timeout for calls to the Shard Manager
• serverVersion: version of the current pod

What's the version?

When performing a rolling update (upgrading all pods one by one without downtime), we want to avoid assigning shards to pods that will be stopped soon. Ideally, we want to move each shard only once during the whole process.

This serverVersion allows the Shard Manager to know which pods are old and which are new. It will then pick the new pods to assign shards.

• entityMaxIdleTime: time of inactivity (without processing any message) after which an entity will be stopped. It can be overridden for each entity type with EntityOptions.
• entityTerminationTimeout: time we give to an entity to finish processing its messages when it is stopped. After that, the messages still running are interrupted and their senders receive EntityInterrupted. Rebalances wait for this, so keep it below your pods' termination grace period.
• sendTimeout: default timeout when sending a message, after which the sender receives SendTimeout. It can be overridden for each messenger.
• refreshAssignmentsRetryInterval: retry interval in case of failure getting shard assignments from storage
• unhealthyPodReportInterval: interval to report unhealthy pods to the Shard Manager (this exists to prevent calling the Shard Manager for each failed message)
• simulateRemotePods: disable optimizations when sending a message to an entity hosted on the local shards (this will force serialization of all messages)
• unregisterRetrySchedule: retry schedule for unregistering the pod from the Shard Manager when it stops
• entityInterceptor: an EntityInterceptor[AnyMessage] applied to the entities of every type, around the interceptor of each type (see Interceptors). It is a no-op by default.

Shard Manager Configuration

Here's the list of things you can configure on the Shard Manager side:

• numberOfShards: a function Role => Int returning the number of shards for a given role (see above)

Why a function?

Because each role has its own pool of shards, you can pick a different number of shards for each role. If you only use the default role (or want the same number of shards everywhere), you can simply pass _ => 300 (or any constant).

• apiPort: port to expose the gRPC API
• rebalanceInterval: interval for regular rebalancing of shards
• rebalanceRetryInterval: retry interval for rebalancing when some shards failed to be rebalanced
• pingTimeout: time to wait for a pod to respond to a ping request
• persistRetryInterval: retry interval for persistence of pods and shard assignments
• persistRetryCount: max retry count for persistence of pods and shard assignments
• rebalanceRate: max ratio of shards to rebalance in a single iteration
• podHealthCheckInterval: interval for checking pod health

Rebalance Rate

The rebalance rate is there to prevent too many shards being assigned immediately to new pods. Instead of reaching a perfect spread right away, we can use several iterations to make sure new pods are able to handle the new shards. This is particularly useful if starting entities has a performance cost (e.g. loading state) and we don't want to start too many at once.

When a pod is leaving, its shards need to be immediately rebalanced so the ratio is not used in that case.

gRPC Configuration

When using shard-protocol-grpc, both pods and the Shard Manager require a GrpcConfig (GrpcConfig.default uses the defaults below):

• maxInboundMessageSize: maximum size of a message received by the gRPC clients and servers (32 MB by default)
• executor: a custom executor given to grpc-java for its clients and servers (none by default)
• shutdownTimeout: time to wait for the gRPC server and channels to stop before forcing them (3 seconds by default)
• streamingPrefetch: number of messages fetched ahead of the consumer in streaming calls (16 by default)
• clientInterceptors: grpc-java ClientInterceptors applied to the calls made by pods and by the Shard Manager, e.g. to add headers for tracing. ShardingClientInterceptor.headersUpdater builds one that updates the headers of each call.
• serverInterceptors: ShardingServerInterceptors applied to the calls received by pods and by the Shard Manager, e.g. for logging or authentication. ShardingServerInterceptor.beforeEach builds one that runs an effect before each call.