Instance events
Nexus is built to work on multiple instances/servers at the same time in a scaled out environment. You can have as many servers running jobs as you like and the system still guarantees that a single job will only run on a single instance at any given time while distributed queue processors can process different leased messages on every instance. Leases prevent simultaneous selection during the lease period; expired leases and retries can cause repeated processing.
In some cases the system needs to communicate with all running instances which is called instance events. At the heart of it is IInstanceEvent and IInstanceEventBroker.
IInstanceEvent represents an event that should be sent to either all or a specific instance. Specific implementations exist for cases such as broadcasting a request to cancel a job or notifying job hosts that queue messages may be ready.
Queue-readiness events are a low-latency nudge, not the durable record of work. Nexus first writes a coalesced activation through IQueueActivationStore, then broadcasts the event. If delivery fails, the activation remains pending and the job engine finds it during reconciliation. This means instance events can improve enqueue-to-start latency without becoming a single point of correctness. Activation storage is configured separately from the broker; process-local memory activation is not durable across restarts.
IInstanceEventBroker is the interface to use when you want to listen to or broadcast these events and it's up to the implementation of it to make sure that the events are sent to the correct instance(s).
Single instance mode
The single-server broker is suitable when only one instance is running. It uses a memory based solution to register listeners and broadcast events. You register it by calling this in your DI setup code:
builder.Services.AddNexus().AddSingleServerInstanceEvents();
Multi instance mode
If you have multiple instances running like in an auto scaled environment where there might be multiple servers jobs running you can't use the single instance mode. Use database polling, Azure Service Bus, RabbitMQ, Redis, Kafka, or a custom broker.
Both the database polling and Azure Service Bus implementation takes an option func that lets you specify BrokerType which is either InstanceEventBrokerType.ProduceAndConsumeEvents or InstanceEventBrokerType.ProduceEvents. For an application that only adds messages to Nexus queues using CommerceMind.Nexus without registering processing you can set the broker type to InstanceEventBrokerType.ProduceEvents. This means that Nexus will skip setting upp polling/subscriptions for that application.
builder.Services.AddNexus().AddDatabasePollingInstanceEvents(options =>
{
options.BrokerType = InstanceEventBrokerType.ProduceEvents;
});
// Or:
builder.Services.AddNexus().AddAzureServiceBusInstanceEvents(options =>
{
options.BrokerType = InstanceEventBrokerType.ProduceEvents;
});
If you're using the CommerceMind.Nexus.ApiClient package that enqueues messages over the HTTP API you don't need to configure instance events at all for that application.
Database polling
If you want to limit external dependencies you can use the database to store instance events and have your servers poll the database for new events. You configure database polling events like this:
builder.Services.AddNexus().AddDatabasePollingInstanceEvents(options =>
{
options.PollDelay = TimeSpan.FromSeconds(1); // Default polling is every second
});
This implementation will check if there's multiple running instances and then save a copy of the event for each running instance except for itself where it broadcasts the event directly. It knows which instances are running by using the INexusInstanceHeartbeatService. Read more about heartbeats here.
RabbitMQ
If you're using RabbitMQ you can use that as your instance event broken. Install the package CommerceMind.Nexus.RabbitMQ and configure the instance event broker:
builder.Services
.AddNexus()
.AddRabbitMq(options =>
{
options.UserName = "...";
options.Password = "...";
options.HostNames = ["localhost"];
// Or:
options.ConnectionString = new Uri("amqp://...");
})
.AddRabbitMqInstanceEvents(options =>
{
// This is the default value, you only need to change this if you're using
// multiple Nexus applications with the same RabbitMQ instance.
options.ExchangeName = "nexus_instance_events";
});
Azure Service Bus
If you're using Azure there's a NuGet package called CommerceMind.Nexus.Azure which uses a service bus topic to broadcast instance events.
After you've installed it you configure it like this in your Program.cs:
builder.Services.AddNexus().AddAzureServiceBusInstanceEvents();
The default name of the topic is nexus-instance-events but that can be configured by passing an options func like this:
builder.Services.AddNexus().AddAzureServiceBusInstanceEvents(options =>
{
options.TopicName = "my-topic-name";
});
By default Nexus will look for the connection string using IConfiguration.GetConnectionString("azureServiceBus") but you can add a loader func on the options object if you need to fetch it from somewhere else:
builder.Services.AddNexus().AddAzureServiceBusInstanceEvents(options =>
{
options.ConnectionStringLoader = (serviceProvider) => "Endpoint=sb://xxx.servicebus.windows.net/;...";
});
Nexus will automatically create topic subscriptions for each instance which works best in an auto scaled environment where you don't know how many instances you will have. In order to be able to create subscriptions on-the-fly you also need to configure an administration connection string. You do that either by setting the connection string azureServiceBusAdmin with IConfiguration or set the AdminConnectionStringLoader property on the options object.
If you're not using an auto scaled environment and you instead want to create the subscriptions yourself you can skip configuring the admin connection string and instead add SubscriptionNameLoader:
builder.Services.AddNexus().AddAzureServiceBusInstanceEvents(options =>
{
options.SubscriptionNameLoader = (serviceProvider) => "my-subscription";
});
Make sure that all instances use a different subscription when you're manually creating them. This setup only works if you have a number of dedicated machines such as VMs that are always running.
Batching events
By default instance events are batched so that events that happen within 0.5 seconds are sent as a single service bus message. This is done to limit the number of messages since Azure Service Bus has a cost per message. If a 0.5 second delay doesn't work for you it's possible to change the delay like this:
builder.Services.AddNexus().AddAzureServiceBusInstanceEvents(options =>
{
options.DebounceDelay = TimeSpan.FromSeconds(5);
options.MaxMessageBatchSize = 10; // The default is 10
});
The MaxMessageBatchSize is to guard against getting a message size that is bigger than what's allowed. So if there's more than 10 events within 0.5 seconds they will get sent as mutiple messages. If you're using the standard tier service bus the limit is 256 kB but if you're using the premium tier the limit is 100 MB.
Redis
If you're using Redis you can use that as your instance event broker. Install the package CommerceMind.Nexus.Redis and configure the instance event broker:
builder.Services
.AddNexus()
.AddRedisInstanceEvents(options =>
{
options.ConnectionString = "localhost:6379";
});
If you already have an IConnectionMultiplexer registered in DI you can reuse it instead of providing a connection string:
builder.Services
.AddNexus()
.AddRedisInstanceEvents(options =>
{
options.ConnectionMultiplexerFactory = sp => sp.GetRequiredService<IConnectionMultiplexer>();
});
The default pub/sub channel name is nexus_instance_events but that can be changed if you're using the same Redis instance for multiple Nexus applications:
builder.Services
.AddNexus()
.AddRedisInstanceEvents(options =>
{
options.ConnectionString = "localhost:6379";
options.ChannelName = "my-app-instance-events";
});
Redis uses pub/sub for broadcasting, so all subscribers on the channel receive every message — no per-instance group or subscription configuration is needed.
Kafka
If you're using Kafka you can use that as your instance event broker. Install the package CommerceMind.Nexus.Kafka and configure both the Kafka connection and the instance event broker:
builder.Services
.AddNexus()
.AddKafka(options =>
{
options.ConsumerConfig.BootstrapServers = "localhost:9092";
options.ConsumerConfig.GroupId = "my-nexus-app";
options.ProducerConfig.BootstrapServers = "localhost:9092";
})
.AddKafkaInstanceEvents();
The default name of the topic is nexus_instance_events but that can be configured:
builder.Services
.AddNexus()
.AddKafka(options => { ... })
.AddKafkaInstanceEvents(options =>
{
// Change this if you're using the same Kafka instance for multiple Nexus applications.
options.TopicName = "my-topic-name";
});
If you need separate producer/consumer configuration specifically for instance events you can override them via the options:
builder.Services
.AddNexus()
.AddKafka(options => { ... })
.AddKafkaInstanceEvents(options =>
{
options.ProducerConfig = new ProducerConfig { BootstrapServers = "localhost:9092" };
options.ConsumerConfig = new ConsumerConfig
{
BootstrapServers = "localhost:9092",
GroupId = "my-nexus-app"
};
});
Because Kafka delivers messages to only one consumer per consumer group, every instance must use a unique GroupId in order to receive all broadcast events. A common pattern is to include the hostname or a random ID in the group ID:
builder.Services
.AddNexus()
.AddKafka(options =>
{
options.ConsumerConfig.BootstrapServers = "localhost:9092";
options.ConsumerConfig.GroupId = $"nexus-instance-events-{Environment.MachineName}";
options.ProducerConfig.BootstrapServers = "localhost:9092";
})
.AddKafkaInstanceEvents();
Custom application instance events
You can use the Nexus instance event infrastructure to broadcast events within your application. To do this, define your own event class that implements the IApplicationInstanceEvent interface and define handlers that implement the IApplicationInstanceEventHandler<TEvent> interface. You can then use the CommerceMind.Nexus.Abstractions.Core.InstanceEvents.IApplicationInstanceEventPublisher to publish events that will be broadcast to all running instances.
Eg:
public class MyEvent : IApplicationInstanceEvent
{
public string? MyProperty { get; set; }
}
public class MyEventHandler : IApplicationInstanceEventHandler<MyEvent>
{
public async Task HandleAsync(MyEvent myEvent)
{
// Do something interesting with the event
}
}
builder.Services.AddScoped<IApplicationInstanceEventHandler<MyEvent>, MyEventHandler>();
var app = builder.Build();
await using var scope = app.Services.CreateAsyncScope();
var publisher = scope.ServiceProvider.GetRequiredService<IApplicationInstanceEventPublisher>();
await publisher.PublishAsync(new MyEvent());
Application event handlers run in a dependency injection scope and can use scoped services. These events use the configured broker's delivery semantics; they are notifications, not a durable application work queue. Use a Nexus queue when the work must be persisted and retried.
Subscribing to instance events
Use IInstanceEventBroker.Subscribe(Func<IInstanceEvent, Task>) for infrastructure subscriptions and dispose the returned IInstanceEventSubscription when the subscriber is disposed. Handlers are asynchronous. A long-lived subscriber should create an operation scope when it needs scoped Nexus/application services instead of capturing them in the broker.
Writing your own broker
If you don't feel like any of the above options works for you then you can always create your own implementation of IInstanceEventBroker.
An important thing to think about is that IInstanceEvent has an InstanceId property that can be null or have a value. If it's null then the event should be sent to all instances. But if it's not null it should only be published as an event on the instance where INexusInstance.InstanceId is the same as IInstanceEvent.InstanceId.
A custom broker can receive a broadcast on every instance and filter by InstanceId before dispatching it. Implement Subscribe() and await asynchronous handlers. The InstanceEventSubscriptions helper manages handler registration, dispatch, and disposable subscriptions:
private readonly InstanceEventSubscriptions _subscriptions = new();
public IInstanceEventSubscription Subscribe(Func<IInstanceEvent, Task> handler)
=> _subscriptions.Subscribe(handler);
// In the transport receive path, after filtering the target instance:
await _subscriptions.PublishAsync(instanceEvent);
The helper handles local subscribers; the broker still owns serialization, transport, instance targeting, and shutdown. Do not settle a transport notification before required asynchronous handling has completed.
When you're done with the implementation all you need to do is register it like this:
builder.Services.AddSingleton<IInstanceEventBroker, MyInstanceEventBroker>();