Cluster Scaling

Cluster mode lets multiple queue:autoscale managers run against the same queues.

Requirements

• Redis must be available to every replica.
• All replicas must share the same Laravel app config and queue backend.
• Enable cluster mode with QUEUE_AUTOSCALE_CLUSTER_ENABLED=true.

If you do not enable cluster mode, Queue Autoscale keeps its single-host behavior and does not require Redis for coordination. In that mode, the default auto signal backends resolve to null/no-op implementations so the manager can run cleanly without Redis.

Managers join the cluster automatically. There is no manual cluster ID, seed list, or host registration step.

Only one queue:autoscale process is allowed per app per host. Starting a second process on the same host will fail fast unless you use queue:autoscale --replace to hand over the local host lock cleanly.

If you must override the generated node identity, set QUEUE_AUTOSCALE_MANAGER_ID. Do that only when you are certain the value is unique per host/pod/node for that app.

If you explicitly force QUEUE_AUTOSCALE_PICKUP_TIME_STORE=redis or QUEUE_AUTOSCALE_SPAWN_LATENCY_TRACKER=redis, those signal backends also require Redis even outside cluster mode.

How it works

Every manager:

• Publishes a Redis heartbeat with its host name, current worker counts, CPU (including fractional core counts from cgroup limits), memory, and local worker capacity.
• Reads the latest per-host worker recommendation from Redis.
• Reconciles only its own local workers to that recommendation.

The elected leader:

• Evaluates queue and group metrics for the whole cluster.
• Computes total target workers per workload.
• Distributes those targets across active hosts.
• Publishes a cluster summary with leader, managers, workloads, capacity, and a host scaling signal.

Inspecting the cluster

php artisan queue:autoscale:cluster
php artisan queue:autoscale:cluster --json

The JSON output is intended for monitoring integrations and web UIs.

Monitoring API

The package exposes the latest cluster snapshot and flattened metrics through the main service / facade:

use Cbox\LaravelQueueAutoscale\Facades\LaravelQueueAutoscale;

$cluster = LaravelQueueAutoscale::cluster();
$metrics = LaravelQueueAutoscale::clusterMetrics();

cluster() returns the latest Redis-backed cluster summary.

clusterMetrics() returns flattened metrics suitable for Prometheus exporters, dashboards, and downstream monitoring packages such as cboxdk/laravel-queue-monitor.

Host scaling signal

The cluster summary includes:

• scale_signal.action
• scale_signal.reason
• scale_signal.current_hosts
• scale_signal.recommended_hosts

Every time the leader publishes a summary it dispatches two events, in this order:

use Cbox\LaravelQueueAutoscale\Events\ClusterScalingSignalUpdated;
use Cbox\LaravelQueueAutoscale\Events\ClusterSummaryPublished;
use Illuminate\Support\Facades\Event;

Event::listen(function (ClusterSummaryPublished $event) {
    $event->clusterId;    // string
    $event->leaderId;     // string
    $event->summary;      // array<string, mixed> — the full summary
    $event->publishedAt;  // int timestamp
});

Event::listen(function (ClusterScalingSignalUpdated $event) {
    $event->clusterId;
    $event->leaderId;
    $event->currentHosts;      // int
    $event->recommendedHosts;  // int
    $event->currentCapacity;   // int — cluster-wide worker capacity
    $event->requiredWorkers;   // int — cluster-wide demand
    $event->action;            // string, 'hold' when the summary carries none
    $event->reason;            // string
});

Two more cluster events fire from every manager, not just the leader:

• ClusterLeaderChanged — clusterId, previousLeaderId, currentLeaderId, observedByManagerId, changedAt
• ClusterManagerPresenceChanged — clusterId, managerIds, addedManagerIds, removedManagerIds, leaderId, observedByManagerId, observedAt

Telemetry

With cboxdk/laravel-telemetry installed, cluster state is also exposed as observable gauges evaluated at scrape time from the Redis summary: queue_autoscale.cluster.managers, .workers, .required_workers, .capacity, .utilization, .recommended_hosts, plus .host.workers and .host.capacity labelled by host. These report nothing in single-host mode. See Monitoring → Telemetry integration.