We are using a RabbitMQ worker (https://github.com/php-enqueue/enqueue-dev/tree/master) on our Laravel 11 system to process job batches. At the end of each job, it checks if any more jobs need adding to the batch. If so, it adds then, if not, the batch then/finally hooks get run.
The queue workers are set to restart every X number of jobs processed using this service provider:
$this->app->extend('queue.worker', function ($worker, $app) {
$extensions = [
new SignalExtension,
new LoggerExtension(Log::getLogger()),
];
$maxJobs = (int) config('queue.connections.rabbitmq.restart_after_jobs');
if ($maxJobs > 0) {
Log::info("Automatic restart after jobs provided via RABBITMQ_RESTART_AFTER_JOBS: restarting every $maxJobs jobs.");
$extensions[] = new LimitConsumedMessagesExtension($maxJobs);
} else {
Log::warning('NO automatic after jobs provided via RABBITMQ_RESTART_AFTER_JOBS');
}
return (new Worker(
$app['queue'],
$app['events'],
$app[ExceptionHandler::class],
function () use ($app) {
return $app->isDownForMaintenance();
}
))->setExtensions($extensions);
});
Often when this happens we see some odd behaviour. The next step will be added to the batch (as far as we can see in the batches table in the db), but no other jobs from that batch will ever be processed.
This is an example of the logging we see:
2025-08-12 07:40:36.452 new batch made 9f9d54ef-357f-4b3f-bb4a-d4b239fbc788
2025-08-12 07:40:36.561 new job added to batch 9f9d54ef-357f-4b3f-bb4a-d4b239fbc788
2025-08-12 07:40:36.563 [Job] Acknowledging message.
2025-08-12 07:40:36.564 [Queue] Pushing new job
2025-08-12 07:40:36.566 [Worker] Job result processed {"job_id":null,"result":"enqueue.already_acknowledged"}
2025-08-12 07:40:36.567 [Worker] Finished processing job.
2025-08-12 07:40:36.568 [SignalExtension] Interrupt execution
Part of the complexity is that we run different jobs on different queues, I'm not sure if that's having any effect. We are using supervisor to control the queue workers:
[supervisord]
nodaemon=true
user=root
logfile=/dev/stdout
logfile_maxbytes = 0
pidfile=/var/run/supervisord.pid
[program:core-worker]
command=/usr/local/bin/php -d variables_order=EGPCS /var/www/html/artisan queue:work --backoff=0 --max-jobs=0 --memory=256 --sleep=3 --timeout=21600 --tries=1 --rest=0
process_name=%(program_name)s_%(process_num)02d
numprocs=15
stopwaitsecs=21600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
stderr_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
The Laravel queue config looks like this:
'rabbitmq' => [
'driver' => 'interop',
'dsn' => 'amqp+rabbitmq://' . env('RABBITMQ_USER') . ':' . env('RABBITMQ_PASSWORD') . '@' . env('RABBITMQ_HOST', 'rabbitmq') . ':' . env('RABBITMQ_PORT', 5672) . '/' . env('RABBITMQ_VHOST', '/') . '?heartbeat=' . env('RABBITMQ_HEARTBEAT', 60),
'queue' => env('REDIS_QUEUE', env('APP_DOMAIN', 'default')),
'restart_after_seconds' => env('RABBITMQ_RESTART_AFTER_SECONDS', 0),
'restart_after_jobs' => env('RABBITMQ_RESTART_AFTER_JOBS', 150),
'restart_after_memory' => env('RABBITMQ_RESTART_AFTER_MEMORY', 0),
],
Here is where we set the restart_after_jobs value instead of supervisor.
We've increased the stopwaitsecs but that doesn't seem to have helped. It's not a consistent problem too, sometimes the queue workers stop and the batch jobs get executed fine by other workers, sometimes not. We expect the next job in the batch to be picked up by another worker and executed.