3

I have RabbitMQ producer and consumer written in PHP (Symfony 4). Consumer is working as custom symfony 4 command along with bundle php-amqplib/rabbitmq-bundle

Here is issue. I want to be able to set consumer to listen and consume tasks instantly after they appear in queue.

I tried to run it as one-shot systemd service but it's not working wery well.

Systemd:

[Unit]
Description=consumer for rabbitmq

[Service]
Type=oneshot
ExecStart=/bin/sh /var/www/public/rabbit.sh
ExecStop=/usr/bin/pkill -f "rabbitmq:consumer"
RemainAfterExit=yes
StandardOutput=journal

[Install]
WantedBy=multi-user.target

If there is better solution than writing daemon please let me know. I just want to be able to set consumer to listen and consume task instantly after it apear in queue.

Improve this question
6
  • You could run the batch consumer command rabbitmq:batch:consumer in the background as daemon, maybe set it to quit after a memory limit is reached and have supervisord automatically restart the deamon after that. When you do this, you will always have a consumer running in the background for new messages. Commented Feb 27, 2019 at 12:14
  • 1
    "I want to be able to set consumer to listen and consume tasks instantly after they appear in queue" - Unless I am missing something here but the consumer command already works that way! e.g. When you run bin/console rabbitmq:consumer -m 5 your_queue the relevant consumer will consume messages as soon as they come in. Check this for supervisor controller RabbitMQ consumers. And inanzzz.com/index.php/posts/rabbitmq Commented Feb 27, 2019 at 23:16
  • @BentCoder but in your case consumer consumes 5 messages and dies, I want it to consume infinite ammount of messages and run it with startup of my system Commented Feb 28, 2019 at 6:47
  • That's why you need to use supervisor. PHP is not the best when it comes to long running processes hence reason run your commands for limited amount numbers/jobs - e.g. 100. When the command dies, supervisor will start it again which is what exactly you need. This is how things are done. All you are goinv to do is, copy and paste job from the first link I've sent you. Commented Feb 28, 2019 at 8:04
  • @BentCoder thank you for help, supervisor is working very well for me :D Commented Feb 28, 2019 at 10:42

1 Answer 1

Reset to default
6

The solution to your "issue" is simple. All you are going to do is, make use of supervisord which will watch your command behind the scene and bring it back up if it goes down.

PHP is not good at long running processes so you want to keep your consumer/worker to consume reasonably enough (not many) messages. e.g. 100 to 200 is good enough.

This is what you are going to do:

  1. Create a supervisor config file for your command - check example below. If you wish go to the doc and read what exactly the properties below do.

  2. Enable this config within the supervisor.

That's all!

[program:name-of-your-command]
command=php bin/console rabbitmq:consumer -m 100 your_queue --env=prod -DFOREGROUND # Your consumer command
directory=/path/to/your/app
autostart=true
autorestart=true
startretries=5
startsecs=0
user=deployer # Your user
numprocs=1 # This tells supervisor to run only one consumer
process_name=%(program_name)s_%(process_num)02d
stderr_logfile=/path/to/your/app/var/logs/%(program_name)s_stderr.log
stderr_logfile_maxbytes=10MB
stdout_logfile=/path/to/your/app/var/logs/%(program_name)s_stdout.log
stdout_logfile_maxbytes=10MB

Examples:

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

This is very useful, thanks. Do you mind if I ask what the -DFOREGROUND does? I haven't seen that before.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.