How to run only one job at time in same laravel queue?

Question

I have saas service which working with API. It has limits so I need that one user account doing only one request at the same time.

For this I queuing with OnQueue($user->name);

then in handle() doing job...

I need only one job can be run in users queue. At the same time may be run only diffent queues 1 job per 1 queue.

Im using redis connection.

This my job class:

 public function __construct(Accounts $acc)
    {
        $this->acc = $acc;
        $this->ownjob = $acc->prepareJobQueue();
    }

    public function handle()
    {
        $acc = $this->acc;
        $job = $this->ownjob;

        $api = new Api([
            'login'     => $acc->login,
            'password'  => $acc->password,
        ]);
        if ($api->checkLogin()) {
            info("{$acc->login} OK Authorized");
            foreach ($job['queue'] as $term) {
                switch($term['type']) {
                    case 'hashtag':
                        info("{$acc->login} Queuing: type - {$term['type']}, value - {$term['value']}");
                        $hashtag = Hashtags::where('cha_name',$term['value'])->first();
                        $answer = $api->getUsersByHashtag($hashtag,50);
                        break;
                    case 'concurency':
                        info("{$acc->login} Queuing: type - {$term['type']}, value - {$term['value']}");
                        $soc_user = Users::where('soc_unique_id',$term['value'])->first();
                        $answer = $api->getUserFollowers($soc_user);
                        break;
                    default:
                        break;
                }
            }
        } else {
            info("{$acc->login} NOT Authorized - STOP JOB");
        }

    }

This is how I dispatching job:

$accounts = Accounts::select(['id', 'login', 'hashtag_filter', 'concurency_filter'])->whereNotNull('hashtag_filter')->get();
        foreach ($accounts as $acc) {
            doFollowing::dispatch($acc)->onQueue($acc->login);
        }

Choraimy · Accepted Answer · 2020-03-23 14:30:34Z

3

You could use Laravel's builtin rate limiting for this (note that it does require Redis).

Inside your job:

Redis::funnel('process-name')->limit(1)->then(function () {
    // Your job logic here
});

Note that if you don't provide a second callback to then, it will throw an exception if it cannot obtain the lock (which will cause your job to fail)

If you're using Laravel 6 or higher you could also opt to do this in a job middleware, rather than in the job itself (handy if you have multiple jobs that share the same lock)

More info on Laravel's rate limiting: https://laravel.com/docs/5.8/queues#rate-limiting

answered Mar 23, 2020 at 14:30

Choraimy

1762 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

xyz · Accepted Answer · 2020-01-03 10:22:17Z

2

Use mxl/laravel-queue-rate-limit Composer package.

It enables you to rate limit Laravel jobs on specific queue without using A third party driver such as Redis.

Install it with:

$ composer require mxl/laravel-queue-rate-limit:^1.0

This package is compatible with Laravel 5.5+ and uses [auto-discovery][1] feature to add MichaelLedin\LaravelQueueRateLimit\QueueServiceProvider::class to providers.
Add rate limit (x number of jobs per y seconds) settings to config/queue.php:
```
'rateLimit' => [
    'mail' => [
        'allows' => 1, // number of jobs
        'every' => 5 // time interval in seconds
    ]
]
```
These settings allow to run 1 job every 5 seconds on mail queue. Make sure that default queue driver (default property in config/queue.php) is set to any value except sync.
Run queue worker with --queue mail option:
```
$ php artisan queue:work --queue mail
```
You can run worker on multiple queues, but only queues referenced in rateLimit setting will be rate limited:
```
$ php artisan queue:work --queue mail,default
```
Jobs on default queue will be executed without rate limiting.

Queue some jobs to test rate limiting:

SomeJob::dispatch()->onQueue('mail');
SomeJob::dispatch()->onQueue('mail');
SomeJob::dispatch()->onQueue('mail');
SomeJob::dispatch();

edited Jan 3, 2020 at 10:22

xyz

5783 silver badges12 bronze badges

answered Jan 3, 2020 at 10:18

Sapnesh Naik

11.7k9 gold badges70 silver badges103 bronze badges

1 Comment

rst630 Over a year ago

I have dynamic queue naming as it's queue per user - I can't always modyfing config for each user. Also Im asking not about limit per time, I want simultaneously run only one job in same queue - does't matter it will be 1 per minute or 100

xyz · Accepted Answer · 2020-01-03 10:15:05Z

1

You could limit numprocs per queue in your Supervisor or Horizon setup.

If you only spawn one queue worker per user I believe you will get your desired behaviour.

answered Jan 3, 2020 at 10:15

xyz

5783 silver badges12 bronze badges

1 Comment

rst630 Over a year ago

It will good work when users is ~100, but what if users 10k? Also as I said - queue name is user name. In your variant I need always modyfing supervisor config?

Collectives™ on Stack Overflow

How to run only one job at time in same laravel queue?

3 Answers 3

Comments

Use mxl/laravel-queue-rate-limit Composer package.

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Use mxl/laravel-queue-rate-limit Composer package.

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related