765 questions
1
vote
1
answer
95
views
Why is the worker's onmessage executing after a macro task?
Event loop execution order
I'm studying how the Javascript event loop works for a future presentation, but I've come across an unexpected behavior with the information I have so far. Simply put, when ...
- 13
1
vote
0
answers
32
views
MongoDB and celery configuration
I am new to celery and there isn't alot of great documentation to follow esepcially with mongodb.
I am essentially trying to have celery post messages to two different collections based on where ...
0
votes
1
answer
142
views
Task Queue: Release all messages at once to a single consumer when threshold is met
I'm looking for a task queue that supports pushing tasks into a queue and only releasing them to a single consumer at a time when a threshold number of messages is reached. When released to the ...
- 53
0
votes
1
answer
53
views
Any HTTP/HTTPs supported Messaging or Queuing system available?
Friends,
I am looking for Messaging/Queuing service, preferably opensource that supports Queuing-Dequeuing over HTTP(its a must). I know there are a lot of Cloud based services(AWS SQS, Azure EventHub)...
- 337
0
votes
1
answer
138
views
How to Implement Conditional Task Execution in Celery from API Registration?
I’m building an API that registers tasks to Celery, and I want to implement a conditional workflow based on the results of the tasks. Specifically, I need to execute Task B if Task A returns a certain ...
0
votes
2
answers
100
views
Job scheduling and deduplicating events in an in-memory task queue pattern with .NET 8
I have a database of messages that I want to send. Each has a unique message identifier and is placed in the database by another process. The only communication between the two processes is via the ...
- 697
-4
votes
2
answers
178
views
Async operation like fetch, async/await are executed after syncronous code is executed then why microtask queue is given more priority?
micro task Queue consist of callbacks/resolved functions from fetch, async/await. but why its given more priority then callback queue because sync code is executed first and then afterward async ...
1
vote
0
answers
289
views
Is the Bull queue in NestJS the best solution for this situation? (best practice)
I have a back-end built with NestJS and MongoDB, where I'm utilizing microservices to communicate with other services.
I have a POST endpoint that handles complex logic:
First, I retrieve data from ...
- 11
-3
votes
1
answer
48
views
Why does 1 is printed before 4 in this execution of js code?
async function check() {
await Promise.resolve(console.log(1));
console.log(2);
}
console.log(3);
check();
console.log(4);
The answer is 3,1,4,2
As I know the async function will go to web API to ...
0
votes
0
answers
89
views
FreeRTOS: Simple Queue program, values of Queue are not being printed on Serial Monitor
I'm using FreeRTOS on the Adafruit HUZZAH32 – ESP32 Feather Board, and programming on the Arduino IDE.
The goal of this simple Queue program is to create a Queue, populate it sending values of an ...
- 8,899
0
votes
1
answer
165
views
Task Queue with cached libraries in Django
I am creating a WEB interface for various Python scripts through Django.
Example in calculation.py I would have :
import datetime
def add_time(a, b):
return = a + b + int(datetime.datetime.now())
...
- 368
0
votes
0
answers
741
views
Celery Alternatives based on DAG
I am building a product that uses Kubernetes. Now a user can submit requests to this cluster, which kicks off a bunch of potentially long-running tasks. I initially had intended to use Celery for this ...
- 133
1
vote
1
answer
200
views
Kubernetes: Infinite Workload Using One Pod Per Task
I’m trying to create something similar to Lambda functions on kubernetes.
I need to run each request on a separate pod. Once the container terminates successfully, the response is returned and the pod ...
- 11
2
votes
0
answers
166
views
Where does the concept of macrotasks come from in the browser? [closed]
In many articles about the browser's event loop, the concept of macrotasks and microtasks is mentioned. But I looked at the description of event loops in the HTML spec, where there is no concept of ...
- 21
0
votes
1
answer
500
views
Is execution of main thread javascript code a macro-task?
I was reading this blog about event loop and micro/macrotask queues in javascript but I have doubts about the statements from the blog:
In JavaScript, no code is allowed to execute until an event has ...
- 11
0
votes
2
answers
446
views
Creating a queue using two stack, but make enqueuing O(1)
Is there a way to implement a queue using two stacks, but with enqueuing 0(1)?
I have been learning how to do a queue and one of the questions that I've been trying to find an answer for is how can I ...
0
votes
1
answer
191
views
How can I get the worker queues for the logged in agent on Twilio Flex
I need to get the Queues assigned to a logged in Agent on Twilio Flex within a Flex Plugin.
I want to add a select box to let the agent select the outbound queue used for calling users, but I'm not ...
- 433
-1
votes
1
answer
120
views
Distributed System: ML Queue
I want to design a distributed queue for a machine learning inference system (using kubernetes). The problem is the following:
Each request also contains a model_id the request should be executed with ...
- 1
5
votes
1
answer
2k
views
Celery Tasks not Running on Mac M2 (Process 'ForkPoolWorker' exited with 'signal 11 (SIGSEGV)')
I am encountering an issue while trying to run Celery tasks on my Mac M1 machine. The error message I'm getting is as follows:
The process has forked and you cannot use this CoreFoundation ...
- 376
0
votes
1
answer
128
views
How do big companies like github queue their emails
I have recently been working on a company where we were sending emails to users, and we had a queue named email, which was specifically used to queue emails in the system. But this was taking too long ...
1
vote
1
answer
435
views
How to get the queued tasks (not yet consumed messages) when using Apache Pulsar as a distributed task queue?
I am currently evaluating the suitability of Apache Pulsar for building a distributed task queue using a "shared" subscription.
One required feature is to get all queued tasks for certain ...
- 11
0
votes
1
answer
74
views
How to get the currently running tasks when using Apache Pulsar as a distributed task queue?
I am currently evaluating the suitability of Apache Pulsar for building a distributed task queue using a "shared" subscription.
One required feature is to get all currently running tasks for ...
- 11
0
votes
1
answer
136
views
How to limit the number of concurrently processed tasks when using Apache Pulsar as a distributed task queue?
I am currently evaluating the suitability of Apache Pulsar to build a distributed task queue using a "shared" subscription.
One required feature is to limit the number of concurrently ...
- 11
0
votes
1
answer
421
views
How to Implement a Sequential Task Queue in Firebase Without Parallel Processing Issues?
I'm new to Firebase and trying to set up a task queue. My objective is to read requests with a 'pending' status from a datastore queue, send an HTTP request to an external third-party server, and then ...
- 1
0
votes
0
answers
71
views
How to implement Live-chat queue management system?
I have cases where I have to implement a queue service for Live-Chat application.
I have operators to support clients via live chat, they communicate via websocket, but I need to define an operator ...
- 31
0
votes
1
answer
131
views
when retrive result of a task with "reply_to" it result tuple. and says " Received and deleted unknown message. Wrong destination?!?"
i am using celery and RabbitMq
create a simple task and send results of the task to the "result" queue
when i connect to this que to processing on the result of the task it say " ...
0
votes
1
answer
155
views
Read data every second cloud function firestore and update data one by one for multiple data
What best choice for read data every second in firestore cloud function and update it continuously if there is data exist with queue.
I am use pubsub in minute for trigger run data in background then ...
- 49
0
votes
1
answer
125
views
Execution order question when processing JavaScript asynchronous functions (promise)
I'd like to know the order in which the JavaScript code below is executed and why. In particular, I would like to learn more about the contents of the code that goes into the microtask queue, focusing ...
- 43
1
vote
0
answers
177
views
Job name is not returning correctly in queue.add() in Bull and Express
I am trying out add queue for several asynchronous requests. And I added few queues properly, but one issue is not clear to me,
On adding new job by queuename.add(someDATA) the response is giving the ...
- 680
1
vote
1
answer
262
views
Can element in JOB QUEUE be completed after element in TASK QUEUE?
community! I've been recently discovering how JavaScript works and came across interesting moment with order of elements being executed in Job Queue and Task Queue.
As we all know, Job Queue has ...
- 13
0
votes
1
answer
70
views
Queue job for multiple email id not working
I try to send multiple email in queue laravel but its not working working single in single email, in my controller
$mail_id = Email::select('email')->whereNotNull('news')->get();
$...
- 39
0
votes
2
answers
433
views
Firebase tasks pass function as queue payload
I'm trying to create a generic task queue in firebase.
Following docs I'm supposed to create a new queue for each function I need but I want to create a generic queue that will accept any function.
I'...
- 1,818
2
votes
0
answers
105
views
A celery task called twice modifying one object
I'm experimenting with celery to understand how it works. I've written a simple worker, which modifies an array:
from .celery import app
import time
arr = []
status = 'waiting'
@app.task
def ...
- 740
4
votes
1
answer
966
views
How to Use Heroku Background Workers with NestJS and Bull?
What is the recommended way of providing Heroku workers for heavy processes that I want running on my queue using NestJS?
I have an HTTP server running on Heroku that executes certain time-consuming ...
- 410
0
votes
1
answer
1k
views
How can I make apply_async to accept messagegroupid parameter for sqs fifo queue
the problem I am having is that celery's apply_async is not delivering a required keyword argument, MessageGroupId, to sendmessage method of boto3 to publish message to sqs fifo queue.
I have been ...
0
votes
0
answers
319
views
Suspend docker-compose container shutdown until result of a condition is true?
I have a docker-compose.yml file that defines a number of services. One is a redis instance, and another is a queue-worker.
The queue-worker fetches jobs from redis and performs the necessary work.
...
- 1,401
0
votes
1
answer
565
views
Is it possible to enqueue an instance method, as opposed to a function, using Python-RQ?
The examples provided in the Python-RQ documentation consistently show functions being enqueued, using queue.enqueue(). For example:
job = q.enqueue(count_words_at_url, 'http://nvie.com')
Is it ...
- 1,401
2
votes
2
answers
965
views
Message Queue VS Task Queue architecture
I want to build a product that can perform some Internet scans (in Python) to collect various kinds of data.
I want to design it with tasks that perform these collecting jobs.
There can be multiple ...
- 31
0
votes
1
answer
253
views
taskqueue is not working google app engine
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/layers/google.python....
- 214
0
votes
1
answer
603
views
What if i say ” sagemaker async is nothing but a task queue which use ML type instances.”
If we run All the inference code within the celery task or another task queue like rabbit MQ, we will get the same performance ( if we ignore the impact of ML type instance ).
What are the major ...
- 75
0
votes
1
answer
67
views
clearTimeout doesn't clear timer, it seems too late to clear it
I'm trying to console.log the input value when the keyup event get fired, but similar to debounce implementation, from all posible events happened within a range of 1000 miliseconds I want the last ...
- 861
0
votes
1
answer
790
views
How does npm bull reads delayed jobs without using cron?
I was recently looking at npm bull package to use for delayed jobs. I can see that for delayed jobs it is using redis sorted set and mapping the value of the job to a hashset in redis. But I fail to ...
- 2,062
0
votes
1
answer
69
views
Does rabbitmq have operations log?
I need to see what operations are happening inside rabbitmq similar to redis-cli monitor or redis aof. Does rabbitmq have this capability?
- 33
1
vote
0
answers
2k
views
Unable to start ARQ worker in FastAPI
I am unable to startup an arq worker in FastAPI even though i was able to create a redis pool and submit a task, but without an active worker, the task would not be executed.
I called the Worker class ...
- 11
0
votes
1
answer
67
views
Given a dynamic list of deadlines, how to trigger events at certain intervals before any deadline?
Say in your DB you have some "deadlines" that are subject to change:
deadline A: 11:30am
deadline B: 4:50pm
deadline C: 6:15pm
At 30 and 120 minutes before any of those deadlines, you want ...
- 793
2
votes
0
answers
224
views
Running Scrapy with a task queue
I built a web crawler with Scrapy and Django and put the CrawlerRunner code into task queue. In my local everything works fine until run the tasks in the server. I'm thinking multiple threads causing ...
- 334
2
votes
1
answer
880
views
How to get the autogenerated task name from Google Cloud Tasks when adding Firebase Task queue functions?
Following the Cloud Functions documentation about how to Enqueue functions with Cloud Tasks it is demonstrated how to create a Firebase task queue function. This works all fine.
However, if I call the ...
- 550
0
votes
2
answers
2k
views
What is the difference between call stack and task queue in javaScript(event-loop)?
I am confuse in differentiating between call stack and task queue in the context of event loop.
Call stack -->Every time a script or function calls a function, it's added to the top of the call ...
- 13
0
votes
1
answer
470
views
How to install Gearman Job Server with PHP
I am trying to install Gearman client worker on Centos server. PHP has been installed and all required gearman-php module also installed. when executing command gearmand -d it gets executed ...
- 709
0
votes
1
answer
846
views
Twilio : Calculating and announcing average wait time
I have a workflow in Twilio which have multiple queues. I need to calculate average wait time for each incoming user based on their position in that specific queue and announce it back to the user.
- 3