100 questions
0
votes
0
answers
63
views
Queue the requests and process one by one
I am exposing an api method which accepts the request from client app. this request will be sent to another COM library where it will be validated . Based on the request data the validation may delay ...
- 171
1
vote
2
answers
231
views
ConcurrentQueue with many enqueuers and single dequeuer
I'm using a ConcurrentQueue<T>. I need:
multiple consumers to enqueue items
a single consumer to dequeue all items in one go
during the dequeue, to lock the queue so other consumers cannot ...
- 22.4k
0
votes
1
answer
118
views
C# Message queues between threads and general threading woes
Before I start, I'll let you know that I'm still just learning (by doing) threading, and I think I'm missing some essential piece of the puzzle.
However, I am having a weird issue. The app I'm writing ...
- 31
2
votes
1
answer
63
views
MPSC-like queue failing with erroneous dequeue
Goal
Implement a MPSC-like queue with mutexes and sempahores.
Issue
Eventually, the consumer will attempt to dequeue an empty queue.
Minimal Reproducible Example
I have edited the question to include ...
- 115
1
vote
0
answers
82
views
Message loss with lock-free MPMC queue when threads exit
Issue
I am getting inconsistent behavior with message loss while attempting to use a MPMC lock-less queue (concurrentqueue) in the implementation of a mutual exclusion algorithm. In particular, I have ...
- 115
0
votes
0
answers
45
views
Why do we need this additional check in this concurrent queue implementation?
Working through a textbook which suggests the following implementation for a concurrent queue. I am wondering why the two if statements, which are highlighted in the code, are necessary. After the ...
- 417
27
votes
1
answer
11k
views
When should System.Threading.Channels be preferred to ConcurrentQueue?
I recently built a consumer/producer system using ConcurrentQueue<T> and SemaphoreSlim. Then made another alternative system utilizing the new System.Threading.Channel class.
After benchmarking ...
- 1,323
0
votes
1
answer
206
views
Using most significant bits of pointers as a tag in non-blocking queue c++
I am trying to implement a concurrent non-blocking queue where the tag is in the 16 most significant bits of the pointer. It follows this algorithm here:
http://www.cs.rochester.edu/research/...
- 169
0
votes
2
answers
1k
views
How do I use ConcurrentQueue for writing to a file in a separate thread?
My application receives data I'm trying to write to a file in a separate thread (writing it in the thread that received it directly caused performance problems). It seems ConcurrentQueue is suitable ...
- 1,767
0
votes
1
answer
133
views
ConcurrentQueue.Count remains zero when socket receive adds data
I have some code that used to work for many years and even now in specific cases it works but in other cases I just cannot understand why it fails.
The following code is part of a Client class that ...
- 1
2
votes
3
answers
1k
views
Difference between Queue's and Concurrent Queue's TryDequeue method
Both collections, Queue and ConcurrentQueue have a method TryDequeue. What is the difference between using TryDequeue with Queue and ConcurrentQueue respectively? Is Queue's TryDequeue method thread-...
- 1,806
-2
votes
2
answers
936
views
How to ensure parallel tasks dequeue unique entries from ConcurrentQueue<T>?
Hi I have a concurrent Queue that is loaded with files from database. These files are to be processed by parallel Tasks that will dequeue the files. However I run into issues where after some time, I ...
- 141
0
votes
1
answer
192
views
Convert a C# collection of arrays to a single array efficiently
I've got a ConcurrentQueue<Vector3[]> that I'm filling up and I want to read out of it in chunks for the sake of performance.
private ConcurrentQueue<Vector3[]> positionsQueue = new ...
- 642
0
votes
2
answers
1k
views
ConcurrentQueue with items that have sequence value
I'm using a ConcurrentQueue<Result> to store results of a certain activity that occurs in my system (ASP.NET Core 6). It works well.
But now I have a new requirement: I need a Result.Id property ...
- 22.4k
3
votes
4
answers
3k
views
When to choose serialQueue over concurrent queue in ios
I was asked to implement a thread safe dictionary in swift, I used the common approach:
class MutableDictionary {
var dictionary: [String : Any] = [:]
var queue = DispatchQueue(label: "...
- 246
0
votes
1
answer
2k
views
C++ moodycamel concurrent queue - enqueue pointers
I am trying to use ConcurrentQueue to log items into a file on a separate thread:
https://github.com/KjellKod/Moody-Camel-s-concurrentqueue
This works:
// declared on the top of the file
moodycamel::...
- 10.6k
1
vote
1
answer
820
views
Working with data held in a ConcurrentQueue<T>
I have a background worker that streams data and saves it to a ConcurrentQueue<T> which is what I need since it is a thread safe First In First Out collection, but I also need to do tasks like ...
- 1,269
6
votes
2
answers
2k
views
Why concurrent queue with sync act like serial queue?
Could anyone help me to understand this code I created:
let cq = DispatchQueue(label: "downloadQueue", attributes: .concurrent)
cq.sync {
for i in 0..<10 {
sleep(2)
print(i)
}
}
...
- 246
1
vote
2
answers
1k
views
How to set maximum concurrent threads while dequeuing ConcurrentQueue?
I have one thread responsible for enqueuing and one thread responsible for dequeuing. However, the frequency of the data being enqueued far surpasses the time needed to dequeue + process the data.
...
- 186
1
vote
0
answers
209
views
I get an ObjectDisposedException when reading a stream in the background service
I want to process a Request.Body in a background-service in ASP.Net Core, so I first add it to a ConcurrentQueue.
public class BackgroundJobs
{
public ConcurrentQueue<Stream> Queue { get; ...
- 1,122
4
votes
1
answer
675
views
What is the difference between TryDequeue and TryTake in a ConcurrentQueue<>?
In the ConcurrentQeueue<> class, and extra method TryDequeue() is defined. But, as it implements IProducerConsumerCollection<>, it also has a TryTake() method. According to the docs, they ...
- 33.8k
0
votes
0
answers
43
views
Way to give queue container to each thread
My purpose is to create queue sets for send data to each thread, assign queue's address to each thread, and then freely pass data to by queue
To do this, what kind of method or container I use?
One of ...
- 57
1
vote
0
answers
439
views
Is there any event which is called whenever an item is added to a concurrent queue?
I am storing some messages in a concurrent queue and I need to show these messages after some delay. So I need an event that gets triggered after an item is added to the queue.
- 93
0
votes
0
answers
275
views
ConcurrentQueue<T> declare static?
I want to use a shared list (ConcurrentQueue<T>) for my project, that includes a listener, a processor and a sender. The listener enqueues items to a list, the sender dequeues them from the ...
- 35
0
votes
1
answer
206
views
Preventing access to concurrent queue while rebuilding
I have a ConcurrentQueue which holds an object with some helpers methods to enqueue and dequeue from the queue. There is a need to remove items from the queue which match some criteria and then ...
- 3,186
0
votes
0
answers
937
views
Can I execute Tasks with ConcurrentQueue<Task> automatically by passing task(method) as a parameter of ConcurrentQueue.Enqueue?
I want to make a queue manager which execute tasks(method) in ConcurrentQueue.
The function of the queue manager is executing any tasks(methods) in queue automatically.
to do this, I try to pass task(...
- 1
0
votes
1
answer
175
views
Thread + While(true) + Entity
I'm building a candle recorder (Binance Crypto), interesting in 1 minute candles, including intra candle data for market study purpose (But eventually I could use this same code to actually be my eyes ...
1
vote
1
answer
267
views
ConcurrentQueue dequeue problem in Azure function
I have declared a ConcurrentQueue and added a list of GUIDs. Adding into the queue is fine, but when I access the queue from inside the TimerTrigger function it seems like it's empty (updateQueue....
- 847
0
votes
0
answers
776
views
Poor Performance BackgroundService dequeuing from Queue
I have a HostedService which inherits from BackgroundService. It loops and dequeues items from a Queue.
These items are placed on the queue via a Controller from a http request.
If I set Postman ...
- 1,447
0
votes
1
answer
198
views
c# 1D-byte array to 2D-double array
I'm dealing with c# concurrent-queue and multi-threading in socket-programming tcp/ip
First, I've already done with socket-programming itself. That means, I've already finished coding about client, ...
- 1
0
votes
0
answers
454
views
How to handle unsent messages in a concurrent queue in a web api
I am using a concurrent queue in my Web API as mentioned in thread
How to maintain state or queue of requests in Web API
But I am running into an issue where if the application pool is stopped or ...
- 11
1
vote
1
answer
541
views
C# Abortable Asynchronous Fifo Queue - leaking massive amounts of memory
I need to process data from a producer in FIFO fashion with the ability to abort processing if the same producer produces a new bit of data.
So I implemented an abortable FIFO queue based on Stephen ...
- 1,285
5
votes
2
answers
7k
views
Concurrent queue usage, with one thread enqueuing and another thread dequeuing
Got a quick question.
Do I have to use a concurrent queue if one thread is enqueuing and other is dequeuing? Is there any race condition/other risk when using regular container in this scenario (1 ...
- 642
5
votes
3
answers
4k
views
Atomically taking everything from a ConcurrentQueue
I have multiple threads generating items and sticking them in a common ConcurrentQueue:
private ConcurrentQueue<GeneratedItem> queuedItems = new ConcurrentQueue<GeneratedItem>();
private ...
- 40.6k
5
votes
1
answer
1k
views
Why can't complexity of ConcurrentLinkedQueue.size() be constant?
Javadoc for ConcurrentLinkedQueue explicitly states that the complexity of the size() method is O(n). I find this surprising. I would follow the stock LinkedList.size() approach by accumulating the ...
- 882
0
votes
1
answer
1k
views
RxJS parallel queue with concurrent workers and handling of each request
I'm having trouble with RxJS and the correct way to handle an array of request.
Let's say I have an array of about 50 requests as follows:
let requestCounter = 0;
function makeRequest(timeToDelay) {
...
3
votes
0
answers
6k
views
ConcurrentQueue size limit and time frame
I am looking for a way to :
read messages from a ConcurrentQueue limited to some size.
read not more then X message at a time frame.
I want to stop the reading from Q once one of the 2 hit,
until ...
0
votes
0
answers
395
views
Comparing ConcurrentQueue.Count to integer returns wrong result
I am writing usual reader-writer functionality with one main thread that equeues and several threads that dequeues. So, there is a part of the code where I am comparing count of items in my ...
- 1
4
votes
2
answers
2k
views
How to release reference from the ConcurentQueue for the item dequeued from the concurrent queue with TryDequeue
I am using ConcurrentQueue (C#, ASP.NET Core) for holding tasks for the upload of the big files. I have a very big memory consumption even after items are dequeued from the concurrent queue. Items are ...
- 163
4
votes
1
answer
878
views
Pattern to remove elements from ConcurrentQueue after peeking [duplicate]
I have one producer and multiple consumers. I am using ConcurrentQueue. I am using C# but I think my problem is language agnostics.
There can be non-unique consumers. i.e. more than one consumers can ...
- 5,321
4
votes
2
answers
2k
views
RxJS parallel queue with concurrent workers?
Let's say I want to I download 10,000 files. I can easily build a queue of those 10,000 files (happy to take advice if any of this can be done better),
import request from 'request-promise-native';
...
1
vote
3
answers
3k
views
How to handle data coming from multiple threads in one thread (with ConcurrentQueue)?
I have orders coming in from multiple threads and I want to process this data in one thread. If I understood it right, the way to do it is with ConcurrentQueue.
I had a look at SO question How to ...
- 5,082
0
votes
1
answer
982
views
Holding requests until a message is in front of ConcurrentQueue not working
I am having an issue where it looks like my ConcurrentQueue in a singleton is not processing items in the correct order. I know it's FIFO, so I am thinking that maybe the queue in memory is not the ...
- 2,455
2
votes
0
answers
383
views
How to output data from Threadpool to ConcurrentQueue (C#)?
I have written an application for my company that essentially sends records from a text or CSV file in arrays of 100 records to a web service, to which it then returns the response, also in arrays of ...
- 29
2
votes
1
answer
125
views
Why weird behavior of concurrent queue?
I am trying to understanding the concurrent queue of iOS GCD.
The I made some code to test it, but found something weird.
code as below:
_syncQueue = dispatch_get_global_queue(...
- 128
2
votes
1
answer
2k
views
What is the Rust equivalent to Intel's tbb::concurrent_queue?
I am looking for an equivalent of the concurrent_queue from Intel's tbb module in Rust. I have found some crates:
multiqueue
two-lock-queue
crossbeam-deque
and even
futures-pool
thread-pool
I feel ...
- 343
2
votes
1
answer
564
views
Split a ConcurrentLinkedQueue into half using Spliterator
I have a ConcurrentLinkedQueue and I want to split it into two halves and let two separate threads handle each. I have tried using Spliterator but I do not understand how to get the partitioned queues....
- 387
0
votes
1
answer
2k
views
When is a ConcurrentQueue needed?
I'm asking this related to this question: Using events on multithreading socket server
I´ve implemented a ConcurrentQueue thinking it would be needed to avoid problems with multi-threading. When ...
- 2,381
3
votes
2
answers
3k
views
Concurrent Queue Issue - iOS/Swift
In my program I need two tasks to run simultaneously in the background. To do that i have used concurrent queues as below,
let concurrentQueue = DispatchQueue(label: "concurrentQueue", qos: .utility,...
- 793
0
votes
0
answers
144
views
How to exit out of a Concurrent Queue
So I have this code that I have written which checks all proxies in my ConcurrentQueue list, but I was wondering what if I just wanted to check if 1 proxy worked and then exit out of it? So I tried ...
- 557