byte array read and write in multithreads

Question

I allocated a very large byte array for threads to read and write bytes in the array. Write operation is to assign the new byte directly to the value in the array like byte[i] = byte2;

I can ensured that there will be no conflict for those operation (same section in the byte array will not write by 2 different threads. when a section in reading, no write operation will perform by other threads). My concern is will the modification to array been instantly available to other threads. I know there might be memory fence, which other threads may still read the old values in the array.

If the problem exists, how to avoid? will volatile byte[] store; works in this circumstance?

Peter Lawrey · Accepted Answer · 2016-02-05 13:37:37Z

4

will the modification to array been instantly available to other threads.

not without a write fence for the writer and a read fence for the reader.

I know there might be memory fence, which other threads may still read the old values in the array.

You will need to use one of those for the reader and writer (not one per byte, one per operation)

will volatile byte[] store; works in this circumstance?

It won't do what you want, though it will slow down your application. When you do

store[i] = b;

This added a read fence as you are reading the store reference to the byte[].

The best solution is to use Unsafe to have complete control over these read write operations. See the code for AtomicIntegerArray.

However, without going down that road you can do a dummy operation like this which is not as efficient/elegant, but a lot simpler to implement.

private final AtomicBoolean fence = new AtomicBoolean();

public void readFence() {
    fence.get();
}

public void writeFence() {
    fence.set(true);
}

For this to work, you must do a writeFence() after all writes, and a readFence() before all reads.

If you are going down the more low level route you might find using off heap memory an advantage. Off heap memory can store very large (TBs) amounts of data without impacting the GC.

edited Feb 5, 2016 at 13:37

answered Feb 5, 2016 at 13:26

Peter Lawrey

535k83 gold badges770 silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

Christophe De Troyer Over a year ago

Why will volatile not work in his case? It will make sure all threads always have the proper vaule, no? Nvm, I found this: stackoverflow.com/questions/2236184/…

Peter Lawrey Over a year ago

@ChristopheDeTroyer correct, all threads will read the right value of the byte[] reference using a read fence. There is no write fence and the contents of the byte[] might no be visible. volatile like final only applies to the reference to the array, not operations on the array. e.g. final bytes[] store = { .. } you can change the contents. You could do a dummy store = store; to trigger a write fence, but I find this more hacky than what proposed. The other downside is there is a memory barrier on every operation which can slow down the copies by an order of magnitude.

Shisoft Over a year ago

Thank you, I think I can find my answer in the AtomicIntegerArray. I still don't understand what is the fence in your example do to the store to make the write fence.

Voo Over a year ago

I know that you know, but I think it'd still be interesting to mention the old "assign volatile array to itself" trick. That's still the only portable way (what with unsafe being an implementation detail theoretically) if you don't want to use the AtomicArray classes.

Shisoft Over a year ago

I have already migrate my application to use unsafe for backend store. It seems more easy to use than my byte array solution since it has internal encoder and decoder for primary types and able to break the 2G size limit.

|

Cortwave · Accepted Answer · 2016-02-05 13:27:19Z

-2

You can use SynchronizedList for this Example

byte[] c = new byte[];
List list = Collections.synchronizedList(Arrays.asList(c));

It will be thread-safely

answered Feb 5, 2016 at 13:27

Cortwave

4,9272 gold badges27 silver badges42 bronze badges

2 Comments

Peter Lawrey Over a year ago

You have to replace the entire byte[] each time for this to work. It wouldn't work for portions of a byte[].

Shisoft Over a year ago

Actually, I don't want to do any copy on the byte array

Collectives™ on Stack Overflow

byte array read and write in multithreads

2 Answers 2

11 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

11 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related