Spawning threads and keeping data from them

Question

This is a super contrived example that shows what I want to do; I have a HashMap that I want to push data in to, but I want to do it in a set of threads - the real example is that I'm connecting to a remote service inside the thread::spawn and so I want to have all of it happen in the background as much as possible so it doesn't block the primary thread.

I'm not sure how to refactor my code to allow what I'm wanting to do - any suggestions would be helpful!

let mut item_array : HashMap<i32, i32> = HashMap::new();
let mut array: [i32; 3] = [0, 1, 2];

for item in &array {
    thread::spawn(|| {
        item_array.insert(*item, *item);
    });
}
println!("{:?}", item_array);

The errors I receive are below

error[E0597]: `array` does not live long enough
   --> src/main.rs:87:17
    |
87  |     for item in &array {
    |                 ^^^^^^
    |                 |
    |                 borrowed value does not live long enough
    |                 argument requires that `array` is borrowed for `'static`
...
153 | }
    | - `array` dropped here while still borrowed

error[E0499]: cannot borrow `item_array` as mutable more than once at a time
  --> src/main.rs:88:23
   |
88 |           thread::spawn(|| {
   |           -             ^^ mutable borrow starts here in previous iteration of loop
   |  _________|
   | |
89 | |             item_array.insert(*item, *item);
   | |             ---------- borrows occur due to use of `item_array` in closure
90 | |         });
   | |__________- argument requires that `item_array` is borrowed for `'static`

error[E0373]: closure may outlive the current function, but it borrows `item`, which is owned by the current function
  --> src/main.rs:88:23
   |
88 |         thread::spawn(|| {
   |                       ^^ may outlive borrowed value `item`
89 |             item_array.insert(*item, *item);
   |                                ---- `item` is borrowed here
   |
note: function requires argument type to outlive `'static`
  --> src/main.rs:88:9
   |
88 | /         thread::spawn(|| {
89 | |             item_array.insert(*item, *item);
90 | |         });
   | |__________^
help: to force the closure to take ownership of `item` (and any other referenced variables), use the `move` keyword
   |
88 |         thread::spawn(move || {
   |                       ^^^^^^^

error[E0373]: closure may outlive the current function, but it borrows `item_array`, which is owned by the current function
  --> src/main.rs:88:23
   |
88 |         thread::spawn(|| {
   |                       ^^ may outlive borrowed value `item_array`
89 |             item_array.insert(*item, *item);
   |             ---------- `item_array` is borrowed here
   |
note: function requires argument type to outlive `'static`
  --> src/main.rs:88:9
   |
88 | /         thread::spawn(|| {
89 | |             item_array.insert(*item, *item);
90 | |         });
   | |__________^
help: to force the closure to take ownership of `item_array` (and any other referenced variables), use the `move` keyword
   |
88 |         thread::spawn(move || {
   |                       ^^^^^^^

error[E0502]: cannot borrow `item_array` as immutable because it is also borrowed as mutable
  --> src/main.rs:92:22
   |
88 |           thread::spawn(|| {
   |           -             -- mutable borrow occurs here
   |  _________|
   | |
89 | |             item_array.insert(*item, *item);
   | |             ---------- first borrow occurs due to use of `item_array` in closure
90 | |         });
   | |__________- argument requires that `item_array` is borrowed for `'static`
91 |       }
92 |       println!("{:?}", item_array);
   |                        ^^^^^^^^^^ immutable borrow occurs here

I think this can be useful to you gradebot.org/doc/ipur/concurrency.html#mutable-shared-data — Đorđe Zeljić
– Đorđe Zeljić, Commented Dec 28, 2019 at 22:04

Đorđe Zeljić · Accepted Answer · 2019-12-28 22:24:15Z

2

Here is very simple example of code based on source you have provieded:

use std::collections::HashMap;
use std::thread;

fn main() {
    let item_array: HashMap<i32, i32> = HashMap::new();
    let array: [i32; 3] = [0, 1, 2];
    let mut handles = vec![];

    let mutex = std::sync::Mutex::new(item_array);
    let arc = std::sync::Arc::new(mutex);

    for item in &array {
        let item = item.to_owned();
        let arc_cloned = std::sync::Arc::clone(&arc);

        let th = thread::spawn(move || {
            let mut guard = arc_cloned.lock().unwrap();

            guard.insert(item, item);
        });

        handles.push(th);
    }

    for handle in handles {
        handle.join().unwrap();
    }

    println!("{:?}", arc.lock().unwrap());
}

And you can play it here on rust playground

answered Dec 28, 2019 at 22:24

Đorđe Zeljić

1,8252 gold badges16 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2020-06-20 09:12:55Z

There are two problems here (which are typical problems with multithreaded Rust):

Your thread cannot borrow any data that may outlive it (which, when using std::thread::spawn, is any data¹).
Multiple thread cannot borrow the same mutable data.

The first problem is typically solved by:

Copying data instead of referencing it. This is particularly useful for primitive types, such as integers.
Using scoped threads.
Using Arc for thread-safe shared pointer, to ensure that data outlives all threads using it.

Not all are possible in all cases, but I'd recommend the above solutions in that order.

The second problem is typically solved with locks such as Mutex or RwLock, which allows only a single thread to get a mutable reference at a time.

Combining these, I'd solve your example like this:

// mutable data is wrapped in a Mutex
let item_array: Mutex<HashMap<i32, i32>> = Mutex::new(HashMap::new());

// immutable data does not have to be wrapped with scoped threads
let array: [i32; 3] = [0, 1, 2];

// using crossbeam::scope from the crossbeam library, which lets
// us reference variables outside the scope (item_array and array)
crossbeam::scope(|s| {
    for item in &array {
        // copy item (since it is an integer)
        let item = *item;
        
        // explicitly reference item_array, since we'll later need to move this
        let item_array_ref = &item_array;
        
        // move item and item_array_ref into the closure (instead of referencing,
        // which is by default)
        s.spawn(move |_| {
            // need to call .lock() to aquire a mutable reference to the HashMap
            // will wait until the mutable reference is not used by any other threads
            let mut item_array_lock = item_array_ref.lock().unwrap();
            
            item_array_lock.insert(item, item);
        });
    }

    // the scope will persist until all threads spawned inside it (with s.spawn) have completed, blocking the current thread,
    // ensuring that any referenced data outlives all the threads
})
.unwrap();

// need to call .lock() here as well to get a reference to the HashMap
println!("{:?}", item_array.lock().unwrap());

Run in playground

¹Except when the data has 'static lifetime, i.e. can exist for the entire lifetime of the program.

Ack! A misconception; 'static does not mean will live for the entire lifetime of the program. It means it could live for that long. Or, in other words, that its existence (And therefore its own lifetime) is not dependent on the lifetime of other things. You can, for example, pass a (Non Copy) struct like File into another thread without issue, but the File is not inlined in the binary. In the case of &'static _ (Where _ is anything), it will live for the entire lifetime of the program, but that's not what 'static says there, 'static is just a side effect of that.

Collectives™ on Stack Overflow

Spawning threads and keeping data from them

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related