Saving results from a parallelized goroutine

Question

I am trying to parallelize an operation in golang and save the results in a manner that I can iterate over to sum up afterwords.

I have managed to set up the parameters so that no deadlock occurs, and I have confirmed that the operations are working and being saved correctly within the function. When I iterate over the Slice of my struct and try and sum up the results of the operation, they all remain 0. I have tried passing by reference, with pointers, and with channels (causes deadlock).

I have only found this example for help: https://golang.org/doc/effective_go.html#parallel. But this seems outdated now, as Vector as been deprecated? I also have not found any references to the way this function (in the example) was constructed (with the func (u Vector) before the name). I tried replacing this with a Slice but got compile time errors.

Any help would be very appreciated. Here is the key parts of my code:

type job struct {
  a int
  b int
  result *big.Int
}

func choose(jobs []Job, c chan int) {
  temp := new(big.Int)
  for _,job := range jobs {
    job.result = //perform operation on job.a and job.b
    //fmt.Println(job.result)
  }
  c <- 1
}

func main() {
  num := 100 //can be very large (why we need big.Int)
  n := num
  k := 0
  const numCPU = 6 //runtime.NumCPU
  count := new(big.Int)

  // create a 2d slice of jobs, one for each core
  jobs := make([][]Job, numCPU)


  for (float64(k) <= math.Ceil(float64(num / 2))) {
    // add one job to each core, alternating so that
    // job set is similar in difficulty
    for i := 0; i < numCPU; i++ {
      if !(float64(k) <= math.Ceil(float64(num / 2))) {
        break
      }
      jobs[i] = append(jobs[i], Job{n, k, new(big.Int)})
      n -= 1
      k += 1
    }
  }

  c := make(chan int, numCPU)
  for i := 0; i < numCPU; i++ {
    go choose(jobs[i], c)
  }

  // drain the channel
  for i := 0; i < numCPU; i++ {
   <-c
   }
  // computations are done

 for i := range jobs {
    for _,job := range jobs[i] {
      //fmt.Println(job.result)
      count.Add(count, job.result)
    }
  }



  fmt.Println(count)
}

Here is the code running on the go playground https://play.golang.org/p/X5IYaG36U-

Mr_Pink · Accepted Answer · 2016-11-16 21:39:54Z

2

As long as the []Job slice is only modified by one goroutine at a time, there's no reason you can't modify the job in place.

for i, job := range jobs {
    jobs[i].result = temp.Binomial(int64(job.a), int64(job.b))
}

https://play.golang.org/p/CcEGsa1fLh

You should also use a WaitGroup, rather than rely on counting tokens in a channel yourself.

answered Nov 16, 2016 at 21:39

Mr_Pink

110k17 gold badges287 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mtonsmann Over a year ago

So the problem was with modifying the job element instead of jobs[i] directly. Classic dumb mistake. Thank you!

Mr_Pink Over a year ago

@mtonsmann: note that using []*Job would have worked too, and may even be preferable depending on what you're actual Job type is.

Collectives™ on Stack Overflow

Saving results from a parallelized goroutine

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related