golang functions: parallel execution with return

Question

How to make two functions calls f1(2) and f1(1) execute in parallel so that all the program would execute for 2 seconds not for 3.

package main

import (
    "fmt"
    "time"
)

// sleeps for `secs` seconds
func f1(secs time.Duration) (result string) {
    fmt.Printf("waiting %V\n", secs)
    time.Sleep(secs * time.Second)
    result = fmt.Sprintf("waited for %d seconds", secs)
    return
}

// prints arg1, arg2
func f2(arg1, arg2 string) {
    fmt.Println(arg1)
    fmt.Println(arg2)
}

// this function executes for 3 seconds, because waits a lot
func runNotParallel() {

    out1 := f1(2)
    out2 := f1(1)
    f2(out1, out2)

}

// golang parallel return functions
// todo: make it run so all the function will executes for 2 seconds not for 3
func runParallel() {
    out1 := f1(2)
    out2 := f1(1)
    f2(out1, out2)
}

func main() {
    runNotParallel()
    runParallel()
}

playground

I guess I can do it only with channels. Should I redefine function f1 or I can leave it as is and change only way I call it?

When using parallel execution to speed-up a program, Amdahl's Law is important and will be one of the factors that determine whether you are successful or just wasting your effort. In Go, concurrency has broader aims than just faster execution. It's important enough to be worth learning about though, so I wish you success. — Rick-777
– Rick-777, Commented Jan 7, 2015 at 23:11
@Rick-777 Amdahl’s Law is more about parallelism than concurrency: in certain cases where latency is a huge factor (think of I/O-bound operations such as making 100 HTTP requests), concurrency can be a huge improvement. — Bora M. Alper
– Bora M. Alper, Commented Jun 21, 2020 at 19:14
Amdahl's law is quite simple and general - it doesn't care about (or even mention) I/o. Nor does it matter about the difference between parallelism and concurrency; these are widely misunderstood terms anyway. — Rick-777
– Rick-777, Commented Jun 22, 2020 at 17:48
You can also check this library github.com/ddelizia/channelify (I am the author :) ) — ddelizia
– ddelizia, Commented Oct 5, 2020 at 16:08

mattn · Accepted Answer · 2015-01-06 05:15:19Z

17

Use chan/goroutine

package main

import (
    "fmt"
    "time"
)

// sleeps for `secs` seconds
func f1(secs time.Duration) (result string) {
    fmt.Printf("waiting %v\n", secs)
    time.Sleep(secs * time.Second)
    result = fmt.Sprintf("waited for %v seconds", secs)
    return
}

// prints arg1, arg2
func f2(arg1, arg2 string) {
    fmt.Println(arg1)
    fmt.Println(arg2)
}

// this function executes for 3 seconds, because waits a lot
func runNotParallel() {
    out1 := f1(2)
    out2 := f1(1)
    f2(out1, out2)

}

// golang parallel return functions
// todo: make it run so all the function will executes for 2 seconds not for 3
func runParallel() {
    out1 := make(chan string)
    out2 := make(chan string)
    go func() {
        out1 <- f1(2)
    }()
    go func() {
        out2 <- f1(1)
    }()
    f2(<-out1, <-out2)
}

func main() {
    runNotParallel()
    runParallel()
}

https://play.golang.org/p/G4RHiq9LJw

answered Jan 6, 2015 at 5:15

mattn

7,77333 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ddelizia Over a year ago

I made a library that can do what you are doing in few lines with reflection. You can check the example github.com/ddelizia/channelify. The logic behind is the same but I think I made it easier to use :)

Raed Shomali · Accepted Answer · 2017-06-07 03:22:01Z

9

Another way you could do it is using WaitGroup

I wrote this utility function to help parallelize a group of functions:

import "sync"

// Parallelize parallelizes the function calls
func Parallelize(functions ...func()) {
    var waitGroup sync.WaitGroup
    waitGroup.Add(len(functions))

    defer waitGroup.Wait()

    for _, function := range functions {
        go func(copy func()) {
            defer waitGroup.Done()
            copy()
        }(function)
    }
}

So in your case, we could do this

value1 := ""
value2 := ""

func1 := func() {
    value1 = f1(2)
}

func2 = func() {
    value2 = f1(1)
}

Parallelize(func1, func2)

f2(out1, out2)

If you wanted to use the Parallelize function, you can find it here https://github.com/shomali11/util

answered Jun 7, 2017 at 3:22

Raed Shomali

1,45516 silver badges8 bronze badges

1 Comment

jsfrz Over a year ago

This is great, can you please explain the code of Parallelize function?

Oleg Kokorin · Accepted Answer · 2021-03-20 17:41:32Z

here is a solution without channels but with the missing f2 synchronization:

package main

import (
    "fmt"
    "sync"
    "time"
)

// sleeps for `secs` seconds
func f1(secs time.Duration, result *string, sg *sync.WaitGroup) () {
    fmt.Printf("waiting %v\n", secs)
    time.Sleep(secs * time.Second)
    *result = fmt.Sprintf("waited for %d seconds", secs)
    if sg!= nil {
        sg.Done()
    }
    return
}

// prints arg1, arg2
func f2(arg1, arg2 string) {
    fmt.Println(arg1)
    fmt.Println(arg2)
}

// this function executes for 3 seconds, because waits a lot
func runNotParallel() {

    var out1, out2 string
    f1(2, &out1, nil)
    f1(1, &out2,nil)
    f2(out1, out2)

}

// golang parallel return functions
// todo: make it run so all the function will executes for 2 seconds not for 3
func runParallel() {
    var sg sync.WaitGroup
    sg.Add(2)
    var out1, out2 string
    go f1(2, &out1, &sg)
    go f1(1, &out2, &sg)
    sg.Wait()
    f2(out1, out2)
}

func main() {
    runNotParallel()
    runParallel()
}

basically, go operator blocks from using/accessing a return values but it could be done using a pointers for the return place holders in the signature

Clay · Accepted Answer · 2022-04-13 21:47:34Z

0

With go 1.18 supporting generics, the channel solution can be made even more readable.

func async[T any](f func() T) chan T {
    ch := make(chan T)
    go func() {
        ch <- f()
    }()
    return ch
}

func main() {
    startTime := time.Now().Local()

    out1 := async(func() string {
        time.Sleep(1 * time.Second)
        return "thing 1"
    })
    out2 := async(func() string {
        time.Sleep(2 * time.Second)
        return "thing 2"
    })

    results := []string{<-out1, <-out2}

    fmt.Printf("results: %v\n", results)
    fmt.Printf("took %v", time.Since(startTime))
}

playground

The lo package provides this along with lots of other generic helper functions.

answered Apr 13, 2022 at 21:47

Clay

11.8k5 gold badges51 silver badges46 bronze badges

Collectives™ on Stack Overflow

golang functions: parallel execution with return

4 Answers 4

1 Comment

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related