Return Values from Powershell Parallel Foreach Loops

Question

I'm trying to code the following situation in Powershell:

Create array of objects

Parallel foreach of array
{
     Perform work on item of array;
     Assign/return value of work;
}

Perform work on results of Parallel foreach loop

I can't find any clear guides on how to do this, just bits and pieces which myself and two other coworkers can't seem to compile into a working solution, even a simple example solution that we can then apply to the actual work we need to do.

Given a simple task, like taking in an array of ints:

$inputArray = @(1..200)

And then, for each int, multiply it by itself, and return the value for each multiplication in a thread safe manner (as, I assume, an array), how would you code this in Powershell?

Edit: Not to confuse any potential answers, but here's what we've tried to create so far.

$inputArray = @()
$resultArray = @()

for($i = 0; $i -le 10; $i++)
{
    $inputArray += Get-Random
}

Write-Output "Workflow"

workflow ExampleWorkflow
{
    param ($items)

    $test = @()

    foreach -Parallel ($item in $items)
    {
        $WORKFLOW:test += ($item * $item)
        $WORKFLOW:test += ($item + $item)
        $WORKFLOW:test += ($item * 3.14)
    }
    Write-Output ("TEST CURRENT: " + $test[0])
}

ExampleWorkflow $inputArray

Write-Output ("This is test: " + $test)

$resultArray | % { Write-Output $_ }

nfernal13 · Accepted Answer · 2015-03-12 21:11:10Z

2

I'm not 100% clear on what your end goal is on this, but from the code you posted it looks like your trying to randomize values in an array and create a new array from the randomized values. If that's the case try something like what I have below. The $resultArray wasn't getting populated before because it was being initialized outside of your workflow.

$inputArray = @()


for($i = 0; $i -le 10; $i++)
{
    $inputArray += Get-Random
}

Write-Output "Workflow"

workflow ExampleWorkflow
{
    param ($items)

    $test = @()

    foreach -Parallel ($item in $items)
    {
        $WORKFLOW:test += ($item * $item)
        $WORKFLOW:test += ($item + $item)
        $WORKFLOW:test += ($item * 3.14)

    }
    [array]$resultArray = @($test)
    Write-Output ("TEST CURRENT: " + $test[0])
    $resultArray


}




ExampleWorkflow $inputArray

edited Mar 12, 2015 at 21:11

answered Mar 11, 2015 at 20:22

nfernal13

444 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ranger Over a year ago

Hey, we've actually looked into workflows. I went ahead and attached what we cooked up so far based on the bits and pieces we Googled, but it doesn't work, and it might be a train wreck.

Jerzy Jerzycki · Accepted Answer · 2021-03-15 17:33:36Z

I have tested the loop behavior in parallel mode in many scenarios. The test showed that each loop call creates its own variable space just like inside the function and the problem is simultaneous editing the same object in parallel mode. For me, a parallel loop in which I wrote to one table worked as expected when I was doing it with a mechanism that accepts parallel writing to the table, i.e. in SQL

    $_ARRAY = $null
    $_ARRAY = @()
    $_ARRAY = @{
            col1 = $somthing1
            col2 = $somthing1
    }
    
    Function _pull_to_SQL($_ARRAY){
        $DB = "DBName"
        $SQLSRVR = "Server\Instance"
        $TABLE = "TableName"
        $SQLCON = New-Object System.Data.SqlClient.SqlConnection("Data Source=$SQLSRVR; `
                    Initial Catalog=$DB;Integrated Security=SSPI")
        $SQLCON.open()
        $SQL = $SQLCON.CreateCommand() 
        $INSERT = "INSERT INTO $TABLE (col1,col2) VALUES ('$($_ARRAY.col1)','$($_ARRAY.col2)')"
        $SQL.CommandText = $INSERT
        $SQL.ExecuteNonQuery() > $null
        $SQLCON.Close()
    }

After the -ThrottleLimit parameter in the table edited in the parallel mode was empty and inside loop it contained the value from only a single call in the loop, so each execution in the loop is performed separately.

An array edited in parallel mode will always be empty after exiting the loop.

Collectives™ on Stack Overflow

Return Values from Powershell Parallel Foreach Loops

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related