2

I am trying to write data to external array while running a powershell job-

This is my code sample that I' m trying-

$datafromJob = @()
$wmijobs = @()
foreach ($c in $computers) {
    $wmijobs += Start-Job -Name WMIInventory -ScriptBlock {
        $jobdata = get-wmiobject -ComputerName $args[0] -Class win32_computersystem -Credential $Cred -ErrorVariable Err -ErrorAction SilentlyContinue
        if ($Err.length) {
            Add-content -Path D:\InventoryError.log -Force -Value $Err
            $details = @{
                Domain       = "Error"
                Manufacturer = "Error"
                Computer     = $args[0]
                Name         = "Error"
            }
            $args[3] += New-Object PSObject -Property $details
        }
        if ($jobdata.length) {
            $details = @{
                Domain       = $jobdata.Domain
                Manufacturer = $jobdata.Manufacturer
                Computer     = $args[2]
                Name         = $jobdata.Name
            }
            $args[3] += New-Object PSObject -Property $details
        }
        -ArgumentList $c, $Cred, "Test", $datafromJob
    }
}

Expecting Output in $datafromJob Variable, but the end of job and loop variable is empty, M not getting how it will work, anyhelp,

Do let me know if any queries on this question

Improve this question

3 Answers 3

Reset to default
3

It is technically possible with threadjobs, but it probably wouldn't work well in your case. Arrays aren't thread safe.

$a = 1,2,3
start-threadjob { $b = $using:a; $b[0] = 2 } | receive-job -wait -auto
$a

2
2
3

Hmm, thread safe updating of a dictionary collection from the bottom here: https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/

$threadSafeDictionary =
[System.Collections.Concurrent.ConcurrentDictionary[string,object]]::new()

Get-Process | ForEach-Object -Parallel {
    $dict = $using:threadSafeDictionary
    $dict.TryAdd($_.ProcessName, $_)
}

$threadSafeDictionary["pwsh"]

"Concurrent bag":

$threadSafeArray =
[System.Collections.Concurrent.ConcurrentBag[object]]::new()

1..10 | foreach-object -parallel { 
    $array = $using:threadSafeArray
    $array.Add($_)
} 

$threadSafeArray

10
9
8
7
6
5
4
3
2
1
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

This answer works extremely well for Powershell 7's foreach-object -parallel! The $using was the trick for me, is this because Powershell considers this a Remote Variable? (learn.microsoft.com/en-us/powershell/module/…)
@aolszowka The foreach parallel loop runs in another runspace like start-threadjob.
2

Background jobs run in a separate (child) process, so you fundamentally cannot directly update values in the caller's scope from them.[1]

Instead, make your job script block produce output that the caller can capture with Receive-Job.

A simple example:

# Create a 10-element array in a background job and output it.
# Receive-Job collects the output.
$arrayFromJob = Start-Job { 1..10 } | Receive-Job -Wait -AutoRemoveJob

Note: If what you output from a background job are complex objects, they will typically not retain their original type and instead be custom-object emulations, due to the limitations of PowerShell's XML-based cross-process serialization infrastructure; only a limited set of well-known types deserialize with type fidelity, including primitive .NET types, hashtables and [pscustomobject] instances (with the type-fidelity limitations again applying to their properties and entries). - see this answer for background information.

A few asides:

  • There is no need to call Start-Job / Get-WmiObject in a loop, because the latter's -ComputerName parameter can accept an array of target computers to connect to in a single call.

    • Since the target computers are then queried in parallel, you may not need a background job (Start-Job) at all.

  • The CIM cmdlets (e.g.,Get-CimInstance) superseded the WMI cmdlets (e.g., Get-WmiObject) in PowerShell v3 (released in September 2012). Therefore, the WMI cmdlets should be avoided, not least because PowerShell [Core] (version 6 and above), where all future effort will go, doesn't even have them anymore.

    • Remote use of the CIM cmdlets by default requires that the target computers be set up for WS-Management connections, as they implicitly are if PowerShell remoting is enabled on them - see about_Remote_Requirements; alternatively, however, you can use the DCOM protocol (which is what the WMI cmdlets used) - see this answer for more information.

Applying the above to your case:

# Create a CIM session that targets all computers.
# By default, the WS-Management protocol is used, which target computers
# are implicitly set up for if PowerShell remoting is enabled on them.
# However, you can opt to use DCOM - as the WMI cmdlets did - as follows:
#   -SessionOption (New-CimSessionOption -Protocol DCOM)
$session = New-CimSession -ComputerName $computers -Credential $Cred

# Get the CIM data from all target computers in parallel.
[array] $cimData = Get-CimInstance -CimSession $session -Class win32_computersystem -ErrorVariable Err -ErrorAction SilentlyContinue |
  ForEach-Object {
    [pscustomobject] @{
      Domain       = $_.Domain
      Manufacturer = $_.Manufacturer
      Computer     = $_.ComputerName
      Name         = $_.Name
    }
  }

# Cleanup: Remove the session.
Remove-CimSession $session

# Add error information, if any.
if ($Err) {
  Set-Content D:\InventoryError.log -Force -Value $Err
  $cimData += $Err | ForEach-Object {
    [pscustomobject] @{
      Domain       = "Error"
      Manufacturer = "Error"
      Computer     = $_.ComputerName
      Name         = "Error"
    }
  }
}

Caveat re targeting a large number of computers at once:

  • As of this writing, neither the Get-CimInstance help topic nor the conceptual about_CimSession topic discuss connection throttling (limiting the number of concurrent connections to remote computers to prevent overwhelming the system).

  • PowerShell's general-purpose Invoke-Command remoting command, by contrast, has a -ThrottleLimit parameter that defaults to 32. Note that PowerShell remoting must first be enabled on the target computers in order to be able to use Invoke-Command on them remotely - see about_Remote_Requirements.

Therefore, to have more control over how the computers are targeted in parallel, consider combining Invoke-Command with local invocation of Get-CimInstance on each remote machine; for instance:

Invoke-Command -ComputerName $computers -ThrottleLimit 16 {
    Get-CimInstance win32_computersystem  
}  -Credential $Cred -ErrorVariable Err -ErrorAction

Also passing a sessions-options object to Invoke-Command's -SessionOption parameter, created with New-PSSessionOption, additionally gives you control over various timeouts.

[1] In a script block executed in a background job, the automatic $args variable contains deserialized copies of the values passed by the caller - see this answer for background information.
Note that the usually preferable, thread-based Start-ThreadJob cmdlet - see this answer - can receive live references to reference-type instances in the caller's scope, though modifying such objects then requires explicit synchronization, if multiple thread jobs access them in parallel; the same applies to the PowerShell 7+ ForEach-Object -Parallel feature.

Improve this answer

2 Comments

hey just an update Get-CimInstance does'nt accepts -Credential parameter, to use custom credential one will have to use New-CimSession to store credential and computer detail, refer this url for reference
I appreciate the correction @Vino - sorry for not checking that myself first; I've updated the answer accordingly. Since I cannot really test these things myself, it would be great if you could let us know if the session-based approach now shown in the answer works, and perhaps also if Get-CimInstance is suitable for targeting 1000+ servers - or whether PS remoting (Invoke-Command) must be used.
0

As per @mklement0 suggestion, have updated my script,

$wmijobs = @()
foreach ($c in $computers) {
    $wmijobs += Start-Job -Name WMIInventory -ScriptBlock {
        $jobdata = get-CimInstance -ComputerName $args[0] -Class win32_computersystem -Credential $Cred -ErrorVariable Err -ErrorAction SilentlyContinue
        if ($Err.length) {
            Add-content -Path D:\InventoryError.log -Force -Value $Err
            $details = @{
                Domain       = "Error"
                Manufacturer = "Error"
                Computer     = $args[0]
                Name         = "Error"
            }
          return New-Object PSObject -Property $details
        }
        if ($jobdata) {
            $details = @{
                Domain       = $jobdata.Domain
                Manufacturer = $jobdata.Manufacturer
                Computer     = $args[2]
                Name         = $jobdata.Name
            }
            return New-Object PSObject -Property $details
        }
        -ArgumentList $c, $Cred, "Test", $datafromJob
    }
}
$wmijobs = $wmijobs | get-job | receive-job -AutoRemoveJob -wait
Improve this answer

1 Comment

Definitely an improvement, but that Get-CimInstance too can accept an array of computer names, which then targets all computers in parallel, so I suspect you don't need Start-Job at all - please see the update to my answer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.