Prevent PowerShell ForEach-Object to flatten the list?

Question

The following code prints the counts if there are more than two items. The .Split(',') was called twice.

'a,b,c', 'x,y', '1,2,3' |
Where-Object { $_.Split(',').Count -gt 2 } |
ForEach-Object { $x = $_.Split(','); $x.Count }

The following code try to call .Split(',') once. It doesn't get any output.

'a,b,c', 'x,y', '1,2,3' |
ForEach-Object { @($_.Split(',')) } | # got a single list instead of `list of list of string`
Where-Object { $_.Count -gt 2 } |
ForEach-Object { $_.Count }

However, ForEach-Object flattens the list of list to list. Is it a way to prevent the flattening?

You could try the unary comma operator instead. ForEach-Object { ,($_.Split(',')) } — AdminOfThings
– AdminOfThings, Commented Sep 13, 2019 at 22:14

mklement0 · Accepted Answer · 2022-03-28 02:37:36Z

4

You can take advantage of the fact that both Where-Object and ForEach-Object run the script blocks passed to them ({ ... }) in the same scope, the caller's scope:

'a,b,c', 'x,y', '1,2,3', 'a,b,c,d' |
  Where-Object { ($count = $_.Split(',').Count) -gt 2 } |
    ForEach-Object { $count }

That is, the $count variable that is assigned to in the Where-Object script block is accessible in the ForEach-Object script block as well, input object by input object.

That said, you can do all you need with ForEach-Object alone:

'a,b,c', 'x,y', '1,2,3', 'a,b,c,d' |
  ForEach-Object { $count = ($_ -split ',').Count; if ($count -gt 2) { $count } }

^{Note that I've switched from the .Split() method to using PowerShell's more flexible -split operator.}

As for what you tried:

Outputting an array (enumerable) to the pipeline causes its elements to be sent one by one rather than as a whole array - see this answer for background information.

The simplest way to avoid that, i.e, to send an array as a whole, is to wrap such an array in an auxiliary single-element wrapper array, using the unary form of ,, the array-construction operator: , $_.Split(',')

Note that enclosing a command in @(...) does not perform the same wrapping, because @(...) doesn't construct an array; loosely speaking, it merely ensures that the output is an array, so if the input already is an array - as in your case - @(...) is - loosely speaking - a (costly) no-op - see the bottom section of this answer for details.

edited Mar 28, 2022 at 2:37

answered Sep 13, 2019 at 22:18

mklement0

453k68 gold badges728 silver badges988 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Slawomir Brzezinski Over a year ago

Re: "Outputting an array ... causes its elements to be sent one by one rather than as a whole array." It's probably useful to clarify: it's not quite outputting just an 'array', but outputting arrays whose some items are arrays, and the 'sending of items one by one' applies to the items - items that are arrays get 'flattened' onto the outer array. The flattening happens to ForEach-Object { @($_.Split(',')) } because it is such a 'array whose items are arrays'. An SO answer here may provide some more explanation of this behavior.

mklement0 Over a year ago

@SlawomirBrzezinski, there are no nested arrays in play in this case, assuming that's what you mean, and discussing the behavior in terms of flattening of arrays rather than enumeration is ultimately confusing. In each ForEach-Object iteration, $_.Split(',') returns a flat array (wrapping it in @(...) is just a costly no-op). On output to the pipeline, the array gets enumerated, so that each iteration outputs individual strings, which, when captured in a variable becomes a regular (in this case flat) PowerShell array, of type [object[]].

Slawomir Brzezinski Over a year ago

I see that, with my suggestion for different description, I was not mindful of the context, which is piping (I came here from discussion of cmdlets' results without pipes). I pointed that ForEach-Object cmdlet produces enumerable, but everything in piping does - enums wrap everything (even $scalar | ...), so you are correct to call items the 'output'. Sorry! Re: "discussing the behavior in terms of ... arrays rather than enumeration is ultimately confusing" I used 'arrays' only in the way you used the term as synonym (in your "array (enumerable)"), else my comment would be too long ;P

Slawomir Brzezinski Over a year ago

As said, your description is indeed enough for pipes, because they add the outer enumerable implicitly, but just to demonstrate the interesting fact about this behavior outside of pipes context, execute this one-liner: function GetSingleResultAsArray { return ,"1" }; "$($(GetSingleResultAsArray).GetType())". It outputs 'string', so the array (forced with ,"1") got flattenned. My SO answer I mentioned may be useful if interested further.

mklement0 Over a year ago

@SlawomirBrzezinski, pipelines also apply without explicit use of |, the pipeline operator: any script or function outputs to the pipeline, so the same rules apply to your GetSingleResultAsArray function: The single-element array , '"1", was enumerated and capturing the output stream that contains that single object therefore captured that object itself. Conceptualizing the PowerShell pipeline in terms of arrays only leads to confusion.

|

Collectives™ on Stack Overflow

Prevent PowerShell ForEach-Object to flatten the list?

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related