Encoding issue piping value to command

Question

I have a head scratcher and I am unsure on how to solve this one. I am looking for an enlightened individual that would know what this is about.

Let's begin with the center piece of this interrogation, this simple piece of code.

Function New-Thingy {
    [CmdletBinding()]
    Param(
        [Parameter(ValueFromPipeline = $true)]$testing
    )
  # Tried playiing with encoding but it didn't do anything.
  #  $Encoding = [System.Text.Encoding]::Default
  #  [Console]::OutputEncoding = $Encoding
  #  [Console]::InputEncoding = $Encoding

      Write-Host $testing
}

'options — test' | New-Thingy

Please note that the problem does not occurs in Powershell ISE but do in VSCODE and Powershell (by invoking the ps1 script containing this code).

From both VS code and Powershell, running the script gives the following output: options â€” test instead

Output should be : options — test

Using F8 on that line of code afterward produce the correct output but F5 execution always output on the wrong codepage.

Furthermore, using an expandable string instead of single quoted produce the following error message :

+ "options â€” test" | New-Thingy
+                  ~~~~~~~~~~~~~~
The string is missing the terminator: ".

How do I get the correct output ? I attempted changing the console output / input encoding but it changes nothing
What is happening with the expandable string and is there any way to prevent it ?

Again, Powershell ISE do the work properly, produce the correct output and does not give any error using expandable string there.

Regular powershell console does and so does VS code.

I am continuing to seek for an answer but I must admin I am a bit lost here.

Maximilian Burszley · Accepted Answer · 2018-08-30 23:36:31Z

2

In your code, you're using a long-dash (—). In Windows PowerShell, if your script is not saved as utf8 with bom, it parses it as "ANSI" encoding which this character falls outside the range of since it takes up more than 1-byte as you can see by the glyph being misrepresented. This is corrected in PowerShell Core where the default is utf8 encoding parsing (versus ANSI).

Saving your Windows PowerShell scripts as utf8bom going forward will remediate this "bug".

In VSCode settings.json with powershell extension:

"[powershell]": {
    "files.encoding": "utf8bom"
}

answered Aug 30, 2018 at 23:36

Maximilian Burszley

19.9k7 gold badges39 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Sage Pourpre Over a year ago

Woah ! Double thanks for the VSCode setting !!! That will be very handy.

Maximilian Burszley Over a year ago

@SagePourpre The only place that will cause friction is if you're writing scripts for powershell core (which won't be very soon in my experience).

Collectives™ on Stack Overflow

Encoding issue piping value to command

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related