How do you create a unix file format in Powershell? I am using the following to create a file, but it always creates it in the windows format.
"hello world" | out-file -filepath test.txt -append
As I understand, the new line characters CRLF make it to be a Windows format file whereas the unix format needs only a LF at the end of the line. I tried replacing the CRLF with the following, but it didn't work
"hello world" | %{ $_.Replace("`r`n","`n") } | out-file -filepath test.txt -append
There is a Cmdlet in the PowerShell Community Extensions called ConvertTo-UnixLineEnding
One ugly-looking answer is (taking input from dos.txt outputting to unix.txt):
[string]::Join( "`n", (gc dos.txt)) | sc unix.txt
but I would really like to be able to make Set-Content do this by itself and this solution does not stream and therefore does not work well on large files...
And this solution will end the file with a DOS line ending as well... so it is not 100%
Set-Content -NoNewLine to avoid this problem.Two more examples on how you can replace CRLF by LF:
Example:
(Get-Content -Raw test.txt) -replace "`r`n","`n" | Set-Content test.txt -NoNewline
Example:
[IO.File]::WriteAllText('C:\test.txt', ([IO.File]::ReadAllText('C:\test.txt') -replace "`r`n","`n"))
Be aware, this does really just replace CRLF by LF. You might need to add a trailing LF if your Windows file does not contain a trailing CRLF.
I've found that solution:
sc unix.txt ([byte[]][char[]] "$contenttext") -Encoding Byte
posted above, fails on encoding convertions in some cases.
So, here is yet another solution (a bit more verbose, but it works directly with bytes):
function ConvertTo-LinuxLineEndings($path) {
$oldBytes = [io.file]::ReadAllBytes($path)
if (!$oldBytes.Length) {
return;
}
[byte[]]$newBytes = @()
[byte[]]::Resize([ref]$newBytes, $oldBytes.Length)
$newLength = 0
for ($i = 0; $i -lt $oldBytes.Length - 1; $i++) {
if (($oldBytes[$i] -eq [byte][char]"`r") -and ($oldBytes[$i + 1] -eq [byte][char]"`n")) {
continue;
}
$newBytes[$newLength++] = $oldBytes[$i]
}
$newBytes[$newLength++] = $oldBytes[$oldBytes.Length - 1]
[byte[]]::Resize([ref]$newBytes, $newLength)
[io.file]::WriteAllBytes($path, $newBytes)
}
make your file in the Windows CRLF format. then convert all lines to Unix format in new file:
$streamWriter = New-Object System.IO.StreamWriter("\\wsl.localhost\Ubuntu\home\user1\.bashrc2")
$streamWriter.NewLine = "`n"
gc "\\wsl.localhost\Ubuntu\home\user1\.bashrc" | % {$streamWriter.WriteLine($_)}
$streamWriter.Flush()
$streamWriter.Close()
not a one-liner, but works for all lines, including EOF. new file now shows as Unix format in Notepad on Win11.
delete original file & rename new file to original, if you like:
ri "\\wsl.localhost\Ubuntu\home\user1\.bashrc" -Force
rni "\\wsl.localhost\Ubuntu\home\user1\.bashrc2" "\\wsl.localhost\Ubuntu\home\user1\.bashrc"
System.IO.StreamWriter type's .NewLine property. As an inconsequential aside: you don't need to call .Flush() before .Close()..Close(). If that weren't the case, it would amount to a serious bug/design flaw. Even in W11 / WinPS v5.1 I don't see a problem with not calling .Flush(), so if there is a problem, after all, it would be good to know what specific OS / PowerShell versions are affected.The following makes sure that the output file has Unix-style line endings (LF) no matter what line endings the input has:
Get-Content input.txt | % { $_ + "`n"} | Set-Content output.txt -NoNewline
Conversely the following makes that the output file has Windows-style line endings (CR LF) no matter what line endings the input has:
Get-Content input.txt| % { $_ + "`r`n"} | Set-Content output.txt -NoNewline
For anyone is searching solutions for pipeline, here is a simple one: just replace io redirector > with the code snippet | % { [Text.Encoding]::UTF8.GetBytes($_ + "`n") } | Set-Content -Encoding Byte
For example:
PS C:\> '你好4536251', 'Mixed With English' > test.txt; bash -c 'file test.txt'
test.txt: Unicode text, UTF-16, little-endian text, with CRLF line terminators
PS C:\> '你好4536251', 'Mixed With English' | % { [Text.Encoding]::UTF8.GetBytes($_ + "`n") } | Set-Content -Encoding Byte test.txt ; bash -c 'file test.txt'
test.txt: Unicode text, UTF-8 text
