How to express Azure File Share location using Powershell

Question

I am trying to modify current Powershell to point to Azure File Share.

Currently, bottom Powershell is used in a local drive, but I would like to change to pointing to Azure File Share.

param ( [String] $Filename  = '*.csv',
        [String] $SourceDir = 'C:\Test\',
        [String] $TargetDir = 'C:\Test\convert\'
)

ForEach ($csvFile in Get-ChildItem -Path $SourceDir -Filter $Filename){
    (Import-Csv $csvFile | 
     ConvertTo-Csv -NoTypeInformation -Delimiter '|') -replace '[,"]' -replace '\|',',' |
         Set-Content (Join-Path $TargetDir $csvFile.Name)                  
}

I have used the bottom Python file that points to Azure File Share like below (for other project).

So, the goal is modify the top Powershell with something similar to Python code below:

How to express URL, account key in the existing Powershell?

from azure.storage.fileshare import ShareServiceClient, ShareClient, 
ShareDirectoryClient, ShareFileClient
import pandas as pd
import io

account_url = "https://example.file.core.windows.net"
account_key = "xxxx=="

service_client = ShareServiceClient(account_url=account_url, credential=account_key)
share_name = "testname"
share_client = service_client.get_share_client(share_name)

# Get a ShareDirectoryClient object to connect to the directory
directory_path = "Parent/Child"
directory_client = share_client.get_directory_client(directory_path)

What I am trying to do is connect the $ctx to the argument (ForEach(...) ):

Bottom is just an example of applying "Remove-AzDtalakeGen2Item" that I found from this YouTube:

$ctx = New-AzStorageContext -StorageAccountName = 'sample' -StorageAccountKey "xxxxxxxxx"
Remove-AzDatalakeGen2Item -Context $ctx -FileSystem "sample" -Path "/Test/" -force
Remove-AzDatalakeGen2Item -Context $ctx -FileSystem "Sample" -Path "/Test/Convert" -force

(1/12/2024): This is an error message:

(1/16/2024):

(1/18/2024):

Passing values directly (1/18/2024)

Have a look at the cmdlets in the Az.Storage module, in particular: Get-AzStorageContainer, Get-AzStorageBlob, Get-AzStorageFile, etc — Santiago Squarzon
– Santiago Squarzon, Commented Dec 13, 2023 at 1:02
@SantiagoSquarzon I updated the content on the top (Update). I guess I need to connect $ctx (context info) with the argument (that starts with "ForEach"). Any suggestion? — Java
– Java, Commented Dec 20, 2023 at 0:07
I guess this is want you are looking for: stackoverflow.com/questions/54110867/… — wp78de
– wp78de, Commented Jan 7, 2024 at 1:42
@wp78de Thank you for your input. I looked at the code on that link, but I am still not sure how I could put all pieces together? I think, regards to $context, I could put username and password and create one, but I am not sure how to apply other folder paths (like $SourceDir, $TargetDir, and $Filename from my code). Can you possibly put the sample codes? — Java
– Java, Commented Jan 8, 2024 at 20:37

Martin Iszac · Accepted Answer · 2024-01-15 00:23:01Z

1

+50

First install Az.Storage using Install-Module -Name Az.Storage -Force -AllowClobber

Here is the script youre looking for:


param (
    [String] $Filename  = '*.csv',
    [String] $SourceDir = 'Test',
    [String] $TargetDir = 'Test/convert'
)

$storageAccountName = "your-storage-account-name"
$storageAccountKey = "your-storage-account-key"
$shareName = "your-share-name"

$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey

$allFiles = Get-AzStorageFile -Context $ctx -ShareName $shareName -Path $SourceDir 
$csvFiles = $allFiles | Where-Object { $_.Name -like $Filename }

ForEach ($csvFile in $csvFiles) {
    $fileExists = Get-AzStorageFile -Context $ctx -ShareName $shareName -Path "$TargetDir/$($csvFile.Name)" -ErrorAction SilentlyContinue

    if ($null -eq $fileExists) {
        $content = Get-AzStorageFileContent -Context $ctx -ShareName $shareName -Path "$SourceDir/$($csvFile.Name)" |
                    Out-String |
                    ConvertFrom-Csv -Delimiter ',' |
                    ConvertTo-Csv -NoTypeInformation -Delimiter '|' |
                    ForEach-Object { $_ -replace '[,"]' -replace '\|',',' }

        Set-AzStorageFileContent -Context $ctx -ShareName $shareName -Path "$TargetDir/$($csvFile.Name)" -Value $content
    } else {
        Write-Host "File $($csvFile.Name) already exists in the Azure File Share."
    }
}

Get-AzStorageFile is used to check if a file with the same name already exists in the Azure File Share. If the file does not exist, Get-AzStorageFile will return $null and the script will upload the local file. If the file does exist, the script will print a message and skip the upload. Feel free to remove this part if you dont care about the files being rewritten.

edited Jan 15, 2024 at 0:23

answered Jan 9, 2024 at 19:22

Martin Iszac

5443 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

24 Comments

Martin Iszac Over a year ago

No problem, I've updated the code for what you need, let me know if that works for you.

Martin Iszac Over a year ago

In Azure Data Factory, you typically don’t need to manually install the Az.Storage module. Azure Data Factory should automatically ensure that these modules are available for your functions. Just in case, make sure you have this setup correctly: stackoverflow.com/questions/57882577/…

Martin Iszac Over a year ago

Yes you're correct.

Martin Iszac Over a year ago

Yes its this module: powershellgallery.com/packages/Az.Storage

Martin Iszac Over a year ago

Yes you need to be an admin on your local PC and run PowerShell elevated, if you don't have permissions then your IT team will need to install it for you.

|

Collectives™ on Stack Overflow

How to express Azure File Share location using Powershell

1 Answer 1

24 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

24 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related