How to use output of a powershell command as parameters in Azure pipeline?

Question

I have a PowerShell script task that gets the names of some files from a folder in my git repo and puts them into a variable. I want to use those file names in parameters and use "each" condition in another task (task: HelmDeploy@0) in order to run that task each time with one of the file names as valueFile variable.

Here is what I have tried, however it gives an error Template-Yaml/deploy-jobs.yaml@pipelinetemplates Expected a sequence or mapping. Actual value '$[dependencies.A.outputs['fileoutput.valuefiles']]' in line ${{each file in parameters.files}}

deploy-jobs.yaml

   parameters:
     files: []

   jobs:
     - job: Deploy
       pool:
         vmImage: 'ubuntu-latest'
       variables:
         filenames: ${{ parameters.files }}
       steps:
       - task: HelmInstaller@1
         displayName: 'Installing Helm'
         inputs:
           helmVersionToInstall: '2.15.1'

       - task: HelmDeploy@0
         displayName: 'Initializing Helm'
         inputs:
           connectionType: 'Azure Resource Manager'
           azureSubscription: $(azureSubscription)
           azureResourceGroup: $(azureResourceGroup)
           kubernetesCluster: $(kubernetesCluster)
           command: 'init'

       - task: AzureCLI@2
         inputs:
           azureSubscription: $(azureSubscription)
           scriptType: 'bash'
           scriptLocation: 'inlineScript'
           inlineScript: 
             echo "##vso[task.setvariable variable=imgtag]$(az acr repository show-tags --name myacr --repository myrepo --orderby time_desc --top 1 | awk ' /[[:digit:]]/ { print $0 } ' | tr -d '[:space:]')"

       - task: Bash@3
         displayName: 'Fetching repo-tag'
         inputs:
           targetType: 'inline'
           script: |
             echo tag=$(imgtag)
             echo filenames=$(filenames) ### output is: **/myfolder/dev/file1.yaml,**/myfolder/dev/file2.yaml

      - ${{each file in parameters.files}}: ##Error 
       - task: HelmDeploy@0
         displayName: 'Upgrading helmchart'
         inputs:
           connectionType: 'Azure Resource Manager'
           azureSubscription: $(azureSubscription)
           azureResourceGroup: $(azureResourceGroup)
           kubernetesCluster: $(kubernetesCluster)
           command: 'upgrade'
           chartType: 'FilePath'
           chartPath: $(chartPath)
           install: true
           releaseName: $(releaseName)
           ##valueFile: $(valuefiles)
           valueFile: ${{ file }}
           arguments: '--set image.tag=$(imgtag) --set domain=$(domain)'

azure-pipeline.yaml file is as following:

  trigger:
    branches:
      include:
        - master
        - refs/tags/v*
    paths:
      exclude:
        - readme.md

  variables:
    azureSubscription: 'myazuresubscription'
    chartPath: '**/mychart'
    containerRegistry: 'mysc'
    repository: 'myrepo'

  resources:
    repositories:
    - repository: pipelinetemplates
      type: github
      name: 'mygitorg/myrepo'
      endpoint: 'mygitorg'

  stages:
    - stage: Deploy_Cluster
      variables:
        azureResourceGroup: 'myresourcegroup'
        kubernetesCluster: 'mycluster'
        releaseName: 'mychartreleasename'
        #valueFile: '**/mychart/values.yaml'
        domain: 'mydomain'
      jobs:

 - job: A
   pool:
    vmImage: 'ubuntu-latest'
   steps:
   - task: PowerShell@2
     displayName: 'Fetching ValueFiles'
     inputs:
      targetType: 'inline'
      script: |
       Write-Host "Fetching value files"
       cd myfolder
       $a=git ls-files
       $List = $a | foreach {'**/myfolder/dev/' + $_}
       Write-Host $List
       $d = '"{0}"' -f ($List -join '","')
       Write-Host $d   ### output is: "**/myfolder/dev/file1.yaml","**/myfolder/dev/file2.yaml"
       Write-Host "##vso[task.setvariable variable=valuefiles;isOutput=true]$d"
     name: fileoutput

      - template: Template-Yaml/deploy-jobs.yaml@pipelinetemplates  ##Error expected a sequence or mapping
        parameters: 
          files : $[dependencies.Deploy.outputs['fileoutput.valuefiles']]

I got some idea from this page: https://www.aaron-powell.com/posts/2019-05-24-azure-pipeline-templates-and-parameters/ regarding using dependencies.

I googled a lot, however I couldn't find a solution to this issue so far, any help would be appreciated.

Tested the replied suggested by Levi:

parameters:
  files: []

jobs:
#- ${{each file in parameters.files}}:
   - job: Deploy
     dependsOn: A
     pool:
       vmImage: 'ubuntu-latest'  
     variables:
       filenames: ${{ parameters.file }} 
     steps:
     - task: Bash@3
       displayName: 'Fetching repo-tag'
       inputs:
         targetType: 'inline'
         script: |
           ##echo files=$(filenames) #output is files=file1.yaml,file2.yaml
           for i in $(filenames)
           do
              echo "valuefiles= $i "
           done

OutPut is valuefiles= files=file1.yaml,file2.yaml

Testing with PowerShell:

 - task: PowerShell@2
   displayName: 'Fetching ValueFiles'
   inputs:
     targetType: 'inline'
     script: |
       foreach ($i in ${{ parameters.files }}) {
           Write-Host "filenames=$i"
       }

Error: ObjectNotFound: ($[dependencies.A.ou\u2026output.valuefiles]]:String) [], ParentContainsErrorRecordException

- task: PowerShell@2
   displayName: 'Fetching ValueFiles'
   inputs:
     targetType: 'inline'
     script: |
       foreach ($i in $(filenames)) {
           Write-Host "filenames=$i"
       }

Error: foreach ($i in ) { + ~ Unexpected token ')' in expression or statement. + CategoryInfo : ParserError: (:) [], ParseException

I would consider to create a script and invoke that with your arguments from the pipeline. For me, the inline PowerShell escape syntax is heavy going... — Martin Brandl
– Martin Brandl, Commented Dec 20, 2019 at 12:33
I would like to use script in the pipeline so that whenever I push a new file.yaml to that folder, the pipeline gets triggered, scans the folder, gets the filename and then run the HelmDeploy task based on that filename. It can be any script language bash or python as long as I know how the algorithm should be. — Matrix
– Matrix, Commented Dec 20, 2019 at 12:37

Levi Lu-MSFT · Accepted Answer · 2019-12-23 09:59:01Z

1

The error occurred is because the files can only be caculated at runtime, but - ${{each file in parameters.files}} is valuated at compile time. Check here for more information about variable syntax.

- ${{each file in parameters.files}} won't work if the dynamic variables have been passed through parameters.

I doesnot know much about kubernetes, if you can manage to use powershell/bash script to do the HelmDeploy task. You can foreach the files inside script and deploy each file.

- powershell: |
          foreach ($i in $(filenames)) {helm.exe upgrade ....}

A similar issue has been submitted to Microsoft Develop team, and it is determined that above issue is by design. But you can still submit a feature request by clicking "Suggest a feature " and choose "azure devops"

Update:

azure-pipeline.yaml

trigger:
    branches:
      include:
        - master
 stages:
  - stage: Deploy_Cluster

    jobs:
    - job: A
      pool:
        vmImage: 'ubuntu-latest'
      steps:
      - task: PowerShell@2
        displayName: 'Fetching ValueFiles'
        inputs:
          targetType: 'inline'
          script: |
            Write-Host "Fetching value files"
            cd '$(system.defaultworkingdirectory)'
            $a=git ls-files
            $List = $a | foreach {'**/myfolder/dev/' + $_}
            Write-Host $List
            $d = '"{0}"' -f ($List -join '","')
            Write-Host $d  

            Write-Host "##vso[task.setvariable variable=valuefiles;isOutput=true]$d"

        name: fileoutput

    - job: B
      dependsOn: A
      pool:
        vmImage: 'ubuntu-latest'
      variables: 
        allfiles: $[dependencies.A.outputs['fileoutput.valuefiles']]
      steps:
      - template: deploy-jobs.yaml
        parameters: 
          files : $(allfiles)

deploy-jobs.yaml

parameters:
  files: []

steps :
- task: PowerShell@2
  displayName: 'Fetching ValueFiles'
  inputs:
    targetType: 'inline'
    script: >

      foreach ($i in ${{parameters.files}})
      {
        Write-Host "filenames=$i"
      }

edited Dec 23, 2019 at 9:59

answered Dec 23, 2019 at 7:41

Levi Lu-MSFT

30.8k3 gold badges42 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Matrix Over a year ago

Thanks for the reply. Yes I can use a bash script in the pipeline but can you please tell me how to do this "You can foreach the files inside script and deploy each file." I have tried many different things but it just keeps giving me error...

Levi Lu-MSFT Over a year ago

To do the helm upgrade work in powershell task, some scripts like above updated powershell scripts

Matrix Over a year ago

so in this line: foreach ($i in $(filenames)) where the filenames are coming from? Doesn't it use any parameters? This is an example of each-loop and parameters which I tried to use however it doesn't work for my case mattvsts.github.io/2019/05/04/…

Levi Lu-MSFT Over a year ago

It comes for the variable filenames: ${{ parameters.files }} you defined in the template. You can also try using foreach ($i in${{ parameters.files }})

Levi Lu-MSFT Over a year ago

You should write the foreach script in one line or check this thread to split a single command. I updated my simple yaml

|

Collectives™ on Stack Overflow

How to use output of a powershell command as parameters in Azure pipeline?

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related