Nextflow: Script of python don't save output

Question

I have problem with a script of Python in Nextflow, my aim is write a file in the script of python and take this with nextflow and save the file in the publishdir (and after I use this file in other process). My process in nextflow is something like this (the files were defined before):

process writefile{
publishDir "${params.output_dir}/formatted", mode: 'copy'
input:
path file from change_file
output:
path "formattedfile.txt" into file_changed
script:
"""
file2formattedfile.py ${file} formattedfile.txt
"""
}

The script of python: (I simplified the real process, but essentiality is something like this), I need obtain in nextflow the file save in output file.

#!/usr/bin/env python3
import argparse
from sys import argv
def main():
   input,output = argv[1:3] 
   out = open(output, "w") 
   #My real operations are here
   out.write("Operations and text") 
   out.close() 

if __name__ == "__main__":
   main()

The problem is the file is don't save in the publish dir, but is in the dir work of nextflow, when i run the workflow the process is completed without error but said DataflowQueue(queue=[])

[e1/74e0ee] process > writefile (DataflowQueue(queue=[])) [100%] 1 of 1 ✔

Thanks!

------------- Update -------------

I changed the input file to a file(). The nextflow.config:

params {
  input_file = 'data/old_file.txt'
  output_dir = 'output_new'
}

The main.nf

change_file = file(params.input_file)
process writefile{
publishDir "${params.output_dir}/formatted", mode: 'copy'
input:
path file from change_file
output:
path "formattedfile.txt" into file_changed
script:
"""
file2formattedfile.py ${file} formattedfile.txt
"""
}

This changed the ouptput of nextflow, but my input file wasn't in the publish dir (but is in the dir work).

[7d/78559b] process > writefile (/home/myuser/Documentos/dir/pipeline_dir/data/old_file.txt) [100%] 1 of 1 ✔

This path after writefile is the path where is my input file, I don't know why (nothing is change in this dir).

There doesn't appear to be anything wrong with your updated code above. You mentioned that your input file wasn't in the publish dir. This is because, by default, input files are not included in process outputs. Try adding path infile to your process 'output' declaration. This will ensure it is captured and let the publishDir directive copy it to your output directory. — Steve
– Steve, Commented Sep 30, 2020 at 0:05
The writefile path on stdout (/home/myuser/Documentos/...) is just your input file. In your actual code (not shown above), you have probably just tagged the process with tag { infile } or something like that. — Steve
– Steve, Commented Sep 30, 2020 at 0:08

Steve · Accepted Answer · 2020-09-29 23:30:38Z

1

Looks like something is wrong with your input using the 'change_file' channel. If 'change_file' should be a value channel, consider the following:

params.change_file = 'my_change_file.txt'
params.output_dir = 'my_output_dir'

change_file = file(params.change_file)


process writefile{

    publishDir "${params.output_dir}/formatted", mode: 'copy'

    input:
    path infile from change_file

    output:
    path "formattedfile.txt" into file_changed

    """
    file2formattedfile.py "${infile}" formattedfile.txt
    """
}

Results:

[d6/5173c1] process > writefile [100%] 1 of 1 ✔

If the above doesn't help, please show how the 'change_file' channel is being created.

edited Sep 29, 2020 at 23:30

answered Sep 28, 2020 at 23:39

Steve

55.1k13 gold badges94 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

JocHwo Over a year ago

Thanks for the answer! I changed the input, and this changed the output of nextflow but my file is not yet in the publish dir. I updated the post with the new information

Steve Over a year ago

@JocHwo do you mean the input file is not in the publishDir or the output file is not in the publishDir? For me, the above works as expected and copies the output file, 'formattedfile.txt', to the publishDir.

Collectives™ on Stack Overflow

Nextflow: Script of python don't save output

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related