Writing console output to a file in python

Question

Sorry for this question, I know that this is a recurrent topic, but I'm incapable of solving my problem, which in fact, is simple to describe: I want to write into a file the output of an execution as it is. I mean, I have an output like this (sorry for the mess):

.251.242.133|:80... connected.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: 2371567 (2.3M) [application/x-gzip]\r\nSaving to: `110907_ERP000591.tar.gz\'\r\n\r\n\r 0% [                                       ] 0           --.-K/s              \r 0% [                                       ] 23,003       104K/s              \r 3% [>                                      ] 82,863       184K/s              \r 8% [==>                                    ] 192,363      282K/s              \r15% [=====>                                 ] 371,943      411K/s              \r26% [=========>                             ] 634,175      563K/s              \r39% [==============>                        ] 925,283      680K/s              \r52% [===================>                   ] 1,250,295    790K/s              \r63% [=======================>               ] 1,497,035    830K/s              \r73% [===========================>           ] 1,732,663    861K/s              \r81% [==============================>        ] 1,937,063    867K/s              \r88% [=================================>     ] 2,099,123    855K/s              \r95% [====================================>  ] 2,268,483    847K/s              \r100%[======================================>] 2,371,567    849K/s   in 2.7s    \r\n\r\n2012-11-01 15:34:10 (849 KB/s) - `110907_ERP000591.tar.gz\' saved [2371567/2371567]\r\n\r\n110907_ERP000591/\r\n110907_ERP000591/1_110907_ERP000591_2_fastq.txt\r\n110907_ERP000591/1_110907_ERP000591_1_fastq.txt\r\n/home/travis/opt/bcbb/nextgen/tests/data/automated/../100326_FC6107FAAXX\r\n--2012-11-01 15:34:10--  http://chapmanb.s3.amazonaws.com/100326_FC6107FAAXX.tar.gz\r\nResolving chapmanb.s3.amazonaws.com (chapmanb.s3.amazonaws.com)... 205.251.242.133\r\nConnecting to chapmanb.s3.amazonaws.com (chapmanb.s3.amazonaws.com)|205.251.242.133|:80... connected.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: 7014592 (6.7M) [application/x-gzip]\r\nSaving to: `100326_FC6107FAAXX.tar.gz\'\r\n\r\n\r 0% [                                       ] 0           --.-K/s              \r 0% [                                       ] 17,163      77.9K/s              \r 0% [                                       ] 64,775       147K/s              \r 2% [                                       ] 174,843      263K/s              \r 5% [=>                                     ] 399,683      456K/s              \r12% [===>                                   ] 866,883      790K/s              \r25% [========>                              ] 1,798,363   1.33M/s              \r45% [================>                      ] 3,178,955   1.90M/s              \r65% [========================>              ] 4,592,803   2.41M/s              \r65% [========================>              ] 4,629,303   2.17M/s              \r67% [=========================>             ] 4,761,595   2.02M/s              \r74% [============================>          ] 5,245,423   2.03M/s              \r83% [===============================>       ] 5,862,435   2.08M/s              \r100%[======================================>] 7,014,592   2.46M/s   in 2.7s    \r\n\r\n2012-11-01 15:34:13 (2.46 MB/s) -

So, as you can see, I have a weirdly formatted output, which is shown like this in the terminal:


/home/travis/opt/bcbb/nextgen/tests/data/automated/../100326_FC6107FAAXX
--2012-11-01 15:34:10--  http://chapmanb.s3.amazonaws.com/100326_FC6107FAAXX.tar.gz
Resolving chapmanb.s3.amazonaws.com (chapmanb.s3.amazonaws.com)... 205.251.242.133
Connecting to chapmanb.s3.amazonaws.com (chapmanb.s3.amazonaws.com)|205.251.242.133|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7014592 (6.7M) [application/x-gzip]
Saving to: `100326_FC6107FAAXX.tar.gz'

100%[======================================>] 7,014,592   2.46M/s   in 2.7s    

2012-11-01 15:34:13 (2.46 MB/s) - `100326_FC6107FAAXX.tar.gz' saved [7014592/7014592]

Much nicer... I want to write the output to a file with this form, so I cant just strip the \r\n or something like that, because this way will appear one line for each step in the progress of the download. I just want to write a file with the final output.

Any help?

EDIT:

Sorry, I should have been clearer: It's not the result of a command but, as commented below, the result of the parsing of a json file. You can reproduce the output:


import urllib
import json

string_to_write = json.loads(urllib.urlopen('https://travis-ci.org/jobs/3019024.json').read())['log']

This should basically work as it is. Meaning, if you got a string s = 'foo\n\rbar' in Python, and write that out to a file with open('outfile.txt, 'w').write(s), the file should contain those line endings and render correctly when cated for example. — Lukas Graf
– Lukas Graf, Commented Nov 2, 2012 at 14:29
Although if you are on Windows, there is a braindead distinction between "binary mode" and "text mode" when writing files. So you might need to use either the 'wb' or 'w' mode, not sure. See the comments in the Python docs about reading and writing files. — Lukas Graf
– Lukas Graf, Commented Nov 2, 2012 at 14:32
Sorry, in fact it's not a command itself, it's the result of parsing the return of [travis-ci api](www.travis-ci.org), concretely, the field ['matrix'][0]['id'] from the json returned by this url — guillemch
– guillemch, Commented Nov 2, 2012 at 14:33
This looks like wget output, which assumes a TTY. Perhaps you should use the --progress=dot switch instead? — Martijn Pieters
– Martijn Pieters, Commented Nov 2, 2012 at 14:34

Zero Piraeus · Accepted Answer · 2012-11-02 15:06:30Z

1

This ought to work:

re.sub(r"(?m)^.*\r(?!$)", "", text)

The (?m) flag indicates that this is a multiline regex, so that ^ and $ match the beginning and end of each line, rather than the whole text.

So, we match everything .* from the start of the line ^ up to and including a carriage return \r, unless that carriage return is immediately followed by the end of the line (?!$) - as it will be if your text contains CRLF linebreaks. And, replace the match with the empty string.

edited Nov 2, 2012 at 15:06

answered Nov 2, 2012 at 14:41

Zero Piraeus

59.7k28 gold badges158 silver badges164 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Liso Over a year ago

I'v got '\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n2012-11-01 15:34:13 (2.46 MB/s) ' is it really good result?

inspectorG4dget · Accepted Answer · 2012-11-02 14:32:35Z

0

Looks like you're on a *NIX system. If this output is generated by running progn (which could really be python myscript.py) on the command line, you could just as easily do this:

progn > logfile

This way, the output of progn is saved to logfile without stripping any characters.

answered Nov 2, 2012 at 14:32

inspectorG4dget

115k30 gold badges159 silver badges253 bronze badges

Comments

Liso · Accepted Answer · 2012-11-02 16:05:48Z

0

If you want just remove what is hidden on terminal then you could try this ->

# t is your text
''.join([i for i in t.split('\r') if i.startswith('\n') or i.startswith('100%')])

Solution is tricky (!) but I think it could probably works for you.

answered Nov 2, 2012 at 16:05

Liso

2,2701 gold badge17 silver badges20 bronze badges

Collectives™ on Stack Overflow

Writing console output to a file in python

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related