Unable to write data across columns in a csv file

Question

I've written a script in python to scrape different names and their values out of a table from a webpage and write the same in a csv file. My below script can parse them flawlessly but I can't write them to a csv file in a customized manner.

What I wish to do is write the names and values across columns which you may see in image 2.

This is my try:

import csv
from bs4 import BeautifulSoup
import requests

res = requests.get("https://www.bloomberg.com/markets/stocks",headers={"User-Agent":"Mozilla/5.0"})
soup = BeautifulSoup(res.text, "lxml")
with open("outputfile.csv","w",newline="") as infile:
    writer = csv.writer(infile)
    for table in soup.select(".data-table-body tr"):
        name = table.select_one("[data-type='full']").text
        value = table.select_one("[data-type='value']").text
        print(f'{name} {value}')
        writer.writerow([name,value])

Output I'm getting like below:

How I wish to get the output is like the following:

Any help to solve this will be vastly appreciated.

Andersson · Accepted Answer · 2018-08-25 14:55:28Z

2

Try to define empty list, append all the values in a loop and then write them all at once:

with open("outputfile.csv","w",newline="") as infile:
    writer = csv.writer(infile)
    names_and_values = []
    for table in soup.select(".data-table-body tr"):
        name = table.select_one("[data-type='full']").text
        value = table.select_one("[data-type='value']").text
        print(f'{name} {value}')
        names_and_values.extend([name,value])
    writer.writerow(names_and_values)

answered Aug 25, 2018 at 14:55

Andersson

52.8k18 gold badges83 silver badges132 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

ThePoetCoder · Accepted Answer · 2018-08-25 14:55:48Z

1

If I understand you correctly, try making just one call to writerow instead of one per loop

import csv
from bs4 import BeautifulSoup
import requests

res = requests.get("https://www.bloomberg.com/markets/stocks",headers={"User-Agent":"Mozilla/5.0"})
soup = BeautifulSoup(res.text, "lxml")
with open("outputfile.csv","w",newline="") as infile:
    writer = csv.writer(infile)
    data = []
    for table in soup.select(".data-table-body tr"):
        name = table.select_one("[data-type='full']").text
        value = table.select_one("[data-type='value']").text
        print(f'{name} {value}')
        data.extend([name, value])
    writer.writerow(data)

answered Aug 25, 2018 at 14:55

ThePoetCoder

1828 bronze badges

1 Comment

SIM Over a year ago

Thanks @ThePoetCoder, your code do the job as well. Wish I could accept both of the answers.

Denziloe · Accepted Answer · 2018-08-25 14:50:28Z

0

That seems like an ugly thing to want to do, are you sure?

Use pandas for getting csvs and manipulating tables. You'll want to do something like:

import pandas as pd

df = pd.read_csv(path)
df.values.ravel()

answered Aug 25, 2018 at 14:50

Denziloe

8,3523 gold badges31 silver badges40 bronze badges

Collectives™ on Stack Overflow

Unable to write data across columns in a csv file

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related