Information in csv file not retrieved properly using python

Question

I am scraping some contents from this site. While writing like the conference head after extracting from the site in csv file the first name is not coming properly, e.g. if the word is microsoft it is coming as osoft but rest all of the words are coming properly

Here is my code:

import csv
import requests
from bs4 import BeautifulSoup

with open('random.csv', 'w') as csvfile:
    a = csv.writer(csvfile)
    a.writerow(["conferenceHead"])

    url = given above      
    r = requests.get(url)
    soup = BeautifulSoup(r.content)
    links = soup.find_all("div")

    r_data = soup.find_all("div",{"class":"conferenceHead"})
    for item in r_data:
        conferenceHead = item.contents[1].text


        with open('random.csv','a') as csvfile:
            a = csv.writer(csvfile)
            data = [conferenceHead]
        a.writerow(data)

JustMe · Accepted Answer · 2016-03-03 10:05:03Z

1

Well, You have three issues in Your code.

dual with open() statements (on the same file)
and second open - in append mode, is in a loop, which makes this even worse
last writerow is out of scope, and csvfile is already closed

This might cause buffer not being written to file, and truncating string You are saving.

After fixing this errors (removing with open('random.csv','a') as csvfile and fixing indentation) code runs and output is not trimmed.

import csv
import requests
from bs4 import BeautifulSoup
with open('random.csv', 'w') as csvfile:
    a = csv.writer(csvfile)
    a.writerow(["conferenceHead"])

    url = "http://www.allconferences.com/search/index"\
          "/Category__parent_id:1/Venue__country:United%20States"\
          "/Conference__start_date__from:01-01-2010/sort:start_date"\
          "/direction:asc/showLastConference:1/page:7/"
    r = requests.get(url)
    soup = BeautifulSoup(r.content)
    links = soup.find_all("div")

    r_data = soup.find_all("div",{"class":"conferenceHead"})

    for item in r_data:
        conferenceHead = item.contents[1].text
        data = [conferenceHead]
        a.writerow(data)

answered Mar 3, 2016 at 10:05

JustMe

7105 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Atul Pant Over a year ago

Is it possible if you can give me your email id and help me with some of the problems.

JustMe Over a year ago

@AtulPant I am sorry, this would be rather unwise to post email. But if You want, enter means of contacting You in Your "about me" page, reply to this comment with \@JustMe and I'll contact You.

Collectives™ on Stack Overflow

Information in csv file not retrieved properly using python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related