Using Python and Mechanize to submit data in the website's html

Question

I have this website and there are four input boxes which are Symbol, Expiry Date, From, To. Now i have written a code to scrape data from the Symbol and Expiry Date which is like this:

import requests
import json
from bs4 import BeautifulSoup

r = requests.get("http://www.mcxindia.com/sitepages/BhavCopyCommodityWise.aspx")
soup = BeautifulSoup(r.content)
pop = []
pop_dates = []
count = 0
print soup.prettify()

option_list = soup.findAll("option")
#print option_list

for value in option_list:
    #print value
    if value.find(text = True):
        text = ''.join(value.find(text = True))
        text1 = text.encode('ascii')

        if count < 32:  
           pop.append(text1) 

    while count == 32 or count > 32:
        pop_dates.append(text1)
        break

    count = count + 1

print pop
print pop_dates

So What i want to do is for From and To i want to give the dates from my code and it will take that input, use it on the website's html and give the output as usual in that website. How can i do this?? I heard mechanize can do this stuffs but how could i use mechanize in this case??

coder006 · Accepted Answer · 2014-06-05 12:28:49Z

1

You can try out something like this:

from mechanize import Browser
from bs4 import BeautifulSoup

br = Browser()
br.set_handle_robots( False )
br.addheaders = [('User-agent', 'Firefox')]
br.open("http://www.mcxindia.com/sitepages/BhavCopyCommodityWise.aspx")
br.select_form("form1")
#now enter the dates according to your choice
br.form["mTbFromDate"] = "date-From"
br.form["mTbFromDate"] = "date-To"
response = br.submit()
#now read the response with BeautifulSoup and do whatever you want
soup = BeautifulSoup(response.read())

answered Jun 5, 2014 at 12:28

coder006

5251 gold badge7 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Using Python and Mechanize to submit data in the website's html

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related