Using mechanize to input username and password

Question

How do I use mechanize to input username and password onto this site?

I deleted and changed my post because my previous one had too much extra information

I've read in other posts that maybe this has to do with javascript, but how do I tell? and what do I do with that information?

import mechanize
import cookielib

url = 'https://www.pin1.harvard.edu/cas/login?service=https%3A%2F%2Fwww.pin1.harvard.edu%2Fpin%2Fauthenticate%3F__authen_application%3DFAS_AC_AUTHENTICATOR'
#req = requests.get(url)
#dom = web.Element(req.text)

#Handles all the browser details 
br = mechanize.Browser()
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
#self.browser = mechanize.Browser(factory=mechanize.RobustFactory())

#Cookie Jar
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)

# Browser options
br.set_handle_equiv(True)
br.set_handle_gzip(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)

br.open(url)


#Select First Form
#br.select_form(nr=1)

#br['username'] = '40839852'

#print list(br.forms())[0] 

for form in br.forms():
    print "Form name:", form.name
    print form
    break

br.select_form(name= formname)
br[searchname] = term
res = br.submit()
content = res.read()
dom = web.Element(content)

TRACEBACK

ParseError: unexpected '/' char in declaration
---> 32 for form in br.forms():
     33     print "Form name:", form.name
     34     print form

UPDATE - BASED ON PACO'S SUGGESTION I ADDED...But i still get a traceback. Python unable to retrieve form with urllib or mechanize

beg =  re.search(t, res.read()).span()[1]
res.set_data(res.get_data()[beg:])
br.set_response(response)
br.select_form(nr=0)


<ipython-input-25-bd1b73406b45> in <module>()
     28 br.set_response(response)
     29 
---> 30 br.select_form(nr=0)
     31 
     32 
ParseError: unexpected '-' char in declaration

long shot, but I'm guessing that br.forms isn't a function. What happens if you remove the parentheses? — Paul H
– Paul H, Commented Apr 18, 2014 at 23:26
Unfortunately, it is a function i think. wwwsearch.sourceforge.net/mechanize if you remove the parens, you get: <bound method Browser.forms of <mechanize._mechanize.Browser instance at 0x1095e7d40>> — user3314418
– user3314418, Commented Apr 18, 2014 at 23:27

taco · Accepted Answer · 2014-04-19 00:05:30Z

1

This is how I selected the first form in my code.

br.select_form(nr=0)
#Form fields to populate
br.form['username'] = username
br.form['password'] = password
#Submit the login form
br.submit()

Modify it to suit your needs. The "nr=0" is probably what you're looking for.

But the problem is the DOCTYPE. I tested the following, and it strips it out.

html = br.response().get_data().replace('<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd >', '')
response = mechanize.make_response(
    html, [("Content-Type", "text/html")],
url, 200, "OK")
br.set_response(response)

I took this straight from the Mechanize FAQ.

edited Apr 19, 2014 at 0:05

answered Apr 18, 2014 at 23:33

taco

1,38317 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user3314418 Over a year ago

unfortunately I still get the parseerror traceback saying unexpected '/' char in declaration. It's something to do with the select_form function --------------------------------------------------------------------------- ParseError Traceback (most recent call last) <ipython-input-11-7eb5e7e1b1f8> in <module>() 23 24 ---> 25 br.select_form(nr=0)

user3314418 Over a year ago

What do I do if I do find that there are errors? (1) Mismatch between Public and System identifiers in the DOCTYPE declaration (2) No Character encoding declared at document level

user3314418 Over a year ago

by strip out, you mean to just delete it?

user3314418 Over a year ago

how did you know that '<!DOCTYPE html PUBLIC...' was the thing to replace?

taco Over a year ago

validator.w3.org will explain why when you put that URL into their validator - "Mismatch between Public and System identifiers in the DOCTYPE declaration", which can lead to a parsing error. Web browsers are more forgiving than Mechanize and other parsers.

Collectives™ on Stack Overflow

Using mechanize to input username and password

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related