1

I'm using Selenium + Python 2.7 stack.
I want to login to this forum and crawl the post data for academic research purposes. I am using the following code to fill the login form :

    username = browser.find_element_by_xpath("//input[@id='navbar_username']")
    username.clear()
    username.send_keys("####")
    password = browser.find_element_by_xpath("//input[@id='navbar_password_hint']")
    password.clear()
    password.send_keys("####")
    loginbutton = browser.find_element_by_xpath("//input[@class='loginbutton']")
    loginbutton.click()

But I'm not able to login when the crawler runs. It is taking me to the wrong username/password page. I tried to login manually using the same information, and was able to login without any errors. While running the crawler, correct values are filled in the username and password fields. I can't understand what might be causing the problem. Any help is deeply appreciated.

1
  • When faced with this kind of issues, the easiest is to use Selenium interactively, and then export the code as Python. When looking at the resulting (working) code, you'd probably discover why your original code failed. Commented Jan 8, 2017 at 21:00

1 Answer 1

1

It seems that you use wrong element as password field. You should try

password = browser.find_element_by_xpath("//input[@id='navbar_password']")

as this element is type="password" while <input id="navbar_password_hint"> is type="text"

As target element is hidden, you can use following to make it visible:

browser.execute_script('document.getElementById("navbar_password").style.display="block";')
Sign up to request clarification or add additional context in comments.

3 Comments

I tried that too, but since tat field is hidden I couldn't send_keys() that field. I checked the form, and on submit it's doing a hash of username and navbar_password field entries. Do you have any idea of how to fill a hidden input text field in python selenium ?
well, your updated answer script made the field visible and I'm able to send the password value to the corresponding field. But it still gives me error on login, i also tried "form.submit()" style method, but to no success.
@tortuga: you've selected this as your accepted answer, but your comment makes it sound like this didn't really solve your problem. Do you still have the problem?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.