Submitting a Form using Selenium in Python

Question

I need to scrape some data behind those hyperlinks from this Site. However, those hyperlinks are javascript function calls, which later submits a form using post method. After some search, selenium seems to be a candidate. So my question is that how should I properly set a value to an input tag and submit the form which does not a submit a button.

from selenium import webdriver

url = "http://www.echemportal.org/echemportal/propertysearch/treeselect_input.action?queryID=PROQ3h3n"
driver = webdriver.Firefox()
driver.get(url)
treePath_tag = driver.find_element_by_name("treePath")

Before submitting the form, I need to assign value to tag <input>. However, I got an error

Message: Element is not currently visible and so may not be interacted with

treePath_tag.send_keys('/TR.SE00.00/QU.SE.DATA_ENV/QU.SE.ENV_ENVIRONMENT_DATA/QU.SE.EN_MONITORING')

IF above is correct, I would like to submit form this way. Is it correct?

selenium.find_element_by_name("add_form").submit()

Below are sources from the web page.

JavaScript function

<script type="text/javascript">
    function AddBlock(path){
        document.add_form.treePath.value=path;
        document.add_form.submit();
    }
</script>

form "add_form"

<form id="addblock_input" name="add_form" action="/echemportal/propertysearch/addblock_input.action" method="post" style="display:none;">
<table class="wwFormTable" style="display:none;"><tr style="display:none;">
  <td colspan="2">
<input type="hidden" name="queryID" value="PROQ3h1w" id="addblock_input_queryID"/>  </td>
</tr>
<tr style="display:none;">
  <td colspan="2">
<input type="hidden" name="treePath" value="" id="addblock_input_treePath"/>  </td>
</tr>
</table></form>

div with javascript call

<div id="querytree">
    <h1>Property Search</h1>
    <h2>Select Query Block Type</h2>
    <p>Select a section for which to define query criteria.</p>
    <div class="queryblocktools"><a href="javascript:document.load_form.submit();"><img style="vertical-align:top;" alt="Load" src="/echemportal/etc/img/load.gif"/>&nbsp;Load Query</a></div>
    <ul class="listexpander">   
    <li>Physical and chemical properties<ul>
        <li><a href="javascript:AddBlock('/TR.SE00.00/QU.SE.DATA_PHYS/QU.SE.PC_MELTING');">Melting point/freezing point</a></li>
        <li><a href="javascript:AddBlock('/TR.SE00.00/QU.SE.DATA_PHYS/QU.SE.PC_BOILING');">Boiling point</a></li>
    </ul>
</div>

Y should you want to assign value to a hidden field? <input type = "hidden"/> won't display in the browser GUI — Aishu
– Aishu, Commented Jul 22, 2016 at 8:33

Saurabh Gaur · Accepted Answer · 2016-07-22 16:57:23Z

7

You are trying to set value on hidden input is which not visible on the page, that's why error has occurred. If you want to set value on hidden field try using execute_script as below :-

treePath_tag = driver.find_element_by_name("treePath")
driver.execute_script('arguments[0].value = arguments[1]', treePath_tag, '/TR.SE00.00/QU.SE.DATA_ENV/QU.SE.ENV_ENVIRONMENT_DATA/QU.SE.EN_MONITORING')

After setting value on hidden field you can use following to submit the form :-

selenium.find_element_by_name("add_form").submit()

Hope it helps..:)

edited Jul 22, 2016 at 16:57

answered Jul 22, 2016 at 8:25

Saurabh Gaur

23.9k11 gold badges56 silver badges74 bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

TTT Over a year ago

Thanks for your suggestion! What is the meaning of arguments[0].value=argument[1]' since I got this error selenium.common.exceptions.WebDriverException: Message: arguements is not defined`

Saurabh Gaur Over a year ago

@tao.hong sorry its arguments[1]..just typo mistake..here arguments[0] is your hidden element and arguments[1] is your value which you want to set...:)

TTT Over a year ago

No worries. But I still think it requires to define arguments since I got this msg selenium.common.exceptions.WebDriverException: Message: arguements is not defined

TTT Over a year ago

Thanks again. But this time, it looks the program was halted... It opened firefox then my python console did not get any return/response...

Saurabh Gaur Over a year ago

@tao.hong Is there any exception?? WHAT you expecte in return?? could you explain more??

|

Collectives™ on Stack Overflow

Submitting a Form using Selenium in Python

JavaScript function

form "add_form"

div with javascript call

1 Answer 1

13 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

JavaScript function

form "add_form"

div with javascript call

1 Answer 1

13 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related