how to get the data from the table with python selenium if the class and xpath are always changing?

Question

<table border="0" cellpadding="0" cellspacing="0" class="A490847cc28c94895bcf96e98abdb2b32209xB">
                  <tbody>
                   <tr>
                    <td style="vertical-align:top">
                     <table border="0" cellpadding="0" cellspacing="0" class="A490847cc28c94895bcf96e98abdb2b32206" cols="8" style="border-collapse:collapse;">
                      <tbody>
                       <tr height="0">
                        <td style="WIDTH:56.91mm;min-width:56.91mm">
                        </td>

                       </tr>
                       <tr valign="top">

                        <td class="A490847cc28c94895bcf96e98abdb2b32126c Pe5cd046a24fa44ef80e79727141463c6_1_r7">
                         <div class="A490847cc28c94895bcf96e98abdb2b32126">
                          Total Tutorials
                         </div>
                        </td>


                        <td class="A490847cc28c94895bcf96e98abdb2b32144c Pe5cd046a24fa44ef80e79727141463c6_1_r7 Pe5cd046a24fa44ef80e79727141463c6_1_r6">
                         <div class="A490847cc28c94895bcf96e98abdb2b32144">
                          <div style="WIDTH:19.98mm;">
                           <div class="A490847cc28c94895bcf96e98abdb2b32141">
                            <span class="A490847cc28c94895bcf96e98abdb2b32140">
                             Attendance
                            </span>
                           </div>
                           <div class="A490847cc28c94895bcf96e98abdb2b32143">
                            <span class="A490847cc28c94895bcf96e98abdb2b32142">
                             %
                            </span>
                           </div>
                          </div>
                         </div>
                        </td>
                       </tr>
                       <tr valign="top">
                        <td class="A490847cc28c94895bcf96e98abdb2b32149cl Pe5cd046a24fa44ef80e79727141463c6_1_r5" style="HEIGHT:6.35mm;">
                         <div class="A490847cc28c94895bcf96e98abdb2b32149">
                          CSFf
                         </div>
                        </td>
                        <td class="A490847cc28c94895bcf96e98abdb2b32153cr">
                         <div class="A490847cc28c94895bcf96e98abdb2b32153">
                          9
                         </div>
                        </td>
                        <td class="A490847cc28c94895bcf96e98abdb2b32157cr">
                         <div class="A490847cc28c94895bcf96e98abdb2b32157">
                          7
                         </div>
                        </td>
                        <td class="A490847cc28c94895bcf96e98abdb2b32161cl">
                         <div class="A490847cc28c94895bcf96e98abdb2b32161">
                          0
                         </div>
                        </td>
                        <td class="A490847cc28c94895bcf96e98abdb2b32165cl">
                         <div class="A490847cc28c94895bcf96e98abdb2b32165">
                          0
                         </div>
                        </td>
                        <td class="A490847cc28c94895bcf96e98abdb2b32169cr">
                         <div class="A490847cc28c94895bcf96e98abdb2b32169">
                          4.0000
                         </div>
                        </td>
                        <td class="A490847cc28c94895bcf96e98abdb2b32173cr">
                         <div class="A490847cc28c94895bcf96e98abdb2b32173">
                          4.0000
                         </div>
                        </td>
                        <td class="A490847cc28c94895bcf96e98abdb2b32177cr Pe5cd046a24fa44ef80e79727141463c6_1_r6">
                         <div class="A490847cc28c94895bcf96e98abdb2b32177">
                          84.62
                         </div>
                        </td>

I am working with python selenium web driver.I want the loop through the table for the text inside the td inside tr tag like this
But problem here is the class name is always changing and xpath and css_selector is also changing.example xpath will be like this .//*[@id='P825048fc6b084257a601fde4805c8c33_1_oReportCell']/table/tbody/tr[2]/td/table/tbody/tr/td/table/tbody/tr[3]/td[8]/div
But the id is changing always.so couldn't apply driver.find_element_by_id().I think regular expressions or BeautifulSoup can be used to solve this.I am beginner to regex.So is there any way this could be solved?

Your html is not complete enough to determine how to do this. Could you instead post the whole table, with only some rows? So <table><tbody><tr><td><table><tbody><tr> <<< that whole table, with only the first two rows on each table included. — Mobrockers
– Mobrockers, Commented Aug 31, 2016 at 8:28
Your xpath already looks way over complicated. What page are you scraping and what do you want? — Padraic Cunningham
– Padraic Cunningham, Commented Aug 31, 2016 at 9:03
@PadraicCunningham hello!that is aspx site in which there is a table and I want to get the data from the table — Sriker Ch
– Sriker Ch, Commented Aug 31, 2016 at 9:06
Try using "//*[contains(@id,'oReportCell')]", I presume only the starting numbers change? — Padraic Cunningham
– Padraic Cunningham, Commented Aug 31, 2016 at 10:27
It worked.I added to your answer //*[contains(@id,'oReportCell')]/table/tbody/tr[2]/td/table/tbody/tr/td/table/tbody/tr[3]/td[1] — Sriker Ch
– Sriker Ch, Commented Aug 31, 2016 at 13:02

Sriker Ch · Accepted Answer · 2016-08-31 13:05:00Z

1

The parent of table id has kind of unique in which every time the number changes but the ReportCell is appended to it so using that we can find its children like this

//*[contains(@id,'oReportCell')]/table/tbody/tr[2]/td/table/tbody/tr/td/table/tbody/tr[3]/td[1]

answered Aug 31, 2016 at 13:05

Sriker Ch

14312 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Mobrockers · Accepted Answer · 2016-08-31 09:02:12Z

0

You might be able to use this xpath, not sure if it's 100% correct:

//div[text() = 'CSFf']/ancestor::tr/td[not(descendant::div[text() = 'CSFf'])]/div

To make it variable:

public String getXpathForTableThing(String searchString) {

    return "//div[text() = '"+searchString+"']/ancestor::tr/td[not(descendant::div[text() = '"+searchString+"'])]/div";
}

I'm not experienced with python but I'm sure you get the idea and can transform this to a python function.

edited Aug 31, 2016 at 9:02

answered Aug 31, 2016 at 8:42

Mobrockers

2,1481 gold badge16 silver badges28 bronze badges

9 Comments

Sriker Ch Over a year ago

I'll try this and let you know

Sriker Ch Over a year ago

working.But not accurately.This selects the div's which shouldn't be included

Sriker Ch Over a year ago

but what if in the place of CSFf there is another text.I have to change the text to the desired string But can there any way to get this without including the text CSFf.?

Mobrockers Over a year ago

What I do when I have a badly made html table like this is create a function for my xpath where I supply the text used to find the rows.

Sriker Ch Over a year ago

what if I want to get them what ever is there in that

|

Collectives™ on Stack Overflow

how to get the data from the table with python selenium if the class and xpath are always changing?

2 Answers 2

Comments

9 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

9 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related