3

I am trying to post value to a form to get data from an aspx page, but I am unable to get data

  1. First Url to be opened: $url="https://www.clarkcountycourts.us/Anonymous/default.aspx" (cookies are set here)

  2. Second url from we have to search data $url3 ="https://www.clarkcountycourts.us/Anonymous/Search.aspx"

When I post data I just get html of page 3 instead of data from page 2

Below is my code, what am I missing? please guide me.

<?php 
$url="https://www.clarkcountycourts.us/Anonymous/default.aspx"; 
$cookie="cookie.txt";
$url3 ="https://www.clarkcountycourts.us/Anonymous/Search.aspx";

$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url); 
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie);  // <-- add this line
curl_setopt ($ch, CURLOPT_REFERER, $url); 
$result = curl_exec ($ch); 



$SearchBy = "2";
$AttorneySearchMode="Name";
$LastName= "Smith";
$FirstName= "William";
$MiddleName ="";
$CaseStatusType= "0";
$SortBy= "fileddate";
$DateFiledOnAfter = "";
$DateFiledOnBefore = "";
$SearchSubmit ="Search";

$fields = array(
    'SearchBy' => urlencode($SearchBy),
    'AttorneySearchMode' => urlencode($AttorneySearchMode),
    'LastName' => urlencode($LastName),
    'FirstName' => urlencode($FirstName),
    'MiddleName' => urlencode($MiddleName),
    'CaseStatusType' => urlencode($CaseStatusType),
    'SortBy' => urlencode($SortBy),
    'DateFiledOnAfter' => urlencode($DateFiledOnAfter),
    'DateFiledOnBefore' => urlencode($DateFiledOnBefore),
    'SearchSubmit' => urlencode($SearchSubmit)
);

$fields_string = "";
foreach ($fields as $key=>$value) {
    $fields_string .= $key.'='.$value.'&';
}
$fields_string = rtrim($fields_string, '&');

curl_setopt($ch, CURLOPT_URL, $url3);
curl_setopt($ch, CURLOPT_HEADER, true );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch,CURLOPT_POST, 1);
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields_string);
$result2 = curl_exec ($ch); 
print_r($result2);
2
  • Is difficult since we are not allowed to view the pages. Commented Apr 27, 2015 at 14:50
  • what's page2 and page3? Your question isn't clear about that. Commented May 3, 2015 at 21:40

3 Answers 3

5
+50

Took a while to figure this one out - but the code below should echo out the results after the search based upon your original query of William Smith as the attorney. Hope this makes sense....

            define('ROOT','c:/wwwroot');
            $url_base='https://www.clarkcountycourts.us';
            $url_login=$url_base.'/Anonymous/Login.aspx?ReturnUrl=/Anonymous/default.aspx';
            $url_start=$url_base.'/Anonymous/default.aspx';
            $url_search=$url_base.'/Anonymous/Search.aspx?ID=200&NodeID=101,103,104,105,500,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,797,798&NodeDesc=All%20Courts';
            $cookiejar=tempnam( sys_get_temp_dir(), 'cookiejar_' );

            $lastname='smith';
            $firstname='william';

            $formparams=array(
                '__VIEWSTATEGENERATOR',
                '__EVENTVALIDATION',
                '__EVENTTARGET',
                '__EVENTARGUMENT',
                '__VIEWSTATE'
            );
            $stdparams=array(
                'SearchBy' => '2',
                'AttorneySearchMode' => 'Name',
                'LastName' => $lastname,
                'FirstName' => $firstname,
                'MiddleName' => '',
                'CaseStatusType' => '0',
                'SortBy' => 'fileddate',
                'DateFiledOnAfter' => '',
                'DateFiledOnBefore' => '',                  
                'SearchSubmit' => 'Search',
                'CaseSearchMode' => 'CaseNumber',
                'CaseSearchValue' => '',
                'CitationSearchValue' => '',
                'CourtCaseSearchValue' => '',
                'PartySearchMode' => 'Name',
                'cboState' => 'AA',
                'DateOfBirth' => '',
                'DriverLicNum' => '',
                'chkCriminal' => 'on',
                'chkFamily' => 'on',
                'chkCivil' => 'on',
                'chkProbate' => 'on',
                'chkDtRangeCriminal' => 'on',
                'chkDtRangeFamily' => 'on',
                'chkDtRangeCivil' => 'on',
                'chkDtRangeProbate' => 'on',
                'chkCriminalMagist' => 'on',
                'chkFamilyMagist' => 'on',
                'chkCivilMagist' => 'on',
                'chkProbateMagist' => 'on',
                'DateSettingOnAfter' => '',
                'DateSettingOnBefore' => '',
                'SearchType' => 'PARTY',
                'SearchMode' => 'NAME',
                'NameTypeKy' => 'ALIAS',
                'BaseConnKy' => 'AT',
                'StatusType' => 'true',
                'ShowInactive' => '',
                'AllStatusTypes' => 'true',
                'CaseCategories' => '',
                'RequireFirstName' => 'False',
                'CaseTypeIDs' => '',
                'HearingTypeIDs' => ''
            );
            /* A fudge I know, manually build this param - I think it might be constructed using javascript on submission */
            $stdparams['SearchParams']='SearchBy~~Search+By:~~Attorney~~Attorney||AttorneyNameOption~~Party+Search+Mode:~~Name~~Name||LastName~~Last+Name:~~'.$lastname.'~~'.$lastname.'||FirstName~~First+Name:~~'.$firstname.'~~'.$firstname.'||AllOption~~Case+Status:~~0~~All||selectSortBy~~Sort+By:~~Filed+Date~~Filed+Date ';




            /* Initialise curl and set basic options */
            $curl=curl_init();
            if( parse_url( $url_base,PHP_URL_SCHEME )=='https' ){
                curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, 2 );
                curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, false );
                curl_setopt( $curl, CURLOPT_CAINFO, realpath( ROOT . '/cacert.pem' ) );
            }
            curl_setopt( $curl, CURLINFO_HEADER_OUT, true );
            curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
            curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, true );
            curl_setopt( $curl, CURLOPT_AUTOREFERER, true );
            curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0' );
            curl_setopt( $curl, CURLOPT_HTTPHEADER, array('Host: '.parse_url( $url_login,PHP_URL_HOST ),'Connection: keep-alive') );


            /* Stage 1 - Get initial page to obtain session cookies etc and store in cookiejar...mmmmm, cookies! */
            curl_setopt( $curl, CURLOPT_URL, $url_login );
            curl_setopt( $curl, CURLOPT_COOKIEJAR, $cookiejar );
            curl_setopt( $curl, CURLOPT_COOKIESESSION, true );
            $res=curl_exec( $curl );



            /* Step 2 - GET form page and analyse input elements */
            curl_setopt( $curl, CURLOPT_URL, $url_search );
            curl_setopt( $curl, CURLOPT_AUTOREFERER, true );
            curl_setopt( $curl, CURLOPT_COOKIEFILE, $cookiejar );
            curl_setopt( $curl, CURLOPT_COOKIESESSION, false );
            $res=curl_exec( $curl );



            /* utility class that simplifies getting DOMDocument with error checking etc */
            $dom=new htmldom( $res );
            $html=$dom->gethtml();
            $col=$html->getElementsByTagName('input');

            /* we need to know values for specific fields - add to array of params to be submitted */
            foreach( $col as $index => $node ){
                if( in_array( $node->getAttribute('name'), $formparams ) ) {
                    $stdparams[ $node->getAttribute('name') ]=$node->getAttribute('value');
                }
            }
            /* Not sure how the 'SearchParams' field value is calculated so calculated manually above..... */




            /* Stage 3 - POST */
            /* Prepare search query */
            $info=$res='';
            $querystring=http_build_query( $stdparams, '', '&' );
            curl_setopt( $curl, CURLOPT_URL, $url_search );
            curl_setopt( $curl, CURLOPT_REFERER, $url_search );
            curl_setopt( $curl, CURLOPT_COOKIEFILE, $cookiejar );
            curl_setopt( $curl, CURLOPT_COOKIESESSION, false );
            curl_setopt( $curl, CURLOPT_POST, true );
            curl_setopt( $curl, CURLOPT_POSTFIELDS, $querystring );
            $res=curl_exec( $curl );
            $info=curl_getinfo( $curl );


            /* The final search results - you could manipulate the dom to get specific items if you wished */
            echo '<pre>';
            print_r( $res );
            echo '</pre>';

            /* Close curl */
            curl_close( $curl );

For completeness, the utility class mentioned above.

class htmldom{
    private $html;
    public function __construct( $data=false, $convert=true ){
        try{
            if( !$data ) return false;
            libxml_use_internal_errors( true );
            $this->html = new DOMDocument('1.0','utf-8');
            $this->html->validateOnParse=false;
            $this->html->standalone=true;
            $this->html->preserveWhiteSpace=true;
            $this->html->strictErrorChecking=false;
            $this->html->substituteEntities=false;
            $this->html->recover=true;
            $this->html->formatOutput=false;

            $this->html->loadHTML( $convert ? mb_convert_encoding( $data, 'utf-8' ) : $data );

            $parse_errs=serialize( libxml_get_last_error() );
            libxml_clear_errors();

        }catch( Exception $e ){
            die( $e->getMessage() );    
        }
    }
    public function gethtml(){
        return $this->html;
    }
}
Sign up to request clarification or add additional context in comments.

Comments

5

You have two things going on here.

  1. You are not using the cookie jar properly. CURLOPT_COOKIEJAR is used when you are wanting to store cookies set from a server. cURL does not function like a programmable browser in this way. If you're wanting to get the session established (cookie) then the first request uses CURLOPT_COOKIEJAR and subsequent requests would use CURLOPT_COOKIEFILE

    CURLOPT_COOKIEJAR cookies are put into the jar from the server.
    CURLOPT_COOKIEFILE cookies are sent from the cookie file back to the server.

    Using CURLOPT_COOKIEJAR beyond the first connection to the server will simply overwrite the cookie jar each time.

Cookies and cURL have been a consistent source of frustration for me so I may be making a bigger deal about this first point.

  1. When I manually do searches on this server, the forms end up being posted to urls like this:

    https://www.clarkcountycourts.us/Anonymous/Search.aspx?ID=200&NodeID=101,103,104,105,500,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,797,798&NodeDesc=All%20Courts

    Furthermore, when I POST the form in my browser, there are lots of other fields that are being sent as well.

Some of these look important - __VIEWSTATE, __EVENTVALIDATION, __VIEWSTATEGENERATOR.

__EVENTTARGET:
__EVENTARGUMENT:
__VIEWSTATE:/wEPDwULLTE3NDQ0MTE0OTgPZBYCZg9kFgICAQ8WAh4HVmlzaWJsZWgWAgIDDw9kFgIeB29ua2V5dXAFJnRoaXMudmFsdWUgPSB0aGlzLnZhbHVlLnRvTG93ZXJDYXNlKCk7ZGSj5Ki6eeFgaBABR8RCqLZPLI/WZw==
__VIEWSTATEGENERATOR:B40748C4
__EVENTVALIDATION:/wEWAgKmuqaTBQKYxoa5CF+xFBuOB81vr2zAb/xD7VZ3+CQV
SearchBy:2
CaseSearchMode:CaseNumber
CaseSearchValue:
CitationSearchValue:
CourtCaseSearchValue:
PartySearchMode:Name
AttorneySearchMode:Name
LastName:Smith
FirstName:William
cboState:AA
MiddleName:
DateOfBirth:
DriverLicNum:
CaseStatusType:0
DateFiledOnAfter:
DateFiledOnBefore:
chkCriminal:on
chkFamily:on
chkCivil:on
chkProbate:on
chkDtRangeCriminal:on
chkDtRangeFamily:on
chkDtRangeCivil:on
chkDtRangeProbate:on
chkCriminalMagist:on
chkFamilyMagist:on
chkCivilMagist:on
chkProbateMagist:on
DateSettingOnAfter:
DateSettingOnBefore:
SortBy:fileddate
SearchSubmit:Search
SearchType:PARTY
SearchMode:NAME
NameTypeKy:ALIAS
BaseConnKy:AT
StatusType:true
ShowInactive:
AllStatusTypes:true
CaseCategories:
RequireFirstName:False
CaseTypeIDs:
HearingTypeIDs:
SearchParams:SearchBy~~Search By:~~Attorney~~Attorney||AttorneyNameOption~~Party Search Mode:~~Name~~Name||LastName~~Last Name:~~Smith~~Smith||FirstName~~First Name:~~William~~William||AllOption~~Case Status:~~0~~All||selectSortBy~~Sort By:~~Filed Date~~Filed Date

Conclusion

I believe you will need to load the search page to get the values to put in some of these hidden fields (parsing the page) and then submit the search page again as a POST with the hidden fields filled in.

Comments

0

Maybe you're being redirected after the initial POST?

Try the following:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);

This should ignore any redirects.

Additional CURL options that might help you debug:

CURLOPT_MAXREDIRS
CURLOPT_POSTREDIR

Reference:

http://php.net/manual/en/function.curl-setopt.php

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.