5

I have an Asp .net page like this simple one http://issamsoft.com/app2/page1.aspx and I want to post to it some data and extract data from the response, by using TIdHttp. I tried to do that in Delphi2009 like this:

Procedure TForm1.Button1Click(Sender: TObject);
Const
VIEWSTATE = '/wEPDwUKMjA3NjE4MDczNmRkSxPt/LdmgqMd+hN+hkbiqIZuGUk=';
EVENTVALIDATION = '/wEWAwL40NXEDALs0bLrBgKM54rGBtmtdOYy+U7IFq8B25bYT1d4o1iK';
FORMPARAMS = 'TextBox1=Issam&Button1=Button';
URL = 'http://issamsoft.com/app2/page1.aspx';
var
http: TIdHttp;
lstParams: TStringList;
begin
 http := TIdHTTP.Create(self);
 lstParams := TStringList.Create;
 try
  lstParams.Add('__VIEWSTATE='+VIEWSTATE);
  lstParams.Add('__EVENTVALIDATION='+EVENTVALIDATION);
  lstParams.Add(FORMPARAMS);
  http.Request.ContentType := 'application/x-www-form-urlencoded';
  Memo1.Lines.Text := http.Post(url,lstParams);
 finally
  http.Free;
  lstParams.Free;
 end;

end;

but TIdhttp always gives an error(HTTP/1.1 500 Internal Server Error.) I read some comments in the idHttp unit talks about problems with http protocol v 1.1 like this one:

Currently when issuing a POST, IdHTTP will automatically set the protocol to version 1.0 independently of the value it had initially, This is because there are some servers that don't respect the RFC to the full extent. In particular, they don't respect sending/not sending the Expect: 100-Continue header. Until we find an optimum solution that does NOT break the RFC, we will restrict POSTS to version 1.0.

is there something wrong with my code or it's TidHttp Bug? and if the problem is in TIdHttp, is there any workaround? or is there other solution using Indy components?

besides. I've made a solution in C# using WebClient and it works very good.

        private void button1_Click(object sender, EventArgs e)
    {
        WebClient myClient = new WebClient();
        string viewstate = HttpUtility.UrlEncodeUnicode(@"/wEPDwUKMjA3NjE4MDczNmRkSxPt/LdmgqMd+hN+hkbiqIZuGUk=");
        string eventvaildation = HttpUtility.UrlEncodeUnicode(@"/wEWAwL40NXEDALs0bLrBgKM54rGBtmtdOYy+U7IFq8B25bYT1d4o1iK");
        string postdata = "__VIEWSTATE=" + viewstate + "&" + 
            "__EVENTVALIDATION=" + eventvaildation + "&TextBox1=Issam&Button1=Button";
        myClient.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
        byte[] responce = myClient.UploadData("http://issamsoft.com/app2/page1.aspx", Encoding.ASCII.GetBytes(postdata));
        txtResponse.Text = Encoding.ASCII.GetString(responce);
    } 

where i can find (good/trusted) class like WebClient in Delphi? free preferred :)

Edit: I hope mechanism of VIEWSTATE,EVENTVALIDATION is clear enough for you, they are hash values generated by server, and they may change(already changed), my orginal project has a piece of code just to extract the current VIEWSTATE,EVENTVALIDATION values, but I omit that part just to make my example simple and clear, so when you want to try the above code you must take VIEWSTATE,EVENTVALIDATION values from the current page source.

3
  • I tried to Encode the VIEWSTATE parameter using TIdURI, and I tried to pass a (pre)Encoded one to TIdhttp. and also it didn't work :( Commented Aug 29, 2009 at 11:26
  • I hope mechanism of VIEWSTATE,EVENTVALIDATION is clear enough for you, they are hash values generated by server, and they may change(already changed), my orginal project has a piece of code just to extract the current VIEWSTATE,EVENTVALIDATION values, but I omit that part just to make my example simple and clear, so when you want to try the above code you must take VIEWSTATE,EVENTVALIDATION values from the current page source. Commented Aug 30, 2009 at 9:20
  • is your question now adequately answered? Commented Sep 1, 2009 at 18:45

3 Answers 3

5

This is most likely caused by the fact that in C#, you use an ampersand (&) characters to concatenate the VIEWSTATE, EVENTVALIDATION and FORMPARAMS.

But in Delphi, you use a TStringList and Add. Add will not put an ampersand (&), but a CR+LF between the additions.

Hence you are sending different data in Delphi than in C#.

If you change your string concatenation logic in Delphi to match the logic in C#, it should work, not matter what kind of WebClient you use on the Delphi side.

--jeroen

Edit: 20090830

This code works; inspecting the C# and IE7 output using Fiddler2, I found out you also need to escape the '/' and '=' characters.

Edit2: 20090831

Somehow the VIEWSTATE and EVENTVALIDATION on the site have changed since yesterday (though I'm not sure why). I have changed the code, and it works at the time of posting my changes to this answer. This is the new request content as captured per Fiddler when posting the page from within Internet Explorer 7:

__VIEWSTATE=%2FwEPDwUKMjA3NjE4MDczNmRk5%2FC2iWwvlAB3L1wYzRpm3KZhRC0%3D&__EVENTVALIDATION=%2FwEWAwLXzuATAuzRsusGAoznisYGSYOqDGy4vMunY6A8xi6ahQEPI5Q%3D&TextBox1=Issam&Button1=Button

This is the new request as when posted form the Delphi example:

__VIEWSTATE=%2FwEPDwUKMjA3NjE4MDczNmRk5%2FC2iWwvlAB3L1wYzRpm3KZhRC0%3D&__EVENTVALIDATION=%2FwEWAwLXzuATAuzRsusGAoznisYGSYOqDGy4vMunY6A8xi6ahQEPI5Q%3D&TextBox1=Issam&Button1=Button

The below sample now gives this result (note the "Issam" in the result):

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >
<head><title>

</title></head>
<body>
    <form name="form1" method="post" action="page1.aspx" id="form1">
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwUKMjA3NjE4MDczNg9kFgICAw9kFgICBQ8PFgIeBFRleHQFDVdlbGNvbWUgSXNzYW1kZGSCDMOkTMjkZJgqLkhpK99twpD5+A==" />

<input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION" value="/wEWAwKr+O/BBQLs0bLrBgKM54rGBlueO5BU/6BAJMZfHNwh5fsQFuAm" />
    <div>

        <input name="TextBox1" type="text" value="Issam" id="TextBox1" />
        <input type="submit" name="Button1" value="Button" id="Button1" />

        <br />

        <span id="Label1">Welcome Issam</span>
        <br />

    </div>
    </form>
</body>
</html>

Edit3: 20090831

And indeed the VIEWSTATE and EVENTVALIDATION changed again: somehow they keep changing: there seems to be a time factor involved.

So I adapted the code, now it it can extract the VIEWSTATE and EVENTVALIDATION from a GET request, then put that into a POST request.

In addition, it appeared that the '+' and ':' also needs to be escaped. So I dug into the ASP.NET page generation logic, found out that they use HttpUtility.UrlEncode to do the escaping, which ultimately checks HttpUtility.IsSafe which characters to escape. So I adapted took the Reflector reverse engineerd Delphi code into a THttpUtility class.

This is the final code:

{$DEFINE FIDDLER}

unit HttpPostExampleFormUnit;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, IdBaseComponent, IdComponent, IdTCPConnection, IdTCPClient, IdHTTP;

type
  THttpPostExampleForm = class(TForm)
    Button1: TButton;
    Memo1: TMemo;
    procedure Button1Click(Sender: TObject);
  private
    procedure Log(const What: string; const Content: string);
    procedure LogClear;
  end;

var
  HttpPostExampleForm: THttpPostExampleForm;

implementation

uses
  HttpUtilityUnit;

{$R *.dfm}

procedure AddFormParameter(const StringStream: TStringStream; const ParameterName: string; const ParameterValue: string);
var
  EncodedParameterValue: string;
begin
  StringStream.WriteString(ParameterName);
  StringStream.WriteString('=');
  EncodedParameterValue := THttpUtility.UrlEncode(ParameterValue);
  StringStream.WriteString(EncodedParameterValue);
end;

procedure AddFormParameterSeparator(const StringStream: TStringStream);
begin
  StringStream.WriteString('&');
end;

function ExtractHiddenParameter(const ParameterName: string; const Request: string): string;
//<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwUKMjA3NjE4MDczNg9kFgICAw9kFgICBQ8PFgIeBFRleHQFDVdlbGNvbWUgSXNzYW1kZGSCDMOkTMjkZJgqLkhpK99twpD5+A==" />
const
  PrefixMask = 'input type="hidden" name="%s" id="%s" value="';
  Suffix = '" />';
var
  Prefix: string;
  PrefixLength: Integer;
  PrefixPosition: Integer;
  SuffixPosition: Integer;
begin
  Prefix := Format(PrefixMask, [ParameterName, ParameterName]);
  PrefixPosition := Pos(Prefix, Request);
  if PrefixPosition = 0 then
    Result := ''
  else
  begin
    PrefixLength := Length(Prefix);
    Result := Copy(Request,
      PrefixPosition + PrefixLength,
      1 + Length(Request) - PrefixPosition - PrefixLength);
    SuffixPosition := Pos(Suffix, Result);
    if SuffixPosition = 0 then
      Result := ''
    else
      Delete(Result, SuffixPosition, 1 + Length(Result) - SuffixPosition);
  end;
end;

procedure THttpPostExampleForm.Button1Click(Sender: TObject);
const
  DefaultVIEWSTATE = '/wEPDwUKMjA3NjE4MDczNmRk5%2FC2iWwvlAB3L1wYzRpm3KZhRC0=';
  DefaultEVENTVALIDATION = '/wEWAwLXzuATAuzRsusGAoznisYGSYOqDGy4vMunY6A8xi6ahQEPI5Q=';
  FORMPARAMS = 'TextBox1=Issam&Button1=Button';
  URL = 'http://issamsoft.com/app2/page1.aspx';
  __VIEWSATE = '__VIEWSTATE';
  __EVENTVALIDATION = '__EVENTVALIDATION';
var
  VIEWSTATE: string;
  EVENTVALIDATION: string;
  getHttp: TIdHttp;
  getRequest: string;
  postHttp: TIdHttp;
  ParametersStringStream: TStringStream;
  postRequest: string;
begin
  LogClear();
  getHttp := TIdHTTP.Create(self);
  try
    getRequest := getHttp.Get(URL);
    Log('GET Request', getRequest);
    VIEWSTATE := ExtractHiddenParameter(__VIEWSATE, getRequest);
    EVENTVALIDATION := ExtractHiddenParameter(__EVENTVALIDATION, getRequest);
    Log('Extracted VIEWSTATE', VIEWSTATE);
    Log('Extracted EVENTVALIDATION', EVENTVALIDATION);
  finally
    getHttp.Free();
  end;

  postHttp := TIdHTTP.Create(self);
{$IFDEF FIDDLER}
  postHttp.ProxyParams.ProxyServer := '127.0.0.1';
  postHttp.ProxyParams.ProxyPort := 8888;
{$ENDIF FIDDLER}
  postHttp.HTTPOptions := postHttp.HTTPOptions + [hoKeepOrigProtocol];
  ParametersStringStream := TStringStream.Create('');
  try
    AddFormParameter(ParametersStringStream, __VIEWSATE, VIEWSTATE);
    AddFormParameterSeparator(ParametersStringStream);
    AddFormParameter(ParametersStringStream, __EVENTVALIDATION, EVENTVALIDATION);
    AddFormParameterSeparator(ParametersStringStream);
    ParametersStringStream.WriteString(FORMPARAMS);
    postHttp.Request.ContentType := 'application/x-www-form-urlencoded';
    ParametersStringStream.Seek(0, soFromBeginning);
    Log('POST Parameters', ParametersStringStream.DataString);
    postRequest := postHttp.Post(url, ParametersStringStream);
    Log('POST Request', postRequest);
  finally
    postHttp.Free;
    ParametersStringStream.Free;
  end;
end;

procedure THttpPostExampleForm.Log(const What, Content: string);
begin
  Memo1.Lines.Add(What + '=');
  Memo1.Lines.Add(Content);
end;

procedure THttpPostExampleForm.LogClear;
begin
  Memo1.Lines.Clear;
end;

end.

And the THttpUtlity class:

unit HttpUtilityUnit;

interface

type
  THttpUtility = class
  private
    class function IsSafe(const ch: Char): boolean;
  public
    class function UrlEncode(s: string): string;
  end;

implementation

uses
  Classes, SysUtils;

class function THttpUtility.IsSafe(const ch: Char): boolean;
begin
  if ((((ch >= 'a') and (ch <= 'z')) or ((ch >= 'A') and (ch <= 'Z'))) or ((ch >= '0') and (ch <= '9'))) then
    Result := True
  else
    case ch of
      '''',
        '(',
        ')',
        '*',
        '-',
        '.',
        '_',
        '!':
        Result := True;
    else
      Result := False;
    end;
end;

class function THttpUtility.UrlEncode(s: string): string;
var
  ch: Char;
  HexCh: string;
  StringStream: TStringStream;
  Index: Integer;
  Ordinal: Integer;
begin
  StringStream := TStringStream.Create('');
  try
    //Note: this is not yet UTF-16 compatible; check before porting to Delphi 2009
    for Index := 1 to Length(s) do
    begin
      ch := s[Index];
      if IsSafe(ch) then
        StringStream.WriteString(Ch)
      else
      begin
        Ordinal := Ord(Ch);
        HexCh := IntToHex(Ordinal, 2);
        StringStream.WriteString('%'+HexCh);
      end;
    end;

    Result := StringStream.DataString;
  finally
    StringStream.Free;
  end;
end;

end.

This should get you going.

--jeroen

Sign up to request clarification or add additional context in comments.

9 Comments

+1.. That should get Issam Ali going. I'd give extra points for trying out and providing a code example, but SO doesn't let me.
Thanks Jeroen, I really appretiate your trying to help, but that doesn't work, if you check the response you'll see that the Label1 value is still "Lable1" while it should be "Welcome Issam". why it doesn't give you Error 500 because you doesn't post a proper parmeters, the parmeters should be like this param1=v1&param2=v2... but actually you post this data: '__VIEWSTATE%3D%2FwEPDwUKMjA3NjE4MDczNmRkSxPt%2FLdmgqMd+hN+hkbiqIZuGUk=&__EVENTVALIDATION%3D%2FwEWAwL40NXEDALs0bLrBgKM54rGBtmtdOYy+U7IFq8B25bYT1d4o1iK&TextBox1=Issam&Button1=Button' so server dosen't see the paremeter __VIEWSTATE
BTW: the Post method of the TIDHttp convert the line break to '&'
As I said in my first comment, I tried to Post Encoded parameters(using C# app to produce it), and tell TIdhttp not to Encode it again(hoForceEncodeParams = false) !, and also that doesn't work :)
((parameters)) not ((parmeters)) sorry :)
|
0

If you include the hoKeepOrigProtocol option in the TIdHTTP instance HTTPOptions property, it will not fall back to HTTP 1.0.

This is documented some lines below the comment text which you cited :)

    // If hoKeepOrigProtocol is SET, is possible to assume that the developer
    // is sure in operations of the server
    if not (hoKeepOrigProtocol in FOptions) then begin
      FProtocolVersion := pv1_0;
    end;

2 Comments

Have you tried a HTTP proxy like Don's Proxy (from Sourceforge, donsproxy.sourceforge.net) to record the HTTP request of Delphi and C# client? This tool is very useful to track down the differences between the requests.
So you can save the requests to files and use a diff tool to find the differences (either in the HTTP header or in the payload) between the Delphi and the C# version.
-1

You need to update the VIEWSTATE and EVENTVALIDATION values.

Running that code causes a 500 error, but if I open the page and use the new __VIEWSTATE and __EVENTVALIDATION values from the webpage source, it no longer causes a 500 error.

I don't know why a C# client would have worked when the Delphi one didn't, but maybe it doesn't work anymore?

This looks like an ASP issue, not Delphi. Both Indy (TIdHTTP) and Synapse are good socket libraries for Delphi.

2 Comments

-1 Actually, you should regard the VIEWSTATE and EVENTVALIDATION as readonly. They are property bags, internally used by ASP.NET to reconstruct the 'before' state when a POST request (with fresh form data) is processed at the server.
Right, the values in the Delphi code just needed to be updated, and then the posted code worked as is. I didn't mean to change the ASP.NET page.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.