Parse (extract) content from a html page using .net

Question

I need to parse/extract information from an html page. Basically what I'm doing is loading the page as a string using System.Net.WebClient and using HTML Agility Pack to get content inside html tags (forms, labels, imputs and so on).

However, some content is inside a javascript script tag, like this:

<script type="text/javascript">
//<![CDATA[
var itemCol = new Array();

itemCol[0] = {
    pid: "01010101",
    Desc: "Some desc",
    avail: "Available",
    price: "$10.00"
};

itemCol[1] = {
    pid: "01010101",
    Desc: "Some desc",
    avail: "Available",
    price: "$10.00"
};

//]]>
</script>

So, how could I parse it to a collection in .NET? Can HTML Agility Pack help with that? I really appreciate any help.

Thanks in advance.

Oliver Bock · Accepted Answer · 2018-01-17 04:38:06Z

1

The HAP will not parse out the javascript for you - the best it will do is parse out the contents of the element.

javascript.net may fit the bill.

edited Jan 17, 2018 at 4:38

Oliver Bock

5,1455 gold badges46 silver badges65 bronze badges

answered Jun 25, 2012 at 20:53

Oded

501k102 gold badges900 silver badges1k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

CodeMaster2008 Over a year ago

For some reason I was unable to install javascript.net (got some errors) but anyways, i was able to do the same with Jint. Thanks.

Kunal Ranglani · Accepted Answer · 2012-06-25 23:53:59Z

1

what part of the content inside the script tag do you want? What kind of collection are you expecting. You can always select script tags using below

  HtmlDocument document = new HtmlDocument();
  document.Load(downloadedHtml);
  XPathNavigator n = document.CreateNavigator();
  XPathNodeIterator scriptTags = n.Select("//script");

  foreach (XPathNavigator nav in scriptTags)
  {
    string innerXml = nav.InnerXml;

    // Parse inner xml using regex
  }

answered Jun 25, 2012 at 23:53

Kunal Ranglani

4083 silver badges15 bronze badges

1 Comment

Kunal Ranglani Over a year ago

using javascript.net using (JavascriptContext context = new JavascriptContext()) { context.SetParameter("data", new MyObject()); StringBuilder s = new StringBuilder(); foreach (XPathNavigator nav in scriptTags) { s.Append(nav.InnerXml); } s.Append(";data.item = itemCol;"); context.Run(s.ToString()); MyObject o = context.GetParameter("data") as MyObject; Then just have a datastructure like class MyObject { public object item { get; set; } }

Kunal Ranglani · Accepted Answer · 2012-06-26 00:21:10Z

1

using the javascript.net library you can get a collection

 using (JavascriptContext context = new JavascriptContext())
  {
    context.SetParameter("data", new MyObject());

     StringBuilder s = new StringBuilder();

    foreach (XPathNavigator nav in scriptTags)
    {
       s.Append(nav.InnerXml);
    }

  s.Append(";data.item = itemCol;");
  context.Run(s.ToString());

  MyObject o = context.GetParameter("data") as MyObject;

Then just have a datastructure like

   class MyObject
   {
     public object item { get; set; }
   }

answered Jun 26, 2012 at 0:21

Kunal Ranglani

4083 silver badges15 bronze badges

Collectives™ on Stack Overflow

Parse (extract) content from a html page using .net

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related