Parse HTML string with Nokogiri

Question

I'm trying to write a ruby script that parses an HTML string and gets some values from specific nodes.

Currently I'm struggling with just reading the string into a Nokogiri document:

This code:

#!/usr/bin/ruby

html_doc = Nokogiri::HTML("<html>  <meta content="text/html; charset=UTF-8"/>  <body style='margin:20px'>    <p>The following user has registered a device, click on the link below to review the user and make any changes if necessary.</p>    <ul style='list-style-type:none; margin:25px 15px;'>      <li><b>User name:</b> Test User</li>      <li><b>User email:</b> [email protected]</li>      <li><b>Identifier:</b> abc123def132afd1213afas</li>      <li><b>Description:</b> Tom's iPad</li>      <li><b>Model:</b> iPad 3</li>      <li><b>Platform:</b> </li>      <li><b>App:</b> Test app name</li>      <li><b>UserID:</b> </li>     </ul>    <p>Review user: https://cirrus.app47.com/[email protected]</p>            <hr style='height=2px; color:#aaa'/>        <p>We hope you enjoy the app store experience!</p>        <p style='font-size:18px; color:#999'>Powered by App47</p>      <img src='https://cirrus.app47.com/notifications/562506219ac25b1033000904/img' alt=''/></body></html>")

Produces this error:

$ ruby emailParser.rb 
emailParser.rb:3: syntax error, unexpected tIDENTIFIER, expecting ')'
...ML("<html>  <meta content="text/html; charset=UTF-8"/>  <bod...
...                               ^
emailParser.rb:3: syntax error, unexpected tSTRING_BEG, expecting end-of-input
...tent="text/html; charset=UTF-8"/>  <body style='margin:20px'...
...                               ^

Note that I have tried the solution here with the same result:

"syntax error, unexpected tIDENTIFIER, expecting $end"

use single quotes inside the HTML or single quotes outside the HTML right now text/html; charset=UTF-8 is not part of the string — engineersmnky
– engineersmnky, Commented Oct 19, 2015 at 17:31

Jordan Running · Accepted Answer · 2015-10-19 17:57:57Z

The problem is that you have double-quotes within your string which are confusing the parser, because you're also using double-quotes to surround the string. To illustrate:

puts "foo"bar"
# => SyntaxError: unexpected tIDENTIFIER, expecting end-of-input
#    puts "foo"bar"
#                 ^

You might intend for this to print foo"bar, but when the parser gets to the second " (after foo) it thinks the string is over, and so the stuff after it causes a syntax error. (Stack Overflow's syntax highlighting even gives you a hint—see how on the first line "foo" is colored differently from bar"? A good syntax-highlighting text editor will do the same thing.)

One solution is to use a single-quote instead:

puts 'bar"baz'
# => bar"baz

That fixes the problem in this case, but won't actually help you because your string also has single-quotes inside it!

Another solution is to escape your quotation marks by preceding them with a \, like so:

puts "foo\"bar"
# => foo"bar

...but that gets a little tedious (and sometimes tricky) for long strings like yours. A better solution is to use a special kind of string called a "heredoc" (for "here document," for what it's worth):

str = <<-END_OF_HTML
  <html>  <meta content="text/html; charset=UTF-8"/>  <body style='margin:20px'>    <p>The following user has registered a device, click on the link below to review the user and make any changes if necessary.</p>    <ul style='list-style-type:none; margin:25px 15px;'>      <li><b>User name:</b> Test User</li>      <li><b>User email:</b> [email protected]</li>      <li><b>Identifier:</b> abc123def132afd1213afas</li>      <li><b>Description:</b> Tom's iPad</li>      <li><b>Model:</b> iPad 3</li>      <li><b>Platform:</b> </li>      <li><b>App:</b> Test app name</li>      <li><b>UserID:</b> </li>     </ul>    <p>Review user: https://cirrus.app47.com/[email protected]</p>            <hr style='height=2px; color:#aaa'/>        <p>We hope you enjoy the app store experience!</p>        <p style='font-size:18px; color:#999'>Powered by App47</p>      <img src='https://cirrus.app47.com/notifications/562506219ac25b1033000904/img' alt=''/></body></html>
END_OF_HTML

html_doc = Nokogiri::HTML(str)

The delimiter "END_OF_HTML" is arbitrary. You could use EOF or XYZZY or whatever suits your fancy instead, although it's a good idea to use something meaningful. (You'll notice that Stack Overflow's syntax highlighting has a little trouble with heredocs; most code editors do fine with them, though.)

You can make this a little more compact like this:

Nokogiri::HTML <<-END_OF_HTML
  <html>  <meta content="text/html; charset=UTF-8"/>  <body style='margin:20px'>    <p>The following user has registered a device, click on the link below to review the user and make any changes if necessary.</p>    <ul style='list-style-type:none; margin:25px 15px;'>      <li><b>User name:</b> Test User</li>      <li><b>User email:</b> [email protected]</li>      <li><b>Identifier:</b> abc123def132afd1213afas</li>      <li><b>Description:</b> Tom's iPad</li>      <li><b>Model:</b> iPad 3</li>      <li><b>Platform:</b> </li>      <li><b>App:</b> Test app name</li>      <li><b>UserID:</b> </li>     </ul>    <p>Review user: https://cirrus.app47.com/[email protected]</p>            <hr style='height=2px; color:#aaa'/>        <p>We hope you enjoy the app store experience!</p>        <p style='font-size:18px; color:#999'>Powered by App47</p>      <img src='https://cirrus.app47.com/notifications/562506219ac25b1033000904/img' alt=''/></body></html>
END_OF_HTML

Or with parentheses (it looks a little odd, but it works, and is sometimes necessary):

Nokogiri::HTML(<<-END_OF_HTML)
  <html>  <meta content="text/html; charset=UTF-8"/>  <body style='margin:20px'>    <p>The following user has registered a device, click on the link below to review the user and make any changes if necessary.</p>    <ul style='list-style-type:none; margin:25px 15px;'>      <li><b>User name:</b> Test User</li>      <li><b>User email:</b> [email protected]</li>      <li><b>Identifier:</b> abc123def132afd1213afas</li>      <li><b>Description:</b> Tom's iPad</li>      <li><b>Model:</b> iPad 3</li>      <li><b>Platform:</b> </li>      <li><b>App:</b> Test app name</li>      <li><b>UserID:</b> </li>     </ul>    <p>Review user: https://cirrus.app47.com/[email protected]</p>            <hr style='height=2px; color:#aaa'/>        <p>We hope you enjoy the app store experience!</p>        <p style='font-size:18px; color:#999'>Powered by App47</p>      <img src='https://cirrus.app47.com/notifications/562506219ac25b1033000904/img' alt=''/></body></html>
END_OF_HTML

You can read more about heredocs, and other ways to represent strings, in the Literals section of the Ruby documentation.

StackyyGuyy · Accepted Answer · 2015-10-19 17:44:56Z

You have to change html string quotes from " to ' and change string quotes inside html to ". Something like this should work:

#!/usr/bin/ruby

html_doc = Nokogiri::HTML('<html>  <meta content="text/html; charset=UTF-8"/>  <body style="margin:20px">    <p>The following user has registered a device, click on the link below to review the user and make any changes if necessary.</p>    <ul style="list-style-type:none; margin:25px 15px;">      <li><b>User name:</b> Test User</li>      <li><b>User email:</b> [email protected]</li>      <li><b>Identifier:</b> abc123def132afd1213afas</li>      <li><b>Description:</b> Tom\'s iPad</li>      <li><b>Model:</b> iPad 3</li>      <li><b>Platform:</b> </li>      <li><b>App:</b> Test app name</li>      <li><b>UserID:</b> </li>     </ul>    <p>Review user: https://cirrus.app47.com/[email protected]</p>            <hr style="height=2px; color:#aaa"/>        <p>We hope you enjoy the app store experience!</p>        <p style="font-size:18px; color:#999">Powered by App47</p>      <img src="https://cirrus.app47.com/notifications/562506219ac25b1033000904/img" alt=""/></body></html>')

Collectives™ on Stack Overflow

Parse HTML string with Nokogiri

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related