5

In my Ruby app, I've used the following method and regular expression to remove all HTML tags from a string:

str.gsub(/<\/?[^>]*>/,"")

This regular expression did just about all I was expecting it to, except it caused all quotation marks to be transformed into &#8220; and all single quotes to be changed to &#8221; .

What's the obvious thing I'm missing to convert the messy codes back into their proper characters?

Edit: The problem occurs with or without the Regular Expression, so it's clear my problem has nothing to do with it. My question now is how to deal with this formatting error and correct it. Thanks!

Improve this question
1
  • Are you using some kind of “quote beautifier”? Commented Feb 13, 2009 at 0:15

5 Answers 5

Reset to default
5

Use CGI::unescapeHTML after you perform your regular expression substitution:

CGI::unescapeHTML(str.gsub(/<\/?[^>]*>/,""))

See http://www.ruby-doc.org/core/classes/CGI.html#M000547

In the above code snippet, gsub removes all HTML tags. Then, unescapeHTML() reverts all HTML entities (such as <, &#8220) to their actual characters (<, quotes, etc.)

With respect to another post on this page, note that you will never ever be passed HTML such as

<tag attribute="<value>">2 + 3 < 6</tag>

(which is invalid HTML); what you may receive is, instead:

<tag attribute="&lt;value&gt;">2 + 3 &lt; 6</tag>

The call to gsub will transform the above to:

2 + 3 &lt; 6

And unescapeHTML will finish the job:

2 + 3 < 6
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

You're going to run into more trouble when you see something like:

<doohickey name="<foobar>">

You'll want to apply something like:

gsub(/<[^<>]*>/, "")

...for as long as the pattern matches.

Improve this answer

1 Comment

Unless you meant that the OP should be prepared to deal with bogus/invalid HTML, you're wrong. This form will never appear in correct HTML.
2

This regular expression did just about all I was expecting it to, except it caused all quotation marks to be transformed into “ and all single quotes to be changed to ” .

This doesn't sound as if the RegExp would be doing this. Are you sure it's different before?

See this question here for information about the problem, it has got an excellent answer:
Get non UTF-8 form fields as UTF-8 in php.

Improve this answer

1 Comment

Holy cow, you're right. I noticed it after adding the Regex, but the effect on characters happens either way. I just hadn't noticed due to it being less obvious. So my question becomes: How do I fix this formatting?
0

I've run into a similar problem with character changes, this happened when my code ran through another module that enforced UTF-8 encoding and then when it came back, I had a different file (slurped array of lines) on my hands.

Improve this answer

Comments

-3

You could use a multi-pass system to get the results you are looking for.

After running your regular expression, run an expression to convert &8220; to quotes and another to convert &8221; to single quotes.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.