Python - Serialise HTML and output as JSON

Question

I have a HTML page that displays a few values. I also have a little app that displays data from some other pages I have, but these other pages are JSON, not HTML. I want to consume these values from the HTML page, convert to JSON, then output.

The reason I want to do this is so that I can simply reuse my code, and just change the URL, or even dynamically create it.

I made the HTML page as plain as possible, so as to strip out all the junk in order to make the regex more basic.

Here is the HTML:

<div class="BlockA">
    <h4>BlockA</h4>
    <div class="name">John Smith</div>
    <div class="number">2</div>
    <div class="name">Paul Peterson</div>
    <div class="number">14</div>
</div>

<div class="BlockB">
    <h4>BlockB</h4>
    <div class="name">Steve Jones</div>
    <div class="number">5</div>
</div>

Both blocks will have varying numbers of elements,depending on a few factors.

Here is my python:

def index(request, toGet="xyz"):
    file = urllib2.urlopen("http://www.mysite.com/mypage?data="+toGet)
    data = file.read()
    dom = parseString(data)
    rows = dom.getElementsByTagName("BlockA")[0]
    readIn = ""
    for row in rows:
        readIn = readIn+json.dumps(
            {'name': row.getAttribute("location"),
            'number': row.getAttribute("number")},
            sort_keys=True,
            indent=4)+","
    response_generator = ( "["+readIn[:-1]+"]" )
    return HttpResponse(response_generator)

So this is basically reading the values (actually, the source is XML in this case), looping through them, and outputting all the values.

If someone can point me in the right direction, it would be much appreciated. For example, reading in the tags like "BlockA" and then the tags "name" and "number".

Thanks.

Josh Smeaton · Accepted Answer · 2011-04-10 22:52:06Z

2

If you truly need to parse an HTML page in Python, you should be using Beautiful Soup. I question whether you really should be doing this though. Are the HTML pages and JSON outputs using the same Django instance? Are they all apart of the same project?

If they are apart of the same project, then you can use something like django-piston which is a RESTful framework for python. This will allow you to define the data that should be exposed, and output in multiple formats such as HTML/Django Template, JSON, XML, or YAML. You can also create your own emitters to output as a different format.

That way, you can expose a particular URL as a regular template, or get the same data as JSON would will be much easier to parse than HTML.

Sorry if I'm misunderstanding your problem. But it really does sound like you want to expose a view as several different formats, and a RESTful framework will help with that.

answered Apr 10, 2011 at 22:52

Josh Smeaton

48.8k24 gold badges137 silver badges166 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

eoinzy Over a year ago

I actually already have 2 "views" defined in my views.py which take data from 2 xml files (1 local, 1 remote). This will be my 3rd view. So I have, for example, mysite.com/view1/21, mysite.com/view2/12 and myste.com/view3/{string}. The "string" view is this one, so I'm already using REST, what I need to do is simply output the HTML so my JSON reader can parse it. Is this what "Beautiful Soup" is for? I've heard of it before and seen some example, but have never used it.

Josh Smeaton Over a year ago

beautiful soup is for parsing HTML or XML structures. It even handles malformed HTML (non-closing blocks) depending on your strategy. But yes, give that a look over.

Collectives™ on Stack Overflow

Python - Serialise HTML and output as JSON

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related