Streaming data with Python and Flask

Question

I can't seem to figure out how to using Flask's streaming. Here's my code:

@app.route('/scans/')
def scans_query():
    url_for('static', filename='.*')
    def generate():
        yield render_template('scans.html')
        for i in xrange(50):
            sleep(.5)
            yield render_template('scans.html', **locals())
    return Response(stream_with_context(generate()))

and in my template:

<p>{% i %}</p>

I would like to see a counter on the page that changes every half second. Instead, the closest I've gotten is the page printing out each number on the next line.

jfs · Accepted Answer · 2012-11-15 17:22:13Z

73

To replace existing content on the page you might need javascript i.e., you could send it or make it to make requests for you, use long polling, websockets, etc. There are many ways to do it, here's one that uses server send events:

#!/usr/bin/env python
import itertools
import time
from flask import Flask, Response, redirect, request, url_for

app = Flask(__name__)

@app.route('/')
def index():
    if request.headers.get('accept') == 'text/event-stream':
        def events():
            for i, c in enumerate(itertools.cycle('\|/-')):
                yield "data: %s %d\n\n" % (c, i)
                time.sleep(.1)  # an artificial delay
        return Response(events(), content_type='text/event-stream')
    return redirect(url_for('static', filename='index.html'))

if __name__ == "__main__":
    app.run(host='localhost', port=23423)

Where static/index.html:

<!doctype html>
<title>Server Send Events Demo</title>
<style>
  #data {
    text-align: center;
  }
</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
<script>
if (!!window.EventSource) {
  var source = new EventSource('/');
  source.onmessage = function(e) {
    $("#data").text(e.data);
  }
}
</script>
<div id="data">nothing received yet</div>

The browser reconnects by default in 3 seconds if the connection is lost. if there is nothing more to send the server could return 404 or just send some other than 'text/event-stream' content type in response to the next request. To stop on the client side even if the server has more data you could call source.close().

Note: if the stream is not meant to be infinite then use other techniques (not SSE) e.g., send javascript snippets to replace the text (infinite <iframe> technique):

#!/usr/bin/env python
import time
from flask import Flask, Response

app = Flask(__name__)


@app.route('/')
def index():
    def g():
        yield """<!doctype html>
<title>Send javascript snippets demo</title>
<style>
  #data {
    text-align: center;
  }
</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
<div id="data">nothing received yet</div>
"""

        for i, c in enumerate("hello"):
            yield """
<script>
  $("#data").text("{i} {c}")
</script>
""".format(i=i, c=c)
            time.sleep(1)  # an artificial delay
    return Response(g())


if __name__ == "__main__":
    app.run(host='localhost', port=23423)

I've inlined the html here to show that there is nothing more to it (no magic). Here's the same as above but using templates:

#!/usr/bin/env python
import time
from flask import Flask, Response

app = Flask(__name__)


def stream_template(template_name, **context):
    # http://flask.pocoo.org/docs/patterns/streaming/#streaming-from-templates
    app.update_template_context(context)
    t = app.jinja_env.get_template(template_name)
    rv = t.stream(context)
    # uncomment if you don't need immediate reaction
    ##rv.enable_buffering(5)
    return rv


@app.route('/')
def index():
    def g():
        for i, c in enumerate("hello"*10):
            time.sleep(.1)  # an artificial delay
            yield i, c
    return Response(stream_template('index.html', data=g()))


if __name__ == "__main__":
    app.run(host='localhost', port=23423)

Where templates/index.html:

<!doctype html>
<title>Send javascript with template demo</title>
<style>
  #data {
    text-align: center;
  }
</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
<div id="data">nothing received yet</div>
{% for i, c in data: %}
<script>
  $("#data").text("{{ i }} {{ c }}")
</script>
{% endfor %}

edited Nov 15, 2012 at 17:22

answered Nov 14, 2012 at 23:04

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

David Marx Over a year ago

The demo you provided (specifically I'm playing with the SSE demo) only works for a single client. If I open up a new browser window and try to access the page streaming data, nothing happens until I close or stop the previous page. And then, the counter starts back up at 0. How would you rework this so that any clients trying to access this would all see the same data/counter, counting up from the time the app is started? I'm assuming you'd have to run the counter in a separate thread, but I'm not sure how to implement this.

jfs Over a year ago

@DavidMarx: there are at least two questions: (1) how to support multiple concurrent clients in flask? — the answer: the same way you do it for any wsgi app e.g., use gunicorn (2) how to provide access to the same counter for multiple clients? — the same way you provide access to shared data in any server program e.g., assuming a single worker: define global iterator and call next(it) in the loop. Anyway, these are separate questions. Ask a new question specific to your particular problem.

Karan Over a year ago

Tried 1st example to read logs file and display in html but html print only first line.. but print() marked with **** printing all line def events(): with open('E:/Study/loggerApp/templates/job.log') as f: data=""; while True: data=f.read() yield "data: "+(data) if data!="": ****print(data)**** time.sleep(1) return Response(events(), content_type='text/event-stream')

jfs Over a year ago

@Karan: I'm surprised the code works at all after 8 years. To send multiline data, you could try sending each line as a separate event, or just send your data as json (newlines are escaped in this case).

David Schumann · Accepted Answer · 2018-10-26 18:01:05Z

7

I think if you're going to use templates like that, you might need to use the stream_template function given here: http://flask.pocoo.org/docs/patterns/streaming/#streaming-from-templates

I didn't test this, but it might look like:

def stream_template(template_name, **context):
    app.update_template_context(context)
    t = app.jinja_env.get_template(template_name)
    rv = t.stream(context)
    rv.enable_buffering(5)
    return rv

@app.route('/scans/')
def scans_query():
    url_for('static', filename='.*')
    def generate():
        for i in xrange(50):
            sleep(.5)
            yield i
    return Response(stream_template('scans.html', i=generate()))

edited Oct 26, 2018 at 18:01

David Schumann

15k13 gold badges83 silver badges106 bronze badges

answered Nov 14, 2012 at 20:53

aezell

1,53014 silver badges17 bronze badges

2 Comments

SpeedCoder5 Over a year ago

Browsing localhoast:5000/scans with this solution gives the following error on my computer: jinja2.exceptions.TemplateSyntaxError: Encountered unknown tag 'i'. Any advice on how I can successfully run it?

aezell Over a year ago

@SpeedCoder5 This code was written back in 2012. I suspect more recent Flask versions and Python versions would require different solutions or at least have syntax changes.

Collectives™ on Stack Overflow

Streaming data with Python and Flask

2 Answers 2

4 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related