Flask Socketio | Update and plot a chart using background tasks created by Flask Executor or ThreadPoolExecutor

Question

I am currently trying to successfully plot a bar chart when the results from the dataframe is updated or when there is any change in the data.

Setup:

Flask-SocketIO==4.3.1
python-engineio==3.13.2
python-socketio==4.6.0

Make sure you restart your machine to not encounter errors.

Python Code -saved in D:\Projects\test_backgroundtask:

from flask import Flask, render_template, request
import pandas as pd
from flask_executor import Executor
import plotly
import plotly.graph_objs as go
from flask_socketio import SocketIO, emit
import json

global test_val
app = Flask(__name__)
socketio = SocketIO(app)
def create_plot(feature_importance):
    feature_importance=feature_importance.reset_index(drop=True)
    feature_importance=feature_importance.iloc[0:5]
    print(feature_importance)

    data = [
        go.Bar(
            x=feature_importance['Age'], # assign x as the dataframe column 'x'
            y=feature_importance['Name'], orientation='h'
        )
    ]

    graphJSON = json.dumps(data, cls=plotly.utils.PlotlyJSONEncoder)

    return graphJSON

@socketio.on("response")
def background_task_func():
    global test_val
    global plot
    socketio.sleep(10)
    data = {'Name': ['Tom', 'Joseph', 'Krish', 'John'], 'Age': [20, 21, 19, 18]}  
    test_val= pd.DataFrame(data)
    bar = create_plot(test_val)
    plot=bar
    if test_val.shape[0]>1:
        print(test_val)
        emit('response_output',plot ,broadcast=True)
        socketio.sleep(1)
        #return render_template('views/index_img_soc.html', plot=bar)
    
@app.route('/', methods=['GET'])
def index():
    global plot
    executor.submit(background_task_func)
    bar = create_plot(test_val)
    
    return render_template('views/index_img_soc.html', plot=bar)



if __name__ == "__main__":
    data ={'Name': [], 'Age': []}  
    test_val= pd.DataFrame(data)   
    executor = Executor(app)
    socketio.run(app)

Html Code (saved in D:\Projects\test_backgroundtask\template\views):

<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.6/d3.min.js"></script>
<div class="chart" id="bargraph">

    <script src="//cdnjs.cloudflare.com/ajax/libs/socket.io/2.2.0/socket.io.js" integrity="sha256-yr4fRk/GU1ehYJPAs8P4JlTgu0Hdsp4ZKrx8bDEDC3I=" crossorigin="anonymous"></script>
        <script type="text/javascript" charset="utf-8">

            var socket = io().connect('http://127.0.0.1:5000');
            socket.emit('response')
            socket.on('response_output', function(receiving_data) {
              var graphs = {{plot | safe}};
              Plotly.plot('bargraph',graphs,{});
            });
    </script>
</div>

Output:

WebSocket transport not available. Install eventlet or gevent and gevent-websocket for improved performance.
 * Serving Flask app "flask_background_app_img_soc2" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [14/Jul/2021 11:45:01] "GET / HTTP/1.1" 200 -
Empty DataFrame
Columns: [Name, Age]
Index: []
2c6e85bb23aa4c83af37bc005a08837b: Sending packet OPEN data {'sid': '2c6e85bb23aa4c83af37bc005a08837b', 'upgrades': [], 'pingTimeout': 60000, 'pingInterval': 25000}
2c6e85bb23aa4c83af37bc005a08837b: Sending packet MESSAGE data 0
127.0.0.1 - - [14/Jul/2021 11:45:03] "GET /socket.io/?EIO=3&transport=polling&t=NgaPyfS HTTP/1.1" 200 -
2c6e85bb23aa4c83af37bc005a08837b: Received packet MESSAGE data 2["response"]
received event "response" from 2c6e85bb23aa4c83af37bc005a08837b [/]
127.0.0.1 - - [14/Jul/2021 11:45:03] "POST /socket.io/?EIO=3&transport=polling&t=NgaPyf-&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -
     Name  Age
0     Tom   20
1  Joseph   21
2   Krish   19
3    John   18
     Name  Age
0     Tom   20
1  Joseph   21
2   Krish   19
3    John   18
emitting event "response_output" to all [/]
2c6e85bb23aa4c83af37bc005a08837b: Sending packet MESSAGE data 2["response_output","[{\"orientation\": \"h\", \"x\": [20, 21, 19, 18], \"y\": [\"Tom\", \"Joseph\", \"Krish\", \"John\"], \"type\": \"bar\"}]"]
127.0.0.1 - - [14/Jul/2021 11:45:13] "GET /socket.io/?EIO=3&transport=polling&t=NgaPyg0&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -
     Name  Age
0     Tom   20
1  Joseph   21
2   Krish   19
3    John   18
     Name  Age
0     Tom   20
1  Joseph   21
2   Krish   19
3    John   18
2c6e85bb23aa4c83af37bc005a08837b: Received packet PING data None
2c6e85bb23aa4c83af37bc005a08837b: Sending packet PONG data None
127.0.0.1 - - [14/Jul/2021 11:45:28] "GET /socket.io/?EIO=3&transport=polling&t=NgaP_9E&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -
127.0.0.1 - - [14/Jul/2021 11:45:28] "POST /socket.io/?EIO=3&transport=polling&t=NgaQ2md&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -
2c6e85bb23aa4c83af37bc005a08837b: Received packet PING data None
2c6e85bb23aa4c83af37bc005a08837b: Sending packet PONG data None
127.0.0.1 - - [14/Jul/2021 11:45:54] "GET /socket.io/?EIO=3&transport=polling&t=NgaQ2nA&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -
127.0.0.1 - - [14/Jul/2021 11:45:54] "POST /socket.io/?EIO=3&transport=polling&t=NgaQ93n&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -
2c6e85bb23aa4c83af37bc005a08837b: Received packet PING data None
2c6e85bb23aa4c83af37bc005a08837b: Sending packet PONG data None
127.0.0.1 - - [14/Jul/2021 11:46:20] "GET /socket.io/?EIO=3&transport=polling&t=NgaQ94a&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -
127.0.0.1 - - [14/Jul/2021 11:46:20] "POST /socket.io/?EIO=3&transport=polling&t=NgaQFPv&sid=2c6e85bb23aa4c83af37bc005a08837b HTTP/1.1" 200 -

First output even though dataframe is updated.

After reloading the page, the page is blank and then displays correct chart.

What I would like to achieve:

Automatically display the bar chart when dataframe is updated without reloading page.

Why does the page need to be reloaded after dataframe is update? Can refreshing a specific div prevent this reload from occurring. If I have to reload an entire page, this will affect other data/tables on this page, especially if the user is required to click a button to view a table - the table will disappear after reload. — Sade
– Sade, Commented Jul 14, 2021 at 10:34

Sade · Accepted Answer · 2021-07-23 07:16:31Z

In function(receiving_data) found in the html page , variable plot is used instead of receiving_data. Variable plot is initialized when render_template is evoked. When data is being emitted via sockets, you do not need to pass variable plot when return render_template('views/index_img_soc.html', plot=bar) is called.

See example script below on how to emit data using sockets to the html script.

The script below will automatically update a bar chart without reloading the page.

The code using chart.js instead of plotly. There are much more examples plotting chart.js in javascript as opposed to plotly.

Updated Python script:

import random
from flask import Flask, render_template, session, request
from flask_socketio import SocketIO
from flask_executor import Executor
import pandas as pd
import json
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
thread = None



@socketio.on("response_demo")
def background_task_func():
    """Example of how to send server generated events to clients."""

    socketio.sleep(5)
    print("send")
   
    data = {'Name': ['Tom', 'Joseph', 'Krish', 'John','Shadz'], 'Age': [20, 21, 19, 18,36]} 

    data_2= pd.DataFrame(data)
    
    df_json=data_2.to_json(orient='records')
    result = {"objects": json.loads(df_json)}
    socketio.emit('my_response',result, broadcast=True)

@app.route('/')
def index():
    executor.submit(background_task_func)
    return render_template('index_2.html')



if __name__ == '__main__':
    executor = Executor(app)
    socketio.run(app)

Updated html script:

<!DOCTYPE HTML>
<html>
<head>
    <title>Flask-SocketIO Test</title>
    <script type="text/javascript" src="//code.jquery.com/jquery-1.4.2.min.js"></script>
    <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/socket.io/1.3.5/socket.io.min.js"></script>
    <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.5.0/Chart.min.js"></script>
    <script src="//cdnjs.cloudflare.com/ajax/libs/socket.io/2.2.0/socket.io.js" integrity="sha256-yr4fRk/GU1ehYJPAs8P4JlTgu0Hdsp4ZKrx8bDEDC3I=" crossorigin="anonymous"></script>

</head>

<canvas id="myChart" width="100" height="100"></canvas>
    <script type="text/javascript" charset="utf-8">

// Chart.js Bar Chart
var ctx = document.getElementById("myChart");
var myChart = new Chart(ctx, {
    type: 'bar',
    data: {
        labels: ["Red", "Blue", "Yellow", "Green", "Purple"],
        datasets: [{
            data: [2, 2,2, 2,2],
            backgroundColor: [
                'rgba(255, 99, 132, 0.2)',
                'rgba(54, 162, 235, 0.2)',
                'rgba(255, 206, 86, 0.2)',
                'rgba(75, 192, 192, 0.2)',
                'rgba(153, 102, 255, 0.2)'
            ],
            borderColor: [
                'rgba(255,99,132,1)',
                'rgba(54, 162, 235, 1)',
                'rgba(255, 206, 86, 1)',
                'rgba(75, 192, 192, 1)',
                'rgba(153, 102, 255, 1)'
            ],
            borderWidth: 1
        }]
    },
    options: {
        scales: {
 
            yAxes: [{
                ticks: {
                    beginAtZero:true
                }
            }]
        }
    }
});


        $(document).ready(function() {

            var socket = io().connect('http://127.0.0.1:5000');
            socket.emit('response_demo')
            socket.on('my_response', function(obj) {
            for (var i=0; i < obj.objects.length; i++) {
             myChart.data.datasets[0].data[i] =obj.objects[i].Age;
             myChart.data.labels[i] =obj.objects[i].Name;
            
            }
            
                myChart.update();
           
            });

        });

</script>
</html>

The code is improved to updated both labels and x-axis values. Emitting list of string resulted in an issue, so I have passed a json object. This has also removed the flickering of the graph. If you find this example helpful, please like.
If you want to use ThreadPoolExecutor instead of Flask Executor, it is simple to update the code above. Add the following to the code -> from concurrent.futures import ThreadPoolExecutor and replace executor = Executor(app) with executor = ThreadPoolExecutor(2). You can then call executor.submit(...) as many times as you want for different functions.

Collectives™ on Stack Overflow

Flask Socketio | Update and plot a chart using background tasks created by Flask Executor or ThreadPoolExecutor

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related