Jupyter/ipython notebook : how to prevent plot redraw using interactive widget

I am struggling to have construct an interactive tool in jupyter notebook. Based on the discussion here : IPython Notebook widgets for Matplotlib interactivity, I've build the following example:

%matplotlib notebook
import matplotlib.pyplot as plt
from ipywidgets import Select,interactive,Dropdown
from IPython.display import display

fig,ax = plt.subplots()
ax.plot(range(5))

vline = ax.axvline(1, color='k')
hline = ax.axhline(0.5, color='k')

def set_cursor(x, y):
    vline.set_xdata((x, x))
    hline.set_ydata((y, y))
    ax.figure.canvas.draw_idle()

interactive(set_cursor, x=ax.get_xlim(), y=ax.get_ylim())

It works pretty well except that that the last line (interactive...) has to be run in a different cell. If I want to launch everything in the same cell (or from the same object if use a Class approach, which is what I want at the end), I have to use the following code:

%matplotlib notebook
import matplotlib.pyplot as plt
from ipywidgets import Select,interactive,Dropdown
from IPython.display import display

fig,ax = plt.subplots()
ax.plot(range(5))

vline = ax.axvline(1, color='k')
hline = ax.axhline(0.5, color='k')

def set_cursor(x, y):
    vline.set_xdata((x, x))
    hline.set_ydata((y, y))
    ax.figure.canvas.draw_idle()
    display(fig)

tool = interactive(set_cursor, x=ax.get_xlim(), y=ax.get_ylim())
display(tool)

But in this case, the complete figure is redraw each time a new value is selected in a widget

Is there any possibility to get everything launch smoothly from the same cell ? Any idea is more than welcome !