Edit : it's in the same notebook !
Yes you can, but not in the markdown cell (as far as I know).
Just save your figure in a figure object (if not already done by fig = plt.figure() )
plt.plot(...)
fig = plt.gcf()
And then call fig wherever in the notebook. To get rid of the input cell after html conversion, see below.
If between 2 distinct notebooks (but also work in the same notebook)
The closest I can get to your request is to use the base64 address of the image :
So far it seems that markdown doesn't deal well with this, but you can put the following in a %%html magic cell.
%%html
<img alt="Image" src="data:image/png;base64,iVBORw0KG ... hEJRU5ErkJggg==" />
With
data:image/png;base64,iVBORw0KG ... hEJRU5ErkJggg==
being your image in base64. You can get this address just by right clicking on the image in the other notebook and select "copy image address" (if it's inline image). Otherwise you can use image to base64 converter.
Here you'll still get the input cell visible, but if you eventually convert your notebook in html, you'll be able to hide the input cells with some Javascript. (See here for how to, disclaimer: it's my answer (to my question)).
At the end, you'll be able to hide the %%htmlmagic cell while keeping the image.
Hope this helps