How to use pandas with matplotlib to create 3D plots

Question

I am struggling a bit with the pandas transformations needed to make data render in 3D on matplot lib. The data I have is usually in columns of numbers (usually time and some value). So lets create some test data to illustrate.

import pandas as pd

pattern = ("....1...."
           "....1...."
           "..11111.."
           ".1133311."
           "111393111"
           ".1133311."
           "..11111.."
           "....1...."
           "....1....")

# create the data and coords
Zdata = list(map(lambda d:0 if d == '.' else int(d), pattern))
Zinverse = list(map(lambda d:1 if d == '.' else -int(d), pattern))
Xdata = [x for y in range(1,10) for x in range(1,10)]
Ydata = [y for y in range(1,10) for x in range(1,10)]
# pivot the data into columns
data = [d for d in zip(Xdata,Ydata,Zdata,Zinverse)]

# create the data frame 
df = pd.DataFrame(data, columns=['X','Y','Z',"Zi"], index=zip(Xdata,Ydata))
df.head(5)

Edit: This block of data is demo data that would normally come from a query on a database that may need more cleaning and transforms before plotting. In this case data is already aligned and there are no problems aside having one more column we don't need (Zi).

So the numbers in pattern are transferred into height data in the Z column of df ('Zi' being the inverse image) and with that as the data frame I've struggled to come up with this pivot method which is 3 separate operations. I wonder if that can be better.

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.cm as cm

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

Xs = df.pivot(index='X', columns='Y', values='X').values
Ys = df.pivot(index='X', columns='Y', values='Y').values
Zs = df.pivot(index='X', columns='Y', values='Z').values

ax.plot_surface(Xs,Ys,Zs, cmap=cm.RdYlGn)

plt.show()

Although I have something working I feel there must be a better way than what I'm doing. On a big data set I would imagine doing 3 pivots is an expensive way to plot something. Is there a more efficient way to transform this data ?

mgc · Accepted Answer · 2020-08-18 20:13:00Z

1

I guess you can avoid some steps during the preparation of the data by not using pandas (but only numpy arrays) and by using some convenience fonctions provided by numpy such as linespace and meshgrid.

I rewrote your code to do so, trying to keep the same logic and the same variable names :

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm

pattern = ("....1...."
           "....1...."
           "..11111.."
           ".1133311."
           "111393111"
           ".1133311."
           "..11111.."
           "....1...."
           "....1....")


# Extract the value according to your logic
Zdata = list(map(lambda d:0 if d == '.' else int(d), pattern))

# Assuming the pattern is always a square
size = int(len(Zdata) ** 0.5)

# Create a mesh grid for plotting the surface
Xdata = np.linspace(1, size, size)
Ydata = np.linspace(1, size, size)
Xs, Ys = np.meshgrid(Xdata, Ydata)

# Convert the Zdata to a numpy array with the appropriate shape
Zs = np.array(Zdata).reshape((size, size))

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

# Plot the surface
ax.plot_surface(Xs, Ys, Zs, cmap=cm.RdYlGn)
plt.show()

answered Aug 18, 2020 at 20:13

mgc

5,4531 gold badge28 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Peter Moore Over a year ago

Thank you @mgc I was hoping to use pandas because of its ability to perform many other functions to clean and transform the data before plotting. The real data would be columns of data millions of rows long in multi dimensions. So operations to groupby or resample data are easier in pandas dataframe as I understand but I'm not yet strong on pandas to say what is the best way to arrive at the plot efficiently.

mgc Over a year ago

My bad, after posting I indeed had the impress that I was modifying the setup of your minimal example instead of really answering it.

Peter Moore Over a year ago

Never the less I appreciate the effort @mgc. I purposefully avoided the meshgrid and linspace to create the example as these are elusive functions and opted for standard python methods that need no lookup. Its easy to confuse a reader that may have a great answer with too many elusive methods.

Collectives™ on Stack Overflow

How to use pandas with matplotlib to create 3D plots

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related