Slice pandas' MultiIndex DataFrame

Question

To keep track of all simulation-results in a parametric run, i create a MultIndex DataFrame named dfParRun in pandas as follows:

import pandas as pd
import numpy as np
import itertools
limOpt = [0.1,1,10]
reimbOpt = ['Cash','Time']
xOpt = [0.1, .02, .03, .04, .05, .06, .07, .08]
zOpt = [1,5n10]
arrays = [limOpt, reimbOpt, xOpt, zOpt]
parameters = list(itertools.product(*arrays))
nPar = len(parameters)

variables = ['X', 'Y', 'Z']
nVar = len(variables)
index = pd.MultiIndex.from_tuples(parameters, names=['lim', 'reimb', 'xMax', 'zMax'])

dfParRun = pd.DataFrame(np.random.rand((nPar, nVar)), index=index, columns=variables)

To analyse my parametric run, i want to slice this dataframe but this seems a burden. For example, i want to have all results for xMax above 0.5 and lim equal to 10. At this moment, the only working method i find is:

df = dfParRun.reset_index()
df.loc[(df.xMax>0.5) & (df.lim==10)]

and i wonder if there is a method without resetting the index of the DataFrame ?

piRSquared · Accepted Answer · 2016-11-14 18:15:04Z

2

option 1
use pd.IndexSlice
caveat: requires sort_index

dfParRun.sort_index().loc[pd.IndexSlice[10, :, .0500001:, :]]

option 2
use your df after having reset_index

df.query('xMax > 0.05 & lim == 10')

setup

import pandas as pd
import numpy as np
import itertools
limOpt = [0.1,1,10]
reimbOpt = ['Cash','Time']
xOpt = [0.1, .02, .03, .04, .05, .06, .07, .08]
zOpt = [1, 5, 10]
arrays = [limOpt, reimbOpt, xOpt, zOpt]
parameters = list(itertools.product(*arrays))
nPar = len(parameters)

variables = ['X', 'Y', 'Z']
nVar = len(variables)
index = pd.MultiIndex.from_tuples(parameters, names=['lim', 'reimb', 'xMax', 'zMax'])

dfParRun = pd.DataFrame(np.random.rand(*(nPar, nVar)), index=index, columns=variables)
df = dfParRun.reset_index()

answered Nov 14, 2016 at 18:15

piRSquared

296k68 gold badges509 silver badges654 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ruben baetens Over a year ago

Thx ! Seems sorting the index was what was missing from my understanding of how to use the DataFrame. Without sorting, the function gave errors.

Collectives™ on Stack Overflow

Slice pandas' MultiIndex DataFrame

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related