Pandas style based on logarithm of value

Question

I'd like to style a Pandas DataFrame display with a background color that is based on the logarithm (base 10) of a value, rather than the data frame value itself. The numeric display should show the original values (along with specified numeric formatting), rather than the log of the values.

I've seen many solutions involving the apply and applymap methods, but am not really clear on how to use these, especially since I don't want to change the underlying dataframe.

Here is an example of the type of data I have. Using the "gradient" to highlight is not satisfactory, but highlighting based on the log base 10 would be really useful.

import pandas as pd
import numpy as np

E = np.array([1.26528431e-03, 2.03866202e-04, 6.64793821e-05, 1.88018687e-05,
       4.80967314e-06, 1.22584958e-06, 3.09260354e-07, 7.76751705e-08])

df = pd.DataFrame(E,columns=['Error'])
df.style.format('{:.2e}'.format).background_gradient(cmap='Blues')

Attack68 · Accepted Answer · 2021-06-07 18:22:46Z

1

Since pandas 1.3.0, background_gradient now has a gmap (gradient map) argument that allows you to set the values that determine the background colors.

See the examples here (this link is to the dev docs - can be replaced once 1.3.0 is released) https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.io.formats.style.Styler.background_gradient.html#pandas.io.formats.style.Styler.background_gradient

answered Jun 7, 2021 at 18:22

Attack68

4,8114 gold badges26 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Donna Over a year ago

Can the gmap argument be set to a function?

Attack68 Over a year ago

not yet but this is a good idea. will add it to the issues and see if I get round to implementing it.

Donna Over a year ago

On second thought, I wonder how useful this would be, given that "apply" is really flexible. Maybe just better documentation on apply? There is also an applyfunc which I didn't look too much into.

Attack68 Over a year ago

all of the built-in methods use apply. the purpose is to make the api more accessible for common functions, and not rely on apply to force common users to code everything themselves.

Donna · Accepted Answer · 2021-06-08 09:16:27Z

I figured out how to use the apply function to do exactly what I want. And also, I discovered a few more features in Matplotlib's colors module, including LogNorm which normalizes using a log. So in the end, this was relatively easy.

What I learned :

Do not use background_gradient, but rather supply your own function that maps DataFrame values to colors. The argument to the function is the dataframe to be displayed. The return argument should be a dataframe with the same columns, etc, but with values replaced by colors, e.g. strings background-color:#ffaa44.
Pass this function as an argument to apply.

import pandas as
import numpy as np

from matplotlib import colors, cm
import seaborn as sns

def color_log(x):
    df = x.copy()
    cmap = sns.color_palette("spring",as_cmap=True).reversed()
    
    
    evals = df['Error'].values
    norm = colors.LogNorm(vmin=1e-10,vmax=1)
    normed = norm(evals)

    cstr = "background-color: {:s}".format
    c = [cstr(colors.rgb2hex(x)) for x in cm.get_cmap(cmap)(normed)]  
    
    df['Error'] = c
    return df

E = np.array([1.26528431e-03, 2.03866202e-04, 6.64793821e-05, 1.88018687e-05,
       4.80967314e-06, 1.22584958e-06, 3.09260354e-07, 7.76751705e-08])

df = pd.DataFrame(E,columns=['Error'])

df.style.format('{:.2e}'.format).apply(color_log,axis=None)

Note (1) The second argument to the apply function is an "axis". By supplying axis=None, the entire data frame will be passed to color_log. Passing axis=0 will pass in each column of the data frame as a Series. In this case, the code supplied above will not work. However, this would be useful for dataframes in which each column should be handled separately.

Note (2) If axis=None is used, and the DataFrame has more than one column, the color mapping function passed to apply should set colors for all columns in the DataFrame. For example,

df[:,:] = 'background-color:#eeeeee'

would sets all columns to grey. Then, selective columns could be overwritten with other colors choices.

I would be happy to know if there is yet a simpler way to do this.

Collectives™ on Stack Overflow

Pandas style based on logarithm of value

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related