Warning with settingsWithCopyWarning , when creating another column

Question

I have data as shown here:

    SampleNumber    X  A  B
0              1  120  2  4
1              2  120  2  4
2              3  120  2  4
3              4  120  2  4
4              5  120  2  4
5              6  120  2  4
6              7    0  2  4
7              8    0  2  4
8              9    0  2  4
9             10    0  2  4
10            11    0  2  4
11            12    0  2  4
12            13    0  2  4
13            14  150  2  4
14            15  150  2  4
15            16    0  2  4
16            17    0  2  4
17            18  135  2  4
18            19  135  2  4
19            20    0  2  4
20            21    0  2  4

I have a column X where most of the values of the same and values sometimes changes I need data when the values in column X changes and it is not 0.

Output: I need samples: 1, 14,18 The way I am doing is

common_t_filtered_df['C'] = common_t_filtered_df['X'].diff()
df_filtered = common_t_filtered_df[common_t_filtered_df['C']!=0]
try:
   df_filtered = df_filtered[df_filtered['X'] != 0]
   df_filtered = (df_filtered[['A','B','X','SampleNumber']]).values

I keep getting this error:

line:err: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy common_t_filtered_df['C'] = common_t_filtered_df['X'].diff()

Can someone let me know how can I get rid of this error or maybe there is some smart idea of getting the values. Btw 120,150,135 are just examples, it can be anything.

David Erickson · Accepted Answer · 2021-01-14 00:10:07Z

1

You are using pandas incorrectly. In this case, I'm not going to fully address the question of why you are getting this particular error as that is all over the internet. However, as a short answer, you can often avoid this error my creating a copy() of the dataframe prior to slicing it (e.g. df1 = df.copy()). But, I would suggest you just use the following code instead of using try statements.

You just have to filter your dataframe for conditions that meet the conditions you have set. You are looking for shift() and when the value is not equal to 0 with ne(0):

df = df[df['X'].ne(0) & df['X'].ne(df['X'].shift())]

Out[1]: 
    SampleNumber    X  A  B
0              1  120  2  4
13            14  150  2  4
17            18  135  2  4

To get just SampleNumber, use:

df['SampleNumber'].tolist()

Out[2]:
[1, 14, 18]

edited Jan 14, 2021 at 0:10

answered Jan 13, 2021 at 22:00

David Erickson

16.7k2 gold badges21 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

saras Over a year ago

Thank you for pointing me in the right direction

Collectives™ on Stack Overflow

Warning with settingsWithCopyWarning , when creating another column

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related