391 questions
2
votes
4
answers
128
views
How to find a common value using if statement
I am still a beginner in python. I am trying to find a common value with if statement,
import pandas as pd
df = pd.read_csv("data.csv")
for n in range(2, len(df)):
if df.loc[n].isin([2]...
- 41
-1
votes
1
answer
81
views
Pandas: Nested use of .loc returns different 'type' for the same field
Why do I get 2 different data types while indexing using .loc, even though i retrieve a single value/cell.
Background: I'm looking up values of df_source in df_map. I do not get string datatype for ...
- 389
2
votes
2
answers
207
views
How can Pandas .loc take three arguments?
I am looking at someones code and this is what they wrote
from financetoolkit import Toolkit
API_KEY = "FINANCIAL_MODELING_PREP_API_KEY"
companies = Toolkit(["AAPL", "MSFT&...
0
votes
1
answer
157
views
What is a quicker way using Pandas to do a .loc on a large dataframe?
I have a large dataframe with customer data. I then have two lists with unique customer IDs and unique backup values. Both lists are the same length. I want a quicker solution to override a column in ...
- 41
0
votes
0
answers
18
views
Setting values of new column df.loc[:,'newcol']=1 [duplicate]
I'd like to create a new column this way:
df['newcol']=1
I get this error: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead
...
- 1,117
0
votes
0
answers
85
views
Why does Pandas loc with multiindex return a matrix with single row
This question is similar to Pandas selecting by label sometimes return Series, sometimes returns DataFrame, however I didn't find a solution there. I have 2 dataframes read from CSV with a multi-index ...
- 1,857
0
votes
2
answers
67
views
How to find rows that fall within time range from a dataframe?
I want to find all rows from my data frame that fall between 7am and 11am inclusive
Using this code I read a csv into a data frame with the relevent data
df = pd.read_csv(info.csv)
amount_df = pd....
- 3
1
vote
1
answer
80
views
Why dtypes are not changing when updating columns in Pandas 2.x but would change in Pandas 1.x?
When changing the values and/or dtypes of specific columns there is a different behaviour from Pandas 1.x to 2.x.
For example, on column e in the example below:
Pandas 1.x: Using pd.to_datetime to ...
- 393
1
vote
0
answers
99
views
Is there any efficient way to replace loc[[bla]] in pandas?
I have a dataframe in pandas (version 1.5.3) and I want to select the records by an index and go through them in a loop. Before I was using df_info = df.loc[[idx]], whose return is a dataframe with ...
0
votes
1
answer
136
views
dataframe is not updated while using loc function
Here below a dataframe.
df_strat = pd.DataFrame({'in_pos': [0, 0, 0, 1, 0, 0, -1, 0, 0, 0],
'in': [0, 0, 0, 1, 0, 0, -1, 0, 0, 0]})
I need to update the 'in' column value, either ...
- 61
1
vote
1
answer
145
views
Beginner Level - Python Pandas query working with Pokemon DB
I am learning Python (Pandas) and working with the Pokemon DB (following Youtube tutorial)
I am currently learning .loc and query methods so that I can access data in different ways.
I encounter an ...
- 13
0
votes
0
answers
20
views
How to modify subsetting and datetime handling with .loc[] to avoid warning? [duplicate]
I try to practice time series with real data. The difficulty is in data wrangling.
This exercise is to show the local passenger departure trend of one of the borders in Hong Kong in 2023.
Jupyter ...
- 393
1
vote
1
answer
1k
views
Issue setting entire column (and changing dtype) with .loc[:,'col'] in pandas 1.5+
I often use df.loc[:,'col'] = arr to reassign columns rather than df['col'] = arr. This was a recommended practice to avoid the fact that, prior to Copy-on-Write changes in pandas 2.0, we couldn't ...
- 21
0
votes
1
answer
78
views
Selecting Rows that only match the column values in another data frame in Python [duplicate]
I have a data frame:
>df_idbank.dtypes
DATASET object
IDBANK object
KEY object
FREQ object
INDICATEUR ...
0
votes
1
answer
28
views
How to stop overwriting values with NA when referenced index missing?
I'm trying to copy the 'Name' column values from two different dataframes stocks and major_indices to another dataframe mport if the index value of mport is present in stocks and major_indices ...
- 13
0
votes
0
answers
21
views
In Python 3.8.8, why does read_csv(StringIO()) behavior change depending on where the string is populated and passed to the function?
Depending on whether I declare a string variable within a function, or pass it from outside the function, changes the behavior of the function. Specifically, declaring a string variable and converting ...
2
votes
1
answer
139
views
Replace values in a dataframe column, based on another dataframe
I have two DataFrames:
df1
PRA_Kod;WorkerId
1049;9024
0014;60260
0608;10506
20270/9;20270
9511;64473
0639;60264
0767;10509
.. .. .. . .
df2
WorkerId;Day;Time;W
1049;11;u.wyp;WE
1049;14;u.wyp;WE
64454;...
- 53
0
votes
1
answer
352
views
Assign subset of rows to a Dataframe using loc / iloc
I am currently having a database with a set of columns X. I am willing to "update" , using .loc (or .iloc) the content of a row for a certain subset of columns (which we can call Y) from X, ...
- 3
0
votes
2
answers
49
views
How can I sum columns in a data frame for rows within a range of values that increases over a specified interval?
Edit: I tried the following code on my actual data and I am getting the incorrect ranges for column 1.
MAX_SIZE = 10_000_000 # max chromosome size
bins = list(range(0, MAX_SIZE, 10_000))
bins[0] = 1
...
- 21
0
votes
0
answers
48
views
Why is df.loc['row', 'col'] the same as df['col'].loc['row']?
I can see theyre both referencing the same cell/value, but why does df['col'].loc[row] even work? I learned to use loc as df.loc[rowname,colname].
I saw it in a list comprehension:
fb['Direction'] = [...
- 167
0
votes
0
answers
31
views
Accessing pandas datframe within function returns an Attribute error: 'function' object has no attribute 'loc'
I wrote a script to update a pandas dataframe by accessing the relevant index using .loc and assigning a desired value to it. This worked fine, but as I put these lines of script into a function which ...
1
vote
2
answers
108
views
Apply definition on multiple columns based on conditions- python
i have dataframe that i want when one column equal to '02' apply the definition to 5 others columns. this is my code:
mask = (df['A']=='02')
z_valid = df[mask]
df.loc[((mask) & (df['B'] !=0)), 'B']...
- 161
1
vote
1
answer
68
views
How filter an inner dataframe based on the row of an outer dataframe
**My goals: **
filter an inner dataframe based on the row (then the values) of an outer dataframe
Let's consider three dataframes:
first a list of shop with detail about the shop:
Shop = pd....
- 11
1
vote
2
answers
277
views
KeyError when filter pandas dataframe by column with particular key:value pair
My df looks like the following
col1
col_x
...
{"key_x":"value1"}
...
None
...
{"key_x":"value2"}
How to select all items with {"key_x":"value1&...
- 3,371
0
votes
2
answers
470
views
time series filtering with .loc with two index DataFrame
I have a time series pandas (df) table with many columns and with 2 indexes "date" and "ticker". I would like to use df.loc to select a specific range of dates , let say ("...
- 11
2
votes
1
answer
879
views
Why does converting a column from datetime to string in pandas not work using .loc?
I'm using pandas to load data from Excel with the resulting DataFrame containing both strings and dates. The columns containing strings are of dtype "object" while the date-columns are of ...
- 43
1
vote
1
answer
48
views
What's the difference between these 2 methods of Series filtering? (with or without lambda)
I have a data Series called Snow (the amount of snow in different months of the year).
These two lines of code produce the same results (at least seems so!)
So I just wanted to know the difference.
...
- 15
-1
votes
1
answer
41
views
Trying to find rows which end in .99 cents // Python // Multiple Conditionals // .loc // Manipulation inside .loc
Trying to find rows of a column which end in .99 cents
len(df_official.loc[(df_official.price_duration_days >= 7) & (df_official.shop.isin(official_stores)) & (df_official['Days From ...
0
votes
1
answer
184
views
.isin() returning a blank dataframe
My DataFrame:
Trying...
Nope...
Still nope...
AND YET...
Any idea why I can't filter this dataframe by a list using isin?
I'm expecting it to return one or more rows where the Chord Notes column ...
0
votes
1
answer
208
views
TypeError: unsupported operand type(s) for &: 'float' and 'bool' pandas
Actually i was trying to copy few columns from a dataframe to another dataframe but got an error unsupported operand type for & float and bool pandas
df
df
mask_actor = df[df.cast.str.contains(&...
-2
votes
1
answer
52
views
using an .loc and If statment
enter image description here
Use the .loc method, select the column rating for the rows of df where company_location equals "U.K."
Store it in a variable called uk_ratings.
uk_ratings = ...
- 3
0
votes
2
answers
642
views
Faster method for complex .loc in large Pandas dataframe?
For each row, I need to get a) the number of and b) the mean score of all rows 28 days prior to that row's date with the same member_ID.
df:
member_ID score date past28
0 ...
- 15
1
vote
1
answer
37
views
How do I change a pandas row value, for a certain column, for a certrain date (datetimeindex) in a dataframe?
I have a pd like this:
DATE delivery
2020-01-01 1
2020-01-01 11
2020-01-01 10
2020-01-01 9
2020-01-01 8
..
2023-03-02 5
2023-03-02 4
2023-03-02 3
...
- 863
0
votes
1
answer
187
views
Unexpected results using pandas .loc - trying to concatenate 2 columns based on a condition
I have a df where I am trying to merge 2 columns based on a condition.
Create df
df = em_df[['Redcap_Case_num', 'EV_EM', 'COMP_EM', 'EV_RND', 'COMP_EM_RND'] ].head(3)
df.to_clipboard(excel = False, ...
1
vote
1
answer
130
views
assigning an object to 1 entry of a pandas dataframe with 2 methods
I would like to share a strange behavior of pandas, and find out the reason :
I assign a numpy array as an object to 1 element (cell, entry) of a pandas dataframe in 2 different ways :
first create a ...
- 43
0
votes
1
answer
77
views
Pandas loc not filtering properly
I am using pandas to read a csv file and then trying to filter by some condition to remove blank/null/NaN values and then sorting. it still does not filter and returns all the contents. Below is the ...
0
votes
2
answers
46
views
How to search for a specific date within concatenated DataFrame TimeSeries. Same Date would repeat several times in a merged df
I downloaded historical price data for ^GSPC Share Market Index (S&P500), and several other Global Indices. Date is set as index.
Selecting values in rows when date is set to index works as ...
- 25
0
votes
0
answers
42
views
.loc only returning first value in a list instead of the full list
I have a data frame and I'm trying to return all the matching values in one column based on another column using loc. The dataframe looks like this.
Col1
Col2
Alpha
Bravo
Alpha
Charlie
Delta
Charlie
...
- 293
0
votes
2
answers
71
views
How do you replace values in one column with values in another when condition is not met with Python
I have some inconsistent data with 5 columns. The two I'm focussing on are - 'Account Name' and 'User ID'. All the Account names should contain a 7 character string. If it doesn't I want to replace if ...
- 15
0
votes
2
answers
80
views
Remove date from datatime index in pandas
I want to remove the date from datetime function in pandas and the following code works just fine.
df= pd.read_csv('data.csv')
df['Value']= df.Value.astype(float)
df['Time'] = pd.to_datetime(df['Time']...
user16593576
2
votes
1
answer
34
views
Mapping value based on two dataframe (error: an only compare identically-labeled Series objects)
Table t1
id mins maxs
0 43852 11 11
1 63087 14 15
2 63106 14 15
3 63155 14 15
Table t2
idx cons
0 1 1.00
1 2 0.95
2 3 0.90
3 4 0.85
4 5 0.80
5 6 ...
- 256
1
vote
1
answer
50
views
Filter data with it correponding value on another column
I have the following datfarame
ind
Q
haul1
haul2
haul3
haul4
haul5
em1
em2
em3
em4
em5
1
1
medio
NaN
NaN
NaN
NaN
0.165850
NaN
NaN
NaN
NaN
2
1
medio
NaN
NaN
NaN
NaN
0.244922
NaN
NaN
NaN
NaN
3
1
medio
...
0
votes
1
answer
102
views
Python Pandas .loc need count of column; null for one column and value in another column
I am trying to create a pandas .loc statement that would get the count of rows in one column where the value is not null and where the value of a different column is 1.
(I could do this same query in ...
- 71
0
votes
1
answer
75
views
In Python, what is the difference between df["row_name"] and df.loc["row_name"]
I am trying to add another row to my data Frame
When I use df["new_row"] = [5, True, "joe", 20] , I get the error
ValueError: Length of values (4) does not match length of index (...
- 47
0
votes
2
answers
399
views
How do I return the rows of DataFrame where every Country in each Continent has a Population of less of than 100?
df = pd.DataFrame({
"Continent": list("AAABBBCCD"),
"Country": list("FGHIJKLMN"),
"Population": [90, 140, 50, 80, 80, 70, 50, 125, 50]})
...
0
votes
0
answers
25
views
Python Pandas DateFrame - SettingWithCopyWarning: [duplicate]
import pandas as pd
class OrdersList:
def __init__(self):
self.shopping_list = pd.DataFrame()
self.shopping_list = self.shopping_list.assign(Order=0, Type=0, Price=0, Quantity=0)
...
- 11
1
vote
1
answer
114
views
Pandas - starting iteration index and slicing with .loc
I'm still quite new to Python and programming in general. With luck, I have the right idea, but I can't quite get this to work.
With my example df, I want iteration to start when entry == 1.
import ...
- 75
0
votes
2
answers
37
views
How to update at a specific row, after finding the same value in two tables
import pandas as pd
data1 = [['xx'], ['4']]
data2 = [['4', 'x0'], ['aa', 'bb'], ['cc', 'dd']]
df1 = pd.DataFrame(data=data1, columns=["isin"])
print(df1)
df2 = pd.DataFrame(data=data2, ...
- 1
0
votes
1
answer
63
views
Locating column of dataframe in a dataframe
I have a dataframe in a dataframe similar to this one (my real one is much larger)
df_peter = pd.DataFrame({"height": [50,np.nan,65], "weight": [20,25,27]})
df_anna = pd.DataFrame({...
- 1
0
votes
1
answer
57
views
Pandas - Blanking out values of one column if value of another column has a specific value
I have the following DataFrame:
import pandas as pd
df = pd.DataFrame({'Status': ['','To Do', '','Completed', 'To Do', 'In Progress', 'Completed'],
'Date': ['','9/1/2022','','12/5/2019','8/12/2020','...
