Dataframe won't append to another dataframe

Question

I have a problem with appending one dataframe to another. The dataframe "dataframe_sbp_scores" is not empty. This is a snippet of it:

cik  nextCik  searchFrac  group
138988  1800   200406    0.026923      1
138989  1800    78003    0.026683      1
138990  1800    14272    0.020913      1
138991  1800    10456    0.018990      1
138992  1800     5187    0.014663      1
138993  1800   310158    0.013702      1
138994  1800    59478    0.013702      1
138995  1800   318154    0.013462      1

When I try to append to the "df_final" it simply returns an empty dataframe, but no error occurs. Does anyone know what is an explanation for this? Below is my code:

for year in range(2004, 2017):
    df_final = pd.DataFrame()
    year_base = year
    year_peers = year - 1
    bases_list = dataframe_bases[f"{year_base}"].tolist()
    counter = 0
    for base in tqdm(bases_list):
        counter += 1
        dataframe_sbp_scores = pd.read_csv(path_to_csv + f"S&P1500 adjusted 95% {year_peers}final.csv")
        dataframe_sbp_scores = dataframe_sbp_scores[dataframe_sbp_scores["cik"] == base]
        dataframe_sbp_scores = dataframe_sbp_scores.head(20)
        dataframe_sbp_scores["group"] = counter
        print(dataframe_sbp_scores)
        df_final.append(dataframe_sbp_scores)
        print(df_final)
        if counter == 10:
            df_final.to_csv(path_save_groups + f"peer_groups_{year_base}.csv", index=False)
            break
    if counter == 10:
        break

Vasilis D · Accepted Answer · 2019-03-24 09:10:11Z

2

The append doesn't happen in-place. Try:

df_final = df_final.append(dataframe_sbp_scores)

edited Mar 24, 2019 at 9:10

answered Mar 24, 2019 at 9:04

Vasilis D

1116 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Adrian Over a year ago

Yes. It works. Thank you! Could you explain a bit more why this behaviour occurs?

Vasilis D Over a year ago

Good to hear :) DataFrame.append() returns, by construction, a new object (a new data frame). If you don't re-assign df_final to this new object, df_final will remain empty.

Adrian Over a year ago

Okay. I think this makes some sense. Thank you for the additional explanation :)

Collectives™ on Stack Overflow

Dataframe won't append to another dataframe

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related