Append Dataframe in Pandas within a function

Question

I'm using this function and APScheduler to add rows to a Dataframe everyhour. The problem is every time it runs it overwrites the previous entry so its not really "appending" anything.

def SMSx(frame):
    SMS(frame)
    frame = frame.append(SMS.SMSdf)
    frame = frame[frame.a != "test"]
    frame = DataFrame(frame, columns=['Time', 'Hour', 'Date', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'Total', 'Bucket'])
    frame.to_excel('Sprint_Log.xlsx', 'Sheet 1', index=False)
    SMSx.frame = frame

If I use the exact same code (below) and run it manually it works just fine. I'm not too sure what is going on here. SMS.SMSdf is a data frame from the SMS function.

SMS(frame)
frame = frame.append(SMS.SMSdf)
frame = frame[frame.a != "test"]
frame = DataFrame(frame, columns=['Time', 'Hour', 'Date', 'Day', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'Total', 'Bucket'])
frame.to_excel('Sprint_Log.xlsx', 'Sheet 1', index=False)
SMSx.frame = frame

Thank you for the help!

Code that worked:

def SMSx(frame_temp, frame_perm):
    SMS(frame_temp)
    try:
        frame_perm = DataFrame.from_csv('Sprint_Log.csv')
    except:
        pass
    frame_perm = frame_perm.append(SMS.SMSdf)
    frame_perm = frame_perm.drop_duplicates()
    frame_perm = frame_perm[frame_perm.Ready != "test"]
    frame_perm = DataFrame(frame_perm, columns=['Time', 'Hour', 'Date', 'Day', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'Total', 'Bucket'])
    frame_perm.to_csv('Sprint_Log.csv')
    SMSx.frame2 = frame_perm

mfitzp · Accepted Answer · 2015-03-01 14:46:21Z

1

The issue I suspect is that you do not return the updated frame variable. While you assign to the SMSx variable in the function scope, as soon as the function exits that is lost. I'm not sure how this is working however, since you do not first define the SMSx variable (it's the name of the current function, or is also a global variable?)

def SMSx(frame):
    SMS(frame)
    frame = frame.append(SMS.SMSdf)
    frame = frame[frame.a != "test"]
    frame = DataFrame(frame, columns=['Time', 'Hour', 'Date', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'Total', 'Bucket'])
    frame.to_excel('Sprint_Log.xlsx', 'Sheet 1', index=False)
    return frame


while True:
     frame = SMSx(frame)  # The returned frame will be used on the next iteration

Without seeing the rest of your code, it's pretty difficult to see what you're trying to do.

answered Mar 1, 2015 at 14:46

mfitzp

15.6k7 gold badges55 silver badges72 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

ik629 Over a year ago

is there a way to return the frame and run the frame = SMSx(frame) line under the SMSx function?

ik629 Over a year ago

just to be clear - your code did in fact do exactly what I wanted it to - I just need it all in one function so I can run it through APScheduler

mfitzp Over a year ago

@ik629 If you run the SMSx(frame) function from inside the SMSx(frame) function you'll end up with an infinite recursion. Looking at APSScheduler I cannot see a way to catch return values from functions. Since you are only running the job every hour, could you instead use a file to store the result? Look at DataFrame methods .to_csv and .from_csv, along with a temporary file.

ik629 Over a year ago

thanks for the help! Now the function will save the dataframe to the .csv file, then recall it everytime it needs to overwrite. Thanks again! The code has been added in the original comment

Collectives™ on Stack Overflow

Append Dataframe in Pandas within a function

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related