0

I cannot find an answer which fits to my problem.

I have a function in python which calculate the energy of a physics system. To optimize the energy I apply SGD to this function. I can record the value of the energy at each SGD step, but I want to evaluate the performance of SGD. I don't need to calculate the total execution time of the code (I know how to do that), but I need to know the evolution of the energy during the execution of SGD. So for instance at each second I need to record the value of the energy calculated so far. I have a code like this

def run_one_SGD(max_steps, lr, precis, al_range, optim):
    al_0 = torch.distributions.Uniform(0, al_range).sample((size_basis,n_pairs)).double().requires_grad_(True)

    if optim == 'SGD':
        optimizer = torch.optim.SGD([al_0], lr = lr)
    elif optim == 'Adam':
        optimizer = torch.optim.Adam([al_0], lr = lr)

    en_hist = [1.0]
    time_hist = []
    i = 0
    precision = 1 
    start_time = time.perf_counter()

    while (i < max_steps and precision > precis):
        optimizer.zero_grad()
        S, K, P, H, e, eigs = find_en(al_0)
        e.backward()  
        optimizer.step()
        en_hist.append(e.clone().detach())
        i += 1
        print(f'At {optim} step {i} the energy is {e}')
        precision = abs((en_hist[-2] - en_hist[-1])/en_hist[-1])
        end_time = time.perf_counter()
        time_hist.insert(i, (end_time - start_time))
    return en_hist, time_hist

With this code I can record the execution time of each SGD step, but I would like to record the energy value e periodically (for instance each second) during the execution of the code.

Thanks in advance for help

Paolo

def run_one_SGD(max_steps, lr, precis, al_range, optim):
    al_0 = torch.distributions.Uniform(0, al_range).sample((size_basis,n_pairs)).double().requires_grad_(True)

    if optim == 'SGD':
        optimizer = torch.optim.SGD([al_0], lr = lr)
    elif optim == 'Adam':
        optimizer = torch.optim.Adam([al_0], lr = lr)

    en_hist = [1.0]
    time_hist = []
    i = 0
    precision = 1 
    start_time = time.perf_counter()

    while (i < max_steps and precision > precis):
        optimizer.zero_grad()
        S, K, P, H, e, eigs = find_en(al_0)
        e.backward()  
        optimizer.step()
        en_hist.append(e.clone().detach())
        i += 1
        print(f'At {optim} step {i} the energy is {e}')
        precision = abs((en_hist[-2] - en_hist[-1])/en_hist[-1])
        end_time = time.perf_counter()
        time_hist.insert(i, (end_time - start_time))
    return en_hist, time_hist
2
  • How do you want to record it? It looks like you're already printing it Commented Aug 24, 2023 at 9:38
  • How long is the execution time of each step? If it's less than the length of the period, keep a running total and record the value of e every time the total exceeds the length. If the step is longer than the desired period, the solution will be more complicated. Multithreading might work, but only if the code executed during each step releases the GIL. Otherwise, you might need to use multiprocessing (assuming you have more than one cpu core available). Commented Aug 24, 2023 at 10:34

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.