Automated concurrent git push

Question

I'd like to setup one simple repository which would contain output from different projects, compiled on Travis CI.

What I can't figure out is the easiest way how to safely push to the repository from the Travis console. If there are two projects building at the same time and both of them happen to push at the same time, one of them will naturally fail with non-updated refs error.

Since paths in each commit are guaranteed to be unique ([project name]/[commit id]) the best I could come up with is a script like this:

while $(git push) not ok {
    git pull --rebase
}

Can you think of something better?

You will need to handle this. Even after that pull, another concurrent job might have started its push and messed up your timeline again. In other words, you will need a slightly smarter script that detects the situation, does the pull and rebase/merge operation and then retries the push, handling the situation yet again if needed. — Lasse V. Karlsen
– Lasse V. Karlsen, Commented Mar 12, 2017 at 8:09
Though, if they are completely separate, why not just have one repository per artifact? — Lasse V. Karlsen
– Lasse V. Karlsen, Commented Mar 12, 2017 at 8:10
@LasseV.Karlsen: that would require more deploy keys, would be harder to maintain/change etc. Too much hassle for a simple "download current snapshot of each project here" kind of website. — Miro Kropacek
– Miro Kropacek, Commented Mar 12, 2017 at 8:36

koppor · Accepted Answer · 2022-02-03 17:06:27Z

2

My solution is to use parallel - as suggested for the general case at https://unix.stackexchange.com/a/471419/18033

parallel --retries 10 --delay 3 ::: "git pull --rebase && git push"

--retries 10 instructs to do 10 retries, --delay 3 states that 3 seconds should be waited between each retry. Adjust to your needs.

answered Feb 3, 2022 at 17:06

koppor

20.9k16 gold badges131 silver badges178 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

VonC · Accepted Answer · 2017-03-12 08:05:44Z

0

A workaround would be to push to two different branches B1 and B2, and have a post-receive hook on the server side which will merge those branches to master.

Since the paths in each commit are guaranteed to be unique, those merges won't have any conflict and can be automated.

Pushing to two different branches means there should no be any concurrency issue.

Since the remote is GitHub, another approach is to merge those branches on the Travis-CI side, and then push only one branch.

edited Mar 12, 2017 at 8:05

answered Mar 12, 2017 at 7:36

VonC

1.4m569 gold badges4.8k silver badges5.7k bronze badges

2 Comments

Miro Kropacek Over a year ago

This would be a cool solution but unfortunately github doesn't allow any custom hooks, only webhooks and for those I need another server to handle them. :-/ I was thinking about letting the branches as they are (i.e. without merging them back to master) but this looked awfully wrong, not to speak about increasing number of branches over the time.

VonC Over a year ago

@MiroKropacek You did not mentioned GitHub ;) I have edited my answer.

Collectives™ on Stack Overflow

Automated concurrent git push

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related