4

What I have tried

git submodule foreach git checkout main
git submodule foreach git add --all
git submodule foreach git diff-index --quiet HEAD || git commit -m "%CommitMessage%"
git submodule foreach git push

This runs command 1 for all submodules, then command 2 for all submodules, etc.

What I would like

I would like to only have one foreach, and do all of the commands for a submodule at once, then move on to the next submodule.

Question

Is there a way to have the git submodule foreach call a method, or in some way call multiple commands at once?

I want to do this within a batch script on Windows.

Improve this question
7
  • @Compo, given that the solution requires batch file-specific syntax in order to pass shell commands to git submodule foreach, I think the batch-file tag is appropriate, and I have restored it. Commented Sep 28, 2023 at 14:13
  • The fact that they have used %CommitMessage% and potentially || should make absolutely no difference. The question is clear "Is there a way to have the git submodule foreach call a method, or in some way call multiple commands at once?". Batch files are irrelevant to that, and a distraction, or potentially another question after this one has been answered. Commented Sep 28, 2023 at 14:55
  • @Compo, the fact that the Unix solution posted below doesn't work from a batch file implies that there are considerations specific to batch files, because the calling shell and its syntax matter when it comes to passing a (POSIX) shell command to git submodule foreach. Explicitly spelling out cmd.exe / batch-file solutions is important for git commands in general, whose sample commands are often written for POSIX-compatible shells, but especially when you have a mix of shell syntaxes, as in this case. Commented Sep 28, 2023 at 16:45
  • Once the OP has an answer to "a way to have the git submodule foreach call a method, or in some way call multiple commands at once". Then they may have an additional question regarding how that can be implemented within a Windows command line environment or script. There should only be one question per submission, and this one is clear. Commented Sep 28, 2023 at 16:57
  • @Compo, I don't think this question is worth answering in the abstract, because it is too closely tied to the syntax of the shell a user may be calling from. Pragmatically speaking, given the limited number of prevalent shells, answers could be given for all of them here, and to a large extent that has already happened, given that there's also larsks' Bash-focused answer. As such, both batch-file and bash tags are now appropriate. I have flagged your tag removal for moderator review - nothing personal, I just think the removal was the wrong thing to do. Commented Sep 28, 2023 at 18:50

2 Answers 2

Reset to default
5

Assuming a Unix-like environment, you can run a shell script with git submodule foreach:

git submodule foreach sh -e -c '
  git checkout main
  git add --all
  git diff-index --quiet HEAD || git commit -m "message"
  git push
  '
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

That does not work for me. It ends up running those commands on the parent repo instead. I also get an error about the quotes: ''' is not recognized as an internal or external command, operable program or batch file.
Then you are mistyping something. Here's an example of what running a similar script would look like in a repository with submodules: asciinema.org/a/GU0qZkV3fd723w0KhpPc10QFi
I copied and pasted what you had directly into my batch script
And yet it demonstrably works, which suggests there is something about your environment which is different. You said "batch script", but this assumes you're running in a Unix shell. Are you using Windows?
Yes, I am using Windows, not Unix. Is there a way to do the same thing within Windows?
|
3

To complement larsks' helpful Unix solution (which would also work in Unix-like environments on Windows, such as Git Bash and WSL) with a solution for Windows, from a batch file (see the next section for PowerShell):

  • As in other cases where git supports passing shell commands, these are evaluated by Git Bash, i.e the Bash implementation that comes bundled with git on Windows. As such, you must use Bash (POSIX-compatible) shell syntax even on Windows.

  • Note: git submodule foreach defines the following (environment) variables to provide information about the submodule at hand, which you may reference in your shell command:

    • $name, $sm_path, $displaypath, $sha1 and $toplevel; as noted, due to having to use Bash syntax in your shell command, you need to reference these variables as shown (e.g. as $name instead of batch-file style %name%)
    • See git help submodule for details.
git submodule foreach "git checkout main && git add --all && git diff-index --quiet HEAD || git commit -m \"%CommitMessage%\" && git push"

  • Use "..." quoting on a single line (cmd.exe / batch files don't support multiline quoted strings, and programs only expect " , not also ' to have syntactic function on their process command lines).

    • The %CommitMessage% batch-style variable reference is expanded up front, by cmd.exe, \"...\" is used to properly escape the embedded "..." around the expanded result.

    • Caveat: If the value of %CommitMessage% contains cmd.exe metacharacters such as | and &, the command will break, because cmd.exe sees these as unquoted, due to not understanding that the surrounding \" are escaped double quotes; as you report, set "CommitMessage=%info1% | %info2% | %info3%" caused a problem in your case, and there are two solution options:

      • Either: If feasible, manually ^-escape the metacharacters; e.g.:

        set "CommitMessage=%info1% ^| %info2% ^| %info3%"

      • Or, as you have done, use delayed variable expansion, which bypasses the problem (but can result in literal ! characters getting eliminated):

        • Use setlocal enableDelayedExpansion at the top of your batch file.
        • Then use !...! instead of %...% to refer to your variable, i.e. !CommitMessage!

  • Chain the commands with &&, so that subsequent commands only execute if the previous ones succeeded.

The PowerShell perspective (on both Windows and Unix-like platforms):

  • PowerShell has flexible string literals, including support for multiline literals.

  • The here-string variant used below helps readability and obviates the need for escaping embedded quotes.

# NOTE: In Windows PowerShell and PowerShell (Core) 7.2-,
#       you must manually \-escape the embedded " chars.
#       (... -m \"$CommitMessage\")
#       $CommitMessage is expanded *by PowerShell*, up front.
git submodule foreach @"
  git checkout main && 
  git add --all && 
  git diff-index --quiet HEAD || git commit -m "$CommitMessage" && 
  git push
"@

Important:

  • As noted in the code comments, Windows PowerShell and PowerShell (Core) versions up to v7.2.x unfortunately require embedded " chars. to be explicitly \-escaped when passing arguments to external programs such as git, which is fortunately no longer need in PowerShell (Core) 7.3+

  • Because PowerShell too uses sigil $ for variable references, you must `-escape any $ characters you want to preserve as such, as part of the (POSIX-compatible) shell command to be executed by git; e.g., in order to pass verbatim $name through in order to refer to the submodule name, use `$name

    • However, this only necessary if "..." quoting is used, i.e an expandable string, which in turn is only necessary if you need PowerShell's string interpolation (expansion), such as to embed the value of PowerShell variable $CommitMessage as shown above.

    • If string interpolation isn't needed, use '...' quoting, i.e. a verbatim string ('...'), in which case pass-through $ chars. need no escaping.

  • larsks' Unix solution can be used as-is in PowerShell (Core) 7.3+, but only from Unix-like environments (including WSL, if you have PowerShell (Core) installed there (too)), given that the standard Unix shell, /bin/sh, is explicitly called.

    • As an aside: This technique therefore involves an extra sh process per submodule, but is convenient, because the -e option - to abort when a command fails - allows specifying the commands individually, without having to chain them with &&
Improve this answer

4 Comments

Good catch, %CommitMessage% was the problem. Since it had | within it I need to set it like so: set "CommitMessage=%info1% | %info2% | %info3%". I didn't realize that I needed to use !CommitMessage! and do setlocal EnableDelayedExpansion. So the final command was git submodule foreach "git checkout main && git add --all && git diff-index --quiet HEAD || git commit -m \"!CommitMessage!\"" Note: This is also why I had \" to escape the quotes. Thanks for the help! I am marking your answer as the solution.
Glad to hear it, @Tyler. The alternative to switching to delayed expansion is to use ^-escaping; please see my update.
Strangely, if I manually escape using ^ it will work fine for the submodules, but then include the ^ in the message when I am committing to main.
@Tyler, that is indeed curious. It doesn't happen for me in this minimal example (run from cmd.exe, as separate commands): set "bar=a ^| b" echo "foo \"%bar%\" baz" - you'll see that the echoed string has no ^

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.