124

First - apologies for asking this question. There are a lot of topics about it already. But I'm not having much luck with them. My lack of familiarity with git is not much help.

I'm moving a folder from one git repo to another (which already exists). e.g.

repo-1
---- dir1
---- dir2
---- dir3
---- dir-to-move
---- dir5

repo-2
---- dir1
---- dir2
---- dir3

In the end I want the repos to look like

repo-1
---- dir1
---- dir2
---- dir3
---- dir-to-move
---- dir5

repo-2
---- dir1
---- dir2
---- dir3
---- dir-to-move

i.e. For a time dir-to-move will exist in both repos. But eventually I'll migrate the latest changes to repo-2 and remove dir-to-move from repo-1.

My initial research made me believe that I needed to use filter-branch. e.g.

How to move files from one git repo to another preserving history using `git format-patch`and `git am`

I've since learnt that subtree superseded that approach. However it's not doing what I expected. I thought I'd be able to do something like

In repo-1 workspace

git subtree split -P dir-to-move -b split

to filter the split branch down to only dir-to-move and it's history. Then in repo-2 workspace

git remote add repo-1 repo-1-url.git
git subtree add --prefix dir-to-move split

This does move the code across. It also, sort of, includes the history

e.g.

cd repo-2
git log

Shows commits from repo-1

but

cd repo-2
git log dir-to-move

Shows only an 'Add dir-to-move from commit ....'

i.e. The history is included but does not show up when checked for the specific files/directories.

How can I do this properly?

Improve this question
1
  • 1
    looks like git fails to follow directory move. If you specify some file instead of the directory, does it follow its move? Commented Jan 24, 2017 at 19:04

12 Answers 12

Reset to default
127

This is indeed possible using git subtree.

In repo-1 create a subtree:

git subtree split -P dir-to-move -b <split>

The split branch will now only contain the dir-to-move directory. You now need to pull the branch from repo-1 into a branch in repo-2.

In case repo-2 happens to be a new repository (e.g. just after git init) Things are as easy as checking out a branch with no history and pulling from repo-1.

cd repo-2
git checkout <branch>
git pull <path-to-repo-1.git> <split>

However if repo-2 is an existing repo where commits have already been made (as is the case in this question), you need to merge from an orphan branch in repo-2:

cd repo-2
git checkout --orphan <temp>                    # Create a branch with no history
git pull <path-to-repo-1.git> <split>           # Pull the commits from the subtree
git checkout <branch>                           # Go back to the original branch
git merge --allow-unrelated-histories <temp>    # Merge the unrelated commits back
git branch -d <temp>                            # Delete the temporary branch
Improve this answer
Sign up to request clarification or add additional context in comments.

15 Comments

That was the most helpful answer for me. I'd like to add that the --allow-unrelated-histories can also just be used on the git pull command directly, which may lead to some merge conflicts if you have the same files in your repo-2 already, but if it's easy to solve those conflicts then this is the fastest solution, imo.
This worked well for me but lost the parent folder. Found a solution here
git pull <path-to-repo-1.git> <split> generates the error fatal: Updating an unborn branch with changes added to the index. Documentation on the error seems unavailable.
Additional point, if you use Azure Devops or similar, when merging to central branches you will frequently find the default merge strategy is "Squash". Do not use squash or all commits will be replaced by a single commit. In other words, you will lose all your history in that final merge to the central branch. Use "Merge (no fast forward)" or something similar for this central branch final merge.
These steps commiting the files to parent folder of new repo however I need to move dir1 from old repo to dir2 of new repo(e.g. newRepo/dir2)..can we achieve this ?
|
38

I can't help you with git subtree, but with filter-branch it's possible.

First you need to create a common repository that will contain both source and destination branches. This can be done by adding a new "remote" beside "origin" and fetching from the new remote.

Use filter-branch on the source branch to rm -rf all directories except dir-to-move. After that you'll have a commit history that can be cleanly rebased or merged into the destination branch. I think the easiest way is to cherry-pick all non-empty commits from the source branch. The list of these commits can be obtained by running git rev-list --reverse source-branch -- dir-to-move

Of course, if the history of dir-to-move is non-linear (already contains merge commits), then it won't be preserved by cherry-pick, so git merge can be used instead.

Example create common repo:

cd repo-2
git remote add source ../repo-1
git fetch source

Example filter branch

cd repo-2
git checkout -b source-master source/master
CMD="rm -rf dir1 dir2 dir3 dir5"
git filter-branch --tree-filter "$CMD"

Example cherry-pick into destination master

cd repo-2
git checkout master
git cherry-pick `git rev-list --reverse source-master -- dir-to-move`
Improve this answer

7 Comments

Appears to be the only solution. I followed the approach here blog.mattsch.com/2015/06/19/…
Notably, if the contents of dir-to-move were ever previously moved within repo-1, this solution will truncate the history of those files to only as long as they've existed in the dir-to-move location within repo-1.
For anyone coming here in 2021 or later, note that in the last couple of years, git convention has changed to use main instead of master, so you may need to replace occurrences of master with main in the code of this answer.
git fetch source --no-tags, if you don't want to pull in tags from repo-1.
error: commit <SHA1> is a merge but no -m option was given. fatal: cherry-pick failed
|
26

FWIW, the following worked for me after a number of iterations.

Clone both the repos to a temporary work area.

git clone <repourl>/repo-1.git 
git clone <repourl>/repo-2.git
cd repo-1
git remote rm origin # delete link to original repository to avoid any accidental remote changes
git filter-branch --subdirectory-filter dir-to-move -- --all  # dir-to-move is being moved to another repo.  This command goes through history and files, removing anything that is not in the folder.  The content of this folder will be moved to root of the repo as a result. 
# This folder has to be moved to another folder in the target repo.  So, move everything to another folder.
git filter-branch -f --index-filter \
'git ls-files -s | /usr/local/bin/sed -e "s/\t\"*/&dir-to-move\//" |
    GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
        git update-index --index-info &&
 mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD
# Above command will go through history and rewrite the history by adding dir-to-move/ to each files.  As a result, all files will be moved to a subfolder /dir-to-move.  Make sure to use gnu-sed as OSX sed doesn't handle some extensions correctly.  For eg. \t

Now switch to target repo and fetch everything from source.

git clone repo-2
git remote add master ../repo-1/
git pull master master --allow-unrelated-histories
git push  # push everything to remote

Above steps assume that master branches are used for both source and targets. However, tags and branches were ignored.

Improve this answer

5 Comments

I had problems with this approach on windows + git bash. I replaced the path to usr\bin path with the actual sed command but I got Rewrite ... mv: cannot stat my-repo-local-path/.git-rewrite/t/../index.new': No such file or directory`
Nice git voodoo for correcting the destination path. Worked flawlessly.
On macOS I had to tweak /usr/local/bin/sed -e "s/\t\"*/&dir-to-move\//" command. Instead of /usr/local/bin/sed just use sed and replace \t with tab provided with ^+V Tab.
What is the ^+V Tab I tried ^+V that does nothing. Then while holding ^+V I push Tab then it switches to a new different tab.
You may need adding "--no-ff" flag to the line: git pull master master --no-ff --allow-unrelated-histories
20

My approach to move dir-to-move to repo 2. Start by cloning the repo 1 in a new location and cd to the repo 1 folder. In my case I am moving the folder from repo 1 branch develop to repo 2 where develop is not existent yet.

git filter-branch --subdirectory-filter dir-to-move -- --all     #line 1
git remote -v                                                    #line 2
git remote set-url origin repo_2_remote_url                      #line 3
git remote -v                                                    #line 4
git push origin develop                                          #line 5

Explanation for each line:

  1. Cleans all commits not related to dir-to-move, removes all files outside that folder, moves all contents in the root folder repo 1. From git docs:

Only look at the history which touches the given subdirectory. The result will contain that directory (and only that) as its project root

  1. You are still pointing to your origin repo 1 url
  2. Replace the origin url for your current folder to point to repo 2
  3. Check the URLs have been updated correctly
  4. Push to repo 2 the (newly created, in this case) develop branch

You can now clone or pull or fetch your repo 2 repository. You will have under repo 2 folder the contents of dir-to-move along with relevant history as expected.

Improve this answer

4 Comments

The problem with this approach is that it will not move all the remote branches and tags.
It only moves one branch at a time, you could do this for all your branches manually if you don't find a better way. (But should you have long living branches?!) The tags are basically commit identifiers so there must be an additional command for that too
It's not about having long living branches but really just remote branches that you have not checked out locally (it might be short lived branches from other people for example). To do that you should do something like this: stackoverflow.com/a/20793890/43046
These commands work for me! If possible, could you please share any command/setting by which I can revert my branch-filter changes after pushing to another branch?
12

Another solution is to use git-filter-repo which git filter-branch officially recommends.

Find a full example including hints about the installation for windows users at my answer on how to push commits affecting a given path to a new origin?

Improve this answer

Comments

9

You can use git format-patch, followed by git am:

mkdir /tmp/mergepatchs
cd ~/repo/org
git format-patch -o /tmp/mergepatchs --root dir_to_move1 dir_to_move2 dir_to_move3
cd ~/repo/dest
git am /tmp/mergepatchs/*.patch

Thank-you to http://blog.neutrino.es/2012/git-copy-a-file-or-directory-from-another-repository-preserving-history/ for suggesting this approach, and to @sean's comment, below, for improving it :)

Improve this answer

4 Comments

simpler with: git format-patch -o /tmp/mergepatchs --root $reposrc
This worked well for me! One slight change is that I used git am -3 /tmp/mergepatchs/*.patch to do a 3-way merge to resolve conflicts. Some docs here: git-scm.com/docs/git-am#Documentation/git-am.txt--3
Thank you @sean . That seems like it should actually be possible to memorize now :) (So far, I just google each time, and just copy-paste from here :) )
git am /tmp/mergepatchs/*.patch This does not work on Windows with cmd, and also not on Windows with powershell, because the wildcard character * is not accepted. You need to call git am with each patch file separately.
5

Split git monorepo into smaller git repositories

(Move git folder from one repo to another while keeping commit history)

cd target-repo

git remote add source ../monorepo
git fetch source

# create new branch from source branch (e.g. master)
git checkout -b source source/master

# filter out git history of a single folder only (e.g. ./dir)
FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch --subdirectory-filter ./dir -- --all

# reapply the commits on top of your target repo history
git rebase master

# create a new Pull Request branch
git checkout master
git checkout -b your_new_branch
git rebase source

Now that you have a fresh new repository, it might be a good time to clean-up some unnecessary files from the history before you publish the new master branch.

FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch --tree-filter 'rm -rf <path_to_file>' HEAD
Improve this answer

1 Comment

Working but timestamps of commit history is overwritten
1

We have two approaches here:

  • git filter-branch
  • git subtree

I used the former approach from https://www.johno.com/move-directory-between-repos-with-git-history and I found some better simplifications (e.g. Basin's approach is great).

Nevertheless, the later approach is much better and I can point some reasons:

  1. "git filter-branch has a plethora of pitfalls that can produce non-obvious manglings of the intended history rewrite (and can leave you with little time to investigate such problems since it has such abysmal performance). These safety and performance issues cannot be backward compatibly fixed and as such, its use is not recommended." - You can read more about the reason why in https://git-scm.com/docs/git-filter-branch

  2. I really disliked the ideia of using git filter-branch when I saw it rewrote the whole history of my repo unnecessarily (i.e. git filter-branch --subdirectory-filter my-dir -- -- all)

  3. When I found git subtree split -P dir-to-move -b which creates a clean branch with only my folder I loved it, so using the approach presented by Thymo really was so clean and I strongly recommend using this approach!

Maybe a need to post a script on githug.

Improve this answer

Comments

0

Building on Thymo's answer, I made a complete executable bash script (there is one command that actually requires bash). It also takes into account the subdirectory in the target repo:

#!/bin/bash
# data to configure individually
pathToSourceRepo="$(pwd)/repo-1"   # The repo the directory should be moved out of. Needs to be an absolute path
pathToTargetRepo="$(pwd)/repo-2"   # The repo the directory should be added into. Needs to be an absolute path
dirToMove=dir-to-move              # path relative to source repo's root
targetPath=dir-to-move-to          # path relative to the target repo's root THIS value needs to be added in line 20 as well!
baseRefInTargetRepo="origin/HEAD"  # ref of the target repo to base the inclusion of the directory on
targetBranch="add-dir-from-repo-2" # name of the branch in the target repo where the commits get merged into (typically a feature branch I suppose)

# Begin of script
# Prepare the subtree with only the directory to be moved in it
cd "$pathToSourceRepo" || exit
splitBranch="split"
git subtree split --prefix="$dirToMove" --branch=$splitBranch
git checkout $splitBranch

# Move the files to the target directory
mkdir --parents "$targetPath"
shopt -s extglob dotglob                  # This is a bash-specific feature!
git mv !(dir-to-move-to|.git) $targetPath # ADD targetPATH literally to the extended glob (did not want to further complicate it with an additional eval)
shopt -u extglob dotglob
git commit -m "Move files to '$targetPath' subdirectory"

# Add the directory to the target repo
cd "$pathToTargetRepo" || exit
git checkout --orphan $splitBranch                                                                    # Create a branch with no history
git pull "$pathToSourceRepo/.git" $splitBranch                                                        # Pull the commits from the subtree
git checkout --no-track -b "$targetBranch" $baseRefInTargetRepo                                       # Go back to the original branch --no-track makes sure the base branch will not be the upstream branch
git merge -m "Merge history of $dirToMove from another repo" --allow-unrelated-histories $splitBranch # Merge the unrelated commits back
git branch --delete $splitBranch                                                                      # Delete the temporary branch
Improve this answer

Comments

-1

You can even make a commit on repo-1 by deleting all directory other than dir-to-move locally (by this way no need to use fitler-branch) and then pull to repo-2.

git clone https://path/to/repo-1
cd repo-1
rm dir1 dir2 dir3 dir5 -rf
git commit -m "Removed unwanted directory for repo-2
cd ../
git clone https://path/to/repo-2
cd repo-2
git remote add repo-1-src ../repo-1
git pull repo-1-src master --allow-unrelated-histories

Note: Make sure git version is 2.9 or more for using --allow-unrelated-histories option

Improve this answer

1 Comment

So, this does work but only does what it says on the tin. Basically the historical commits from repo-1 leading up to the commit in which everything else got removed will become jammed into your repo-2. I reckon this could be used together with filter-branch so that repo-2's history stays intact. @donnie and @basin answers are the full package.
-1

I got hard time to try different approaches, but this works just fine:

git log --pretty=email --patch-with-stat --reverse --full-index --binary -m --first-parent -- your-folder/ > your-folder-history.patch

then you'll have a big your-folder-history.patch file, which you can apply to any another repo with

mv /old/repo/path/your-folder-history.patch .
git apply your-folder-history.patch
Improve this answer

Comments

-12

dumb manual solution if all other fails:

  1. Download all files from first repo
  2. Create a new repo
  3. Clone the second repo to disk.
  4. Unpack the first repo files you want to keep (new structure)
  5. Push the changes to second repo.
  6. Verify that you have what you want
  7. Delete the files/folders from first repo that you dont want.

You lose history however

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.