Merging two, three or more git repositories keeping the log history

Wed, Oct 1, 2014

If you ask anyone, merging several git repositories into a single one would usually be considered a bad strategy: as a rule of thumb, your code base should have several components isolated and then use a dependency manager (e.g: Composer) to bring them together.

Nevertheless, there are always many different scenarios and with them come different use cases, weird situations or just plain normal activities that would justify this procedure. In any case if you ever need to merge several repos into one, and still want to keep the git history this is an easy way to do it.

The following technique is not rocket science and it's based in the simple addition of remotes and branch merging. If you expected something fancier I am sorry to tell you you'll find this boring.

Assumptions

• You have two, three (or N) repositories you want to merge inside a single (new) one.
• All the commit history of the three will be kept and browseable in the new one
• You want to merge everything in an empty repo named repository-all-merged
• repository,repository2 and repository3 are the ones you want to glue together
• Every merged repo will end-up inside a new folder, wherever it is. If you do not want this, just skip the early steps.
• The master branch or any others will be present in the new repo as well.

Preparing the repositories

As we said, we are going to put every repo in a different folder inside the new one. This is what you need to do (you can copy sequentially the commands writing the right paths):

Create a new directory and clone everything inside. The final repo and the existing ones. E.g:

mkdir ~/all_repos_merged
cd all_repos_merged
git clone git@bitbucket.org:alombarte/repository-all-merged.git
git clone git@bitbucket.org:alombarte/repository.git
git clone git@bitbucket.org:alombarte/repository2.git
git clone git@bitbucket.org:alombarte/repository3.git

Now go to the first repository you want to import and create a new folder, and git-move all the content inside. The folder structure you create now is the folder you'll see in the merged repo. In the following example this first repo would appear under the path/to/repository folder.

cd repository/
mkdir -p path/to/repository
git mv -k * path/to/repository/
git commit -m "Merging repositories like a boss"

Note that the -k flag ignores errors such as moving the newly created folder to itself. Before moving on with the next operation you might want to check that you don't have in each repos conflicting files, such as the .gitignore. You can delete those files and restore them later or you just fix the conflicts when they arise (edit the file, git add and then git commit: conflict solved).

Now we prepare the second, third, forth... repository. Repeat the same N more times:

    cd ../repository2/
    mkdir -p path/to/repository2
    git mv -k * path/to/repository2/
    git commit -m "Merging repositories like a boss"

    cd ../repository3/
    mkdir -p path/to/repository3
    git mv -k * path/to/repository3/
    git commit -m "Merging repositories like a boss"

CAUTION: You should never PUSH this changes to origin.

Commits are local in your computer and remember you want a new repo, not mess around with the existing ones.

Now the three repos are "ready". Meaning they are all inside a new folder and content is not mixed.

Merging all the repos together in the new one

Next steps are having the just prepared repositories available for importing code in the final repository. With git you can reference any other external repository with the "remotes".

Go to the final repo:

cd ../repository-all-merged/

Add all the remotes for merging named repository, repository2, repository3... Note that although being a "remote" this repos are the ones you cloned in your local computer:

    git remote add repository ~/all_repos_merged/repository
    git remote add repository2 ~/all_repos_merged/repository2/
    git remote add repository3 ~/all_repos_merged/repository3/

And this is the moment when we actually bring the changes to the final repo:

    git checkout master
    git pull repository master

Now we have in the master branch all the changes of repository/master branch. Conflicts might arise at this point (specially if you didn't do anything with the gitignore), but fixing them is as easy as:

    vi .gitignore
    git add .gitignore
    git commit

And repeat for every other repo:

    git pull repository2 master
    git pull repository3 master
    ...

Check that everything is in place:

git log --oneline --graph --decorate --all

The repository-all-merged contains now all the code with all the history of the master branches. If you for instance need to import more branches it would be as easy as:

git checkout another_branch
git pull repositoryN another_branch

If this merged repo has a remote, you can clean and push it now:

git remote rm repository
git remote rm repository2
git remote rm repository3
git push

The rest of the repositories can be deleted, so you don't push by mistake later on.