Conflict Scope

If you’ve worked with any other source management systems, you’re probably familiar with a scenario like the one just mentioned. In those systems, the typical granularity is a file and the system records the delta change (Figure 13.1). The conflict usually occurs when you make a change to a file that someone else has also changed. They check in their change before you do. Then, when you check in your change to the same file, the system tells you there is a merge conflict.

The idea in Git is similar, but the scope is wider. Recall that Git records changes as commits (or snapshots) of entire trees with files and directories. As a result, Git operates based on the idea of commits (snapshots) versus files for scope of change (Figure 13.2).

This means that if anything (any file) has been changed by someone else within the scope of the commit (snapshot) since you started making changes, you get a merge conflict—that is, you are rejected—when you go to push your changes over to the remote repository.

The idea you have to get used to is that you can get a merge conflict (inability to do a fast-forward) even if both you and the other user have changed entirely different files, even in different directories. To illustrate this, consider a situation with two users working in Git. To start with, both users have cloned the remote repository down to their local environment (Figure 13.3).

Now, let’s suppose that User 1 modifies files A2 and C2, then stages and commits them. User 1 then pushes those changes back over to the remote repository. Because User 1 is the first of the two users to push their changes in, they can get their changes in without any merge issues, as shown in Figure 13.4.

While this is occurring, User 2 is working on modifications to the same commit in their working directory. However, they are only modifying file B1—the one file that User 1 did not touch. When they are done, they commit the change and attempt to push it into the remote repository. At this point, Git rejects the push (as indicated by the X in Figure 13.5).

When trying to do this push that Git can’t fast-forward, the user generally sees a message like this:

$ git push
To C:/Program Files/Git/./calc2.git
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'C:/Program Files/Git/./calc2.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull …') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Notice that even though User 2 has changed a file that User 1 did not touch (and that User 2 hasn’t changed any of the same files that User 1 did touch), Git still sees this as a merge conflict. Again, this is because multiple users changed some content within the same snapshot (commit).

As far as the remote side is now concerned, there is a merge conflict, and it’s up to the user to resolve the problem in a local environment and then push the merged content again. (The presumption is then that the merged content can be fast-forwarded.)

Resolving the Conflict

Here’s an example of what that resolution process could look like. In Figure 13.6, User 2 has changes they cannot currently push. It is therefore up to them to get the latest changes and merge them locally into their changes. Getting the latest updates into the local environment can be done with a fetch into the repository and then a merge of the remote-tracking branch into the current branch. It can also be done with a pull operation, because the pull operation updates the local repository and attempts to merge in the changes in the working directory as well.

For simplicity, I’ll illustrate resolving this conflict in Figure 13.6 with User 2 doing a pull operation.

Once this pull is complete, the local merge completes without any problems because the same files weren’t changed. However, what if the same files were changed? In that case, more manual merging would probably be required. There are two basic scenarios here: merging after a basic pull operation and merging after a pull with the --rebase option. Let’s take a look at how each of these merge operations might be used in practice.

Dealing with Merges after a Pull

Suppose you have two users making changes to an instruction file for an application. One user is working on chapter 1 and the other is working on chapter 2. The initial version of the file that was pushed looks like this:

$ cat instructions.txt
User Instructions

Both users have cloned the repository down with the initial version of the file. User 1 makes two changes, stages and commits them, and then pushes them.

$ echo "Chapter 1" >> instructions.txt

$ git commit -am "Add chapter 1 heading"
[master 6d30ad0] Add chapter 1 heading
 1 file changed, 1 insertion(+)

$ echo "Welcome" >> instructions.txt

$ git commit -am "Add chapter 1 title"
[master ae619aa] Add chapter 1 title
 1 file changed, 1 insertion(+)

$ git push
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (6/6), 546 bytes | 0 bytes/s, done.
Total 6 (delta 2), reused 0 (delta 0)
To C:/Program Files/Git/./calc2.git
   b7f554d..ae619aa  master -> master

The log now looks like this:

$ git log --oneline

ae619aa Add chapter 1 title
6d30ad0 Add chapter 1 heading
b7f554d Add initial instructions file

User 2 also makes two changes, then stages and commits them.

$ echo "Chapter 2" >>> instructions.txt

$ git commit -am "Add chapter 2 heading"
[master 5446cab] Add chapter 2 heading
 1 file changed, 1 insertion(+)

$ echo "Next steps" >> instructions.txt

$ git commit -am "Add chapter 2 title"
[master c6495d9] Add chapter 2 title
 1 file changed, 1 insertion(+)

User 2’s log now looks like this:

$ git log --oneline
c6495d9 Add chapter 2 title
5446cab Add chapter 2 heading
b7f554d Add initial instructions file

User 2 then attempts to push their changes over.

$ git push
To C:/Program Files/Git/./calc2.git
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'C:/Program Files/Git/./calc2.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull …') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

User 2 is rejected because User 1 got their changes in first. User 2 can now choose to use the default git pull merge behavior or the git pull --rebase option to do a rebase. Let’s see what each operation looks like. First, I’ll describe the default merge behavior of the basic pull operation.

Pull Alone

User 2 does a pull.

$ git pull
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 6 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From C:/Program Files/Git/./calc2
   b7f554d..ae619aa  master     -> origin/master
Auto-merging instructions.txt
CONFLICT (content): Merge conflict in instructions.txt
Automatic merge failed; fix conflicts and then commit the result.

As expected, there is a merge conflict.

$ cat instructions.txt
User Instructions
<<<<<<< HEAD
Chapter 2
Next steps
||||||| merged common ancestors
=======
Chapter 1
Welcome
>>>>>>>> ae619aa1f2e09c7b98fefdaabf392765c43df61f

User 2 fixes the merge,

$ cat instructions.txt
User Instructions
Chapter 1
Welcome
Chapter 2
Next steps

then stages and commits it.

$ git add .

$ git commit -am "Merged chapter 2 content"
[master 934258f] Added chapter 2 content

After the commit, the log looks like this:

$ git log --oneline
934258f Merged chapter 2 content
c6495d9 Add chapter 2 title
5446cab Add chapter 2 heading
ae619aa Add chapter 1 title
6d30ad0 Add chapter 1 heading
b7f554d Add initial instructions file

Notice that you now have the new “merge commit” in your log. Now you’ll see how this process differs when you use the rebase option.

Pull with the Rebase Option

Assume you’ve reset back to the point where User 2 has attempted to push and has received the rejection message. User 2’s log looks like this:

$ git log --oneline
c6495d9 Add chapter 2 title
5446cab Add chapter 2 heading
b7f554d Add initial instructions file

User 2 now does the pull with the rebase option.

$ git pull --rebase
From C:/Program Files/Git/./calc2
 + 934258f…ae619aa master     -> origin/master  (forced update)
First, rewinding head to replay your work on top of it…
Applying: Add chapter 2 heading
Using index info to reconstruct a base tree…
M       instructions.txt
Falling back to patching base and 3-way merge…
Auto-merging instructions.txt
CONFLICT (content): Merge conflict in instructions.txt
error: Failed to merge in the changes.
Patch failed at 0001 Add chapter 2 heading
The copy of the patch that failed is found in: .git/rebase-apply/patch

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".

Notice that you see the familiar rebase messages about replaying changes, and then the merge conflict as Git tries to do the first step of the rebase. (Note the 0001 reference in “Patch failed at 0001…”, which refers to the first commit you’re trying to apply.)

You are now in a rebasing state until you complete the merges needed for a successful rebase or abort. Let’s take a look at the log at this intermediate point:

$ git log --oneline
ae619aa Add chapter 1 title
6d30ad0 Add chapter 1 heading
b7f554d Add initial instructions file

From the log, you can see that the process started with the content from the master branch and was at the step of trying to apply the first new commit on top of that content. Looking at the file with conflicts, you can see the details:

$ cat instructions.txt
User Instructions
<<<<<<< ae619aa1f2e09c7b98fefdaabf392765c43df61f
Chapter 1
Welcome
||||||| merged common ancestors
=======
Chapter 2
>>>>>>> Add chapter 2 heading

After fixing the merge conflict for this first change, User 2 stages it and then tells the rebase to continue.

$ git add .

$ git rebase --continue
Applying: Add chapter 2 heading
Applying: Add chapter 2 title
Using index info to reconstruct a base tree…
M       instructions.txt
Falling back to patching base and 3-way merge…
Auto-merging instructions.txt
CONFLICT (content): Merge conflict in instructions.txt
error: Failed to merge in the changes.
Patch failed at 0002 Add chapter 2 title
The copy of the patch that failed is found in: .git/rebase-apply/patch

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".

You have a merge conflict again, but notice that this is for the second change (0002) by User 2. Again, the contents of the file with the conflicts reflect this state:

$ cat instructions.txt
User Instructions
Chapter 1
Welcome
Chapter 2
<<<<<<< 5a81fc4b7a7d02947fb5de6c7af148fb921d484b

||||||| merged common ancestors
=======
Next steps
>>>>>>> Add chapter 2 title

User 2 now fixes this conflict.

$ cat instructions.txt
User Instructions
Chapter 1
Welcome
Chapter 2
Next steps

Still in the rebasing state, User 2 stages the change and then tells the rebase to continue.

$ git add .

$ git rebase --continue
Applying: Add chapter 2 title

After the rebase is complete, the log shows the commits in the order you expect. Note that unlike the default merge outcome, you do not have a new, separate merge commit in the log.

$ git log --oneline
7effec1 Add chapter 2 title
5a81fc4 Add chapter 2 heading
ae619aa Add chapter 1 title
6d30ad0 Add chapter 1 heading
b7f554d Add initial instructions file

So, either the merge or rebase option of the pull command can incorporate other people’s changes. The merge option may be simpler because it only requires one merge in some cases. However, it also introduces a merge commit into the history. The rebase option can require more merging and care to resolve conflicts, but it also gives a cleaner history.

Pushing Updated Content

You now have the latest changes from the remote repository incorporated into your changes in the local environment. This makes the content in the remote repository an ancestor to your changes. You can now try again to push your changes back to the remote repository, as shown in Figure 13.7.

The changes are merged successfully. This represents the typical model of working with Git and multiple users. If you try to push content and are rejected because of a non-fast-forward situation, you should pull or fetch the latest code, take care of any merge conflicts locally, and then try the push again. Of course, if someone else has made additional changes in the remote repository between when your push was initially rejected and when you try again, you may encounter new conflicts.

In the next section, I’ll talk about an approach to how remote repositories are handled on many public hosting sites, and how it facilitates contributing to other projects.

The Fork-and-Pull Model

In the fork-and-pull model, each developer or contributor has their own space where their remote repositories exist independent of the remote repositories of other users. If a user wants to just update their own projects, they can work with their remote repositories as usual.

However, if a user wants to contribute to another project (as is the case with most open-source projects), then they utilize a different workflow. To start with, the user who wants to contribute to the project owned by another user forks the owner’s remote repository on the site. In this case, fork means getting a copy of the owner’s repository (as it is at that point in time) and putting it in the contributor’s own space. So, the contributor ends up with their own personal copy of the repository to use for developing their potential contributions. Figure 13.8 illustrates this process.

They can then use the standard clone, add, commit, push, or other workflow to create a local environment and make changes to their copy of the owner’s remote repository.

Sound familiar? In some sense, you can think of this as cloning a remote repository from one user to another. The result is that the contributor can then develop whatever content they want without disturbing or interfering with the owner’s repository. (For ease of later incorporation and changes, it is a good idea to make these changes on a topic branch.) Figure 13.9 illustrates this part of the process.

So, once the contributor has completed their development and made the changes they think would be useful to the owner’s project, how do they incorporate their updates into the owner’s project? This is where the controlled process comes in. When the contributor is ready, they can send a request to the owner to merge their change in. This is called a pull request and can be as simple as a personal e-mail (although some sites provide an interface to create and send the request from directly within the browser, as shown in Figure 13.10.)

The owner can then review the candidate change from the contributor’s repository and review it before agreeing to incorporate it. They can communicate with the contributor to ask questions, request changes, or offer feedback. If the owner decides that the change is a good addition to the overall project, and is of the quality that they expect, they can merge the change from the contributor into the primary repository for the project, as shown in Figure 13.11. (Depending on the site or package, there may be built-in mechanisms to do this on the site, or it may be necessary to clone, merge, and push.) However, if the change isn’t appropriate or doesn’t meet their standards, the owner can decline it or request changes to make it suitable for incorporation.

The fork operation to create a copy of the owner’s remote repository in the contributor’s space, coupled with the request to pull the change and incorporate it, give the fork-and-pull model its name. This model provides the project owner with a high degree of oversight (and responsibility) because they are able to review and decide whether or not to take changes from others. The criteria for accepting a change could be anything from a visual inspection to a group code review to requiring passage of a set of automated testing. The end benefit is that the primary project repository (that of the owner) does not have any changes merged into it except those that the owner allows and that pass any quality gates that may be set up (such as having unit tests, adhering to a coding style, and so on).

In some cases, applications may be so big or have so many contributors that even the fork-and-pull approach becomes difficult to manage. In that scenario, one strategy is to break the application up into multiple projects (assuming it isn’t already) and assign different owners to oversee contributions to each of the parts. This is also sometimes done in a hierarchical fashion where owners of lower-level components oversee updates to those components and then submit pull requests to owners of higher-level sections to pull their collected changes in, and so on. This is similar to using military ranks for the different levels of owners—sergeants, lieutenants, generals, and so on. You may sometimes see these terms used to describe the hierarchy of owners for different parts of large projects.

Use of Multiple Remotes

When someone is working on a contribution for another project, there is a useful strategy that can help them keep their content in sync with the primary project, and guarantee that the owner of the project will be more likely to easily merge their changes. After forking the owner’s project into their own space and cloning a local environment from it, the contributor has a remote reference that points back to the forked repository. However, during the period when the contributor is working on their changes, it’s likely that other changes and contributions are being made to the owner’s primary repository. If the contributor doesn’t keep up with those changes in their forked repository, they can end up with a very different code base (aside from their changes) and significant merge issues when they are done. This can delay or prohibit their changes from being accepted. Put another way, the copy doesn’t stay in sync with the original while new development is going on.

To avoid this problem, a simple strategy is to create a second remote reference to the original repository (the one the fork was created from) and periodically fetch updates from there. This is simple to do with the git remote command, which I discuss in Chapter 12. To use an illustration based on a GitHub repository, you may have forked a repository into your space and then cloned from it. As a result, you could have an origin like this:

$ git remote -v
origin  https://github.com/contributor/project1 (fetch)
origin  https://github.com/contributor/project1 (push)

To create a connection to the owner’s project, you could create another remote reference like this:

$ git remote add primary https://github.com/owner/project1.git

You can then have access to both of these remotes:

$ git remote -v
origin  https://github.com/contributor/project1 (fetch)
origin  https://github.com/contributor/project1 (push)
primary https://github.com/owner/project1 (fetch)
primary https://github.com/owner/project1 (push)

The benefit of having this additional remote is that now, while you are working on your changes, you can periodically fetch the latest changes from the owner’s repository.

$ git fetch primary <branch>

You can then use the git merge command to merge those latest changes in with your current work at your discretion. When you are ready to submit the pull request, if you’ve been diligent about fetching and merging the latest updates from the owner’s repository, the owner should find it fairly easy to merge your changes—perhaps even doing a fast-forward. All of this contributes to the likelihood that your change will be incorporated quickly if it is suitable.

Having multiple remotes is an example of where you may want to frequently update your content in the local repository and the working directory, so that you can keep up to date with the changes being made in the remote of the other repository while you are working on your own changes. That way, you don’t diverge too far from the original code base as it continues to be updated.

If the updates overlap your local changes, this can be problematic. You need to have a way to manage this process; I offer one possible strategy in the next section.

Managing Local Updates

In Chapter 12, I describe how the pull command fetches updates from the remote repository into the local repository and then attempts a merge. If the merge is successful, the working directory is updated as well as the local repository. In some cases, such as the one I previously described, automatically merging updates into the working directory may not always be what you want.

Suppose that you are implementing a new feature that involves changing files A, B, and C. You haven’t completed your changes to A, B, and C when you learn that a key security fix has been made in the original code base that you need to incorporate in short order. So, you do a pull from the updated code base. The pull brings down the security fix and merges the changed files into the contents of your working directory. As it turns out, the security fix also involved changes to files A, B, and C. The merge changed the versions of A, B, and C that you were working on in a way that breaks your new feature significantly. You’re now faced with trying to untangle what the automatic merge did after the fact to get back to a working state.

A better approach is to have control over the merge before it occurs. Using my example, that approach would look like this:

1. Save off your changes that are in progress in A, B, and C before you do the pull. You do this using the git stash command that I cover in Chapter 11. Now you’ve stashed your changes that are in progress, effectively resetting your working directory and staging area back to the way they were at the last commit.
2. Do the pull operation to get the updates with the security fix into your local environment.
3. When you are ready, pop or apply your work from the stash. When you try to do this and there are differences between the stashed contents and the working directory contents, Git stops and informs you about the differences.
4. Make the necessary changes in your working directory, or even create another branch to do the changes in.

So, one model of incorporating updates from others while you have work in progress is to stash your changes, do a pull, and then apply or pop the changes from the stash. At that point, Git lets you know if there are merge conflicts, and you can resolve them manually instead of trying to unravel what the pull may have changed automatically.