The Hackerlab at `regexps.com`

## Implementing Development Policies

up: arch next: The Theory of Patches and Revisions prev: Getting Started with arch

Different projects have different policies for managing the "main development path" as distinguished from various kinds of release (such as candidate releases, experimental releases, and stable releases).

`arch` is flexible enough to allow many such policies to be implemented in a direct way. Here are some examples and hints about using `arch` .

### Milestone/Numbered Versions

One development policy is based on milestones . The team works on one milestone goal at a time. When they have just about reached that milestone, they make candidate releases of the milestone for people to test. After fixing bugs, a milestone release is made.

After several milestones, the milestone release gets a version number, and becomes a version release .

Beginning from scratch on the first milestone, the developers create the first development path -- in which to work on reaching the first milestone:

         mozilla--devo--1.0
         ------------------
         base-0
         patch-1
         patch-2
         patch-3
         patch-4

After some time, they are ready to make some candidate releases and enter a bug-fixing cycle. They'll use tags for that -- making the bug fixes in the `devo` branch:

         mozilla--devo--1.0
         ------------------
         base-0
         patch-1
         patch-2                       mozilla--milestone--1.0
         patch-3                       -----------------------
         patch-4 --------------------->base-0 (tag -- 1st release candidate)
         patch-5               .------>patch-1 (tag -- 2nd candidate)
         patch-6 --------------   ---->patch-3 (tag -- 3rd candidate)
         patch-7                 |
         patch-8 ----------------

In the diagram, patches `5..8` in the `devo` branch are fixes made based on reports from candidate releases. We'll suppose that after `patch-8` , the first milestone appears to be stable. The developers want to make the milestone release, and begin work on milestone `2` :

         mozilla--devo--1.0
         ------------------
         base-0
         patch-1
         patch-2                       mozilla--milestone--1.0
         patch-3                       -----------------------
         patch-4 --------------------->base-0 (tag -- 1st release candidate)
         patch-5               .------>patch-1 (tag -- 2nd candidate)
         patch-6 --------------   ---->patch-3 (tag -- 2rd candidate)
         patch-7                 |     version-0 (milestone release)
       --patch-8 ----------------
      |
      |  mozilla--devo--2.0  (development path for the second milestone)
      |  ------------------
       ->base-0 (continuation)

That pattern can continue indefinately, eventually resulting in a versioned release, (again using tags):

         mozilla--devo--1.0            mozilla-milestone--1.0
         ------------------            ----------------------
         ...
       --patch-N---------------------->...
      |                                version-0 (milestone 1 release)
      |
      |
      |  mozilla--devo--2.0            mozilla-milestone--2.0
      |  ------------------            ----------------------
      |  ...
       ->patch-N---------------------->...
      |                                version-0 (milestone 2 release)
      |
      |
      |  mozilla--devo--3.0            mozilla-milestone--3.0
      |  ------------------            ----------------------
      |  ...
       ->patch-N---------------------->...
      |                                version-0 (milestone 3 release)
      |          ...
      |
      |  mozilla--devo--10.0           mozilla-milestone--10.0
      |  -------------------           -----------------------
      |  ...
       ->patch-N---------------------->...
                                     --version-0 (milestone 10 release)
                                    |
                                    |  mozilla-1.0
                                    |  -----------
                                     ->base-0 (tag)
                                       version-0 (versioned release)

That pattern of `arch` usage is very simple because the developers stay in sync, following a strict cycle of working on a milestone, making candidate releases and bug-fixing, making a milestone releases and starting on the next milestone. Less synchronized development can be much more complicated, as illustrated in the next example:

### Even/Odd Versions

Another, more intricate policy is based on version numbers. Odd numbered versions are the "leading-edge" of development -- often unstable, but having the very latest sources. Even numbered versions are "stable" -- lagging behind the leading-edge, but containing only code known to work reasonably well.

The `arch` concept of a "continuation version" is ideal for this, because there is no requirement that a continuation version be a continuation of the immediately preceeding version.

A series of diagrams can help to illustrate this usage of `arch` and some of the subtleties that can arise. We'll start with just the initial leading-edge development path

         linux--0.1
         ----------
         base-0
         patch-1
         patch-2
         patch-3
         patch-4
         version-0

At that point, the developers decide to make the first stable release:

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |
         patch-4          |
         version-0  ------

We can suppose that `patch-1` of `linux--0.2` is just a quick change to the top level `README` file, and that after that -- the new stable version is sealed (creating `version-0` ) and released. We never particularly want to merge `patch-1` of `linux-0.2` back on to the odd-numbered versions.

Naturally, some bugs are detected in the stable release -- three in rapid succession. The developers fix these on the leading edge branch, planning to merge them with the stable tree later:

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |
         patch-4          |
         version-0  ------
         versionfix-1
         versionfix-2
         versionfix-3

They wait a week for some testing to occur. The bug fixes are looking ok, so the developers decide to start work on the next leading-edge version:

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |
         patch-4          |
         version-0  ------
         versionfix-1
         versionfix-2
       --versionfix-3
      |
      |
      |  linux--0.3
      |  ---------
       ->base-0 (continuation)
         patch-1

Just as the developers are about to merge the bug fixes with the stable version, one more bug report trickles in. Fortunately, it's a trivial bug -- so the developers are confident about making the fix in the leading edge, but immediately releasing it in the stable version.

Here's a catch, though -- the latest leading edge version (`linux--0.3` ) has already diverged from the stable version because of `patch-1` . The developers definately don't want `patch-1` of `0.3` in the the stable `0.2` yet, so they fix the newly reported bug in `0.1` , then merge all four bug fixes with the stable tree in the usual way. Meanwhile, separate work also continues on `0.3` :

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |    ->versionfix-1 (merge)
         patch-4          |   |
         version-0  ------    |
         versionfix-1\        |
         versionfix-2 |-------
       --versionfix-3 |
      |  versionfix-4/
      |
      |
      |
      |  linux--0.3
      |  ---------
       ->base-0 (continuation)
         patch-1
         patch-2
         patch-3

It's worth noting at that point that `linux--0.3` is missing a bug-fix (`versionfix-4` ) from `linux--0.1` :

         % cd ~/wd/linux--0.3

         % larch whats-missing linux--0.1
         versionfix-4

We'll eventually do an update to pick up that bug fix, but first, let's make the situation more complicated.

Suppose, some mailing lists get wind of `patch-3` in `linux--0.3` . Soon `Slashdot` and `Newsforge` pick up the story. It turns out that `patch-3` is a very desirable feature and the implementation appears to be clean and stable. People are clammering for its appearence in a stable release, and the developers happen to think it is a good idea.

So, they start the next stable revision:

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |    ->versionfix-1 (merge)
         patch-4          |   |     |
         version-0  ------    |     |
         versionfix-1\        |     |
         versionfix-2 |-------      |    linux--0.4
       --versionfix-3 |             |    ----------
      |  versionfix-4/               --->base-0 (continuation)
      |
      |
      |
      |  linux--0.3
      |  ---------
       ->base-0 (continuation)
         patch-1
         patch-2
         patch-3

They only want `patch-3` of `0.3` for `0.4` -- not anything else. That's a job for `replay --exact` :

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |    ->versionfix-1 (merge)
         patch-4          |   |     |
         version-0  ------    |     |
         versionfix-1\        |     |
         versionfix-2 |-------      |    linux--0.4
       --versionfix-3 |             |    ----------
      |  versionfix-4/               --->base-0 (continuation)
      |                               =->patch-1 (replay --exact merge)
      |                              |   version-0
      |                              .
      |  linux--0.3                  |
      |  ---------                   .
       ->base-0 (continuation)       |
         patch-1                     .
         patch-2                     |
         patch-3-=--=--=--=--=--=--=-
         ...

After merging the much-desired `patch-3` of `0.3` into `0.4` , the developers seal `0.4` and make the stable release.

Let's suppose that development on `0.3` continues for a while. After some time, the developers decide that `0.3` has aquired enough new features. They want to do two things: start `0.5` , and start getting the stable `0.6` release ready:

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |    ->versionfix-1 (merge)
         patch-4          |   |     |
         version-0  ------    |     |
         versionfix-1\        |     |
         versionfix-2 |-------      |    linux--0.4
       --versionfix-3 |             |    ----------
      |  versionfix-4/               --->base-0 (continuation)
      |                               =->patch-1 (replay --exact merge)
      |                              |   version-0
      |                              .
      |  linux--0.3                  |
      |  ---------                   .
       ->base-0 (continuation)       |
         patch-1                     .
         patch-2                     |
         patch-3-=--=--=--=--=--=--=-
         ...                            linux--0.6
       -version-0                       ----------
      |
      |
      |
      |  linux--0.5
      |  ----------
       ->base-0

Notice that they haven't created the base revision for `0.6` yet. There's a choice here. They could make `0.6` a continuation of `0.4` , then merge in all the patches they're missing from `0.3` . On the other hand, they could make `0.6` a continuation of some `0.3` and pick up all those missing patches "the easy way".

But, oops, when someone checks out `version-0` from `0.3` and runs `whats-missing` , they find out that the `0.3` branch never picked up `versionfix-4` from `0.1` . That's easily fixed by updating a `0.3` tree against `0.1` , and checking in the resulting merge to `0.3` . The resulting merge is also the revision that will become the base revision for `0.6` :

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |    ->versionfix-1 (merge)
         patch-4          |   |     |
         version-0  ------    |     |
         versionfix-1\        |     |
         versionfix-2 |-------      |    linux--0.4
       --versionfix-3 |             |    ----------
      |  versionfix-4/               --->base-0 (continuation)
      |            |                  =->patch-1 (replay --exact merge)
      |             -----            |   version-0
      |                  |           .
      |  linux--0.3      |           |
      |  ---------       |           .
       ->base-0 (continuation)       |
         patch-1         |           .
         patch-2         |           |
         patch-3-=--=--=--=--=--=--=-
         ...             |              linux--0.6
       -version-0        V              ----------
      | versionfix-1 (0.1 update)------>base-0 (continuation)
      |                                 version-0
      |
      |  linux--0.5
      |  ----------
       ->base-0
         patch-1
         patch-2

Meanwhile, new work continues on `0.5` .

But now, the `0.5` tree is in an interesting state. If a developer checks out the latest `0.5` and asks:

         % larch whats-missing linux--0.1
         linux--0.1--versionfix-4

If the developer asks `whats-missing` from `0.3` , the answer is:

         % larch whats-missing linux--0.3
         linux--0.3--versionfix-1

If those two patches were unrelated -- there would be no problem: simply update from both branches and check the result into `0.5` .

In fact, though, `versionfix-1` from `0.3` is really the same change as `versionfix-4` from `0.1` (look back at how `versionfix-1` was created). Let's also suppose that when the fix was merged into `0.3` , a slight change had to be made -- to resolve a merge conflict.

So if the developer just blindly updates from `0.1` , then from `0.3` , the second update will result in new conflicts. That might not be so bad if we're only talking about two patches -- but if we were talking about `20` or `200` , a lot of needless work would be called for.

Fortunately, `arch` can help. First, the developer gets the latest `0.5` revision:

         % larch get linux--0.5 ~/wd/linux--0.5

Then gets a list of all patches missing from `0.1` and `0.3` :

         % cd ~/wd/linux--0.5

         % larch whats-missing --full linux--0.1 linux--0.3
         archive@kernel.org--primary/linux--0.1--versionfix-4
         archive@kernel.org--primary/linux--0.3--versionfix-1

That list can be piped into the `reconcile` tool:

         % ... | larch reconcile
         archive@kernel.org--primary/linux--0.3--versionfix-1

What happened? `reconcile` figured out that `versionfix-1` from `0.3` already includes `versionfix-4` from `0.1` -- there's no need to apply both patches. So `patch-plan` reported the list of patches that do need to be applied, and in this case, there's only one.

In a more complicated situation, `patch-plan` would print a list of patches in the order they should be applied. In general, it will be a subset of the patches in its input, but applying that subset will have the same effect as applying all of the patches (but hopefully with fewer conflicts).

The developer uses `larch replay --list` to process that list, finally winding up with:

         linux--0.1              linux--0.2
         ----------              ----------
         base-0            ----> base-0 (continuation)
         patch-1          |      patch-1
         patch-2          |      version-0
         patch-3          |    ->versionfix-1 (merge)
         patch-4          |   |     |
         version-0  ------    |     |
         versionfix-1\        |     |
         versionfix-2 |-------      |    linux--0.4
       --versionfix-3 |             |    ----------
      |  versionfix-4/               --->base-0 (continuation)
      |            |                  =->patch-1 (replay --exact merge)
      |             -----            |   version-0
      |                  |           .
      |  linux--0.3      |           |
      |  ---------       |           .
       ->base-0 (continuation)       |
         patch-1         |           .
         patch-2         |           |
         patch-3-=--=--=--=--=--=--=-
         ...             |              linux--0.6
       -version-0        V              ----------
      | versionfix-1 (0.1 update)------>base-0 (continuation)
      |                    |            version-0
      |                    |
      |  linux--0.5        |
      |  ----------        |
       ->base-0            |
         patch-1           |
         patch-2           V
         patch-3 (0.1/0.3 reconciliation)

Now if someone gets the latest revision of `0.5` and asks:

         % larch whats-missing --full linux--0.1 linux--0.3
         [no output]

Isn't `reconcile` handy?

arch: The arch Revision Control System
The Hackerlab at `regexps.com`