Saving and restoring the state of a muddle build

So over the last week or so I've been working on adding commands to muddle so that we can store and restore "version" information for a build tree.

The basic idea is to be able to save enough information to a file to allow reconstructing the main build structure, and the content of every checkout.

A build description doesn't necessarily suffice for this, firstly because you don't actually have to specify a revision id for a particular checkout (one can default to HEAD), and secondly because the user of a build might have updated checkouts since the build description was first used to extract it.

So, here is some documentation for the result (so far, at least).

The following applies to muddle revision 317 and after. I do not promise that details will not change (but I don't think they will). I also assume there may be bugs.

If you have comments/suggestions, please let me know.

Muddle version stamping

Build names

First of all, it is now possible to give a (hopefully) meaningful name to a build - for instance, one might call the project "Thing" build on Intel 'ProjectThing_Intel'.

This is done by adding a build_name property to the Builder class. What this means in practice is that in the build description you can now do:

builder.build_name = 'ProjectThing_Intel'

The name may only contain alphanumerics, underscores and hyphens.

If no name is given, the name of the build description file (without the ".py" extension) is used - so, for instance, the ProjectThing build might typically default to being called '01'.

Build names are intended to be more generic than the build description file name - after all, one of our projects in particular has progressed from 01.py to 02.py and may one day change again. It would make sense for the build name to stay the same throughout, however.

(Note that since the build name is set in the build description, it need not necessarily be a constant - so one could conceive of automatically naming 'ProjectThing_ARM' and 'ProjectThing_Intel' builds.)

You can find out the current build name with:

muddle query name

Stamp files

A stamp file is a description of a current build. Specifically, it describes the overall build (root repository and description), the names of any domains in the build, and a full description of each checkout used by the build.

For each checkout, all of the information needed to retrieve exactly the version in use (from its remote repository) is remembered.

If any checkouts contain uncommitted data, or do not appear to match an exact revision in their remote repository (in other words, if it appears that it would not be possible to retrieve them as-is from the remote repository), then the stamping process will warn the user, and will create a "partial" stamp file (extension .partial, instead of .stamp).

Note the file is a simple text file (it's actually a Windows-style .INI file, in fact), which means that it is possible to edit it with a text editor, if wished.

Muddle provides several commands for handling stamp files:

• muddle stamp save creates a stamp file.
• muddle stamp diff compare two stamp files. (This could do with providing the ability to compare with the current build, and also with a remote stamp file - these should be fairly easy to add.)
• muddle unstamp creates a build from a stamp file. It understands how to retrieve a stamp file over the internet in various ways, as well as how to use a local file.
• muddle stamp restore is a synonym for muddle unstamp - it's not entirely clear to me if muddle unstamp would be better replaced with this alias, so that all "stamp" related actions are subcommands of muddle stamp.
• muddle stamp version is a specialisation of muddle stamp save. and is discussed below.

Version stamp files

A new conventional muddle directory is introduced, versions/.

This lives at the same level as .muddle/, src/ and domains/, and is covnentionally used to store version stamps.

A version stamp is created with muddle stamp version. It writes a stamp file called <build_name>.stamp to the versions/ directory.

The assumption is that one does muddle stamp version to create a new versions/ directory and put the (nicely named) stamp file there. One can then commit this versions/ directory to whatever repository one wishes, just as is done for the src/builds/ directory.

muddle unstamp then understands how to do:

muddle unstamp  bzr+ssh://kynesim.co.uk/repo  versions/ProjectThing.stamp

to clone the directory versions/ from bzr+ssh://kynesim.co.uk/repo/versions, and then unstamp the stamp file indicated. This is deliberately very similar to the way that muddle init works.

A developer may also, of course, make copies of a version stamp file and commit them - for instance:

cd versions/
cp ProjectThing.stamp ProjectThing-v1_0.stamp

The file must, however, still have extension .stamp.

More information

More information is available by doing:

muddle help stamp
muddle help unstamp

What is tested

I've tested stamping and unstamping with local stamp files.

I've tested version stamping, and then unstamping with file+file:// URLs, both direct to the stamp file and also via the "init-like" mechanism.

I've tested version stamping, committing to a local BZR repository, and then unstamping with a file+bzr:// URL, both direct to the stamp file and via the "init-like" mechanism.

Subversion and git are not tested.

Direct file unstamping is not supported for git (there seems to be no way to "extract" a single file without doing a proper clone). I assume if one wishes to do this one would use a URL provided by a webserver.

With thanks to

The Python library, and particularly to @property, collections.MutableMapping, ConfigParser, difflib, hashlib, urllib, urlparse and all the usual suspects.