Wednesday 29 September 2010

muddle and its directories - part 5 - adding and stamps

Adding a new checkout/package

Let's suppose we want to add a new package, based on a new checkout. There is more than one way to do this, but the following shows a sensible approach.

We shall create our new package by just copying one we've already got:

$ cp -a src/cpio_co src/fred

We must remember to remove the outdated repository information:

$ rm -rf src/fred/,svn/

And make this checkout do something different:

$ mv src/fred/hello_world.c src/fred/bye_world.c
$ sed -e s/hello_world/bye_world/g --in-place src/fred/Makefile
$ sed -e s/Hello/Byebye/g --in-place src/fred/bye_world.c

muddle has no knowledge of this checkout yet, so we need to add it to the build description. We can simply edit src/builds/01.py to add the lines:

muddled.checkouts.simple.relative(builder, "fred")

and:

muddled.pkgs.make.simple(builder, "fred", "x86", "fred")

(yes, that leaves us with a package and checkout with the same name, but that's not a problem, and is closer to the more conventional usage).

That's enough to give us:

$ m3 query checkouts
builds
cpio_co
fred

and:

$ m3 query deps deployment:cpio_dep/deployed
Build order for deployment:cpio_dep/deployed ..
checkout:fred/checked_out
checkout:cpio_co/checked_out
package:pkg_cpio{x86}/preconfig
package:fred{x86}/preconfig
package:pkg_cpio{x86}/configured
package:fred{x86}/configured
package:fred{x86}/built
package:pkg_cpio{x86}/built
package:fred{x86}/installed
package:pkg_cpio{x86}/installed
package:pkg_cpio{x86}/postinstalled
deployment:cpio_dep/deployed

However, if we were to try to build:

$ m3
Building deployment:cpio_dep/deployed
> Building checkout:fred/checked_out
> svn checkout  http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/fred fred
svn: URL 'http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/fred' doesn't exist
Can't build deployment:cpio_dep/deployed - Command 'svn checkout  http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/fred fred' execution failed - 1

This makes sense, because (a) we have not told muddle that the new checkout is present in its directory structure, and (b) we have not actually checked it in to the subversion repository it wants to look in.

Since we've not yet finished developing this new package, we don't want to check it in to the repository yet (and, in particular in our case, we do not want to permanently add it to our example).

So we need to make muddle believe that it has been checked out.

As you might expect, there are two ways to do this. The one that you may guess is to manipulate the .muddle/` directory structure directly - so we could just do:

$ mkdir .muddle/tags/checkout/fred
$ touch .muddle/tags/checkout/fred/checked_out

However, that's a bit clumsy to type, so muddle provides a convenient command for asserting a particular tag (just assuming you are comfortable with the label syntax):

$ m3 assert checkout:fred/checked_out
> Make directory /home/tibs/sw/m3/example3/.muddle/tags/checkout/fred

and now the tag file is present:

$ ls .muddle/tags/checkout/fred
checked_out

and we can then do:

$ m3
Building deployment:cpio_dep/deployed
> Building package:fred{x86}/preconfig
> Make directory /home/tibs/sw/m3/example3/obj/fred/x86
> Make directory /home/tibs/sw/m3/example3/.muddle/tags/package/fred
> Building package:fred{x86}/configured
> make  -f Makefile config
Nothing to do
> Building package:fred{x86}/built
> make  -f Makefile
cc -o /home/tibs/sw/m3/example3/obj/fred/x86/bye_world bye_world.c
> Building package:fred{x86}/installed
> make  -f Makefile install
if [ ! -d /home/tibs/sw/m3/example3/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example3/install/x86/bin; fi
install -m 0755 /home/tibs/sw/m3/example3/obj/fred/x86/bye_world /home/tibs/sw/m3/example3/install/x86/bin/bye_world
install -m 0644 bye_world.c /home/tibs/sw/m3/example3/install/x86/bye_world.c
> Building deployment:cpio_dep/deployed
> Make directory /home/tibs/sw/m3/example3/deploy/cpio_dep
Collecting package:*{x86}/*  for deployment to / ..
h = ---Roots---
/ -> [ / (fs /home/tibs/sw/m3/example3/install/x86) mode = 40755 uid = 7007 gid = 7007 kids = /bin /hello_world.c /bye_world.c]
---Map---
/bin/bye_world -> [ /bin/bye_world (fs /home/tibs/sw/m3/example3/install/x86/bin/bye_world) mode = 100755 uid = 7007 gid = 7007 kids = ]
/bin -> [ /bin (fs /home/tibs/sw/m3/example3/install/x86/bin) mode = 40755 uid = 7007 gid = 7007 kids = /bin/bye_world /bin/hello_world]
/hello_world.c -> [ /hello_world.c (fs /home/tibs/sw/m3/example3/install/x86/hello_world.c) mode = 100644 uid = 7007 gid = 7007 kids = ]
/bin/hello_world -> [ /bin/hello_world (fs /home/tibs/sw/m3/example3/install/x86/bin/hello_world) mode = 100755 uid = 7007 gid = 7007 kids = ]
/bye_world.c -> [ /bye_world.c (fs /home/tibs/sw/m3/example3/install/x86/bye_world.c) mode = 100644 uid = 7007 gid = 7007 kids = ]
/ -> [ / (fs /home/tibs/sw/m3/example3/install/x86) mode = 40755 uid = 7007 gid = 7007 kids = /bin /hello_world.c /bye_world.c]
---

base = /
Scanning instructions for role x86, domain None ..
> Writing /home/tibs/sw/m3/example3/deploy/cpio_dep/my_archive.cpio ..
> Packing / ..
> Packing /bin ..
> Packing /bin/bye_world ..
> Packing /bin/hello_world ..
> Packing /hello_world.c ..
> Packing /bye_world.c ..
> Packing TRAILER!!! ..
> Make directory /home/tibs/sw/m3/example3/.muddle/tags/deployment/cpio_dep

This just leaves actually adding the new package to revision control somewhere - in this case, the build description is saying (implicitly) that this package is stored in the same subversion repository as the rest of the example, and so one would need to add it there by the normal means (which is a topic for another time).

(The fact that this is much easier to do for distributed revision control systems like bazaar, mercurial or git is in itself a good reason for using them!)

Version stamps

Version stamps allow one to produce a simple text file (actually, an INI file) representing the current state of a build. This can be useful for various purposes, but its main intent is to allow saving a build state so that it can be accurately recreated at a later date (for instance, so one can rebuild an earlier release to customers).

The muddle stamp and unstamp commands have documentation via muddle help stamp and help unstamp.

However, it is worth giving a quick example of creating a version stamp:

$ m3 stamp version
Finding all checkouts... found 2
Processing Svn checkout 'builds'
Processing Svn checkout 'cpio_co'
Creating directory /home/tibs/sw/m3/example/versions
Writing to /home/tibs/sw/m3/example/versions/_temporary.stamp
Wrote revision data to /home/tibs/sw/m3/example/versions/_temporary.stamp
File has SHA1 hash 189085d413217cb1865868b5d9916085d22e6f50
Renaming /home/tibs/sw/m3/example/versions/_temporary.stamp to /home/tibs/sw/m3/example/versions/01.stamp

As indicated above, we now have a new versions/ directory:

$ ls -AFt
versions/  deploy/  install/  obj/  src/  .muddle/

containing the stamp file:

versions
`-- 01.stamp

Modern build descriptions are encouraged to specify a build name (this is as simple as adding a line of the form:

builder.build_name = 'ExampleBuild'

to the Python code). If this is given, then the version stamp file will use that build name as its filename, otherwise it defaults to the filename of the main build description file (01 in this case).

The versions/ directory is suitable for putting into a version control system, as one might expect, and future versions of muddle are likely to provide more support for handling this (see issue 117 for some ideas on this).

The contents of a stamp file is deliberately human readable:

[ROOT]
description = builds/01.py
repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio

[CHECKOUT builds]
name = builds
relative = builds
repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio
revision = 458

[CHECKOUT cpio_co]
name = cpio_co
relative = cpio_co
repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio
revision = 458

As normal. muddle will not stop you writing or editing stamp files. This may conceivably be useful.

No comments:

Post a Comment