muddle - CPIO support in packages

There is a new version of muddle, which addresses issue 201: Allow CPIO file creation by a package, not just by a deployment, and also has some internal amendments which should not be visible.



This new version of muddle is tagged package-cpio-support. It is available on both Google Code and github, in the normal manner (just "git pull").

The traditional way of creating a CPIO archive with muddle is via a deployment, something like:

          import muddled.deployments.cpio as cpio

        fw = cpio.create(builder, 'firmware.cpio', 'deployment-name')
        fw.copy_from_role(role1, '', '/')
        fw.copy_from_role(role2, 'bin', '/bin')
        fw.done()

which creates a file called deploy/deployment-name/firmware.cpio.

This is well and good, but sometimes a build wants the CPIO archive to use in a package. The most obvious example is when one builds a kernel and wants to attach a CPIO archive (or worse, one of several CPIO archives) to it as an initrd filesystem.

In the previous versions of muddle, the only way to do this was to make the kernel-building package depend upon a deployment which built the CPIO archive. This doesn't work very well, as muddle was never designed to support packages depending on deployments (see issue 235). Dependency tracking "through" the deployment doesn't work as one might hope, so doing a proper rebuild of the kernel-building package can be quite difficult.

One solution would be to fix muddle to handle dependency tracking through deployments, but that's non-trivial, not least because one would need to work out what it means in the general case.

Luckily, it turns out to be simple to enhance the CPIO creation mechanism to support packages.

The second argument of cpio.create is amended so that it can take a deployment name (as before), or a deployment or package label. If it is given a deployment label, it acts just as before, but if it is given a package label:

          from muddled.depend import package

        fw = cpio.create(builder, 'firmware.cpio', package('firmware', 'firmware-role'))
        fw.copy_from_role(role1, '', '/')
        fw.copy_from_role(role2, 'bin', '/bin')
        fw.done()

then it now creates the CPIO file as install/firmware-role/firmware.cpio.

Note that it is still possible to specify a subdirectory in the CPIO file name, so 'fred/firmware.cpio' would create install/firmware-role/fred/firmware.cpio.

The "virtual" package package:firmware{firmware-role}/* is created automatically, and depends on all the packages in roles role1 and role2, just as was the case when using a deployment. This does, however, mean that the firmware package cannot be in the same role as any of the packages it depends on, or we would end up with a circular dependency, which is not allowed. I think that I would recommend having a specific role for CPIO file generation, and if multiple CPIO files are to be constructed, disambiguate them via name and subdirectory (using subdirectories to split then will work well if one is trying to deploy them via a collecting deployment).

The other change is a restructuring of the internals of checkout/VCS handling within muddle. This should be entirely transparent to the normal user (although anyone using upstream repositories should keep a careful eye on their use, as that's the part of the code I'm least confident of). It's been done in aid of the lifecycle-branching work I'm doing in a development branch, but I believe is a general improvement in the code as well.