Contents
Domains
Domains are part of advanced muddle use, and are probably not relevant to most builds. Regardless, it is probably worth reading at least What domains are for from this section, as useful background, and so that you can recognise when they might be applicable.
The way domains work follows naturally from the rest of muddle, but the implementation is still being tidied up - hence the muddle3_label branch.
What domains are for
Domains allow one to include one build description inside another, in much the same way as a Python module can import another.
A simple example might be when building a system with WebKit and X11 support. We might have two builds, constructed as follows:
def UI_Build: import WebKit_build import X11_build
and:
def Login_build: import X11_build
Alternatively, we might have some high-level software for handling internet television, and want to build two systems on different architectures. In this case, our contrasting builds might be:
def IPTV_stack: import ARM_based_stack
and:
def IPTV_stack: import MIPS_based_stack
How domains work
A build includes another build (a subdomain) using the include_domain() function. This takes the same arguments as the muddle init command (that is, a repository specification and a build description therein), plus a name for the domain.
The following sequence of actions is then performed:
- Create a directory called domains/<name>/, where <name> is the name of the new domain.
- cd into that new directory, and perform (the equivalent of) a muddle init command therein, using the repository specification and build description given.
- Read in this new build, giving a stand-alone build tree datastructure.
- Find all of the labels in the new build tree datastructures, and change them to add the domain name. So, for instance, checkout:fred/checked_out would become checkout:(<name>)fred/checked_out.
- Incorporate this amended build tree datastructure into the original ("top level") build.
- Create a .muddle/am_subdomain file in the domain. This allows muddle to distinguish the actual (final) top-level build from any subdomains within it, which is a useful optimisation.
Note that the build description for a domain does not itself know that it is not the top-level of a muddle build. Indeed, this is an important property of domains - it means that any build description can potentially be included in any other.
For most purposes, the subdomain works just as if it were a "normal" top-level build. In particular, it uses its own .muddle/ directory to record its build state, and writes things to its own obj/ and install/ directories.
Also note that, as a consequence of the way domains work (and this is a good thing), it is only possible for a build to refer to labels in subdomains, not to those in sibling or parent build trees.
To allow for more flexibility in how domains are used, there are tools in the muddled package for manipulating the merged label-space. For instance, if one is using subdomains that originated as two different set-top-box builds, both might provide a linux kernel, and the top-level build probably wants to amend the labels such that both subdomains use one of the kernel packages (broadly, saying "this label should be used instead of that label").
A worked example of using domains
So, we shall start in a new empty directory:
$ cd .. $ mkdir example2 $ cd example2
and start with the same build that we used before:
$ m3 init svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio builds/01.py > Make directory /home/tibs/sw/m3/example2/.muddle Initialised build tree in /home/tibs/sw/m3/example2 Repository: svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio Build description: builds/01.py Checking out build description .. > Make directory /home/tibs/sw/m3/example2/src > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/builds builds A builds/01.py Checked out revision 459. > Make directory /home/tibs/sw/m3/example2/.muddle/tags/checkout/builds Done.
So, just as before, we now have the following directory structure:
. |-- .muddle/ | |-- Description | |-- RootRepository | `-- tags/ | `-- checkout/ | `-- builds/ | `-- checked_out `-- src/ `-- builds/ |-- 01.py `-- 01.pyc
We can now edit the src/builds/01.py file to include a subdomain as part of the build. As described above, we need to specify how to retrieve the domain, giving the same information as for the "muddle init" command. We shall call our domain b (after the example we're taking it from):
from muddled.mechanics import include_domain include_domain(builder, domain_name = "b", domain_repo = "svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b", domain_desc = "builds/01.py")
We also have to say how to include the result into our deployment - a simple way to do this is:
import muddled.deployments.collect as collect collect.deploy(builder, "everything") collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "", domain = None) collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "usr", domain = "b")
and then deploy the new deployment "everything", instead of the old "cpio_dep". Note that we're not deploying to a CPIO archive this time, but just as normal files.
We should probably also change the introductory comment to:
# An example of building with a subdomain
So the "top level" build description is now:
#! /usr/bin/env python # # An example of building with a subdomain import muddled import muddled.pkgs.make import muddled.deployments.cpio import muddled.checkouts.simple import muddled.deployments.collect as collect from muddled.mechanics import include_domain def describe_to(builder): # Checkout .. muddled.checkouts.simple.relative(builder, "cpio_co") muddled.pkgs.make.simple(builder, "pkg_cpio", "x86", "cpio_co") include_domain(builder, domain_name = "b", domain_repo = "svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b", domain_desc = "builds/01.py") collect.deploy(builder, "everything") collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "", domain = None) collect.copy_from_role_install(builder, "everything", role = "x86", rel = "", dest = "usr", domain = "b") builder.invocation.add_default_role("x86") builder.by_default_deploy("everything") # End file.
If we now do a "checkout _all":
$ m3 checkout _all > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle Initialised build tree in /home/tibs/sw/m3/example3/domains/b Repository: svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b Build description: builds/01.py Checking out build description .. > Make directory /home/tibs/sw/m3/example3/domains/b/src > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/b/builds builds A builds/01.py Checked out revision 459. > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle/tags/checkout/builds There is no rule to build label checkout:b_co/checked_out > Building checkout:cpio_co/checked_out > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio/cpio_co cpio_co A cpio_co/hello_world.c A cpio_co/Makefile Checked out revision 459. > Make directory /home/tibs/sw/m3/example3/.muddle/tags/checkout/cpio_co
giving us:
$ ls -AF domains/ .muddle/ src/
or, in more detail (omitting .svn directories):
. |-- domains/ | `-- b/ | |-- .muddle/ | | |-- am_subdomain | | |-- Description | | |-- RootRepository | | `-- tags/ | | `-- checkout/ | | `-- builds/ | | `-- checked_out | `-- src/ | `-- builds/ | |-- 01.py | `-- 01.pyc |-- .muddle/ | |-- Description | |-- RootRepository | `-- tags/ | `-- checkout/ | |-- builds/ | | `-- checked_out | `-- cpio_co/ | `-- checked_out `-- src/ |-- builds/ | |-- 01.py | `-- 01.pyc `-- cpio_co/ |-- hello_world.c `-- Makefile
Apart from the am_subdomain file in its .muddle/ directory, the domains/b/ directory looks like a perfectly normal build.
If we then do:
$ m3 build _all Building package:(b)pkg_b{x86}/postinstalled package:pkg_cpio{x86}/postinstalled > Building checkout:(b)b_co/checked_out > svn checkout http://muddle.googlecode.com/svn/trunk/muddle/examples/b/b_co b_co A b_co/hello_world.c A b_co/Makefile Checked out revision 459. > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle/tags/checkout/b_co > Building package:(b)pkg_b{x86}/preconfig > Make directory /home/tibs/sw/m3/example3/domains/b/obj/pkg_b/x86 > Make directory /home/tibs/sw/m3/example3/domains/b/install/x86 > Make directory /home/tibs/sw/m3/example3/domains/b/.muddle/tags/package/pkg_b > Building package:(b)pkg_b{x86}/configured > make -f Makefile config Nothing to do > Building package:(b)pkg_b{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example3/domains/b/obj/pkg_b/x86/hello_world hello_world.c > Building package:(b)pkg_b{x86}/installed > make -f Makefile install install -m 0755 /home/tibs/sw/m3/example3/domains/b/obj/pkg_b/x86/hello_world /home/tibs/sw/m3/example3/domains/b/install/x86/hello_world > Building package:(b)pkg_b{x86}/postinstalled > Building package:pkg_cpio{x86}/preconfig > Make directory /home/tibs/sw/m3/example3/obj/pkg_cpio/x86 > Make directory /home/tibs/sw/m3/example3/install/x86 > Make directory /home/tibs/sw/m3/example3/.muddle/tags/package/pkg_cpio > Building package:pkg_cpio{x86}/configured > make -f Makefile config Nothing to do > Building package:pkg_cpio{x86}/built > make -f Makefile cc -o /home/tibs/sw/m3/example3/obj/pkg_cpio/x86/hello_world hello_world.c > Building package:pkg_cpio{x86}/installed > make -f Makefile install if [ ! -d /home/tibs/sw/m3/example3/install/x86/bin ]; then mkdir /home/tibs/sw/m3/example3/install/x86/bin; fi install -m 0755 /home/tibs/sw/m3/example3/obj/pkg_cpio/x86/hello_world /home/tibs/sw/m3/example3/install/x86/bin/hello_world install -m 0644 hello_world.c /home/tibs/sw/m3/example3/install/x86/hello_world.c > Building package:pkg_cpio{x86}/postinstalled
we end up with the subdomain built in its directory structure:
domains/ `-- b/ |-- install/ | `-- x86/ | `-- hello_world* |-- .muddle/ | |-- am_subdomain | |-- Description | |-- RootRepository | `-- tags/ | |-- checkout/ | | |-- b_co/ | | | `-- checked_out | | `-- builds/ | | `-- checked_out | `-- package/ | `-- pkg_b/ | |-- x86-built | |-- x86-configured | |-- x86-installed | |-- x86-postinstalled | `-- x86-preconfig |-- obj/ | `-- pkg_b/ | `-- x86/ | `-- hello_world* `-- src/ |-- b_co/ | |-- hello_world.c | `-- Makefile `-- builds/ |-- 01.py `-- 01.pyc
and the top-level build in its:
`-- install/ | `-- x86/ | |-- bin/ | | `-- hello_world* | `-- hello_world.c |-- .muddle/ | |-- Description | |-- RootRepository | `-- tags/ | |-- checkout/ | | |-- builds/ | | | `-- checked_out | | `-- cpio_co/ | | `-- checked_out | `-- package/ | `-- pkg_cpio/ | |-- x86-built | |-- x86-configured | |-- x86-installed | |-- x86-postinstalled | `-- x86-preconfig |-- obj/ | `-- pkg_cpio/ | `-- x86/ | `-- hello_world* `-- src/ |-- builds/ | |-- 01.py | `-- 01.pyc `-- cpio_co/ |-- hello_world.c `-- Makefile
If we ask after packages, we have both sets:
$ m3 query packages pkg_b pkg_cpio
(this should arguably report the domain of each package name as well - there should probably be an option to select this).
If we ask what domains we have, we get:
$ m3 query domains b
(the current printout of domains includes the "empty" or top-level domain in its listing, which is slightly unobvious, and will probably be fixed at some time).
Our dependency rules are now extended to include those from the subdomain as well:
$ m3 depend user-short ----- checkout:builds/changes_committed <-VcsCheckoutBuilder-- [ checkout:builds/up_to_date[T] ] checkout:builds/changes_pushed <-VcsCheckoutBuilder-- [ checkout:builds/changes_committed ] checkout:builds/pulled <-VcsCheckoutBuilder-- [ checkout:builds/checked_out, checkout:builds/up_to_date[T] ] checkout:builds/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:builds/checked_out ] checkout:cpio_co/changes_committed <-VcsCheckoutBuilder-- [ checkout:cpio_co/up_to_date[T] ] checkout:cpio_co/changes_pushed <-VcsCheckoutBuilder-- [ checkout:cpio_co/changes_committed ] checkout:cpio_co/pulled <-VcsCheckoutBuilder-- [ checkout:cpio_co/checked_out, checkout:cpio_co/up_to_date[T] ] checkout:cpio_co/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:cpio_co/checked_out ] checkout:(b)b_co/changes_committed <-VcsCheckoutBuilder-- [ checkout:(b)b_co/up_to_date[T] ] checkout:(b)b_co/changes_pushed <-VcsCheckoutBuilder-- [ checkout:(b)b_co/changes_committed ] checkout:(b)b_co/pulled <-VcsCheckoutBuilder-- [ checkout:(b)b_co/checked_out, checkout:(b)b_co/up_to_date[T] ] checkout:(b)b_co/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:(b)b_co/checked_out ] checkout:(b)builds/changes_committed <-VcsCheckoutBuilder-- [ checkout:(b)builds/up_to_date[T] ] checkout:(b)builds/changes_pushed <-VcsCheckoutBuilder-- [ checkout:(b)builds/changes_committed ] checkout:(b)builds/pulled <-VcsCheckoutBuilder-- [ checkout:(b)builds/checked_out, checkout:(b)builds/up_to_date[T] ] checkout:(b)builds/up_to_date[T] <-VcsCheckoutBuilder-- [ checkout:(b)builds/checked_out ] deployment:everything/deployed <-CollectDeploymentBuilder-- [ package:*{x86}/postinstalled, package:(b)*{x86}/postinstalled ] package:pkg_cpio{x86}/built <-MakeBuilder-- [ package:pkg_cpio{x86}/configured ] package:pkg_cpio{x86}/configured <-MakeBuilder-- [ package:pkg_cpio{x86}/preconfig ] package:pkg_cpio{x86}/installed <-MakeBuilder-- [ package:pkg_cpio{x86}/built ] package:pkg_cpio{x86}/postinstalled <-MakeBuilder-- [ package:pkg_cpio{x86}/installed ] package:pkg_cpio{x86}/preconfig <-MakeBuilder-- [ checkout:cpio_co/checked_out ] package:(b)pkg_b{x86}/built <-MakeBuilder-- [ package:(b)pkg_b{x86}/configured ] package:(b)pkg_b{x86}/configured <-MakeBuilder-- [ package:(b)pkg_b{x86}/preconfig ] package:(b)pkg_b{x86}/installed <-MakeBuilder-- [ package:(b)pkg_b{x86}/built ] package:(b)pkg_b{x86}/postinstalled <-MakeBuilder-- [ package:(b)pkg_b{x86}/installed ] package:(b)pkg_b{x86}/preconfig <-MakeBuilder-- [ checkout:(b)b_co/checked_out ] -----
and our deployment has the expected build order:
$ m3 query deps deployment:everything/deployed Build order for deployment:everything/deployed .. checkout:cpio_co/checked_out checkout:(b)b_co/checked_out package:(b)pkg_b{x86}/preconfig package:pkg_cpio{x86}/preconfig package:(b)pkg_b{x86}/configured package:pkg_cpio{x86}/configured package:(b)pkg_b{x86}/built package:pkg_cpio{x86}/built package:(b)pkg_b{x86}/installed package:pkg_cpio{x86}/installed package:(b)pkg_b{x86}/postinstalled package:pkg_cpio{x86}/postinstalled deployment:everything/deployed
Which means that when we deploy:
$ m3 deploy Building deployment:everything/deployed > Building deployment:everything/deployed > Make directory /home/tibs/sw/m3/example3/deploy/everything Copying /home/tibs/sw/m3/example3/install/x86/ to /home/tibs/sw/m3/example3/deploy/everything/ Copying /home/tibs/sw/m3/example3/domains/b/install/x86/ to /home/tibs/sw/m3/example3/deploy/everything/usr > Make directory /home/tibs/sw/m3/example3/.muddle/tags/deployment/everything
we end up with everything deployed in the deploy/ directory at the top-level:
deploy/ `-- everything/ |-- bin/ | `-- hello_world* |-- hello_world.c `-- usr/ `-- hello_world*
bin/hello_world comes from the top-level build, and usr/hello_world from the subdomain. The hello_world.c is also from the top-level build (the example Makefile for the cpio package copies the source file into the install/ directory).
Version stamps and domains
Version stamps work with domains as well, although one may only generate them for the top-level in a particular build tree.
Thus we can try:
$ m3 stamp version Finding all checkouts... found 4 Processing Svn checkout 'builds' builds: 'svnversion' reports checkout has revision '459M' Processing Svn checkout 'cpio_co' Processing Svn checkout '(b)b_co' Processing Svn checkout '(b)builds' Found domains: set([DomainTuple(name='b', repository='svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b', description='builds/01.py')]) Unable to work out revision ids for all the checkouts - although we did work out 3 of 4 Problems were: * builds: 'svnversion' reports checkout has revision '459M' Problems prevent writing version stamp file
OK, that makes sense, since we had indeed edited that checkout, and had not commited the changes. Luckily, it is sufficient for our purposes to save the state without that edit:
$ m3 stamp save -force Forcing original revision ids when necessary Finding all checkouts... found 4 Processing Svn checkout 'builds' builds: 'svnversion' reports checkout has revision '459M' Processing Svn checkout 'cpio_co' Processing Svn checkout '(b)b_co' Processing Svn checkout '(b)builds' Found domains: set([DomainTuple(name='b', repository='svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b', description='builds/01.py')]) Unable to work out revision ids for all the checkouts - although we did work out 3 of 4 Problems were: * builds: 'svnversion' reports checkout has revision '459M' Writing to working.stamp Wrote revision data to working.stamp File has SHA1 hash 6425284f002c81c9849f2f23add025a710207509 Renaming working.stamp to 6425284f002c81c9849f2f23add025a710207509.partial
and that at least shows us that the stamp file does record details of any subdomains:
[ROOT] description = builds/01.py repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio [DOMAIN b] description = builds/01.py name = b repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b [CHECKOUT (b)b_co] domain = b name = b_co relative = b_co repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b revision = 459 [CHECKOUT (b)builds] domain = b name = builds relative = builds repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/b revision = 459 [CHECKOUT cpio_co] name = cpio_co relative = cpio_co repository = svn+http://muddle.googlecode.com/svn/trunk/muddle/examples/cpio revision = 459 [PROBLEMS] problem1 = builds: 'svnversion' reports checkout has revision '459M'
No comments:
Post a Comment