...and muddle continues to evolve

muddle v2.3.1 fixes a couple of bugs, and beyond that, head-of-tree adds "muddle --version", which uses git to report a sensible version string, and also the directory that muddle is being run from. For instance:

$ muddle --version
muddle v2.3.1-3-g34519ca in /home/tibs/sw/muddle


(That's the last tag (v2.3.1), the number of commits since then (3) and the start of the SHA1 string for this commit (34519ca))

KBUS on Google Code now using Git

The KBUS repositories on http://code.google.com/p/kbus/ have now been moved from Mercurial to Git.

The "default" (userspace) and kernel repositories are now recombined. The changes from the kernel submission work on github have also been folded in.

See http://code.google.com/p/kbus/wiki/DevelopmentHistory for more information.

Version 2.3 of muddle is released

I took a branch of muddle out in September 2011 to write tests for domain support, which had "bit rotted" through disuse. As it turned out, my need for domain support went away again, and instead I started to use the branch for rewriting the help texts for each command. Of course, in order to do that, I had to understand exactly what each command did, which led to rewriting the main command structure so that this was consistent, and hopefully also predictable, and thus easier to document. So it all took rather longer than planned.
Meanwhile, another project recently necessitated adding support for git repositories on Google Code, which in turn required rewriting the internal repository and checkout support. So that also got folded in.
This version of muddle is now stable enough to merge back into the mainline. I've been using it for real development work throughout its development, and there are more tests than there used to be (although still not enough subdomain tests!). However, clearly, if you come across a bug or infelicity that I've introduced, do let me know (preferably by raising an issue at http://code.google.com/p/muddle/issues/list, but otherwise by email to tibs@kynesim.co.uk).
Since there's a large amount of sudden change, much of it to significant internal structure, I'm giving this new version of muddle a version tag, and calling it version 2.3 (SVN revision 638 (back when muddle was stored in subversion) had tag 2_2).

(The tag "end_of_v2.2" marks the end of version 2.2, and there is also a branch "v2.2-legacy" from the same location, which can be conveniently used to checkout the end of the 2.2 sequence, and also provides a place for any maintenance changes if such prove necessary.)

The main changes are:

• Much of the the internals of the command line processing has been rewritten, and this should give more robust command handling. It also means that commands of a common "category" (see 'muddle help categories') share much more common code, and thus have better consistency.
• Much of the help text has been reworked, and there are new topics:
  • 'muddle help categories'
  • 'muddle help labels'
  • 'muddle help subdomains'
  Also, help text is now paged (much as is done by git), using the pager specified by $PAGER, or else 'more'. There is a command line switch, '--nopager', to rever to the old behaviour.
• An attempt has been made to make the "bare" muddle command (i.e., 'muddle' with no arguments) follow the DWIM (Do What I Mean) principles described in the documentation. See 'muddle help labels' for more information on how this works.
• Similarly, muddle checkout/package/dependency commands with no label arguments (e.g., 'muddle build') now try to decide consistently what to do. Again, see 'muddle help labels'.
• All checkout/package/dependency commands now know how to cope with any type of label. See 'muddle help labels' for details.
• Most commands that take labels as arguments can now take partial labels (label fragments) instead, which is a lot simpler for a human being. This includes the 'visdep' script in the sandbox. See 'muddle help labels' for more information.
• 'muddle -n' now works for all commands that do something (it used to be very unpredictable about this). It is now a reliable tool, and may be used as such. Also, note that it reports a single label per line, which makes it much easier to read.
• As well as the special '_all' argument, there are now also '_default_deployments' and '_default_roles', which expand to all the default deployment labels and all the package labels in the default roles, respectively. Thus the top-level "bare" 'muddle' command is identical to 'muddle buildlabel _default_deployments _default_roles'.
  However, note that '_all' is now treated as a special value for use on the commandline, and not as a "pseudo-wildcard" label name. Thus it is no longer meaningful to ask for package:_all{x86} when one meant package:*{x86} (the former will report that there is no package called _all{x86}).
• What 'muddle where' says has changed in detail, and there is now 'muddle where -detail' for use in scripts.
• When building things, or "killing" things, muddle is significantly less verbose - it doesn't normally report ALL the labels it will affect.
• More and hopefully more useful error messages have been introduced. In particular, I've tried to catch occasions where muddle unexpectedly does nothing, and suggest why this might be (liking trying to do 'muddle build' in a checkout directory when the only role for the corresponding package is not one of the default roles - this one has taken me too long to work out in the past). Some of these error reports are more verbose, or not as easy to understand, as they should be, but this is hoped to be an important improvement.
• When a syntax or other error is found in a build description, it is no longer buried in a traceback concerned with how the build description was being imported.
• muddle commands now try harder to catch cases where the user has specified a role for a checkout or deployment label, and grumble.
• Various command aliases have been removed:
  • 'muddle query deps' - use 'needed-by'
  • 'muddle query instructions' - use 'inst-files'
  • 'muddle query makeenv' - use 'make-env'
  • 'muddle query preciseenv' - use 'precise-env'
  • 'muddle query results' - use 'needs'
  Note that 'muddle query envs' is still an alias for 'muddle query all-envs', and this is still different than 'muddle query env'.
  Also, as it turned out, 'muddle uncheckout' and 'muddle removed' did exactly the same thing. 'muddle removed' has thus been removed.
• Some commands have been changed:
  • 'muddle dependencies' is replaced by 'muddle query dependencies'
  • 'muddle depends' is replaced by 'muddle query depends' (this is actually the same as the previous item)
  • 'muddle vcs' is replaced by 'muddle query vcs'
  • 'muddle root' has gone. Instead use 'muddle query root', which already existed. 'muddle query root' now just reports the root (of the build tree), and not the "default domain", since this concept has essentially gone away.
  • 'muddle query default-labels' is now the more accurate 'muddle query default-deployments'
• 'muddle query package-roles' will also include domains in the reported names, if appropriate.
• 'muddle pull' is once again preferred over 'muddle fetch' or 'muddle update', but the other variants are still there.
• The 'utils.Directory' classes now set $PWD (as top-level muddle itself does, and for the same reason). They also now have a useful 'join' method (see the classes for its use). A version of these classes has been pulled out to a separate repository on github - see https://github.com/tibs/withdir
• There is now more testing! (particularly of the 'muddle -n' commands, and of subdomain handling - necessary because the latter had "bit rotted").
• Various obscure bugs have been found and fixed by the new tests. Some of them were really quite obscure.
• Various things that were to be deprecated have now been removed. Hopefully no-one will notice.
• 'NullPackage' and 'null_package()' have been added to muddled/pkg.py. Their docstrings explain why they might be useful.
• 'Rule' classes now have an 'action' member, instead of an 'obj' member. I believe this to be much clearer, but it is an unannounced incompatible change.
• Repository handling has been rewritten, separating out the concerns of naming a (remote) repository and naming (the location of) a checkout.
  In particular, and importantly, it now understands how to use git repositories at google code, where the repository name is appended to the "main" URL with a dot, rather than a "/".
  The old commands for setting things up (checkouts.twolevel and so on) are still there, but one can now do:
  

  from muddled.repository import Repository
  from muddled.version_control import checkout_from_repo
  
  repo = Repository('git', 'https://example.com', 'useful-stuff',
                    branch='very-useful')
  checkout_from_repo(builder, 'useful-stuff', repo)
  

  althogh there's actually more to it than that - in particular, one can specify a different <co_dir> and <co_leaf> to checkout_from_repo, giving one full control over where the repository is actually checked out:
  

  checkout_from_repo(builder, 'useful-stuff', repo, co_dir='jim',
                     co_leaf='fred')
  

  checks the repository out into 'src/jim/fred', but still using the label 'checkout:useful-stuff/checked_out'. I'd claim this is significantly easier to understand than the twoleve/multilevel functions.
  See the documentation for Repository via "muddle doc repository.Repository", or in its source file.
  There is now thus a new command "muddle query checkout-repos" to report on the repositories. See its help for details.
  This new mechanism is believed to understand muddle stamp files generated by previous versions of muddle, and to be able to produce muddle stamp files that previous versions of muddle should understand.
• Internal changes that may be of interest:
  • 'builder.invocation.default_labels' is now 'builder.invocation.default_deployment_labels'.
  • Similarly, the method 'add_default_label()' becomes 'add_default_deployment_label()'. It also now checks that the label is a deployment label.

Also, of course, many minor code tidies have been made.

Particular issues fixed

• Fixes issue 4. Bugs in description files are not handled elegantly
• Fixes issue 67. Checkout and package inference should know about domains
• Fixes issue 104. No feedback on undefined label name
• Fixes issue 112. Python 2.7 (I'm now using that for testing as well)
• Fixes issue 126. Provide coherent branch support in the VCS plugins
• Fixes issue 166. test_checkouts.py broken by fix for issue 161
• Fixes issue 175. Repository name should be independent of local checkout name
• Fixes issue 177. 'muddle checkout package:xxx' should checkout what is needed for package xxx
• Fixes issue 179. revisit the implementation of the simple/twolevel/multilevel checkout code
• Fixes issue 184. All the muddle commands that take labels should support domains
• Fixes issue 185. Muddle unstamp should check its build against domain/checkout name, not just checkouts name
• Fixes issue 186. Deprecate use of 'all_packages', use 'all_package_labels' instead
• Fixes issue 189. Make sure all commands do something vaguely sensible for '-n'
• Fixes issue 194. 'muddle pull' of a toplevel checkout in a subdomain attempts to clone
• Fixes issue 195. If there's nothing to do, say so
• Fixes issue 198. Muddle should actually honour the DWIM section from the README
• Fixes issue 199. Helpful error behaviour of fetch/pull unexpected (well, ok, this is in master as well)
• Fixes issue 200. Find all targets for deployments
• Fixes issue 207. Muddle assumes too much about how RootRepository and Description are joined

Updating to the new version

...is simple, just cd into your muddle source directory and git pull in the normal manner.

If you find some serious bug and need to go back to version 2.2, then cd into the muddle source directory and do git checkout v2.2-legacy (and, obviously, report the problem).