Some notes on cross-compiling in a muddled world

Obviously one of the main aims of developing muddle has been to make it easier to build cross-compiled systems, since that's what I spend a lot of my time doing.

Lately I've been taking a DEB-based build and turning it into a source-based build. This article is some notes on things I've learnt (or had reinforced) in the process.

Makefiles

For the purpose of this post, I'm specifically talking about use of GNU make - other make programs may have different requirements (and portability between make programs is a different issue).

So, some tips for making makefiles that will play well with muddle (whether they are Makefile.muddle files, called directly by muddle itself, or new makefiles for a particular project).

Additive assignment

Always use CFLAGS+= and not CFLAGS= in your makefile (and similarly for other variables that may be inherited from "outside", such as LDFLAGS, CPPFLAGS and CXXFLAGS).

Consider as an example if the makefile does not specify -Wall in its C flags, but the user wishes to do so. If the makefile sets:

CFLAGS = -Iinclude -o2

then this is difficult, but if instead the makefile does:

CFLAGS += -Iinclude -o2

then the caller can safely add their own flag(s).

Don't be overspecific

For somewhate similar reasons, never, ever actually set CC and its ilk in the Makefile.

Too often, the developer of a system may need to do the equivalent of:

CC = "gcc -m32"

or even:

LD=$(CC)

Whilst there are ways of overriding CC=xxx settings in the makefile, these are unobvious, and if the setting was amending the value (i.e., if the makefile itself did CC=gcc -m32) then it is bound to go wrong...

Immediate versus deferred

Remember the difference (we're assuming GNU make) between = and :=, and which one will get evaluated multiple times and thus slow things down.

Specifically, if you're going to do:

PKG_OBJDIR := $(shell $(MUDDLE) query objdir package:pkg{role}/built)

then it is worth knowing that := will evaluate this once, and = will cause it to be evaluated each time $(PKG_OBJDIR) is used. Clearly, we assume that a package's objdir is not going to change, and we would rather not run the muddle command line program multiple times if we don't need to.

(Here I recommend reading the GNU make documentation, which can be found at http://www.gnu.org/software/make/manual/make.html. Look up "immediate" versus "deferred" expansion.)

$(MUDDLE), not muddle

Speaking of which, in a muddle-based world, always use $(MUDDLE) in a makefile, instead of an explicit muddle. muddle itself sets this environment variable to the muddle command-line program, which may not actually be called muddle (in my muddle3 development environment, I use m3), and also may not be on the PATH.

(This is similar to the strong recommendation to always say $(MAKE) instead of make, although this latter carries more implications of what happens to your environment than is the case with muddle.)

The evils of TARGET_ARCH

And finally, don't try to use an environment variable called "TARGET_ARCH".

If $(TARGET_ARCH) has a value, then GNU make will insert it onto the $(CC) command line (and maybe other comamnd lines, as well). This is not documented behaviour (it is not mentioned in the above referenced manual), although one can find out about it by searching the net. It looks as if it is expecting this to be some sort of flags for the compilation (so it should be called TARGET_ARCH_FLAGS), but no-one seems to know or remember why it was introduced.

In fact, GNU make treats a lot of environment variables specially, many of which are not documented. You can use:

make -p

to get a listing of these, as well as the actual implicit rules used (for $(CC) command lines, etc.), allowing one to determine where $(TARGET_ARCH) might actually be used.

Random other notes

libicu

Many packages and libraries are difficult to build with cross-compilation - it's not something best served by the autotools framework, and it's not something that most package builders will have needed to worry about.

There are exceptions, however, and one of the nicest may be libicu (libicu4c, from http://site.icu-project.org/). Its documentation clearly explains how to cross-compile it (basically, build it once on the host, and then use that host build as an enabler for building for the target), and the mechanism to do the building works well.

Assembler, C and underscores

Various mechanisms need to know whether external C symbols need an extra underscore (_) prepending for use from assembler. configure can't decide this, because it wants to do it by trying to build and run a program (which won't work for cross compilation!). In the better packages, it just refuses to guess, and tells the user to find out and define a value to let it know. In the less helpful packages, it silently defaults to one choice or the other.

The former choice is a pain, but clearly the correct thing to do.

The latter choice, found, for instance, in libgcrypt11, is unhelpful. In this particular instance, if it can't tell (because it is cross compiling) it defaults to "yes", which was the wrong choice. This took a while to debug.

gcc as linker

It is an advantage to use gcc or g++ as the linker program ($(LD)), because they implicitly know where their libraries are. Some packages insist on using ld instead, which does not.

One package tried to decide a value by parsing the output of ld's "help" output, which failed in various ways.

muddle and pkg-config

Whilst I may grumble about autotools in general, it should be noted that pkg-config does not seem to be too great a problem, as it is quite possible to amend the text files it uses to describe libraries.

muddle does this quite well, but only if you tell it (via the appropriate arguments to make.twolevel() or whatever) that this should be done. In the current version of muddle, this means specifying the rewriteAutoconf and usesAutoconf arguments as True.

libtool grumbles

libtool itself may or may not be so bad, but when it is used without consideration for future cross-compilation, it can be a real problem.

For the moment:

• BAD packages that insist on putting /lib on the library search path explicitly. Bad libtools for not handling this. There may be a fix round the corner, but of course this will never appear in many packages, since they will not be updating their autotools setup...

• The whole libtool and can't install to library ending with problem, which is a bug in design, and I've still not figure out how to fix in any clean way...

  (if you haven't come across this, it's a particular problem for some gstreamer plugins, and seems in the short term to be a show-stopper)