WITH[OUT]_BMAKE

While the original goal was to achieve cutover within a release cycle, this proved impractical, so a BMAKE option was added to select which make should be installed as /usr/bin/make. The knob defaults to "yes".

The top-level makefiles, have to pay attention to WITH[OUT]_BMAKE and ensure that the correct make is used.

This is a short term thing, since to keep WITHOUT_BMAKE working, requires anyone making changes to makefiles (at least src/share/mk/* and src/Makefile*) to test both ways.

job tokens

When you do make -j8, the goal is that at most 8 jobs will run in parallel. To achieve this most modern make's use a shared token pool, each sub-make waits for a token before running a job.

In fmake the token pool is managed as a named FIFO. If fmake does not find the appropriate variable in its environment, it creates and initializes the FIFO and exports its name.

In bmake the token pool is a pipe, with the open descriptors passed to sub-makes only for targets that are tagged with .MAKE. Sub-makes know that they are such due to the -J in,out arg they get. If the descriptors named are invalid, bmake behaves as for -B (compat mode).

Both designs have their benefits. The difference though can lead to problems when mixing the two.

If the first make instance is fmake, it creates its FIFO and passes -jN to sub-makes. If these are bmake the lack of -J causes them to create their own token pool - oversubscription.

Similarly if the first make is bmake and sub-makes are fmake problems can arise since bmake does not export the name of FIFO, so each sub-make will create its own. This case is less likely to happen accidentally, since -J will cause an error for fmake and thus needs to be explicitly filtered. Which implies the mixing is deliberate. An example of this is in src/Makefile when building WITHOUT_BMAKE.

.MAKE     Execute the commands associated with this target even if the -n
          or -t options were specified.  Normally used to mark recursive
          bmake's.

MAKE_JOB_ERROR_TOKEN

Normally on failure during a jobs-mode build, bmake puts an error token into the job token pool. This causes the build to be abandoned immediately, which is normally exactly what is desired.

For make universe however it is desired that each branch of the build that can continue, do so. This is facilitated by the following in src/Makefile:

.if make(universe)
# we do not want a failure of one branch abort all.
MAKE_JOB_ERROR_TOKEN= no
.export MAKE_JOB_ERROR_TOKEN
.endif

errCheck

Fmake runs target scripts with set -e in effect. This means that the script fails if any statement in a multi-statement command line fails. This means that the command:

cd /no/such/dir; rm -rf *

will not do any harm.

By contrast, bmake runs scripts such that the command line rather than individual statements is the unit of failure. This means that the above command should be written as:

cd /no/such/dir && rm -rf *

while fixing all the makefiles in base is feasible, fixing ports is another matter, and then there is the question of makefiles written by FreeBSD users.

The practical solution is to provide a .SHELL description that replicates the fmake behavior:

# By default bmake does *not* use set -e
# when running target scripts, this is a problem for many makefiles here.
# So define a shell that will do what FreeBSD expects.
.ifndef WITHOUT_SHELL_ERRCTL
.SHELL: name=sh \
        quiet="set -" echo="set -v" filter="set -" \
        hasErrCtl=yes check="set -e" ignore="set +e" \
        echoFlag=v errFlag=e \
        path=${__MAKE_SHELL:U/bin/sh}
.endif

Quoted strings in .for loops

The following:

OPTIONS= FOO "Description of FOO" FOO_DEFAULT
.for i in ${OPTIONS}

is expected to iterate 3 times. Quoted strings isn't something NetBSD's make (hence bmake) supported. This has been added. Using bmake the above could be coded as:

.for opt description default in ${OPTIONS}

allowing a much easier to follow makefile.

Bootstrapping bmake for old versions of FreeBSD

A bigger issue for ports though is that it isn't branched and must remain buildable on the oldest supported release. Thus it is important to not use enhancements such as multiple iterators in .for loops until the oldest supported release uses bmake.

The reason being that in order to build ports on an old release, which neither has bmake nor a make which has the compatibility modifiers (:tu etc), we need ports to be able to build and install bmake for itself automatically.

This can be done with the following in Mk/bsd.ports.mk:

.if !defined(.PARSEDIR) && ${.MAKEFLAGS:M-V} == ""
.MAIN: all
# We are not bmake
# bmake-sh will invoke bmake (installing it if needed).
all ${.TARGETS}: use-bmake
.if !target(use-bmake)
use-bmake:
        +sh ${PORTSDIR}/Mk/bmake-sh ${.MAKEFLAGS} ${.TARGETS}
.endif
.else                           # Not bmake

The script Mk/bmake-sh will simply invoke bmake if it exists, and otherwise will build/install it - using a copy of Mk/*mk converted back to the old make.

I won't bore you with the details, since this approach was not adopted.

Hack to allow old modifiers

Turns out that the way portmgr build ports cannot accommodate the approach outlined above. So a knob was added to allow bsd.port.mk to:

# tell bmake we use the old :L :U modifiers
.MAKE.FreeBSD_UL= yes

support for this will removed once 8.3 is EOL and portmgr convert ports to use :tl and :tu.

With this as part of a simple patch to bsd.port.mk, portmgr were able to get ports building with bmake, and thus close this issue.

Portmgr were very helpful during this process.

Default -V behavior

Ports uses make -V FOO extensively. The old FreeBSD make outputs a fully expanded value for FOO so:

FOO_VAR!= ${MAKE} -C foo -V VAR

is commonly used.

However, with the above, bmake will show its literal value, which might be ${DESTDIR}foo - not what is desired.

To ensure full expansion bmake -V '${FOO}' is used. In a makefile that typically needs to be coded as:

FOO_VAR!= ${MAKE} -C foo -V '$${VAR}'

But those doubled $$ get complex fast when commands get put in other variables. Thus NetBSD pkgsrc uses:

FOO_VAR!= ${MAKE} -C foo show-var VARIABLE=VAR

to avoid complexities when such things need to be used indirectly. Using this model, will work equally well regardless of make version, but only handles a single variable.

$ bmake -V FOO_DIR
/opt/bin/foo
$ bmake -dV -V FOO_DIR
${DESTDIR}/opt/bin/foo

$ FOO='${DESTDIR}${.CURDIR:H}' bmake -V FOO .MAKE.EXPAND_VARIABLES=yes
/usr/src/usr.bin
$ FOO='${DESTDIR}${.CURDIR:H}' bmake -V FOO .MAKE.EXPAND_VARIABLES=no
${DESTDIR}${.CURDIR:H}
$ FOO='${DESTDIR}${.CURDIR:H}' bmake -dV -V FOO .MAKE.EXPAND_VARIABLES=yes
${DESTDIR}${.CURDIR:H}

.for loop iterators

With fmake bsd.port.mk used a rather complex dance involving nested .for loops with heavily escaped iterator variables to handle building the MLINKS list for multiple languages.

In bmake loop iterators are substituted as ${:Uvalue} which avoids lots of issues but won't work with the above. Fortunately a one line nested inline-loop does the job:

.if defined(.PARSEDIR)
# inline loops are simpler
_MLINKS=    ${_MLINKS_PREPEND} \
        ${MANLANG:S,^,man/,:S,/"",,:@m@${MLINKS:@p@${MAN${p:E}PREFIX}/$m/man${p:E}/$p${MANEXT}@}@}

.else

There are two loops there, one with iterator m for ${MANLANG} resulting in values like man man/fr etc., and another p for ${MLINKS}, thus for:

MANLANG= "" fr
MLINKS= foo.1 goo.2

we get:

/usr/local/man/man1/foo.1
/usr/local/man/man2/goo.2
/usr/local/man/fr/man1/foo.1
/usr/local/man/fr/man2/goo.2

exactly the same as the 20 or so original lines produce.