Yeah unfortunately Automake doesn't (yet) have a way to ask `SUBDIRS` to be built in parallel. It would indeed work for GP's top-level Makefile, but unfortunately there's no easy way to do that. Non-recursiveness is also (slightly) more efficient than recursiveness just because it wastes less time spawning processes, analyzing files mdates and reading Makefiles, but that's probably less of a concern -- although on my machine (no SSD) running `make` on GP when there's nothing to do still takes around 3.5s (parallelized or not).
All this said, when I was writing the initial comment on SUBDIRS not being parallelizable, I had an idea for a hack to work around this. It's not really beautiful, but it does the job: ```diff $ git diff diff --git a/Makefile.am b/Makefile.am index 8cf1634..148c67e 100644 --- a/Makefile.am +++ b/Makefile.am @@ -176,3 +176,10 @@ EXTRA_DIST = \ wscript \ README.waf \ README.windows + + +all-recursive: $(SUBDIRS) +.PHONY: $(SUBDIRS) + +$(SUBDIRS): + $(am__cd) $@ && $(MAKE) $(AM_MAKEFLAGS) all ```
What it does is recursing in each `$(SUBDIRS)` first, before making all-recursive. This obviously isn't prefect, as it is effectively slower when there's nothing to do, as it'll recurse in each `$(SUBDIRS)` twice; but it will parallelize the first run (as per `-j` option), so the second one won't do anything, meaning the actual build will be parallelized. On my machine (2 physical, 4 logical cores), such an empty run takes around 7s when unparallelizd, and less than 5s when parallelized (`-j4`). OTOH, a full build with `-j4` goes from around 68s to around 53s (-15s, 22%). FTR, with CCache (so the actual build time is a LOT smaller), it goes from around 24s to around 16s (-8s, 33%).
This can also be done in a slightly more hackish way outside of Automake's Makefile, meaning it doesn't change how an Automake's Makefile run behaves. Using another Makefile (I named it *makefile.parallel* locally), like that: ```make #!/usr/bin/make -f # # Parallelizes the top-level Makefile of a SUBDIR-based Automake build system. # # What it actually does is recursing in $(builddir) for each target, and in # each $(SUBDIRS) for target 'all'. 'all' has a dependency on $(SUBDIRS) so # they are recursed in first, respecting `-j` option as any target. # # Note that this means that each $(SUBDIRS) is first built in parallel, and # then again through the top-level Makefile for target 'all'. This is normally # not so much of a problem, as the targets should be up-to-date the second # time, so you only get the Make overhead but no actual building. # This is needed to "recurse" in '.' in case it's useful. # # As this makes uses of normal Make features, it is very possible to force some # ordering by declaring explicit dependencies between some elements of # $(SUBDIRS); for example if sub-directory 'po' depends on recursing in another # one first, i.e. 'src', declare '$(builddir)/po: $(builddir)/src' (not # forgetting '$(builddir)/' prefix), so 'src' is build before 'po'. # # Only 'all' is parallelized. First, this is because it's likely to be the # only target that really makes sense to parallelize, as it's the one building # real things. Also, as the parallelized targets are run twice (see above), # a second run should have low overhead, so it doesn't make sense to e.g. # parallelize i.e.g 'check' the same way, as it would run the checks twice -- # although parallelization would slightly help when building test programs. # This could be addressed by omitting recursion in '.', but this can be a # problem if './Makefile' contains anything beside recursion. # Additionally, there's a technical difficulty in knowing which target was # actually asked for, as each $(SUBDIRS) is a target itself, so $@ can't be # relied on. # # Limitations: # * Inspecting the value of Automake's SUBDIRS variable is naive, and doesn't # support expansions and the like. This means that if $(SUBDIRS) from # Makefile.am contains variable references it'll most likely will break.
# user can set either builddir (when running from the source directory) or # srcdir (when running from the build directory). If builddir==srcdir, no # overrides are needed. builddir ?= . srcdir ?= .
upper_level = $(srcdir)/Makefile.am
SED ?= sed
# extract SUBDIRS SUBDIRS != $(SED) -ne '/SUBDIRS/s%^.*= (.*$)%$(builddir)/\1%p' $(upper_level)
# MAKEFLAGS minus builddir= and srcdir= filtered_makeflags = `echo $(MAKEFLAGS) | \ $(SED) -e 's/(build|src)dir=[^ ]*//g'`
# all targets we want to forward somehow TARGETS := all check clean distclean dist distcheck
all: $(SUBDIRS)
$(TARGETS): $(MAKE) -C $(builddir) $(filtered_makeflags) $@
$(SUBDIRS): $(MAKE) -C $@ $(filtered_makeflags)
.PHONY: $(SUBDIRS) $(TARGETS) ```
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany-plugins/pull/277#issuecomment-161035919
github-comments@lists.geany.org