BTW, the unparallelized case can be optimized by only recursing first if Make was passed a `-j` option:
```diff
diff --git a/Makefile.am b/Makefile.am
index 8cf1634..9023c10 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -176,3 +176,12 @@ EXTRA_DIST = \
wscript \
README.waf \
README.windows
+
+
+all-recursive: $(SUBDIRS)
+.PHONY: $(SUBDIRS)
+
+$(SUBDIRS):
+ @if (target_option=j; $(am__make_running_with_option)); then \
+ $(am__cd) $@ && $(MAKE) $(AM_MAKEFLAGS) all; \
+ fi
```
---
Reply to this email directly or view it on GitHub:
https://github.com/geany/geany-plugins/pull/277#issuecomment-161038082
Yeah unfortunately Automake doesn't (yet) have a way to ask `SUBDIRS` to be built in parallel. It would indeed work for GP's top-level Makefile, but unfortunately there's no easy way to do that.
Non-recursiveness is also (slightly) more efficient than recursiveness just because it wastes less time spawning processes, analyzing files mdates and reading Makefiles, but that's probably less of a concern -- although on my machine (no SSD) running `make` on GP when there's nothing to do still takes around 3.5s (parallelized or not).
All this said, when I was writing the initial comment on SUBDIRS not being parallelizable, I had an idea for a hack to work around this. It's not really beautiful, but it does the job:
```diff
$ git diff
diff --git a/Makefile.am b/Makefile.am
index 8cf1634..148c67e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -176,3 +176,10 @@ EXTRA_DIST = \
wscript \
README.waf \
README.windows
+
+
+all-recursive: $(SUBDIRS)
+.PHONY: $(SUBDIRS)
+
+$(SUBDIRS):
+ $(am__cd) $@ && $(MAKE) $(AM_MAKEFLAGS) all
```
What it does is recursing in each `$(SUBDIRS)` first, before making all-recursive. This obviously isn't prefect, as it is effectively slower when there's nothing to do, as it'll recurse in each `$(SUBDIRS)` twice; but it will parallelize the first run (as per `-j` option), so the second one won't do anything, meaning the actual build will be parallelized.
On my machine (2 physical, 4 logical cores), such an empty run takes around 7s when unparallelizd, and less than 5s when parallelized (`-j4`).
OTOH, a full build with `-j4` goes from around 68s to around 53s (-15s, 22%). FTR, with CCache (so the actual build time is a LOT smaller), it goes from around 24s to around 16s (-8s, 33%).
This can also be done in a slightly more hackish way outside of Automake's Makefile, meaning it doesn't change how an Automake's Makefile run behaves. Using another Makefile (I named it *makefile.parallel* locally), like that:
```make
#!/usr/bin/make -f
#
# Parallelizes the top-level Makefile of a SUBDIR-based Automake build system.
#
# What it actually does is recursing in $(builddir) for each target, and in
# each $(SUBDIRS) for target 'all'. 'all' has a dependency on $(SUBDIRS) so
# they are recursed in first, respecting `-j` option as any target.
#
# Note that this means that each $(SUBDIRS) is first built in parallel, and
# then again through the top-level Makefile for target 'all'. This is normally
# not so much of a problem, as the targets should be up-to-date the second
# time, so you only get the Make overhead but no actual building.
# This is needed to "recurse" in '.' in case it's useful.
#
# As this makes uses of normal Make features, it is very possible to force some
# ordering by declaring explicit dependencies between some elements of
# $(SUBDIRS); for example if sub-directory 'po' depends on recursing in another
# one first, i.e. 'src', declare '$(builddir)/po: $(builddir)/src' (not
# forgetting '$(builddir)/' prefix), so 'src' is build before 'po'.
#
# Only 'all' is parallelized. First, this is because it's likely to be the
# only target that really makes sense to parallelize, as it's the one building
# real things. Also, as the parallelized targets are run twice (see above),
# a second run should have low overhead, so it doesn't make sense to e.g.
# parallelize i.e.g 'check' the same way, as it would run the checks twice --
# although parallelization would slightly help when building test programs.
# This could be addressed by omitting recursion in '.', but this can be a
# problem if './Makefile' contains anything beside recursion.
# Additionally, there's a technical difficulty in knowing which target was
# actually asked for, as each $(SUBDIRS) is a target itself, so $@ can't be
# relied on.
#
# Limitations:
# * Inspecting the value of Automake's SUBDIRS variable is naive, and doesn't
# support expansions and the like. This means that if $(SUBDIRS) from
# Makefile.am contains variable references it'll most likely will break.
# user can set either builddir (when running from the source directory) or
# srcdir (when running from the build directory). If builddir==srcdir, no
# overrides are needed.
builddir ?= .
srcdir ?= .
upper_level = $(srcdir)/Makefile.am
SED ?= sed
# extract SUBDIRS
SUBDIRS != $(SED) -ne '/SUBDIRS/s%^.*= \(.*\$\)%$(builddir)/\1%p' $(upper_level)
# MAKEFLAGS minus builddir= and srcdir=
filtered_makeflags = `echo $(MAKEFLAGS) | \
$(SED) -e 's/\(build\|src\)dir=[^ ]*//g'`
# all targets we want to forward somehow
TARGETS := all check clean distclean dist distcheck
all: $(SUBDIRS)
$(TARGETS):
$(MAKE) -C $(builddir) $(filtered_makeflags) $@
$(SUBDIRS):
$(MAKE) -C $@ $(filtered_makeflags)
.PHONY: $(SUBDIRS) $(TARGETS)
```
---
Reply to this email directly or view it on GitHub:
https://github.com/geany/geany-plugins/pull/277#issuecomment-161035919
> Recursive automake processes SUBDIRS in a for loop which can't be done in parallel. This PR does away with SUBDIRSand instead creates proper make targets with complete dependencies.
So yeah, an (auto)Make-isms.
---
Reply to this email directly or view it on GitHub:
https://github.com/geany/geany-plugins/pull/277#issuecomment-161002245
I have read http://www.geany.org/manual/#various-preferences and http://wiki.geany.org/howtos/configurebuildmenu . I am wanting to have additional entries in the build-menu besides the default amount provided. I have tried setting "number_ft_menu_items" to "7" in geany.conf and then adding the below code to filetypes.c.
```FT_03_LB=Splint
FT_03_CM=splint -I. -hints -checks "%d/%f"
FT_03_WD=```
I need more build-menu entries that I can configure. I have many syntax/code checkers, lint programs, and analyzers that are specific to various programming languages. Having three filetype-specific and four non-filetype-specific build-menu entries is not enough for me.
---
Reply to this email directly or view it on GitHub:
https://github.com/geany/geany/issues/782
Am 1. Dezember 2015 02:19:10 MEZ, schrieb Matthew Brush <notifications(a)github.com>:
>> depenencies are globally known to make; for G-P this means that
>plugins don't need to be build in a specific order, but simply as soon
>as possible
>
>Is that really a thing even for GP? I have not much clue about how Make
>works, but it seems like since none of the plugins depend on each
>other, they can all already be parallelized and not build in a
>particular order. Is it a Make-ism why this isn't the case?
>
>---
>Reply to this email directly or view it on GitHub:
>https://github.com/geany/geany-plugins/pull/277#issuecomment-160816125
The default recursive automake setup which GP uses prevents this. This is exactly the purpose of this PR, enabling full parallel build.
Recursive automake processes SUBDIRS in a for loop which can't be done in parallel. This PR does away with SUBDIRSand instead creates proper make targets with complete dependencies. Then the recipes can be build in parallel.
---
Reply to this email directly or view it on GitHub:
https://github.com/geany/geany-plugins/pull/277#issuecomment-160866530
> depenencies are globally known to make; for G-P this means that plugins don't need to be build in a specific order, but simply as soon as possible
Is that really a thing even for GP? I have not much clue about how Make works, but it seems like since none of the plugins depend on each other, they can all already be parallelized and not build in a particular order. Is it a Make-ism why this isn't the case?
---
Reply to this email directly or view it on GitHub:
https://github.com/geany/geany-plugins/pull/277#issuecomment-160816125