Makefile performance: built-in rules
August 19, 2009 — Eric MelskiLike any system that has evolved over many years, GNU Make is rife with appendages of questionable utility. One area this is especially noticeable is the collection of built-in rules in gmake. These rules make it possible to do things like compile a C source file to an executable without even having a makefile, or compile and link several source files with a makefile that simply names the executable and each of the objects that go into it.
But this convience comes at a price. Although some of the built-in rules are still relevant in modern environments, many are obsolete or uncommonly used at best. When’s the last time you compiled Pascal code, or used SCCS or RCS as your version control system? And yet every time you run a build, gmake must check every source file against each of these rules, on the off chance that one of them might apply. A simple tweak to your GNU Make command-line is all it takes to get a performance improvement of up to 30% out of your makefiles. Don’t believe me? Read on.
Let’s look at a trivial example:
all: input @echo done
Touch the file input, then run gmake with the -d option, so you can see as gmake tries each of the built-in rules. GMake will ramble on for hundreds of lines, as you’ll see. Here’s a sample of that output:
Considering target file `all'. File `all' does not exist. Considering target file `input'. Looking for an implicit rule for `input'. ... many lines omitted ... Trying pattern rule with stem `input'. Trying implicit prerequisite `RCS/input,v'. Trying pattern rule with stem `input'. Trying implicit prerequisite `RCS/input'. Trying pattern rule with stem `input'. Trying implicit prerequisite `s.input'. Trying pattern rule with stem `input'. Trying implicit prerequisite `SCCS/s.input'. Trying pattern rule with stem `input'. ... hundreds more lines omitted ... No implicit rule found for `input'. Finished prerequisites of target file `input'.
What’s going on here? Well, we didn’t provide a rule describing how to build the file input, so gmake is checking to see if any of the built-in rules could be used to generate it. Of course none of them do, so this is all wasted effort. Lucky for us, a single command-line option is all you need to tell gmake not to bother with the default built-in rules: -r. Try that trivial makefile again, this time with -d -r:
Considering target file `all'. File `all' does not exist. Considering target file `input'. Looking for an implicit rule for `input'. No implicit rule found for `input'. Finished prerequisites of target file `input'.
All the extra nonsense is gone! And even on this toy example, there is a measurable performance improvement: originally, this makefile runs in about 0.015s (average over three runs); with the built-in rules disabled, it’s just 0.012s. But I can see you won’t be convinced by such a trivial example. So let’s try something a bit bigger:
SOURCES:=$(wildcard sub/*.x)
TARGETS:=$(SOURCES:.x=.o)
all: $(TARGETS)
@echo done
%.o: %.x
@echo $@
The directory sub contains 15,000 files named 00001.x through 15000.x. With the built-in rules (and redirecting output to /dev/null), this makefile runs in about 60.2s; without the built-in rules, 42.9s — 28% faster.
Finally, let’s try this optimization on a real build. I built one component of the Accelerator project completely, then ran “no-op” builds (ie, no work to be done, just checking that everything is up-to-date). With built-in rules, this took 6.0s; without, it took 5.2s — 13% faster:
| Large test, with built-ins: |
60.2s
|
| Large test, no built-ins; |
42.9s
|
| No-op build, with built-ins: |
6.0s
|
| No-op build, no built-ins: |
5.2s
|
Now, if your build actually relies on built-in rules obviously you can’t simply disable them. But you could explicitly define just those rules that you require and disable the rest. For example, if you use the default %.o: %.cpp rule, you could add just that rule to your makefiles:
%.o: %.cpp $(COMPILE.cpp) $(OUTPUT_OPTION) $<
Once you’ve done that, you can add -r to your command-line and enjoy the benefits. If you go this route, you can see the list of built-in rules by running gmake -p; the built-ins are marked as “built-in” in that output.
Conclusion
Disabling gmake’s array of built-in rules is an easy way to squeeze extra performance out of your makefiles, particularly on large builds and on no-op builds. All you have to do is add -r to your commmand-line. (NB: If you prefer more descriptive command-lines, you can use –no-builtin-rules instead!)
This article is the latest of several looking at different aspects of makefile performance. If you liked this article, you may enjoy the others in the series:
- Makefile performance: recursive make
- Makefile performance: $(shell)
- Makefile performance: pattern-specific variables
- Makefile performance: built-in rules (this post)
Is there a way to turn on -r from inside the Makefile? Otherwise, it depends on the invoker remembering to use -r.
Ciao!
Put MAKEFLAGS=-r in your Makefile to turn this on by default. (You still pay some extra startup cost relative to passing -r on the command line, but it’s a fraction of a second rather than the huge difference you observe between having -r and not.)
Adding MAKEFLAGS=-r to your Makefile disables some of the built-in rules, but not all. Try running gmake -d on a Makefile that has set MAKEFLAGS=-r and you’ll still see boatloads of extra rules being checked.
However, you can explicitly disable each of the built-in rules by adding empty pattern rules. For example, to disable the %.o: %.f built-in, just add the following to your makefile:
Note that the rule deliberately has no commands! Follow this same pattern for each of the built-ins and you will achieve the same effect as setting -r on the command-line.
What about .SUFFIXES?
@Jean: .SUFFIXES can be another unexpected source of performance problems for makefiles. I plan to cover that in a future post.
thanks for this great tip
Hi. I read a few of your other posts and i wanted to say thank you for the informative posts.
Hi. If you concerned about GNU make speed you might be interested in Fastmake – continuation of GNU make with speed and syntax improvements
Thanks for the link, Vitaly. Fastmake has some interesting ideas, but unfortunately because of it’s incompatibilities with gmake, I was unable to compare performance on any real build. Also, some of your “syntax improvements” seem to be identical in function to existing features — for example, .FOREACH seems to do exactly what standard gmake $(foreach) does. Anyway, good luck to you, and thanks for stopping by!
Hi. Just want to say thanks for the post, and that we’re seeing an even greater gain than 30%. Our up-to-date builds used to take 8 seconds, and now they take 2. Great tip!
That’s great John! Thanks for commenting!