A Shallow Dive into GNU Make
GNU Make is a popular and commonly used program for building C language software. It is used when building the Linux kernel and other frequently used GNU/Linux programs and software libraries.
Most embedded software developers will work with GNU Make at some point in their career, either using it to compile small libraries or building an entire project. Though there are many, many alternatives to Make, it’s still commonly chosen as the build system for new software given its feature set and wide support.
This article explains general concepts and features of GNU Make and includes recommendations for getting the most out of a Make build! Consider it a brief guided tour through some of my favorite/most used Make concepts and features 🤗.
If you feel like you already know Make pretty well, feel free to skip the tutorial portion and jump to my personal recommendations.
Table of Contents
- What is GNU Make?
- When to choose Make
- Invoking Make
- Anatomy of a Makefile
- Advanced Topics
- Debugging Makefiles
- Full Example
What is GNU Make?
GNU Make is a program that automates the running of shell commands and helps with repetitive tasks. It is typically used to transform files into some other form, e.g. compiling source code files into programs or libraries.
It does this by tracking prerequisites and executing a hierarchy of commands to produce targets.
Although the GNU Make manual is lengthy, I suggest giving it a read as it is the best reference I’ve found: https://www.gnu.org/software/make/manual/html_node/index.html
Let’s dive in!
When to choose Make
Make is suitable for building small C/C++ projects or libraries that would be included in another project’s build system. Most build systems will have a way to integrate Make-based sub-projects.
For larger projects, you will find a more modern build system easier to work with.
I would suggest a build system other than Make in the following situations:
- When the number of targets (or files) being built is (or will eventually be) in the hundreds.
- A “configure” step is desired, which sets up and persists variables, target definitions, and environment configurations.
- The project is going to remain internal or private and will not need to be built by end users.
- You find debugging a frustrating exercise.
- You need the build to be cross platform that can build on macOS, Linux, and Windows.
In these situations, you might find using CMake, Bazel, Meson, or another modern build system a more pleasurable experience.
make will load a file named
Makefile from the current directory
and attempt to update the default goal (more on goals later).
Make will search for files named
Makefile, in that order
You can specify a particular makefile with the
$ make -f foo.mk
You can specify any number of goals by listing them as positional arguments:
# typical goals $ make clean all
You can pass Make a directory with the
-C argument, and this will run Make as if it first
cd‘d into that directory.
$ make -C some/sub/directory
gitalso can be run with
-Cfor the same effect!
Make can run jobs in parallel if you provide the
-l options. A
guideline I’ve been told is to set the job limit to 1.5 times the number of
processor cores you have:
# a machine with 4 cores: $ make -j 6
Anecdotally, I’ve seen slightly better CPU utilization with the
limit” option, vs. the
-j “jobs” option. YMMV though!
There are a few ways to programmatically find the CPU count for the current
machine. One easy option is to use the python
function to get the number of threads supported by the system (note on a system
with hyper-threading, this will use up a lot of your machine’s resources,
but is probably preferable to letting Make spawn an unlimited number of jobs).
# call the python cpu_count() function in a subshell $ make -l $(python -c "import multiprocessing; print(multiprocessing.cpu_count())")
Output During Parallel Invocation
If you have a lot of output from the commands Make is executing in parallel, you
might see output interleaved on
stdout. To handle this, Make has the option
I recommend using
--output-sync=recurse, which will print the entire
output of each target’s recipe when it completes, without interspersing other
It also will output an entire recursive Make’s output together if your recipe is using recursive make.
Anatomy of a Makefile
A Makefile contains rules used to produce targets. Some basic components of a Makefile are shown below:
# Comments are prefixed with the '#' symbol # A variable assignment FOO = "hello there!" # A rule creating target "test", with "test.c" as a prerequisite test: test.c # The contents of a rule is called the "recipe", and is # typically composed of one or more shell commands. # It must be indented from the target name (historically with # tabs, spaces are permitted) # Using the variable "FOO" echo $(FOO) # Calling the C compiler using a predefined variable naming # the default C compiler, '$(CC)' $(CC) test.c -o test
Let’s take a look at each part of the example above.
Variables are used with the syntax
FOO is the variable name.
Variables contain purely strings as Make does not have other data types. Appending to a variable will add a space and the new content:
FOO = one FOO += two # FOO is now "one two" FOO = one FOO = $(FOO)two # FOO is now "onetwo"
In GNU Make syntax, variables are assigned with two “flavors”:
variable = expressionThe expression on the right hand side is assigned verbatim to the variable- this behaves much like a macro in C/C++, where the expression is evaluated when the variable is used:
FOO = 1 BAR = $(FOO) FOO = 2 # prints BAR=2 $(info BAR=$(BAR))
variable := expressionThis assigns the result of an expression to a variable; the expression is expanded at the time of assignment:
FOO = 1 BAR := $(FOO) FOO = 2 # prints BAR=1 $(info BAR=$(BAR))
$(info ...)function is being used above to print expressions and can be handy when debugging makefiles!*`
Variables which are not explicitly, implicitly, nor automatically set will evaluate to an empty string.
Environment variables are carried into the Make execution environment. Consider the following makefile for example:
$(info YOLO variable = $(YOLO))
If we set the variable
YOLO in the shell command when running
make, we’ll set the value:
$ YOLO="hello there!" make YOLO variable = hello there! make: *** No targets. Stop.
Note: Make prints the “No targets” error because our makefile had no targets listed!
If you use the
?= assignment syntax, Make will only assign that value if the
variable doesn’t already have a value:
# default CC to gcc CC ?= gcc
We can then override
$(CC) in that makefile:
$ CC=clang make
Another common pattern is to allow inserting additional flags. In the makefile, we would append to the variable instead of directly assigning to it.
CFLAGS += -Wall
This permits passing extra flags in from the environment:
$ CFLAGS='-Werror=conversion -Werror=double-promotion' make
This can be very useful!
A special category of variable usage is called overriding variables. Using this command-line option will override the value set ANYWHERE ELSE in the environment or Makefile!
# the value passed in the make command will override # any value set elsewhere YOLO = "not overridden" $(info $(YOLO))
# setting "YOLO" to different values in the environment + makefile + overriding # variable, yields the overriding value $ YOLO="environment set" make YOLO='overridden!!' overridden!! make: *** No targets. Stop.
Overriding variables can be confusing, and should be used with caution!
These variables are only available in the recipe context. They also apply to any prerequisite recipe!
# set the -g value to CFLAGS # applies to the prog.o/foo.o/bar.o recipes too! prog : CFLAGS = -g prog : prog.o foo.o bar.o echo $(CFLAGS) # will print '-g'
These are pre-defined by Make (unless overridden with any other variable type of the same name). Some common examples:
$(CC)- the C compiler (
$(AR)- archive program (
$(CFLAGS)- flags for the C compiler
Full list here:
These are special variables always set by Make and available in recipe context. They can be useful to prevent duplicated names (Don’t Repeat Yourself).
A few common automatic variables:
# $@ : the target name, here it would be "test.txt" test.txt: echo HEYO > $@ # $^ : name of all the prerequisites all.zip: foo.txt test.txt # run the gzip command with all the prerequisites "$^", outputting to the # name of the target, "$@" gzip -c $^ > $@
See more at: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html
Targets are the left hand side in the rule syntax:
target: prerequisite recipe
Targets almost always name files. This is because Make uses last-modified time to track if a target is newer or older than its prerequisites and whether it needs to be rebuilt!
When invoking Make, you can specify which target(s) you want to build as the
goals by specifying it as a positional argument:
# make the 'test.txt' and 'all.zip' targets make test.txt all.zip
If you don’t specify a goal in the command, Make uses the first target specified in the makefile, called the “default goal” (you can also override the default goal if you need to).
Sometimes it’s useful to have meta-targets like
test, etc. In these cases, you don’t want Make to check for a file named
Make provides the
.PHONY target syntax to mark a target as not pointing to a
# Say our project builds a program and a library 'foo' and 'foo.a'; if we want # to build both by default we might make an 'all' rule that builds both .PHONY: all all: foo foo.a
If you have multiple phony targets, a good pattern might be to append each to
.PHONY where it’s defined:
# the 'all' rule that builds and tests. Note that it's listed first to make it # the default rule .PHONY: all all: build test # compile foo.c into a program 'foo' foo: foo.c $(CC) foo.c -o foo # compile foo-lib.c into a library 'foo.a' foo.a: foo-lib.c # compile the object file $(CC) foo-lib.c -c foo-lib.o # use ar to create a static library containing our object file. using the # '$@' variable here to specify the rule target 'foo.a' $(AR) rcs $@ foo-lib.o # a phony rule that builds our project; just contains a prerequisite of the # library + program .PHONY: build build: foo foo.a # a phony rule that runs our test harness. has the 'build' target as a # prerequisite! Make will make sure (pardon the pun) the build rule executes # first .PHONY: test test: build ./run-tests.sh
.PHONYtargets are ALWAYS considered out-of-date, so Make will ALWAYS run the recipe for those targets (and therfore any target that has a
.PHONYprerequisite!). Use with caution!!
Implicit rules are provided by Make. I find using them to be confusing since there’s so much behavior happening behind the scenes. You will occasionally encounter them in the wild, so be aware.
Here’s a quick example:
# this will compile 'test.c' with the default $(CC), $(CFLAGS), into the program # 'test'. it will handle prerequisite tracking on test.c test: test.o
Full list of implicit rules here:
Pattern rules let you write a generic rule that applies to multiple targets via pattern-matching:
# Note the use of the '$<' automatic variable, specifying the first # prerequisite, which is the .c file %.o: %.c $(CC) -c $< -o $@
The rule will then be used to make any target matching the pattern, which above
would be any file matching
If you use those
.o files mentioned above to build a program:
OBJ_FILES = foo.o bar.o # Use CC to link foo.o + bar.o into 'program'. Note the use of the '$^' # automatic variable, specifying ALL the prerequisites (all the OBJ_FILES) # should be part of the link command program: $(OBJ_FILES) $(CC) -o $@ $^
As seen above, these are targets that Make will check before running a rule. They can be files or other targets.
If any prerequisite is newer (modified-time) than the target, Make will run the target rule.
In C projects, you might have a rule that converts a C file to an object file, and you want the object file to rebuild if the C file changes:
foo.o: foo.c # use automatic variables for the input and output file names $(CC) $^ -c $@
A very important consideration for C language projects is to trigger
recompilation if an
#include header files change for a C file. This is done
-M compiler flag for gcc/clang, which will output a
.d file you
will then import with the Make
.d file will contain the necessary prerequisites for the
.c file so any
header change causes a rebuild. See more details here:
The basic form might be:
# these are the compiler flags for emitting the dependency tracking file. Note # the usage of the '$<' automatic variable DEPFLAGS = -MMD -MP -MF $<.d test.o: test.c $(CC) $(DEPFLAGS) $< -c $@ # bring in the prerequisites by including all the .d files. prefix the line with # '-' to prevent an error if any of the files do not exist -include $(wildcard *.d)
These prerequisites will only be built if they don’t exist; if they are newer than the target, they will not trigger a target re-build.
A typical use is to create a directory for output files; emitting files to a
directory will update its
mtime attribute, but we don’t want that to trigger a
OUTPUT_DIR = build # output the .o to the build directory, which we add as an order-only # prerequisite- anything right of the | pipe is considered order-only $(OUTPUT_DIR)/test.o: test.c | $(OUTPUT_DIR) $(CC) -c $^ -o $@ # rule to make the directory $(OUTPUT_DIR): mkdir -p $@
The “recipe” is the list of shell commands to be executed to create the target. They are
passed into a sub-shell (
/bin/sh by default). The rule is considered
successful if the target is updated after the recipe runs (but is not an error
if this doesn’t happen).
foo.txt: # a simple recipe echo HEYO > $@
If any line of the recipe returns a non-zero exit code, Make will terminate and
print an error message. You can tell Make to ignore non-zero exit codes by
prefixing with the
.PHONY: clean clean: # we don't care if rm fails -rm -r ./build
Prefixing a recipe line with
@ will disable echoing that line before
clean: @# this recipe will just print 'About to clean everything!' @# prefixing the shell comment lines '#' here also prevents them from @# appearing during execution @echo About to clean everything!
Make will expand variable/function expressions in the recipe context before
running them, but will otherwise not process it. If you want to access shell
variables, escape them with
USER = linus print-user: # print out the shell variable $USER echo $$USER # print out the make variable USER echo $(USER)
These features are less frequently encountered, but provide some powerful functionality that can enable sophisticated behavior in your build.
Make functions are called with the syntax:
arguments is a comma-delimited list of arguments.
There are several functions provided by Make. The most common ones I use are for text manipulation: https://www.gnu.org/software/make/manual/html_node/Text-Functions.html https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html
FILES=$(wildcard *.c) # you can combine function calls; here we strip the suffix off of $(FILES) with # the $(basename) function, then add the .o suffix O_FILES=$(addsuffix .o,$(basename $(FILES))) # note that the GNU Make Manual suggests an alternate form for this particular # operation: O_FILES=$(FILES:.c=.o)
You can define your own functions as well:
reverse = $(2) $(1) foo = $(call reverse,a,b)
A more complicated but quite useful example:
# recursive wildcard (use it instead of $(shell find . -name '*.c')) # taken from https://stackoverflow.com/a/18258352 rwildcard=$(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d)) C_FILES = $(call rwildcard,.,*.c)
You can have Make call a shell expression and capture the result:
TODAYS_DATE=$(shell date --iso-8601)
I’m cautious when using this feature, though; it adds a dependency on whatever programs you use, so if you’re calling more exotic programs, make sure your build environment is controlled (e.g. in a container or with Conda).
Make has syntax for conditional expressions:
FOO=yolo ifeq ($(FOO),yolo) $(info foo is yolo!) else $(info foo is not yolo :( ) endif # testing if a variable is set; unset variables are empty ifneq ($(FOO),) # checking if FOO is blank $(info FOO is unset) endif
The “complex conditional” syntax is just the
# "complex conditional" ifeq ($(FOO),yolo) $(info foo is yolo) else ifeq ($(FOO), heyo) $(info foo is heyo) else $(info foo is not yolo or heyo :( ) endif
You can import other Makefile contents using the
SOURCE_FILES := \ bar.c \ foo.c \
include sources.mk OBJECT_FILES = $(SOURCE_FILES:.c=.o) %.o: %.c $(CC) -c $^ -o $@
Invoking Make from a Makefile should be done with the
somelib.a: $(MAKE) -C path/to/somelib/directory
This is often used when building external libraries. It’s also used heavily in Kconfig builds (e.g. when building the Linux kernel).
Note that this approach has some pitfalls:
- Recursive invocation can result in slow builds.
- Tracking prerequisites can be tricky; often you will see
More details on the disadvantages here:
eval directive allows us to generate Make syntax at runtime:
# generate rules for xml->json in some weird world FILES = $(wildcard inputfile/*.xml) # create a user-defined function that generates rules define GENERATE_RULE = $(eval # prereq rule for creating output directory $(1)_OUT_DIR = $(dir $(1))/$(1)_out $(1)_OUT_DIR: mkdir -p $@ # rule that calls a script on the input file and produces $@ target $(1)_OUT_DIR/$(1).json: $(1) | $(1)_OUT_DIR ./convert-xml-to-json.sh $(1) $@ ) # add the target to the all rule all: $(1)_OUT_DIR/$(1).json endef
# produce the rules .PHONY: all all: $(foreach file,$(FILES),$(call GENERATE_RULE,$(file)))
Note that approaches using this feature of Make can be quite confusing, adding helpful comments explaining what the intent is can be useful for your future self!
VPATH is a special Make variable that contains a list of directories Make
should search when looking for prerequisites and targets.
It can be used to emit object files or other derived files into a
directory, instead of cluttering up the
# This makefile should be invoked from the temporary build directory, eg: # $ mkdir -p build && cd ./build && make -f ../Makefile # Derive the directory containing this Makefile MAKEFILE_DIR = $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST)))) # now inform Make we should look for prerequisites from the root directory as # well as the cwd VPATH += $(MAKEFILE_DIR) SRC_FILES = $(wildcard $(MAKEFILE_DIR)/src/*.c) # Set the obj file paths to be relative to the cwd OBJ_FILES = $(subst $(MAKEFILE_DIR)/,,$(SRC_FILES:.c=.o)) # now we can continue as if Make was running from the root directory, and not a # subdirectory # $(OBJ_FILES) will be built by the pattern rule below foo.a: $(OBJ_FILES) $(AR) rcs $@ $(OBJ_FILES) # pattern rule; since we added ROOT_DIR to VPATH, Make can find prerequisites # like `src/test.c` when running from the build directory! %.o: %.c # create the directory tree for the output file 👍 echo $@ mkdir -p $(dir $@) # compile $(CC) -c $^ -o $@
I recommend avoiding use of
VPATH. It’s usually simpler to achieve the same
out-of-tree behavior by outputting the generated files in a build directory
You may see the
touch command used to track rules that seem difficult to
otherwise track; for example, when unpacking a toolchain:
# our tools are stored in tools.tar.gz, and downloaded from a server TOOLS_ARCHIVE = tools.tar.gz TOOLS_URL = https://httpbin.org/get # the rule to download the tools using wget $(TOOLS_ARCHIVE): wget $(TOOLS_URL) -O $(TOOLS_ARCHIVE) # rule to unpack them tools-unpacked.dummy: $(TOOLS_ARCHIVE) # running this command results in a directory.. but how do we know it # completed, without a file to track? tar xzvf $^ # use the touch command to record completion in a dummy file touch $@
I recommend avoiding the use of
touch. However there are some cases where it
might be unavoidable.
I typically use the Make equivalent of
functions, for small problems, for example when checking conditional paths that
ifeq ($(CC),clang) $(error whoops, clang not supported!) endif
For debugging why a rule is running when it shouldn’t (or vice versa), you can
I recommend redirecting stdout to a file when using this option, it can produce a lot of output.
For profiling a make invocation (e.g. for attempting to improve compilation times), this tool can be useful:
Check out the tips here for compilation-related performance improvements:
Using a Verbose Flag
If your project includes a lot of compiler flags (search paths, lots of warning flags, etc.), then you may want to simplify the output of Make rules. It can be useful to have a toggle to easily see the full output, for example:
ifeq ($(V),1) Q := else Q := @ endif %.o: %.c # prefix the compilation command with the $(Q) variable # use echo to print a simple "Compiling x.c" to show progress @echo Compiling $(notdir @^) $(Q) $(CC) -c $^ -o $@
To enable printing out the full compilation commands, set the
variable like so:
$ V=1 make
Here’s an annotated example of a complete build process for an example C project. You can see this example and the source tree here.
# Makefile for building the 'example' binary from C sources # Verbose flag ifeq ($(V),1) Q := else Q := @ endif # The build folder, for all generated output. This should normally be included # in a .gitignore rule BUILD_FOLDER := build # Default all rule will build the 'example' target, which here is an executable .PHONY: all: $(BUILD_FOLDER)/example # List of C source files. Putting this in a separate variable, with a file on # each line, makes it easy to add files later (and makes it easier to see # additions in pull requests). Larger projects might use a wildcard to locate # source files automatically. SRC_FILES = \ src/example.c \ src/main.c # Generate a list of .o files from the .c files. Prefix them with the build # folder to output the files there OBJ_FILES = $(addprefix $(BUILD_FOLDER)/,$(SRC_FILES:.c=.o)) # Generate a list of depfiles, used to track includes. The file name is the same # as the object files with the .d extension added DEP_FILES = $(addsuffix .d,$(OBJ_FILES)) # Flags to generate the .d dependency-tracking files when we compile. It's # named the same as the target file with the .d extension DEPFLAGS = -MMD -MP -MF $@.d # Include the dependency tracking files -include $(DEP_FILES) # List of include dirs. These are put into CFLAGS. INCLUDE_DIRS = \ src/ # Prefix the include dirs with '-I' when passing them to the compiler CFLAGS += $(addprefix -I,$(INCLUDE_DIRS)) # Set some compiler flags we need. Note that we're appending to the CFLAGS # variable CFLAGS += \ -std=c11 \ -Wall \ -Werror \ -ffunction-sections -fdata-sections \ -Og \ -g3 # Our project requires some linker flags: garbage collect sections, output a # .map file LDFLAGS += \ -Wl,--gc-sections,-Map,$@.map # Set LDLIBS to specify linking with libm, the math library LDLIBS += \ -lm # The rule for compiling the SRC_FILES into OBJ_FILES $(BUILD_FOLDER)/%.o: %.c @echo Compiling $(notdir $<) @# Create the folder structure for the output file @mkdir -p $(dir $@) $(Q) $(CC) $(CFLAGS) $(DEPFLAGS) -c $< -o $@ # The rule for building the executable "example", using OBJ_FILES as # prerequisites. Since we're not relying on an implicit rule, we need to # explicity list CFLAGS, LDFLAGS, LDLIBS $(BUILD_FOLDER)/example: $(OBJ_FILES) @echo Linking $(notdir $@) $(Q) $(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ # Remove debug information for a smaller executable. An embedded project might # instead using [arm-none-eabi-]objcopy to convert the ELF file to a raw binary # suitable to be written to an embedded device STRIPPED_OUTPUT = $(BUILD_FOLDER)/example-stripped $(STRIPPED_OUTPUT): $(BUILD_FOLDER)/example @echo Stripping $(notdir $@) $(Q)objcopy --strip-debug $^ $@ # Since all our generated output is placed into the build folder, our clean rule # is simple. Prefix the recipe line with '-' to not error if the build folder # doesn't exist (the -f flag for rm also has this effect) .PHONY: clean clean: - rm -rf $(BUILD_FOLDER)
A list of recommendations for getting the most of Make:
- Targets should usually be real files.
- Always use
$(MAKE)when issuing sub-make commands.
- Try to avoid using
.PHONYtargets. If the rule generates any file artifact, consider using that as the target instead of a phony name!
- Try to avoid using implicit rules.
- For C files, make sure to use
.dautomatic include tracking!
- Use metaprogramming with caution.
- Use automatic variables in rules. Always try to use
$@for a recipe output path, so your rule and Make have the exact same path.
- Use comments liberally in Makefiles, especially if there is complicated behavior or subtle syntax used. Your co-workers (and future self) will thank you.
- Use the
-loptions to run Make in parallel!
- Try to avoid using the
touchcommand to track rule completion
I hope this article has provided a few useful pointers around GNU Make!
Make remains common in C language projects, most likely due to its usage in the Linux kernel. Many recently developed statically compiled programming languages, such as Rust or Go, provide their own build infrastructure. However, when integrating Make-based software into those languages, for example when building a C library to be called from Rust, it can be surprisingly helpful to understand some Make concepts!
You may also encounter automake in
open source projects (look for a
./configure script). This is a related tool
that generates Makefiles, and is worth a look (especially if you are writing
C software that needs to be very widely portable).
There are many competitors to GNU Make available today, I encourage everyone to look into them. Some examples:
- CMake is pretty popular (the Zephyr project uses this) and worth a look. It makes out-of-tree builds pretty easy
- Bazel uses a declarative syntax (vs. Make’s imperative approach)
Meson is a meta-builder like
cmake, but by default uses Ninja as the backend, and can be very fast
Good detailed dive into less common topics (shout out on
Mix of very exotic and simpler material:
Very nice summary: