Repository Return

Introduction

The purpose of this post is to introduce Make and Makefiles. The goal of this post is to follow the progression in complexity that a developer may face moving from introductory to professional, production software. Along the way, I am going to include real-world use-cases and techniques I have come across in my career, in the hope that the reader will learn the WHY and not just the HOW of using makefiles. I have created a repository with the source code and example makefile that can be cloned or downloaded here.

Audience, Disclaimer

This post is written from a c/c++ development perspective and it is not intended as a thorough tutorial or introduction to Make. There are countless, fantastic resources for learning Make elsewhere. The source code within is extremely simplistic, but a general understanding of c++ development is expected.

I'm not entirely sure who will find this tutorial useful, if anyone. Instead, I'm writing the makefile introduction that I wish I had been given when I graduated college and began working in the software engineering industry. I don't claim to be an expert, and I don't claim that the practices within are best-practices. However, this post contains makefiles that are similar to those I've seen used in successful, professional production environments.

Introduction to Make

Obviously there are endless resources for reading about Make, Makefiles, and the purpose for automated build processes. If the brief introduction I provide here is insufficient, you should have no trouble finding countless resources online/YouTube videos etc.

Make is a build automation tool typically used for compiling source code into an executable. Make itself does not perform compilation; rather, make manages the files, directories, and toolchain required to compile complex applications.

A Simple Program

To begin, we will start with a simple program and a make-less way of compiling it. Our simple program just prints "Hello":

This is in a file called main.cpp in a directory by itself. Here is our directory structure:

A program like this can be compiled with g++, and then executed using commands like this:

The g++ command compiles an executable from the main.cpp file. The -o main option tells g++ to name the resulting executable main. I then executed this program with ./main and we can see our output.

Let’s see how this same procedure can be performed with a basic make file. First we create a file named Makefile in the directory alongside main.cpp. This is a special filename that Make will automatically look for when it is executed. This is the contents of that makefile:

all:
    g++ main.cpp -o main

In the makefile, all is called a target. Targets are typically an action to carry out, a stage of the build process, or the name of a resulting executable. all is a special target name that is called if no target is supplied to the make command. This will make more sense later on; for now, just know that it's called a target.

The action that our all target will carry out is just our compilation command from above. To run this makefile, we just type "make" in the directory that it is located.

So what is the point of this? Well, right now, not much. This build is far too simplistic to require build automation. Let’s continue to build on this foundation.

A Slightly More Complex Program

The remainder of this tutorial will build on a slightly more complex program that has multiple directories, source files, and header files. To make things clear moving forward, lets look at the program. The directory structure and files within is shown below:

Our main method is in main.cpp still. Inside main, we include the header helloprinter.h that declares the HelloPrinter class. main creates a HelloPrinter object and calls print() on it.

The HelloPrinter class is declared in includes/helloprinter.h and defined in source/helloprinter.cpp. It includes simplemath.h that declares the SimpleMath class. It contains a method called print that creates a SimpleMath object. It calls the add method of this object to obtain a result (in this case, 2+2 which is 4) that is included with a message that is printed to std out.

Lastly, the SimpleMath class is declared in includes/simplemath.h and defined in source/simplemath.cpp. It contains a method called add that adds two integers and returns the result.

Here is the output of that program:

Building on the Basic Makefile

Let’s look at the command that might be used to compile this program.

Even with the introduction of just a few more files and directories, the command is already starting to get unmanageable. Typing a command like this that includes hundreds of files in dozens of folders is completely impractical. Let's see how a makefile might help. Here is what we did before:

all:
    g++ main.cpp source/helloprinter.cpp includes/helloprinter.h source/simplemath.cpp includes/simplemath.h -o main

Now, this would work, but is it really any better? We are still using a single, ugly long command. Let's improve on that by introducing a new feature of Makefiles, variables.

In the make file above, we've introduced two variables, one named SOURCES and the other HEADERS. We set these variables equal to a list of files, source files and header files respectively. The \ on each line (notice, it's not on the final line) tells Make to treat that text as if it is the same line. This is done for readability.

For example:

SOURCES:= test.cpp \
another.cpp

is exactly equivalent to:

SOURCES:= test.cpp another.cpp

In the command at the bottom, we use the variables by following the $(<variable name>) syntax. When expanded, the command here is functionally identical to the ugly one from earlier. Let's look at the output from make:

When developers work on this project and need to add files or directories, it will be much easier for them to append them to these two variables than to modify a long compilation command. This one small change has significantly improved the readability and modifiability of the compilation command.

Let's introduce a small mistake into the program: an unused variable:

Normally a mistake like this wouldn't matter, but in production software "warn-able" items like this are unacceptable. Let's add a compilation flag to check for this. We could simply add the flags to our compilation command like this:

g++ -Wall $(SOURCES) $(HEADERS) -o main

But what if we want to have multiple flags? or we want to easily modify the flags in use, without digging around for the compilation command? Let's add another variable for flags:

Now our compiler will catch the unused variable:

Using variables for common sets of compiler flags, options and configurations is an extremely common technique; for example, setting library paths and including libraries, or toggling platform dependent resources. (platform independent makefiles are something I will not cover in this tutorial, because I believe it is beyond the scope of hand-made makefiles and should be handled with tools like Cmake, which I hope to cover in a future post).

Let's add another class to our project, and reorganize the structure a little. We'll add a new HarderMath class that does multiplication, and we will put all of our math-related classes in a math folder, both in source and in includes. This is a very common type of structure, with related classes grouped into sub-directories. Here is our folder structure and output now:

We need to include these new files into our Makefile. We could just add them to the list of files in our two variables, but as we add more folders and more files, this will become tedious. There is a better way. Let's introduce a shell command to list the files for us.

In the Makefile above, we've replaced our variables with a $(shell) command. The shell inside of the parenthesis tells make we want to execute the shell command. The command we've chosen is the find command present on most linux systems. This functions identically to how it does when you run find by hand. We pass the root folder to search, then use the -name and a query string *.<file extension> to list all the files with the given extension in the given root folder. The list returned by find is stored in the variable, and the variables are later expanded in our compilation command. We then appended main.cpp to our list of sources using the += syntax.

At this point, it is clear that Makefiles take complex compilation operations and boil them down to a single make command. It's also clear that modifying the project structure and compilation process becomes much easier with the introduction of a makefile. Let's keep improving.

Building on the more complex Makefile

So far we've compiled all of our source files into one executable, main. Let's add some object files to our compilation process.

Let's break this down because there are several changes at once. First, we have the new variable named OBJECTS. OBJECTS is set on this line:

OBJECTS=$(SOURCES:.cpp=.o)

The expression inside the parenthesis can be explained as: Take our list of source file names, and create a list of object names by replacing .cpp with .o. The resulting list is identical to our list of sources, except each ends in .o.

Next, we added $(OBJECTS) to our all target with this line:

all: $(OBJECTS)

This can be described as: For each object file, execute our compilation command.

Finally, we replaced our $(SOURCES) with our object files in the compilation command. Let's run this and see the output.

We can see that our compilation command is called once for each of our source files, and then the list of source files and the headers are called to compile our final executable. Let's look at the resulting files in our file structure.

We can see we now have a bunch of object files alongside each of our source files. Let's run make again and observe what happens:

So what happened here? Why didn't our object files get compiled? This brings up an extremely important feature of Make. The object files weren't recompiled because the corresponding source files were not changed. Let's change helloprinter.cpp and see what happens:

Only our helloprinter object file was re-compiled. In a production software project, with dozens or even hundreds of source files, re-compiling everything can take minutes or even hours. Re-compiling everything everytime is not feasible for a production team. Make optimizes this process for us.

But what if we WANT to re-compile everything? One example might be a parameter external to the source code changing. Since this change doesn’t touch our source files, Make doesn't know we need to recompile but we do nonetheless. This might be a bit confusing but I will touch on a concrete example soon. For now, let's use this scenario as a segue into a new Makefile concept: additional targets.

Remember those .o object files laying around in our file system? If we remove those, Make will recompile the corresponding source for us. Let's add another extremely common target to our Makefile, clean:

Let's break down what this does. Here we've added a new target named clean. When that target executes, it passes our list of object files to the rm command to remove them. Notice the - before the rm. That dash tells Make to ignore any failures in the rm command. If you ran this target twice in a row, the second time would fail because the files were already deleted. The - allows us to ignore that failure.

Remember that the target name all is a special name that Make automatically executes. That means the commands make and make all are the same. To run a different target, we specify the target name. To run our clean target, we use:

make clean

Here is the output:

The RM command has run, and all of the .o files are gone from our project. Now if we run make again, everything will re-compile.

I want to demonstrate a common "gotcha" with targets. Lets' create an empty file just named "clean" in our project and run make clean and see what happens.

So what is going on here? Instead of running our clean target, Make sees the file named "clean" and treats it as a regular source file. (in this case, it doesn't think it needs to be updated because it's unchanged). To fix this, we need to add a .PHONY declaration. "Phony" targets are Make targets that perform operations without producing artifacts. If we explicitly declare them, we can avoid the problem we just had. Let's add that to our Makefile:

The .PHONY line above simply says: clean is a target, don't treat it as a file, even if it exists in our project.

Let's add a couple more targets that are very common in "real" Makefiles.

Ok, there is a lot going on here, but it's very simple. First, we added a target named deploy. This target executes a couple of commands that copy our executable, and a readme file named app_readme.txt into a folder named deploy and then tar it into a compressed archive so it's easy to distribute out to users. (I added app_readme.txt just for this step, its just an empty text file).

Notice that this target calls our all target and a new target named distclean using the syntax seen here:

deploy: all distclean

This means that every time the deploy target is run, the all target and then the distclean target are also run. deploy is dependent on the all target because it needs to have the main executable built before it can package it for distribution. These types of linked targets are how dependencies like that can be enforced.

Next we added a target named distclean, short for "distribution clean" that removes our deployment directory and our archive. This can be used to clean up old deployment artifacts as the "deploy" command above does.

Lastly, we added an allclean target that calls clean and distclean respectively. This wipes out all of our build and deployment artifacts and restores our project to a clean state. Some developers may want to remove the executable that was created as well; this would just be a matter of adding it to the list of files removed by the clean target.

These targets can be named anything you want, they are not pre-defined. The target names I've chosen here are ones I've encountered frequently in production environments. Notice I added each target name to the .PHONY list.

For the final feature I want to include in this tutorial, let's try adding some versioning to our project. In a real-world environment, this step would involve a continuous integration system or some other build-management process assigning unique versions to the project when it is built. Let's simulate that with an environment variable. First, let's modify our makefile:

Starting at the top, we've added a variable named VERSION with a slightly different syntax. The ?= says that if VERSION isn't defined, default to the value 0.0.

Next we added the flag -DVERSIONMACRO='"$(VERSION)"' to our CPPFLAGS variable. This line will expand to a flag that passes the value of VERSION to our compiler and it can be expanded during compilation as a macro named VERSIONMACRO in our source code. Let's add it to our HelloPrinter class:

Notice the single quotes around the double quotes on the variable in the 3rd line of the makefile. This is to escape the version as a String for use in the code.

Lastly, we added the version to the resulting executable name, preceded by an underscore:

all: $(OBJECTS)
    g++ $(CPPFLAGS) $(OBJECTS) $(HEADERS) -o main_$(VERSION)

Let's set an environment variable with a version and compile our application using Make:

Another way to set this variable is to pass it to Make as a command line argument. For example:

Here, the VERSION variable is passed after the make <target> command using the syntax <var name>=<value>

Remember earlier when I mentioned situations where you may want to force Make to recompile? If we change the value of the VERSION environment variable, we have to force a re-compilation. We can do this by cleaning away the object files using our clean target. Another option for forcing the HelloPrinter class to re-compile is to use the linux touch command on the helloprinter.cpp source file. This will tell Make that the file has been changed and force a re-compile.

touch helloprinter.cpp

We can see that the version number is captured from our environment variable (or argument). The version is passed to each compilation command, and the macro is expanded within the source code so the software prints the correct version number. Lastly, our application is named with the version included. In a real development environment, this type of automatic versioning is common with continuous integration systems or other build automation infrastructure.

Summary

Make has improved our workflow in multiple ways. Compilation is much simpler because the necessary files are managed for us, and our flags and options are organized in one place. Versioning and packaging our application for distribution was semi-automated in our make file. A developer can now make an update to the software and run a single command to build and package the application.

Next Steps

There are still plenty of topics and features of Make I didn't cover here. Moving forward I would like to discuss some more advanced features of Make in a future post. I also hope to cover tools that simplify and automate complex makefiles such as CMake. Lastly, I'd like to cover continuous integration and build automation using tools like Gitlab. Keep an eye out for those topics in future posts.

Get honeypotted? I like spam. Contact Us Contact Us Email Email ar.hp@outlook.com email: ar.hp@outlook.com