To understand how specs are found you have to look at the driver source code. Specifically the method driver::set_up_specs.

Near the beginning of the method you'll see that it looks for a file called specs in the startfile prefixes. You can find the startfile prefixes by running gcc -print-search-dirs and looking at the "libraries" entry. See the code in driver::maybe_print_and_exit for a reference.

The output of -print-search-dirs can be a hard to read (this is a theme of GCC output) so you can use the following to simplify it, preserving the search order.

gcc -print-search-dirs \
  | sed -n -e '/^libraries: =/ {s/.*: =//; s/:/\n/g; p}' \
  | xargs realpath -q \
  | awk '{if (!seen[$0]) print; seen[$0] = 1;}'

Once you have this list you might think that you can put a file named specs in one of the directories and it will override the built-in specs.

Is it really that simple? Of course not.

First of all, the list of startfile prefixes at the time this spec file is read is not the same as the one printed via -print-search-dirs. The directory that probably will work is going to be the same one that shows up in the "install" entry. You'll know you've found it when the initial output from -v or -### starts with "Reading specs from" instead of "Using built-in specs".

And more importantly, this file won't actually override the built-in specs!

When a spec file is read the driver eventually calls set_spec. In that function if the global array of specs is not initialized it will initialize it to the set of static built-in ones. So you can't actually override the built-in specs unless you override each entry.

This is a lot of work to do manually, so if you really want to change them all it's best to save the built-in ones and alter them.

You can always pass specs using the --specs option (these are "user specs"), but you may want to use the specially placed spec files if you have some site-wide change you want to make, or you need to set specs early and affect spec processing that happens before user specs are processed.

Any argument to the --specs option will be searched for in the library directories if the name of the file is not an absolute path. This means you can ship spec files with the compiler and use them by name instead of using full pathnames. If you have the Arm GCC cross compiler installed you probably use this feature when linking bare-metal applications. In that case you are likely passing --specs=nosys.specs to link with a dummy syscall library. So this mechanism does get used in actual compiler distributions.

It's also worth noting that you can't use -dumpspecs to see the specs that you've set. When dumping specs the driver always uses the built-in ones and exits before any spec files are read. If you want to view the specs that get set by any files you provide you have to build the driver with -DDEBUG_SPECS added to CXXFLAGS and use -v when running.

Wholesale replacement of the built-in specs is generally not a good idea. What you'll likely want to do is alter an existing spec by adding to the beginning or the end. There are standard ways to do this.

To add to the beginning you have to rename a spec. There is an example in the documentation. Here is another that will add --special-option before any other assembler arguments in the asm spec.

%rename asm old_asm

*asm:
--special-option %(old_asm)

To add to the end of a spec, use the plus sign (+) followed by a space. The documentation commits a sin of omission here because it says that the spec only has to start with the + symbol for it to be appended.

The example below would enable stack protector for every C file you compile.

*cc1:
+ -fstack-protector

Altering the full contents of a spec string without copying it to a file and modifying it requires messing around with spec functions and changing the source code. It's not something you'll want to do unless you have some really specific requirements. (And if you think you really need to do this, I recommend you keep thinking.)

Enterprising readers may have noticed that you can override the specs for a suffix, which means you can change the spec used for file types. If you want to do this effectively you need to study the default file type specs used by the driver. Those are found in the array default_compilers.

To get your imagination going, here's how you can change the assembler to run a different assembler. The conditionals that would prevent the assembler from running have been omitted for clarity.

.s:
custom-as %(asm_debug) %(asm_options) %i %A 

If you do this you may notice that compiling a .c file does not result in your custom assembler being used. This is because spec processing doesn't actually go in "compiler-assembler-linker" order; it goes in "compiler-linker" order. It is up to the spec for the language to invoke a series of commands to create an object file for the linker. This is what the driver considers to be a "compiler". The assembler spec we have defined above is the compiler for files given to the driver that end with .s.

If you look at the spec for the .c files in the driver source it will be associated with the @c entry which is the actual spec used. Reading it will eventually reveal it uses the invoke_asm spec to call the assembler. Doing a full override of specs is tough: there are many cases to consider!

(Side note: you can override @ entries. They are considered to be special suffixes. The above example could be written with @assembler: as the first line. In this case it would work for any file suffix that is aliased to the assembler.)

There is an interesting entry in the -dumpspecs output that will almost certainly be empty in your installation: self_spec.

What is interesting about the self spec is that it is applied to the arguments for the driver. This means you can change the driver arguments from within the driver itself.

The driver has built-in self specs that can only be altered by rebuilding from source. They are defined by the target backend. However, you can define user self specs which are applied after all other specs have been read.

Use of self specs tend to be target-specific or needed for highly customized situations. Anything added via a self spec has to be an argument that starts with a hyphen (-).

For an illustrative—but not really motivational—example, suppose we changed the driver options such that all instances of -m32 are removed and we always add -m64 to the arguments. It can be done this way.

*self_spec:
%<m32 -m64

Changing the driver arguments affects all subprocesses and must be done with care. You can cause a lot of trouble with this; it's much like eval in dynamic languages.

When the driver runs a subprocess the environment variable COLLECT_GCC_OPTIONS contains all the driver arguments after all the self specs have been processed. When running with verbose output use it to debug any use of self specs.

In the previous article I said this:

All directives, once substituted, have spaces around them so you can't piece together multiple arguments into one argument.

That's not entirely true. You can concatenate multiple arguments into a single one, but it will not always work as expected. (There were multiple counterexamples in the previous article, if you looked closely.)

The official documentation says this about directives.

Note that spaces are not generated automatically around the results of expanding these sequences. Therefore you can concatenate them together or combine them with constant text in a single argument.

Let's test that out using the following spec file, assuming I have a file test.sh in the current directory that only prints its arguments.

.xx:
./test.sh /%{O1:1/opt}/%{g3:3/debug}

Trying this out, I get

% gcc --specs=local.specs -c t.xx -O1 -g3
/1/opt/3/debug

Great! But this is specific to the -O and -g levels. Let's say I want to generalize it and base if off the values passed to those options.

.xx:
./test.sh /%{O*:%*/opt}/%{g*:%*/debug}

Now I get something different.

% gcc --specs=local.specs -c t.xx -O1 -g3
/1/opt /3/debug

The %* substitution will cause a space to be added because of the code in give_switch.

While it is true that directives are substituted without adding a space (notwithstanding the example above), a spec could have leading or trailing spaces. Consider the following example.

# There is a space after the word "middle"
*stuff:
%{dummy} middle 

.xx:
./test.sh BEGIN%(stuff)END

% gcc --specs=local.specs -c t.xx
BEGIN middle END

In this case %{dummy} expands to the empty string which is followed by a space, and the spec for stuff ends in a space. (Note that leading spaces in a spec are skipped when it is read but not when it is processed.) Sure, the directives are concatenated. But that does not mean that putting two specs directly beside each other will result in a contiguous string with no space separating them. This means if you are trying to create pathnames using substitutions, for example, it can be tricky.

Spec functions are a generic mechanism for providing specs. They must be defined in source code. They deserve their own article so I will not be covering them much here.

Spec functions can be used in two ways: for direct substitution or conditional substitution. Most spec functions only make sense in one of those contexts. In a conditional context, if a spec function returns NULL it is considered false. If it returns any string (even empty) it is considered true.

Target backends may define their own spec functions but these will not be in the official documentation. You have to go into the source to find them.

Here's an example that adds the -g3 argument for cc1 if the last -O argument is -O2 or higher.

*cc1:
%{%:gt(%{O*:%*;:0} 1):-g3}

The semantics of spec functions are where specs get really complicated and bizarre. Stay tuned for another article that will delve into that mess.

Have you ever passed -v to the driver and studied the output? There's a lot of stuff in there that is revealing. It's also a good way to understand what specs are and why they exist.

Because the output from -v is so extensive I'm going to use one example, broken up into parts. Not all of it is relevant for understanding specs, but it may be of general interest to some.

To get the output, I ran the following command on an amd64 machine running Debian 12, which is using GCC 12.2.

gcc -v t.c

The content of the source file is not important, only that the program is in a single file. (Use int main (void) {} if you have nothing sitting around.) I've reformatted the output where needed. In some cases I've omitted output and indicate it with (...).

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/12/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v (...)
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (Debian 12.2.0-14) 
COLLECT_GCC_OPTIONS='-v' '-mtune=generic' '-march=x86-64' '-dumpdir' 'a-'

The first line says where the driver is getting its specs. It's possible to override the builtin specs but that is for a later discussion.

What follows are some environment variables that are set by gcc to communicate with the subprograms and various other information. There is also the "Configured with" line that tells you how GCC was configured, which is useful if you want to build it yourself.

In our example the driver has to compile the source using the C compiler, assemble it into an object file, then link the object into an executable. The first subprogram run is the compiler. cc1 is the C compiler, cc1plus is the C++ compiler. Note that subprogram invocations in the output are lines that start with a single space.

 /usr/lib/gcc/x86_64-linux-gnu/12/cc1 \
   -quiet \
   -v \
   -imultiarch x86_64-linux-gnu \
   t.c \
   -quiet \
   -dumpdir a- \
   -dumpbase t.c \
   -dumpbase-ext .c \
   -mtune=generic \
   -march=x86-64 \
   -version \
   -fasynchronous-unwind-tables \
   -o /tmp/ccJOdUuR.s
(...)

We only provided the arguments -v and t.c to the driver yet the compiler has many more. Some are even provided twice. If you look at the documentation for options you won't find the "dump" ones listed, for example, so the compiler doesn't use the same options as the driver. That said, some of them match. -fasynchronous-unwind-tables is a code generation option helpful when debugging if the target machine supports it. -march and -mtune control what kind of code is generated for the processor. These are options specific to the target or host. It would be tedious to specify them every time.

I've trimmed all the compiler output because it's mostly version info, although it does print the C standard it is using, some of the compiler's heuristic data, and the compiler executable checksum. It will also print the search order for headers which can be very handy when debugging header problems or just knowing where system headers reside.

As a side note, it's worth pointing out that you can run the compiler directly if you want. You can even pass it --help to see the extensive set of options it accepts. Running the compiler directly isn't something you need to do very often, even when debugging it, because gcc has a -wrapper option to do this for you.

The compiler outputs assembler source, so the next step is for the driver is to run the assembler.

 as -v --64 -o /tmp/ccj7Fe2p.o /tmp/ccJOdUuR.s
(...)

Note that the input is a temporary file, as is the output. The driver has to manage all these temporaries and coordinate them across the subprograms. If you pass -save-temps then it will change the way this is done and use more intuitive names.

Finally, we have the linker ^{invocation, complete with the messiest and most complex set of arguments.}

/usr/lib/gcc/x86_64-linux-gnu/12/collect2 \
   -plugin /usr/lib/gcc/x86_64-linux-gnu/12/liblto_plugin.so \
   -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/12/lto-wrapper \
   -plugin-opt=-fresolution=/tmp/cc7cMZRs.res \
   -plugin-opt=-pass-through=-lgcc \
   -plugin-opt=-pass-through=-lgcc_s \
   -plugin-opt=-pass-through=-lc \
   -plugin-opt=-pass-through=-lgcc \
   -plugin-opt=-pass-through=-lgcc_s \
   --build-id --eh-frame-hdr -m elf_x86_64 \
   --hash-style=gnu --as-needed \
   -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie \
   /usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/Scrt1.o \
   /usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/crti.o \
   /usr/lib/gcc/x86_64-linux-gnu/12/crtbeginS.o \
   -L/usr/lib/gcc/x86_64-linux-gnu/12 \
   -L/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu \
   -L/usr/lib/gcc/x86_64-linux-gnu/12/../../../../lib \
   -L/lib/x86_64-linux-gnu \
   -L/lib/../lib \
   -L/usr/lib/x86_64-linux-gnu \
   -L/usr/lib/../lib \
   -L/usr/lib/gcc/x86_64-linux-gnu/12/../../.. \
   /tmp/ccj7Fe2p.o \
   -lgcc \
   --push-state --as-needed -lgcc_s --pop-state \
   -lc -lgcc \
   --push-state --as-needed -lgcc_s --pop-state \
   /usr/lib/gcc/x86_64-linux-gnu/12/crtendS.o \
   /usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/crtn.o

What the options mean is beyond the scope of this article, but it's worth noting what arguments are needed. There are object files (the .o suffix) and libraries (-l options) and library search paths (-L options). The order of these options matters, too, and the driver has to get this right. Furthermore, which system object files to use (found in /usr/lib in this example) can be non-obvious, so we should appreciate all the work the driver does behind the scenes. Manually specifying a link is tedious.

To round out the discussion of the files that the driver has to manage, all the temporary files created during the compile/assemble/link sequence are deleted before the driver exits.

All the arguments passed to the subprograms and the management of files are done with specs.

The driver determines what subprograms to run based on the arguments it is given. For example, if you give it a file name with the .c suffix it will know to run the C compiler for that file; give it a file with a .cpp suffix and it will use the C++ compiler. It recognizes many file extensions and you can specify the language it should use with the -x option. Some options tell it not to run the assembler or the linker.

The driver determines how to run a subprogram based on a spec for the language. Each language has its own spec that has the name of the subprogram and the arguments to pass to it. A spec is a string made up of zero or more lines. Each line corresponds to a program to run so it's possible for you to run more than one program for each language or "stage" being handled.

Since the arguments needed for a subprogram can vary significantly from one invocation to the next they are rarely given directly. Instead, they are provided with directives that expand to arguments based on what the user supplied. The syntax of these directives is inspired by the ones used for printf, albeit with very different semantics.

This may sound somewhat elegant but let me dissuade you of any notion of elegance right now. We are about to wade into an ad hoc mini-language where the only reliable way to get the result you want is to experiment to find out what happens, although admittedly the lack of consistent (and documented) semantics only tends to show up when you start doing the advanced stuff.

We're going to focus on the assembler invocation since it is the subprogram with the fewest arguments and is the easiest to follow. The goal is to trace through how it gets its arguments.

To do this you have to learn how to read a spec. This is accomplished by looking at the spec documentation and casting your eyes on arguably the ugliest output GCC can produce: gcc -dumpspecs.

The full output of -dumpspecs is a beast, so here's a snippet of the dumped specs that provides one of the above options. I've reformatted the output slightly to put each directive from the spec string on its own line so it's easier to read, but keep in mind that you're not going to get output this agreeable most of the time.

*asm:
%{m16|m32:--32} \
  %{m16|m32|mx32:;:--64} \
  %{mx32:--x32} \
  %{msse2avx:%{!mavx:-msse2avx}}

The *asm: line declares this is a named spec with the name asm. This happens to be the builtin spec name for options passed to the assembler. Confusingly, this is not how all the options are passed, but we'll get to that shortly.

Let's look closely at the following directive, which is how the --64 option is passed to the assembler. It's one pattern you're going to see a lot.

%{m16|m32|mx32:;:--64}

This says that if any of the arguments -m16, -m32, or -mx32 has been passed to the driver then substitute nothing, otherwise substitute in the string --64. Let's go over it in more detail.

The general form is %{S:X;:D}. This says that if the argument -S has been given then substitute X, otherwise substitute D. (Clearly, S can also be a combination of arguments separated with the pipe symbol |.) This is a conditional and is the primary way you control the way arguments are passed.

In the above case X is empty which is why it looks so confusing. The first : indicates the end of the condition. The ; indicates the end of the substitution. Since there is nothing there, nothing (well, the empty string) is substituted. The last : indicates the start of the last substitution. You can add white space to make it more readable, but it may or may not help. You can nest them and when they nest deeply (which they will), they can be very difficult to follow.

Our example at the beginning only passed the arguments -v and t.c. Since none of -m16, -m32, or -mx32 was passed, --64 is passed to the assembler.

If you look at the other directives in the asm spec you'll see other forms of conditionals.

• %{m16|m32:--32} means that if either -m16 or -m32 was passed, substitute --32. Otherwise, substitute nothing. This is a conditional with only one clause and is very common. You could write it as %{m16|m32:--32;:} but that adds needless line noise so it's better not to.
• %{mx32:--x32} should be clear by now: substitute --x32 if -mx32 was given.
• %{msse2avx:%{!mavx:-msse2avx}} is interesting because it is how you do conjunctions. The other examples we've looked at were disjunctions. Conjunctions are done by nesting. This is where things can get really hairy. This one says that if -msse2avx was given and not -mavx then substitute -msse2avx. (You may think "oh, conjunctions aren't too bad". I invite you to take a look at the link_command spec. Come back and we can share a story or two.)

There are few other points to note about these directives.

• The options in the condition are written without leading hyphens. This is a good thing because adding them would make reading a spec even harder. Internally, the driver has already decoded the options when matching happens so the hyphens are gone.
• The substitution doesn't have to be an option, it can be any string. Usually you want it to be an option, but it can also be another directive. If it is another directive, it will be processed as one.
• You have negation with the ! syntax. There are other "operators" but they are not common.
• You can substitute an option directly if it was given, or nothing if it wasn't, by writing %{S} where S is the option. This is a simple short form for %{S:-S;:}.
• The full conditional syntax allows for any number of clauses. For example, you could write %{m16|m32|mx32:;:--64} as %{m16:; m32:; mx32:; :--64}. I call this form the "maximal confusion" form because if it's used it's rarely short and always difficult to read.
• There doesn't seem to be a way to match option patterns or option arguments. There is a way, but we haven't gotten there yet.

We've covered all this ground and so far we've only explained one argument. And it seems to get there by magic. The documentation says that the asm spec specifies the options to pass to the assembler. Clearly there are more than just the one. What gives?

What gives is that the documentation does not tell the whole story. The full spec for the assembler is actually in the source code. It's not large so I'll reproduce it here.

%{!M:%{!MM:%{!E:%{!S:as %(asm_debug) %(asm_options) %i %A }}}}

Here you can see the messiness of specs starting to show up. If we strip away the conditionals ^{we get something more manageable.}

as %(asm_debug) %(asm_options) %i %A

The %(asm_debug) and %(asm_options) directives reference named specs. They substitute in the value of that spec for further processing. If you look at the output of -dumpspecs you should find both asm_debug and asm_options. On my system they are long and mostly of no consequence so I won't completely reproduce them here, but we can go through the ones that actually produce the options. Then we can look at a few other notable ones.

asm_debug doesn't substitute anything in our running example, so let's turn our attention to asm_options.

Here's an abbreviated version of the spec. Parts that are omitted are denoted with (...). You should see something similar if you are using a recent version of GCC.

*asm_options:
(...) %{v} (...) %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O}

Not all of these substitute anything, but they are notable so I have included them.

To see how we go from spec to assembler invocation, I will show the conceptual version of the full assembler spec at this point with the final assembler invocation. The conceptual version bypasses asm_debug since it substitutes nothing and substitutes in the abbreviated version of the asm_options spec.

as %{v} %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O} %i %A

as -v --64 -o /tmp/ccj7Fe2p.o /tmp/ccJOdUuR.s

We will go through each directive and show what happens.

• %{v} should be clear from earlier discussion: substitute -v because -v was passed.
• %a and %Y are special builtin specs. %a substitutes the asm spec. (Why have a special one and not just say %(asm)? I don't know.) This is how we get the --64 option as described in the previous section. %Y is used to substitute any assembler arguments that are passed via -Wa, which is the way you tell the driver that an argument is meant specifically for the assembler. It's how you pass through an argument without the driver validating it. We did not pass any of those so it substitutes nothing.
• The next directive is conditional and only substitutes if -c was passed to the driver. It substitutes nothing in our example. I included it, however, because it is written directly beside the next spec. This may seem significant but it actually isn't. All directives, once substituted, have spaces around them so you can't piece together multiple arguments into one argument. Why was this not written as a conditional with an else clause? Probably as a matter of style.
• The directive %{!c:-o %d%w%u%O} is going to give us the arguments for the output file, but it looks like random gibberish so take it one step at a time and look them up in the documentation.
  • %d marks the argument as a temporary file which will be deleted when the driver is done. It substitutes nothing.
  • %w marks the argument as the designated output file and also substitutes nothing.
  • %u%O substitutes the file name. %u generates and substitutes the temporary file name with the suffix substituted by %O.
• After the directives that give the output we have %i which substitutes the input file. This is the assembler file generated by the compiler.
• Finally, %A substitutes the asm_final spec. This lets you run some post processing on the assembler. It does nothing in our case, but if you're using the -gsplit-dwarf option with a recent version of GCC you might want to check it out.

That's it! That's the (mostly) complete trace of how the driver determines how to call the assembler.

At this point we've traced through how the assembler got its arguments from the driver in some detail—and there is plenty more that we could cover. I'm going to do that in a later article, but in the meantime I can show you the easiest way to experiment with specs that doesn't involve writing them or changing the source.

If you pass the option -### to the driver it will print all the commands it would run without actually running them. Using this "dry run" mode you can see how options are substituted without having to bother with making the subprogram actually accept them.

Earlier in the asm_options spec I omitted an interesting directive: %{I*}. This is a spec that matches any option that starts with -I and its option. In this case it will substitute all the -I options that were passed. For example, if we had passed -I dir1 and -Ipath/to/headers to the driver, both of them would be substituted in for %{I*}. You can see this by running

gcc '-###' -I dir1 -Ipath/to/headers t.c

and looking for the assembler command. The -### command is quoted to prevent the shell from possibly interpreting it as a comment or a pattern.

% gcc '-###' t.c -I dir -Ipath/to/headers 2>&1 | grep '^ as'
 as -I dir -I path/to/headers --64 -o /tmp/ccNz0vob.o /tmp/ccVtbBuj.s

If you want to explore specs a bit more and don't want to start changing the compiler, a good way is to use a custom spec that will show you what gets substituted by defining a new file type.

You can do this easily with the following spec file.

.xx:
./test.sh %i

This will register a new file extension .xx and run the compiler ./test.sh passing it the input file as the argument. All test.sh has to do is echo its arguments.

You can run the driver like this to use your custom specs and "compiler".

gcc -c -specs=custom.specs file.xx

You want to pass -c so that the driver does not try to run any other subprograms. Change the custom spec to your liking. Here's an example that will transform all options that start with -m to ones that start with -k. See the documentation for more details.

.xx:
./test.sh %{m*:-k%*}

If you pass -m32 -march=blah it will be passed to your "compiler" as -k32 -karch=blah.

The GCC driver coordinates and runs multiple subprograms. To manage how it specifies all the arguments to these programs it uses something called specs. These are strings with printf-like directives that are processed and substituted with values based on the arguments passed to the driver and various contextual information.

You can view most, but not all, of the builtin specs to the driver by running gcc -dumpspecs. To get the full spec for a subprogram you need to look in the driver source code.

Once you have these specs you can trace the logic behind an option, but it is tedious work. Specs are not known for their readability. And it's pretty uncommon for anyone to have to debug the driver.

Nevertheless, specs can help you understand what the driver is doing. It's one of those odd mysteries of a common tool that aren't really explained anywhere.

In a future article I'll show some of the more esoteric parts of spec usage, how to use specs to affect the driver itself, and the idiosyncrasies of writing spec functions.

(Thanks to Matt L. Holmes for reviewing this.)