Compiler Warnings: Use Them, Don't Trust Them
Compiler warnings are useful, but they are highly unreliable. In addition, they are no substitute for language subsetting, as mandated by most functional safety standards and typically achieved by complying to the MISRA coding standards. Even compilers with MISRA-checking capabilities are normally unreliable as a consequence of how the analysis complexity/precision trade-off is dealt with.
Book a Free MISRA Strategy Call
Register for Next MISRA Training
Turning On All Warnings Is Definitely a Good Thing
Most compilers provide useful warning messages that inform about circumstances that may not correspond to the intentions of the programmer. In most environments, where code quality and low defect rates are important, a rule requiring the code to compile without warnings when all the compiler warnings are enabled is increasingly being enforced. The rationale is that most compiler warnings flag code that is truly problematic. When this is not the case, the compiler is confused by the flagged section of code because it is extremely convoluted, and, as such, there is a good chance it will also confuse fellow programmers: in both cases, the code should be modified.
Turning On All Warnings Is Not Enough
The fact that more and more people compile with all warnings enabled is definitely a good thing. However, a significant number of developers seem to assume that, because the compiler can detect some instances of issue X, the compiler must be able to detect all instances of issue X. Unfortunately, this is false more often than it is true. There are several intertwined reasons which compound to that effect; some are of technological nature and others are sociological.
Technological Reason: Compilers Issue Warnings Because They Can
C/C++ Compilers, at least since the DEC C compiler, have traditionally provided warnings because they have some of the information required to provide many of them. They build and visit abstract syntax trees and, for the purpose of optimization, they perform static analysis of the code. Thus, incorporating some diagnostic facilities into compilers as a byproduct of other activities is a natural thing to do.
Sociological Reason: Compilers Have To Be Fast
Compilers have to be fast, because people using them want them to be fast. This is only partly justifiable but it is a fact (too many developers recompile their code very often due to uncertainty on the language syntax and semantics; a deeper knowledge of the language would result in less recompilation and increased productivity). The compiler can be excused for higher compilation times only when optimizations are turned on. For this reason, optimizations are usually turned off during large parts of development (and, in some industry sectors, they are always turned off).
Technological Reason: Optimizations and Analyses Are Optimized
When optimizations are turned off, the corresponding static analyses are also turned off for the sake of compilation speed. Hence, when optimizations are turned off, the information needed to provide high quality warnings is in scarce supply. When optimizations are turned on, more static analyses are active and more information is available. However, compilation time is also an issue when optimizing code and this has two consequences:
optimization is geared towards high-reward opportunities which tend to occur often in real code;
static analyses that enable such optimizations are complexity-throttled in the same way; the analysis targets the very same opportunities and the algorithms employed are quick to give up when, based on heuristics, such opportunities are unlikely to be present.
In summary: turning optimization off means little information; turning optimization on means more information according to some heuristics whose objective is achieving a good complexity/precision trade-off from the point of view of optimization, not from the point of view of consistently generating high-quality warnings.
When information is insufficient, any algorithm for the generation of diagnostics will have false positives (warnings are given for instances that are not problematic), false negatives (warnings are not given for non-problematic instances), or both. Moreover, when information is insufficient (and excluding buggy algorithms), reduction of the number of false positives is demonstrably only possible by increasing the number of false negatives and the other way around.
Sociological Reason: False Positives Make for Angry Developers
Developers do not tolerate false positives. This is only partly justifiable, but it is a fact (developers should pay more attention to the consequences of false negatives). This is what happens as a result: a false positive in a project generates complaints, then compiler developers tweak the algorithm to suppress it, and, more often than not this results in false negatives. If you follow the development of GCC and CLANG (as the author has done since their inception) you will see this pattern at work (if you have a few hours to spare, you can see it yourself: "git clone https://github.com/llvm/llvm-project.git ; cd clang ; git log -p | less", then search for keywords like "suppress"). Or you can try some simple experiments: for example. you will see that warnings may be given or not given depending on factors that have nothing to do with the existence of a problem, whether part of the syntax involved comes from macro expansion or not, or whether the problematic construct is within a loop or not, and so on. An example is the following:
#define FORCED_STOP (0)
static int state[2][2];
static void foo(int action) {
const int stop = FORCED_STOP;
if (action == 2) {
for (;;) {
if (stop || !state[0]) {
break;
}
}
}
}
int main() {
foo(2);
return 0;
}
$ gcc-10 -O3 -W -Wall -Wunreachable-code r.c
$ clang-14 -O3 -W -Wall -Weverything -Wunreachable-code r.c
$ eclair_env -enable=MC3R1.R2.1 -enable=B.REPORT.TXT -- gcc r.c
r.c:10.9-10.13: violation for rule MC3R1.R2.1: (required) A project shall not contain unreachable code. Loc #1 [culprit: `break' statement is unreachable]
break;
<~~~>
$
GCC and CLANG report many instances of unreachable code but they fail to report many other instances (even though, in this specific case, their optimizers know what is going on: you will see it if you examine the produced assembly code). The same is true for many other warnings.
Conclusion
Quoting G. J. Holzmann, "The power of ten — Rules for developing safety critical code", IEEE Computer, 39(6):95--97, 2006:
All code must be compiled, from the first day of development, with all compiler warnings enabled at the compiler’s most pedantic setting. All code must compile with these setting without any warnings. [...] There simply is no excuse for any software development effort not to make use of this readily available technology.
Assuming compilers can consistently cover well-defined classes of potential defects is a big mistake: they simply have not been designed for this. Even compilers that claim to (partially) cover the MISRA standards do so with many false positives and tons of false negatives. In contrast, a well-designed static analysis tool will unambiguously define what it is checking (e.g., MISRA C:2012), the precision guarantees it provides for each rule (e.g., no false negatives and/or no false positives) and will be consistent with such definitions.
In conclusion, on the theme of warning messages, our advice to those who develop code with safety and/or security concerns is the following: enable all compiler warnings and avoid all of them. In addition, select a well-defined and well-maintained coding standard (i.e., one of the MISRA standards or BARR-C:2018) and a state-of-the-art static source code analyzer to properly support it; base systematic code reviews upon the findings of the tool, while making sure each report is carefully considered and dealt with by changing the code or by filing a deviation.
Roberto Bagnara, PhD, is co-founder of BUGSENG, software verification expert and evangelist, and professor of Computer Science at the University of Parma. He is also a member of the ISO/IEC JTC1/SC22/WG14 - C Standardization Working Group and of the MISRA C Working Group.
Patricia M. Hill, PhD, is co-founder and director of BUGSENG, software verification expert, and former senior researcher at the University of Leeds. She is also a member of the MISRA C++ Working Group.
Join our LinkedIn community to keep up to date with all our news.