Software gets bigger and bigger, this is certainly the case for Linux as well. In addition, Linux is configurable so not all code is built. Different kinds of developers are involved: casual contributors, maintainers and janitors.
Janitors clean up other people’s messes. They know coding style conventions and API changes. However, they don’t know the subsystem they affect deeply, and often they don’t have the possibilities to test well. There is a risk of a silent compiler failure: the janitor modifies some code, compilation succeeds, but the compiler actually didn’t build the modified code because it was configured out.
JMake handles the silent compiler failure to improve the reliability of janitor code. This grows out the coccinelle work: when doing this kind of tree-wide change, it is hard to make sure that you’re actually testing what you changed. People want to have immediate feedback of what they do, so an online tool that sends a mail is not appropriate. Even under allyesconfig, some parts are not built.
JMake looks at a diff and what gets built, and reports any modified line that did not get built. JMake can also find that the line can be built by compiling allyesconfig for a different architecture.
Tools available to JMake: make.cross and allyesconfig, in-tree defconfigs. Trying all that would take too long, so JMake has heuristics. For files in arch/, allyesconfig is used. For drivers etc. it is alsways x86/allyesconfig. If that fails, look in the Makefile if there is a CONFIG variable associated with the C file. A final heuristic is to use the same arch as for the other lines in the patch.
Extra challenge for .h file, because you don’t know where it will be used. Additional complication that a header file with the same name is often used for different arches. Also conditional complication is especially heavily used for .h files, i.e. not including unless some symbol is defined.
To find out which lines are actually compiled, you could look at the line numbers in the compiled code (.lst file) (but this doesn’t work for macros); introduce a syntax error in the modified line and check that an error is reported (but not reliable and compiler-specific); mutate source code and verify that the mutation is in the .i file (preprocessed code), and if yes also verify that the (unmodified) source file actually gets built. The mutation is done by adding a string (which never gets modified by preprocessor) surrounded by characters that are not valid in C. If the mutation is in a macro, it will end up on a different line but it will end up somewhere.
To run jmake, you give it a commit ID or range of commit IDs to look at. It then goes through the above process. It looks at changes in blocks that are not interrupted by #ifdef. It reports for each file how it managed to build the modification: “make” = x86/allyesconfig; “make.cross ARCH=…” = ARCH/allyesconfig; “make.cross ARCH=foo:bar_defconfig” = with a different defconfig than allyesconfig; Failure = needs to be looked at manually. For a commit that affects 83 files it took about 8.5 minutes on Julia’s laptop.
Julia ran it on 11K commits. 96% of the modified non-arch files are visible in x86/allyesconfig. 365 .c files and 75 .h files outside of arch are not visible in x86 but are in some other arch, typically arch. 415 of .c files doe compile but not all modified lines are compiled; 54 of these can be found in other arches, but 361 cases JMake fails.
- Config options that are never set.
- Changes that are done both in #ifdef and #else can never work.
- Changes in #ifdef MODULE because not testing modules.
Julia made an objective definition of what a janitor is based on some metrics about the type of commits people make. Basically a janitor makes changes in a lot of different files, in different subsystems. She detected 21 janitor commits that do not get built on x86/allyesconfig.
The tool works well when you’re reacting to dependencies, e.g. adding new arguments to a function. It does not work when you create dependencies, e.g. adding const to a declaration – the latter would need to build all the users of that function, but JMake doesn’t do that.