kefir: Independent C17/C23 compiler

c1c07bf Implement in-place target IR instruction construction 23 hours ago 634d272 Add LTO build support to Makefile a day ago Kefir is an independent compiler for the C17/C23 programming language, developed by Jevgenij Protopopov. Kefir has been validated with a test suite of 100 software projects, among which are GNU core- and binutils, Curl, Nginx, OpenSSL, Perl, Postgresql, Tcl and many others. The compiler targets x86_64 architecture and System-V AMD64 ABI, supporting Linux, FreeBSD, NetBSD, OpenBSD and DragonflyBSD. The project intends to provide a well-rounded, compatible and compliant compiler, including SSA-based optimization pipeline, debug information generation, position-independent code support, and bit-identical bootstrap. Kefir integrates with the rest of system toolchain --- assembler, linker and shared library.#At a glanceKefir: Supports the C17 standard -- including complex and imaginary numbers, atomics, variable-length arrays, etc. (see Implementation quirks). Supports the C23 standard -- including bit-precise integers and _Decimal floating-point support (see Implementation quirks). Supports some of widespread GNU C built-ins, certain extensions, inline assembly, 128 bit integers. Is written in C11 -- runtime dependencies are limited to the standard library, bits of POSIX and the shell. Targets x86_64 and System-V ABI -- primarily Linux (both glibc and musl libc), secondarily FreeBSD, NetBSD, OpenBSD and DragonflyBSD (see Supported environments). Is extensively validated on real-world open source software test suites -- including dozens of well-known projects (see Testing and validation). Implements SSA-based optimization pipeline with two SSA phases -- primarily targetting local scalars: local variable promotion to registers, dead code elimination, constant folding, global value numbering, loop-invariant code motion, function inlining, tail-call optimization, but also providing conservative global memory access optimization, as well as target-specific optimizations (see Optimization and codegen). Supports DWARF5 debug information, position-independent code, AT&T and Intel syntaxes of GNU As and has limited support for Yasm.

Implements bit-identical bootstrap -- within fixed environment, Kefir produces identical copies of itself. Is able to generate freestanding assembly code -- with the exception for thread-local storage (might require external calls per ABI), _Decimal floating-point numbers, and atomic operations of non-platform native sizes, which require libatomic-compatible library. Provides cc-compatible command line interface. Is able to output internal representations (tokens, abstract syntax tree, intermediate representation) in machine-readable JSON form. Provides auditable logs and all build artifacts for pre-release testing of the most recent release. Licensed under GNU GPLv3 (only) terms for the compiler, and BSD-3 terms for runtime includes (see License). Is written by a single developer. Is named after fermented milk drink -- no other connotations are meant or intended. Important note: as the project is developed and maintained by a single person, unfunded and in spare time, the author warns that uses of Kefir in production settings might be undesirable due to insufficient level of support the author can provide.Important note #2: to the best of author's knowledge, all of the claims above are true (and many are reproducibly demonstrated by the test suite). Yet even with full rigour, many bugs, unintended omissions, inconsistencies and misunderstandings may slip through. The author intends to faithfully represent capabilities of the project, and is especially sensitive to any overstatements in this regard. If you have any doubts, objections or otherwise disagree with the above, please do not hesitate contact the author (see Author and contacts) -- corrections will be issued immediately after identifying the deficiency.#Installation and usageOn supported platforms, Kefir is built and tested as follows:make test all # Linux glibc make test all USE_SHARED=no CC=musl-gcc KEFIR_TEST_USE_MUSL=yes # Linux musl gmake test all CC=clang # FreeBSD gmake test all CC=clang AS=gas # OpenBSD gmake test all CC=gcc AS=gas # NetBSD gmake test all LD=/usr/local/bin/ld AS=/usr/local/bin/as # DragonflyBSD The installation is done via (g)make install prefix=.... The default prefix is /opt/kefir.Kefir build time dependencies are: C11 compiler -- tested with gcc and clang.

Bash GNU Make GNU Coreutils Groff m4 mandoc Kefir runtime dependencies are: The C standard library and POSIX Shell Furthermore, for correct end-to-end compilation, Kefir requires: External assembler -- full support for GNU As, limited support for Yasm External linker -- GNU ld External libc -- glibc or musl libc on Linux, system libc on *BSD systems. Note that this might be different from the libc Kefir itself is linked with. External startfiles -- crti.o, Scrt.o, etc. libatomic-compatible library (i.e. libatomic of gcc or compiler_rt of clang). libgcc in case decimal floating-number support is desired. Users can consult dist/Dockerfile* files that document the necessary environment for Ubuntu (base target), Fedora and Alpine, respectively, as well as dist/PKGBUILD for Arch Linux. For *BSD systems, consult respective .builds/*.yml files.Note: upon build, Kefir detects host system toolchain (assembler, linker, include and library paths) and configures itself respectively. Upon update of the toolchain, Kefir provides kefir-detect-host-env --environment command whose output shall be placed into $(prefix)/etc/kefir.local file.Note #2: aforementioned dependencies do not include optional development and full test suite dependencies. For these, please consult dist/Dockerfile dev and full targets.At the moment, Kefir is automatically tested in Ubuntu 24.04, FreeBSD 14.x, OpenBSD 7.7 and NetBSD 10.x environments; Arch Linux used as the primary development environment. DragonflyBSD support is tested manually prior to release.#Decimal floating-point supportKefir provides support for _Decimal floating-point numbers relying on libgcc arithmetic routines. In order to enable the support, Kefir shall be compiled directly or transitively (i.e. bootstrapped) by gcc host compiler. Decimal arithmetic code produced by Kefir requires linkage with libgcc; if conversion between bit-precise integers and decimal floating-point numbers is desired, libgcc of version 14 or newer is required.Both BID and DPD encodings are supported, BID being the default one.

To enable DPD, pass the following Make option when building Kefir: EXTRA_CFLAGS="-DKEFIR_PLATFORM_DECIMAL_DPD".Kefir can bootstrap libgcc version 4.7.4 automatically:make bootstrap_libgcc474 -j$(nproc) #LibatomicKefir can build required libatomic routines from compiler_rt project via:make build_libatomic -j$(nproc) #UsageKefir implements cc-compatible command line interface and therefore can be used as a near-drop-in replacement (see Implementation quirks) of cc in standard compilation pipelines:which kefir # Should output correct path to kefir after installation # Example usage kefir -O1 -g -fPIC -o hello_world ./hello_world.c ./hello_world Furthermore, kefir provides a manual page that documents command-line options and environment considerations:man kefir # Make sure that kefir installation directory is available to man kefir --help # Identical to the manual page contents #Portable KefirKefir provides scripts to build portable and standalone Kefir distribution package for Linux. The package includes statically-linked Kefir C compiler, musl libc, and selected tools from GNU Binutils. The package is intended to provide a minimalistic C development toolchain independent of host system tooling.make portable_bootstrap -j$(nproc) # Build artifact is located in bin/portable/kefir-portable-*.tar.gz #Supported environmentsKefir targets x86_64 instruction set architecture and System-V AMD64 ABI. Supported platforms include modern versions of Linux (glibc & musl libc), FreeBSD, OpenBSD, NetBSD and DragonflyBSD operating systems. A platform is considered supported if: Kefir can be built with system compiler and successfully executes own test suite (see Testing and validation). Kefir can be built with itself and successfully executes own test suite. Kefir passes c-testsuite and gcc-torture tests (see Testing and validation for the exact tests, conditions, etc). Kefir can compile Lua and run its base test suite. Kefir can perform reproducible (i.e. bit-identical bootstrap) of itself within fixed environment. To claim a platform supported, no other requirements are imposed.

Other tests and validations described in Testing and validation section are focused predominantly on Linux to ensure overall compilation process correctness. In general, there are very few differences between Linux and BSD system code generation, thus full testing and validation sequence shall suffice only on a single platform. Please note that libc header quirks are generally the main offender of compatibility, thus additional macro definitions or individual header overrides might be necessary. Musl libc provides the most smooth experience, however Kefir has accumulated sufficient support for GNU C extensions to use glibc and BSD libc implementations reasonably (consult Implementation quirks and the external test suite part of Testing and validation, as well as respective .build/*.yml files for platform of choice for detailed examples).As mentioned in the Installation section, Kefir detects system toolchain configuration on build and uses it later. The compiler also supports a set of environment variables that take precedence over the built-in configuration. Consult respective section of the manual page for details of supported environment variables.#Standard library considerationsOn Linux, Kefir works with both glibc and musl libc. Musl headers are more standards-compliant and generally provide smoother compatibility. glibc, by contrast, may introduce incompatibilities with non-mainstream compilers (see Implementation quirks).On FreeBSD, OpenBSD, and NetBSD, the system standard library can be used, though additional macro definitions (e.g. __GNUC__, __GNUC_MINOR__) may be required for successful builds.#Implementation quirksThe following details need to be taken into account: Kefir implementation of C23 standard provides the support for _Decimal floating-point numbers relying on libgcc routines for decimal arithmetics (see Installation and usage). The C23 standard mandates use of Unicode for char8_t, char16_t and char32_t types and literals. Kefir relies on the standard library wide character encoding facilities, and thus implements this requirement under condition that the system locale is Unicode-based. The author believes that this is a reasonable assumption. In general, the author has much higher confidence in compatibility with features of C17 and earlier versions. As of current version of external test suite (see Testing and validation), the absolute majority of third-party projects do not rely on any of C23 features, which makes external validation of C23 support much more limited.