By Ted Pudlik
Published 2024-05-31
Let’s say you’re migrating your build system to Bazel. Your project heavilyrelies on preprocessor defines to configure its code.
-DBUILD_FEATURE_CPU_PROFILE-DBUILD_FEATURE_HEAP_PROFILE-DBUILD_FEATURE_HW_SHA256
In your source files, you use these preprocessor variables to conditionallycompile some sections, via #ifdef. When building the same code fordifferent final product configurations, you want to set different defines.
How do you model this in Bazel?
This post discusses three possible approaches:
More power with no code changes: Platform-based Skylib flags
Error-preventing approach: Chromium-style build flag pattern
Which one to choose? If you have the freedom to refactor your code to use theChromium pattern, give it a try! It is not difficult to maintain onceimplemented, and can prevent real production issues by detecting typos.
Let’s start with the simplest approach: you can put the compiler options intoyour bazelrc configuration file.
# .bazelrccommon:mydevice_evt1 --copts=-DBUILD_FEATURE_CPU_PROFILEcommon:mydevice_evt1 --copts=-DBUILD_FEATURE_HEAP_PROFILEcommon:mydevice_evt1 --copts=-DBUILD_FEATURE_HW_SHA256# and so on
Then, when you build your application, the defines will all be applied:
bazel build --config=mydevice_evt1 //src:application
Configs are expanded recursively, allowing you to group options together andreuse them:
# .bazelrccommon:full_profile --copts=-DBUILD_FEATURE_CPU_PROFILEcommon:full_profile --copts=-DBUILD_FEATURE_HEAP_PROFILE# When building for mydevice_evt1, use full_profile.common:mydevice_evt1 --config=full_profile# When building for mydevice_evt2, additionally enable HW_SHA256.common:mydevice_evt2 --config=full_profilecommon:mydevice_evt1 --copts=-DBUILD_FEATURE_HW_SHA256
While it is simple, the config-with-copts approach has a few downsides.
If you misspell BUILDDD_FEATURE_CPU_PROFILE [sic!] in your .bazelrc,the actual BUILD_FEATURE_CPU_PROFILE variable will take the default valueof 0. So, although youintended to enable this feature, it will just remain disabled!
This isn’t just a Bazel problem, but a general issue with the simpleBUILD_FEATURE macro pattern. If you misspell BUILD_FEATUER_CPU_PROFILE[sic!] in your C++ file, you’ll get it to evaluate to 0 in any build system!
One way to avoid this issue is to use the “Chromium-style” build flagpattern, BUILDFLAG(CPU_PROFILE). If youdo, a misspelled or missing define becomes a compiler error. However, theconfig-with-copts approach is a little too simple to express this pattern,which requires code generation of the build flag headers.
Bazel allows you to perform multi-platform builds. For example, in a singleBazel invocation you can build a Python flasher program (that will run on yourlaptop) which embeds as a data dependency the microcontroller firmware to flash(that will run on the microcontroller). We do this in Pigweed’s ownexamples.
Unfortunately, the config-with-copts pattern doesn’t work nicely withmulti-platform build primitives. (Technically, the problem is that Bazeldoesn’t support transitioning on –config.) If youwant a multi-platform build, you need some more sophisticated idiom.
This approach doesn’t introduce any variable that can be used within the buildsystem to e.g. conditionally select different source files for a library,choose a different library as a dependency, or remove some targets from thebuild altogether. We’re really just setting preprocessor defines here.
The .bazelrc files are not automatically inherited when another repodepends on yours. They can be imported, but it’s an all-or-nothingaffair.
Let’s address some shortcomings of the approach above by representing the buildfeatures as Skylib flagsand grouping them through platform-based flags.(Important note: this feature is still under development! See the Appendix for workarounds for older Bazel versions.)
The platform sets a bunch of flags:
# //platform/BUILD.bazel# The platform definitionplatform( name = "mydevice_evt1", flags = [ "--//build/feature:cpu_profile=true", "--//build/feature:heap_profile=true", "--//build/feature:hw_sha256=true", ],)
The flags have corresponding code-generated C++ libraries:
# //build/feature/BUILD.bazelload("@bazel_skylib//rules:common_settings.bzl", "bool_flag")# I'll show one possible implementation of feature_cc_library later.load("//:feature_cc_library.bzl", "feature_cc_library")# This is a boolean flag, but there's support for int- and string-valued# flags, too.bool_flag( name = "cpu_profile", build_setting_default = False,)# This is a custom rule that generates a cc_library target that exposes# a header "cpu_profile.h", the contents of which are either,## BUILD_FEATURE_CPU_PROFILE=1## or,## BUILD_FEATURE_CPU_PROFILE=0## depending on the value of the cpu_profile bool_flag. This "code# generation" is so simple that it can actually be done in pure Starlark;# see below.feature_cc_library( name = "cpu_profile_cc", flag = ":cpu_profile",)# Analogous library that exposes the constant in Python.feature_py_library( name = "cpu_profile_py", flag = ":cpu_profile",)# And in Rust, why not?feature_rs_library( name = "cpu_profile_rs", flag = ":cpu_profile",)bool_flag( name = "heap_profile", build_setting_default = False,)feature_cc_library( name = "heap_profile_cc", flag = ":heap_profile",)bool_flag( name = "hw_sha256", build_setting_default = False,)feature_cc_library( name = "hw_sha256_cc", flag = ":hw_sha256",)
C++ libraries that want to access the variable needs to depend on thecpu_profile_cc (or heap_profile_cc, hw_sha256_cc) library.
Here’s one possible implementation of feature_cc_library:
# feature_cc_library.bzlload("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo")def feature_cc_library(name, build_setting): hdrs_name = name + ".hdr" flag_header_file( name = hdrs_name, build_setting = build_setting, ) native.cc_library( name = name, hdrs = [":" + hdrs_name], )def _impl(ctx): out = ctx.actions.declare_file(ctx.attr.build_setting.label.name + ".h") # Convert boolean flags to canonical integer values. value = ctx.attr.build_setting[BuildSettingInfo].value if type(value) == type(True): if value: value = 1 else: value = 0 ctx.actions.write( output = out, content = r"""#pragma once#define {}={}""".format(ctx.attr.build_setting.label.name.upper(), value), ) return [DefaultInfo(files = depset([out]))]flag_header_file = rule( implementation = _impl, attrs = { "build_setting": attr.label( doc = "Build setting (flag) to construct the header from.", mandatory = True, ), },)
A neat feature of the simple config-based approach was that configs could becomposed through recursive expansion. Fortunately, platforms can be composed,too! There are two mechanisms for doing so:
Use platforms’ support for inheritance.This allows “subplatforms” to override entries from “superplatforms”. But,only single inheritance is supported (each platform has at most one parent).
The other approach is to compose lists of flags directly, through concatenation:
FEATURES_CORTEX_M7 = [ "--//build/feature:some_feature",]FEATURES_MYDEVICE_EVT1 = FEATURES_CORTEX_M7 + [ "--//build/feature:some_other_feature",]platform( name = "mydevice_evt1", flags = FEATURES_MYDEVICE_EVT1,)
Concatenation doesn’t allow overriding entries, but frees you from thesingle-parent limitation of inheritance.
Tip
This approach can also be used to define custom host platforms:HOST_CONSTRAINTS in @local_config_platform//:constraints.bzlcontains the autodetected @platform//os and @platforms//cpuconstraints set by Bazel’s default host platform.
How do you actually associate the platform with a binary you want to build? Oneapproach is to just specify the platform on the command-line when building acc_binary:
bazel build --platforms=//platform:mydevice_evt1 //src:main
But another approach is to leverage multi-platform build, throughplatform_data:
# //src/BUILD.bazelload("@rules_platform//platform_data:defs.bzl", "platform_data")cc_binary(name = "main")platform_data( name = "main_mydevice_evt1", target = ":main", platform = "//platform:mydevice_evt1",)
Then you can keep your command-line simple:
bazel build //src:main_mydevice_evt1
You can make various features of the build conditional on the value of theflag. For example, you can select different dependencies:
# //build/feature/BUILD.bazelconfig_setting( name = "hw_sha256=true", flag_values = { ":hw_sha256": "true", },)# //src/BUILD.bazelcc_library( name = "my_library", deps = [ "//some/unconditional:dep", ] + select({ "//build/feature:hw_sha256=true": ["//extra/dep/for/hw_sha256:only"], "//conditions:default": [],})
Any Bazel rule attribute described as “configurable” can take a value thatdepends on the flag in this way. Library header lists and source lists arecommon examples, but the vast majority of attributes in Bazel are configurable.
If you used “Chromium-style” build flagsyou would be immune to dangerous typos when using this Bazel pattern. Butuntil then, you still have this problem, and actually it got worse!
If you forget to #include "build/features/hw_sha256.h" in the C++ file thatreferences the preprocessor variable, the build system or compiler will stillnot yell at you. Instead, the BUILD_FEATURE_HA_SHA256 variable will takethe default value of 0.
This is similar to the typo problem with the config approach, but worse, because it’s easier to missan #include than to misspell a name, and you’ll need to add these#include statements in many places.
One way to mitigate this problem is to make the individualfeature_cc_library targets private, and gather them into one big librarythat all targets will depend on:
feature_cc_library( name = "cpu_profile_cc", flag = ":cpu_profile", visibility = ["//visibility:private"],)feature_cc_library( name = "heap_profile_cc", flag = ":heap_profile", visibility = ["//visibility:private"],)feature_cc_library( name = "hw_sha256_cc", flag = ":hw_sha256", visibility = ["//visibility:private"],)# Code-generated cc_library that #includes all the individual# feature_cc_library headers.all_features_cc_library( name = "all_features", deps = [ ":cpu_profile_cc", ":heap_profile_cc", ":hw_sha256_cc", # … and many more. ], visibility = ["//visibility:public"],)
However, a more satisfactory solution is to adopt Chromium-style buildflags, which we discuss next.
The Skylib bool_flag that represents the build flag within Bazel has abuild_setting_default attribute. This attribute is mandatory.
This may be a disappointment if you were hoping to provide no default, and haveBazel return errors if no value is explicitly set for a flag (either via aplatform, through .bazelrc, or on the command line). The Skylib build flagsdon’t support this.
The danger here is that the default value may be unsafe, and you forget tooverride it when adding a new platform (or for some existing platform, whenadding a new flag).
There is an alternative pattern that allows you to define default-less buildflags: instead of representing build flags as Skylib flags, you can representthem as constraint_setting objects. I won’t spell this pattern out inthis blog post, but it comes with its own drawbacks:
The custom code-generation rules are more complex, and need to parse theconstraint_value names to infer the build flag values.
All supported flag values must be explicitly enumerated in the BUILDfiles, and the code-generation rules need explicit dependencies on them.This leads to substantially more verbose BUILD files.
On the whole, I’d recommend sticking with the Skylib flags!
This pattern builds on More power with no code changes: Platform-based Skylib flags byadding a macro helper for retrieving flag values that guards against typos. TheBUILD.bazel files look exactly the same as in the previous section, but:
Users of flags access them in C++ files via BUILDFLAG(SOME_NAME).
The code generated by feature_cc_library is a little more elaborate thana plain SOME_NAME=1 or SOME_NAME=0, and it includes a dependency onthe Chromium build flag header.
Here’s the feature_cc_library implementation:
load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo")def feature_cc_library(name, build_setting): """Generates a cc_library from a common build setting. The generated cc_library exposes a header [build_setting.name].h that defines a corresponding build flag. Example: feature_cc_library( name = "evt1_cc", build_setting = ":evt1", ) * This target is a cc_library that exposes a header you can include via #include "build/flags/evt1.h". * That header defines a build flag you can access in your code through BUILDFLAGS(EVT1). * If you wish to use the build flag from a cc_library, add the target evt1_cc to your cc_library's deps. Args: name: Name for the generated cc_library. build_setting: One of the Skylib "common settings": bool_flag, int_flag, string_flag, etc. See https://github.com/bazelbuild/bazel-skylib/blob/main/docs/common_settings_doc.md """ hdrs_name = name + ".hdr" flag_header_file( name = hdrs_name, build_setting = build_setting, ) native.cc_library( name = name, hdrs = [":" + hdrs_name], # //:buildflag is a cc_library containing the # Chromium build flag header. deps = ["//:buildflag"], )def _impl(ctx): out = ctx.actions.declare_file(ctx.attr.build_setting.label.name + ".h") # Convert boolean flags to canonical integer values. value = ctx.attr.build_setting[BuildSettingInfo].value if type(value) == type(True): if value: value = 1 else: value = 0 ctx.actions.write( output = out, content = r"""#pragma once#include "buildflag.h"#define BUILDFLAG_INTERNAL_{}() ({})""".format(ctx.attr.build_setting.label.name.upper(), value), ) return [DefaultInfo(files = depset([out]))]flag_header_file = rule( implementation = _impl, attrs = { "build_setting": attr.label( doc = "Build setting (flag) to construct the header from.", mandatory = True, ), },)
If you have the freedom to refactor your code to use the Chromium pattern,Bazel provides safe and convenient idioms for expressing configuration throughbuild flags. Give it a try!
Otherwise, you can still use platform-based Skylib flags, but beware typos andmissing #include statements!
A couple “deep in the weeds” questions came up while this blog post was beingreviewed. I thought they were interesting enough to discuss here, for theinterested reader!
If you made it this far you might be wondering, why is the code listing forfeature_cc_library even here? Why isn’t it just part of Pigweed, and usedin our own codebase?
The short answer is that Pigweed is middleware supporting multiple buildsystems, so we don’t want to rely on the build system to generate configurationheaders.
But the longer answer has to do with how this blog post came about. Some timeago, I was migrating team A’s build from CMake to Bazel. They used Chromiumbuild flags, but in CMake, so to do a build migration they needed Bazel supportfor this pattern. So I put an implementation together. I wrote a designdocument, but it had confidential details and was not widely shared.
Then team B comes along and says, “we tried migrating to Bazel but couldn’tfigure out how to support build flags” (not the Chromium flags, but the “naive”kind; i.e. their problem statement was exactly the one the blog opens with). SoI wrote a less-confidential but still internal doc for them saying “here’s howyou could do it”; basically, More power with no code changes: Platform-based Skylib flags.
Then Pigweed’s TL comes along and says “Ted, don’t you feel like spending a dayfighting with RST [the markup we use for pigweed.dev]?” Sorry, actually theysaid something more like, “Why is this doc internal, can’t we share this morewidely”? Well we can. So that’s the doc you’re reading now!
But arguably the story shouldn’t end here: Pigweed should probably provide aready-made implementation of Chromium build flags for downstream projects. Seeissue #342454993 to check out how that’sgoing!
If you are a Bazel expert, you may ask: do we need to have Bazel write out theactual header files, and wrap those in a cc_library? If we’re alreadywriting a custom rule for feature_cc_library, can we just set -Ddefines by providing CcInfo? That is, do something likethis:
define = "{}={}"..format( ctx.attr.build_setting.label.name.upper(), value)return [CcInfo( compilation_context=cc_common.create_compilation_context( defines=depset([define])))]
The honest answer is that this didn’t occur to me! But one reason to preferwriting out the header files is that this approach generalizes in an obviousway to other programming languages: if you want to generate Python or Golangconstants, you can use the same pattern, just change the contents of the file.Generalizing the CcInfo approach is trickier!
This blog focused on describing approaches that rely on platform-basedflags.But this feature is very new: in fact, as of this writing, it is still underdevelopment, so it’s notavailable in any Bazel version! So what can you do?
One approach is to define custom wrapper rules for your cc_binary targetsthat use a transition to set theflags. You can see examples of such transitions in the Pigweed examplesproject.
https://bxt.org/wdepb