benchfmt.rs: Bringing Go's Benchmarking and benchstat Workflow to Rust
Pangram verdict · v3.3
We believe that this text is a mix of AI and human-written content.
AI likelihood · overall
MixedArticle text · 1,363 words · 19 segments analyzed
TL;DR: I built benchfmt.rs, a zero-dependency Rust benchmark runner and Criterion converter for Go’s benchstat.
It adds /key=value projections, heap tracking for allocated bytes (B/op) and objects (allocs/op) per op, and pprof profiling.
In this post, I am benchmarking varint and Prometheus Remote Write 2.0 decoding across Go and Rust to showcase the benefits of benchstat-like flow in Rust. I’m also sharing some fun learnings on the way (e.g., yes Rust is faster). Recently, I’ve been working on a few Rust projects across Google and open source. I came to Rust after years of writing performance-sensitive Go in Prometheus, Thanos, and teaching Go in the “Efficient Go” book. For me, much of Rust’s performance story is really good, until…. I wanted to run A/B microbenchmarks. Perhaps I was spoiled by the Go benchmarking ecosystem, with the go tool -bench and benchstat in the center. To me, Rust standard benchmarking tools are lacking: Standard format for case comparison projections. Standard heap allocation measurements. Standard pprof support. Simple to understand reliability scoring. DISCLAIMER: A standard here means something that will be familiar to large number of developers, and I biased on Go experience for sure! As a result, I asked myself: can we get benchstat-compatible output for Rust microbenchmarks?
Wouldn’t that make A/B testing easier and unlock fun Go vs Rust comparisons? Here we are!
I built benchfmt.rs, a zero-dependency Rust benchmark runner with a thread-local heap allocation counter, pprof profiler integration, and the same CLI flags as go test -bench. For those who don’t trust a new benchmarking runtime (you shouldn’t from the start!), it also ships a Criterion and libtest converter that emits Go’s benchfmt format directly. In this blog post, we will implement the same pieces of code in Go and Rust, benchmark them across different Rust frameworks, and compare the results. We might learn something new along the way (I did when building this!).
Enjoy! All benchmark code and raw outputs live in bwplotka/benchmarks/benchmarks/rust-go-decode.
Benchmarking two workloads Let’s pick two workloads to implement and benchmark in both Go and Rust. I am particularly curious about protobuf decoding performance, which, funny enough, is often one of the main costs of running distributed systems these days.
Thus, let’s decode our Prometheus Remote Write 2.0 protocol.
Because protobuf code generators have different maturity and trade-offs across languages, let’s also measure something simpler alongside it, a standalone varint decoder.
Workload details: DecodeVarints.
A 36-line unsigned LEB128 varint stream decoder.
Varints are the integer encoding underneath Protocol Buffers.
Our fixture varints_2000.bin is 4,798 bytes and packs 2,000 uint64 values mixing 1-byte symbol indices, 2-byte bucket offsets, 4-byte deltas, and 6-byte millisecond Unix timestamps. The function writes into a preallocated 2,000-element slice with zero heap allocations.
DecodeRemoteWrite2.
A full HTTP request body decoder for Snappy-compressed Remote Write 2.0 (io.prometheus.write.v2.Request) payloads. We test batches of 200 and 2,000 series, where 2,000 matches Prometheus’s default max_samples_per_send. The 2,000-series fixture mixes counters, gauges, classic and native histograms, 1,000 trace exemplars, and metric metadata. Remote Write 2.0 interns all strings into a top-level symbols table and stores packed varint labels_refs per series. That shrinks the 2,000-series batch to 244.9 KiB uncompressed and 57.0 KiB Snappy-compressed.
Implementing and benchmarking in Go Let’s start with Go.
First, DecodeVarints writes straight into a caller-provided slice, with a fast path for single-byte values below 0x80: func DecodeVarints(src []byte, dst []uint64) (int, error) { n := 0 i := 0 for i < len(src) { if n >= len(dst) { return n, ErrDstTooSmall } b := src[i] i++ if b < 0x80 { dst[n] = uint64(b) n++ continue } x := uint64(b & 0x7f) shift := uint(7) for { if i >= len(src) { return n, ErrUnexpectedEOF } b = src[i] i++ x |= uint64(b&0x7f) << shift if b < 0x80 { break } shift += 7 if shift >= 64 { return n, ErrOverflow } } dst[n] = x n++ } return n, nil } Second, for DecodeRemoteWrite2, we tune the Go implementation the way Prometheus runs in production rather than benchmarking slow reflection-based unmarshaling. We pair github.com/golang/snappy, which uses x86-64 assembly on linux/amd64, with vtprotobuf and its generated, optimized UnmarshalVT decoder: func DecodeRemoteWrite2(compressed []byte) (*writev2.Request, error) { raw, err := snappy.Decode(nil, compressed) if err != nil { return nil, err } req := &writev2.Request{} if err := req.UnmarshalVT(raw); err != nil { return nil, err } return req, nil } Now let’s measure the efficiency of our Go code! Notice Go’s /key=value sub-benchmark names (samples=2000/lang=go), which let Go’s official benchstat tool pivot across /lang or /samples later, along with b.SetBytes, b.ReportAllocs, and for b.Loop(): func BenchmarkDecodeVarints(b *testing.B) { manifest := ensureFixtures(b) raw, err := os.ReadFile(filepath.Join("testdata", "varints_2000.bin")) if err != nil { b.Fatalf("read varints: %v", err) } dst := make([]uint64, manifest.VarintsCount) // Encode dimensions as /key=value so benchstat can pivot across /samples or /lang. b.Run("samples=2000/lang=go", func(b *testing.B) { b.SetBytes(int64(len(raw))) b.ReportAllocs() // NOTE: b.Loop() resets the timer on its first call, stops it on exit, and keeps // call arguments and results alive without package-level sink variables.
for b.Loop() { n, err := DecodeVarints(raw, dst) if err != nil || n != manifest.VarintsCount { b.Fatalf("unexpected decode result: n=%d err=%v", n, err) } } }) } func BenchmarkDecodeRemoteWrite2(b *testing.B) { ensureFixtures(b) for _, samples := range []int{200, 2000} { file := fmt.Sprintf("rw2_%d.snappy", samples) compressed, err := os.ReadFile(filepath.Join("testdata", file)) if err != nil { b.Fatalf("read %s: %v", file, err) } b.Run(fmt.Sprintf("samples=%d/lang=go", samples), func(b *testing.B) { b.SetBytes(int64(len(compressed))) b.ReportAllocs() for b.Loop() { req, err := DecodeRemoteWrite2(compressed) if err != nil || len(req.Timeseries) != samples { b.Fatalf("unexpected v2 decode: err=%v", err) } } }) } } Finally, let’s run 6 benchmark runs (“epochs”) for 2s each at -cpu 2 on a 24-core Intel Xeon 2.20GHz Linux workstation (Go 1.26.8, Rust 1.99.1): go test -run '^$' -bench '^BenchmarkDecode' \ -benchtime 2s -count 6 -cpu 2 -timeout 999m -benchmem \ | tee results/go.txt One epoch looks as follows: goos: linux goarch: amd64 pkg: github.com/bwplotka/benchmarks/benchmarks/rust-go-decode cpu: Intel(R) Xeon(R) CPU @ 2.20GHz BenchmarkDecodeVarints/samples=2000/lang=go-2 279861 8514 ns/op 563.52 MB/s 0 B/op 0 allocs/op BenchmarkDecodeRemoteWrite2/samples=200/lang=go-2 8467 264321 ns/op 31.43 MB/s 195296 B/op 2189 allocs/op BenchmarkDecodeRemoteWrite2/samples=2000/lang=go-2 976 2448633 ns/op 23.84 MB/s 1643970 B/op 19297 allocs/op Implementing and benchmarking in Rust with Criterion Now let’s port both decoders and benchmarks to Rust.
First, decode_varints mirrors the Go loop line for line: pub fn decode_varints(src: &[u8], dst: &mut [u64]) -> Result<usize, VarintError> { let mut n = 0usize; let mut i = 0usize; while i < src.len() { if n >= dst.len() { return Err(VarintError::DstTooSmall); } let mut b = src[i]; i += 1; if b < 0x80 { dst[n] = u64::from(b); n += 1; continue; } let mut x = u64::from(b & 0x7f); let mut shift = 7u32; loop { if i >= src.len() { return Err(VarintError::UnexpectedEof); } b = src[i]; i += 1; x |= u64::from(b & 0x7f) << shift; if b < 0x80 { break; } shift += 7; if shift >= 64 { return Err(VarintError::Overflow); } } dst[n] = x; n += 1; } Ok(n) } Second, decode_remote_write2 pairs the snap crate with prost compile-time generated Message structs: pub fn decode_remote_write2(compressed: &[u8]) -> Result<proto::v2::Request, String> { let raw = snap::raw::Decoder::new() .decompress_vec(compressed) .map_err(|e| e.to_string())?; proto::v2::Request::decode(raw.as_slice()).map_err(|e| e.to_string()) } Now, let’s write both Rust benchmarks using Criterion, the default choice for stable-Rust microbenchmarking. We keep the same /key=value benchmark IDs and configure Throughput::BytesDecimal to match Go’s decimal MB/s: fn bench_varints(c: &mut Criterion) { // Resolve testdata/ from the crate directory baked in at compile time. let root = Path::new(env!("CARGO_MANIFEST_DIR")).join("testdata"); let raw = fs::read(root.join("varints_2000.bin")).unwrap(); // Preallocate the output buffer outside the timed loop so each op does 0 heap allocs. let mut dst = vec![0u64; 2000]; // Group + function ID join into "DecodeVarints/samples=2000/lang=rust" to match Go's b.Run. let mut group = c.benchmark_group("DecodeVarints"); // Report decimal MB/s (1e6 B/s) to match Go's b.SetBytes; Throughput::Bytes prints MiB/s. group.throughput(Throughput::BytesDecimal(raw.len() as u64)); group.bench_function("samples=2000/lang=rust", |b| { // b.iter runs the closure across Criterion's warmup and 100 linear-ramp samples. b.iter(|| { // std::hint::black_box is an LLVM optimization barrier: Unlike Go's b.Loop(), Rust // would constant-fold known inputs or delete unused work without black_box hints. let n = decode_varints(black_box(&raw), black_box(&mut dst)).unwrap(); assert_eq!(n, 2000); // Observe the written buffer so LLVM cannot optimize away stores into dst.