Benchmark: Kafka builds 6.6x faster on Avrea than GitHub Actions

Why Kafka

Apache Kafka is a large JVM project: dozens of Gradle modules, a long test suite, heavy javac. Gradle supports a build cache for task outputs.

This combination was a good fit for benchmarking raw runner performance and Avrea's Gradle remote caching.

How we ran the test

We ran ./gradlew assemble on both sides. Same runner (2 vCPU, 7.8 GiB). Temurin 21. Same Kafka commit.

For the cached run, Gradle's remote cache was pointed at Avrea's cache. On the no-cache run, build caching was disabled entirely. The GitHub Actions side had no remote cache hooked up on either run.

Below is a self-contained example workflow of what we are measuring. The Gradle remote cache endpoint is configured automatically on Avrea runners, we only had to flip --build-cache in the build. -PcommitId=benchmark keeps the git SHA out of task inputs so the cache isn't invalidated on every commit.

name: "Kafka"

on:
  workflow_dispatch:

env:
  PROJECT_REPO: "https://github.com/apache/kafka.git"
  PROJECT_TAG: "4.2.0"
  JAVA_VERSION: "21"

jobs:
  build:
    name: "Build"
    runs-on: avrea-ubuntu-latest-2-vcpu
    steps:
      - name: Clone project
        run: git clone --depth 1 --branch ${{ env.PROJECT_TAG }} ${{ env.PROJECT_REPO }} kafka

      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: ${{ env.JAVA_VERSION }}

      - name: Build
        working-directory: kafka
        run: ./gradlew assemble --build-cache --no-daemon -PcommitId=benchmark

Results

Hardware alone buys 2.6x. Turning on Gradle's remote cache against Avrea's cache layer gets the warm number to 52 seconds, for 6.6x total.

Why it's faster

A no-cache Kafka build spends most of its time in javac and incremental class compilation across modules. CPU-bound work.

Avrea's 2 vCPU runner on dedicated hardware pulls the 2.6x on its own. Same mechanics as every other CPU-bound JVM build.

Gradle's remote cache stores task outputs (compiled class files, test results, etc.) keyed by the hash of each task's inputs. On a cached run, if a task's inputs match, Gradle pulls the output and skips the task entirely.

Warm hit rate is a low 23%. Kafka's assemble involves generated sources and Scala compilation that consume most other modules' outputs, which can cause cache invalidations / misses. Improving the build to be more cache friendly could speed it up massively.

That's why Kafka's 6.6x is more modest than the 39x on Bazel or the 142x on Next.js in this series of blog posts. 6.6x is still nothing to sneeze at if you get it out-of-the-box without any changes to your workflow.

Other benchmarks in this series

• Bazel (39.7x)
• Linux kernel (68x)
• Ghostty (27x)
• Next.js (142x)
• RisingWave (11.5x)

What Avrea is

Avrea is a label swap, not a platform migration. runs-on: avrea-ubuntu-latest and your Gradle pipeline runs on our hardware.

Every job comes with CPU and memory at step granularity, searchable log history, and SSH into the running VM when you need it. Flaky JVM tests are where the SSH access earns its keep. A test that only fails on the CI box usually needs jstack or a heap dump, which you can only get from the running VM.

Every job runs in an isolated ephemeral VM. ISO 27001:2022 certified.

See pricing or start free on Avrea →