RESTHeart Build and Release Process Overview

In this blog post, I'll take you through how we handle the build and release processes for the RESTHeart project. RESTHeart is a robust and fairly complex Java application that serves as a high-performance backend for data-centric services. It is built using a multi-module Maven POM structure, which allows it to efficiently manage multiple components and dependencies.

This architecture supports its flexible integration capabilities, enabling RESTHeart to handle REST APIs over MongoDB with ease. The complexity of the project, combined with its multi-module nature, makes it a perfect candidate for automated build, test, and deployment processes that ensure quality and consistency across releases.

Key Components of the Build and Release Workflows

The entire flow is managed using GitHub Actions, offering a detailed orchestration from code checkout to final deployment. This approach ensures consistent, repeatable, and efficient releases, with careful integration of various components, including multi-architecture Docker images and native images using GraalVM.

The processes can be broadly divided into two key workflows: the tags.yml workflow, which handles the core build, testing, Docker image creation, and standard deployments, and the native-image-release.yml workflow, which deals with building and releasing native images across different operating systems.

1. Triggering Mechanisms

The workflows are triggered either on pushes to tagged branches or via manual intervention using workflow dispatch. This is configured to ignore the master branch and to initiate builds for any other tag, ensuring that the main development branch is not disturbed by premature release actions. Additionally, workflows can be invoked by other workflows using workflow_call, which allows seamless chaining between the different components.

2. Build and Test Phase

The initial phase of the tags.yml workflow focuses on building and testing the project using different MongoDB versions (5.0, 6.0). Here is a fragment of the workflow that defines this phase:

jobs:
  build:
    if: "!contains(github.event.head_commit.message, 'skip ci')"
    runs-on: ubuntu-latest
    strategy:
      matrix:
        mongodb-version: ["5.0", "6.0"]
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4

      - name: Set up JDK 21
        uses: actions/setup-java@v4
        with:
          distribution: "temurin"
          java-version: "21"
          cache: "maven"

      - name: Build and Test
        run: |
          mvn -B clean verify -Dmongodb.version="${{ matrix.mongodb-version }}"

Here, we leverage a matrix strategy to run tests against multiple MongoDB versions, ensuring that RESTHeart maintains compatibility across a diverse range of MongoDB setups. This build process also makes use of JDK 21 for compilation, with caching enabled for Maven dependencies to speed up the process.

Notably, both the build and deploy steps include an if condition to avoid running if the commit message contains "skip ci," providing a way to bypass CI/CD for minor changes that don’t need full verification.

3. Deployment to Maven Central and DockerHub

The deployment phase handles uploading built artifacts to both Maven Central and DockerHub. Here is the relevant fragment from tags.yml:

  deploy:
    if: "!contains(github.event.head_commit.message, 'skip ci')"
    runs-on: ubuntu-latest
    strategy:
      matrix:
        mongodb-version: ["7.0"]
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@v4

      - name: Set up JDK 21
        uses: actions/setup-java@v4
        with:
          distribution: "temurin"
          java-version: "21"
          cache: "maven"

      - name: Build and Test
        run: mvn -B clean verify -Dmongodb.version="${{ matrix.mongodb-version }}"

      - name: Import private gpg key
        run: |
          printf "%s" "$GPG_PRIVATE_KEY" > private.key
          gpg --pinentry-mode=loopback --batch --yes --fast-import private.key
        continue-on-error: true
        env:
          GPG_PRIVATE_KEY: ${{ secrets.GPG_PRIVATE_KEY }}

The first task is to import a private GPG key to sign artifacts, which is an important step for Maven Central. Maven deploy commands are then executed with appropriate Maven settings and additional JVM options to avoid class access restrictions.

For DockerHub, the process begins by setting up QEMU and Docker Buildx, which allows for multi-architecture support:

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to DockerHub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USER }}
          password: ${{ secrets.DOCKER_TOKEN }}

We then create a series of tags for the Docker images:

      - name: Set Docker Tags for standard images
        id: set_tags
        run: |
          # Construct all tags based on the MAJOR.MINOR.PATCH format
          TAGS="softinstigate/restheart:latest,softinstigate/restheart:${{ env.MAJOR }},softinstigate/restheart:${{ env.MAJOR }}.${{ env.MINOR }},softinstigate/restheart:${{ env.MAJOR }}.${{ env.MINOR }}.${{ env.PATCH }}"
          echo "TAGS=$TAGS" >> $GITHUB_ENV

The multi-architecture images are built for amd64, arm64, ppc64le, and s390x platforms, making RESTHeart versatile across different server environments.

4. GitHub Release

Once all images are built and pushed, the workflow creates a GitHub release:

      - name: Upload GitHub release
        uses: softprops/action-gh-release@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          body: |
            # Release ${{ env.VERSION }}
          files: |
            core/target/restheart.tar.gz
            core/target/restheart.zip
          draft: true
          prerelease: false

The release includes binaries and packaged versions of RESTHeart for easy download. A release can be marked as either a draft or a stable version, giving us flexibility in managing pre-releases and public stable releases.

5. Native Image Build and Release

The native-image-release.yml workflow is triggered by the main workflow using workflow_call to build native images using GraalVM. Here is a fragment showing part of the configuration:

jobs:
  build-and-upload:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        include:
          - os: ubuntu-latest
            arch: linux-amd64
          - os: windows-latest
            arch: windows-amd64
          - os: macos-13
            arch: "darwin-amd64"
          - os: macos-latest
            arch: "darwin-arm64"

This process covers four target environments:

Linux (amd64)
Windows (amd64)
macOS (Intel and Apple Silicon)

Each platform has a tailored build script, using specific commands to compile RESTHeart into a native executable:

      - name: Build Linux native
        if: matrix.arch == 'linux-amd64'
        run: |
          echo "GRAALVM_HOME: $GRAALVM_HOME"
          echo "JAVA_HOME: $JAVA_HOME"
          java --version
          native-image -march=list
          mvn package -Pnative -DskipTests -Dnative.march="-march=x86-64"

After successful compilation, the native binary is uploaded as a GitHub artifact and then to the release corresponding to the tag.

6. Docker Publishing for Native Images

Following the native build, a separate job runs to publish Docker images based on the native binaries. These Docker images are built using multi-architecture support to ensure consistency with other Docker releases. Here is the relevant fragment:

  docker-publish-linux:
    runs-on: ubuntu-latest
    needs: build-and-upload
    if: needs.build-and-upload.outputs.ubuntu == 'true'
    steps:
      - uses: actions/checkout@v4

      - name: Download Binary Artifact
        uses: actions/download-artifact@v4
        with:
          name: restheart-linux-amd64
          path: core/target/restheart

      - name: Build and Push multi-arch native Docker images
        uses: docker/build-push-action@v6
        with:
          context: ./core/
          file: ./core/Dockerfile.native
          platforms: |
            linux/amd64,
            linux/arm64,
            linux/ppc64le,
            linux/s390x
          push: true
          pull: true
          tags: ${{ env.TAGS }}

Best Practices and Learnings from Our Approach

Multi-Architecture Support: One of the central aspects of our release process is to ensure all Docker images are built to support different architectures. This helps us cater to diverse hosting environments without requiring additional steps from end users.
Native Image Builds for Performance: Building native images provides considerable performance benefits. With GraalVM, we are able to create versions of RESTHeart that have significantly reduced startup time and memory footprint, enhancing their utility for serverless or performance-critical environments.
Version Tagging and Consistency: Careful version tagging ensures that both users and developers can easily understand the lineage of a given release. We automatically generate and apply version tags to Docker images and binaries, which creates a predictable and user-friendly experience for version control.
Automation Through Chained Workflows: Leveraging workflow_call allows us to create modular workflows that can be reused or invoked as necessary. This keeps our main workflows streamlined and helps minimize repetition in configuration.

Conclusion

Our CI/CD process for RESTHeart is designed to make releasing new versions as smooth and efficient as possible while maintaining high standards of compatibility and performance. From managing multi-version builds and multi-architecture Docker images to creating optimized native binaries, our GitHub Actions workflows automate every part of the process in a scalable and transparent manner.

We hope these insights into our build and release process will help you design efficient CI/CD workflows for your own projects. If you have any questions or suggestions for improvement, feel free to write us.