Open in app

Sign in

Write

Sign in

GitOps with Flux: Leveraging OCI Registry as a State Store

Stéphane Este-Gracias
8 min readMar 20, 2023

--

Flux — https://www.cncf.io/projects/flux/

Introduction

GitOps does not simply mean using Git with YAML configuration files and Kubernetes operators.

This article will explore what GitOps truly entails, its use cases, and available tools.

Furthermore, it will demonstrate how an OCI registry can be an alternative to a Git repository as a desired state store.

OpenGitOps — https://opengitops.dev/

What’s GitOps

According to OpenGitOps (CNCF Sandbox project owned by the GitOps Working Group), the GitOps Principles 1.0.0 are the following:

  1. Declarative: A system managed by GitOps must have its desired state expressed declaratively.
  2. Versioned and Immutable: Desired state is stored in a way that enforces immutability, versioning and retains a complete version history.
  3. Pulled Automatically: Software agents automatically pull the desired state declarations from the source.
  4. Continuously Reconciled: Software agents continuously observe actual system state and attempt to apply the desired state.

To better understand these important concepts, OpenGitOps has provided a list of terms to help for understanding the main ideas and details of each term. By using this list, these concepts can be applied in your work and can be explained to others more easily.

  • Continuous: “Continuous” is intended to match the industry standard term: reconciliation continues to happen, not that it must be instantaneous.
  • Declarative Description: A configuration that describes the desired operating state of a system without specifying procedures for how that state will be achieved. This separates configuration (the desired state) from the implementation (commands, API calls, scripts etc.) used to achieve that state.
  • Desired State: The aggregate of all configuration data that is sufficient to recreate the system so that instances of the system are behaviourally indistinguishable. This configuration data generally does not include persistent application data, eg. database contents, though often does include credentials for accessing that data, or configuration for data recovery tools running on that system.
  • Drift: When a system’s actual state has moved or is in the process of moving away from the desired state, this is often referred to as drift.
  • Reconciliation: The process of ensuring the actual state of a system matches its desired state. Contrary to traditional CI/CD where automation is generally driven by pre-set triggers, in GitOps reconciliation is triggered whenever there is a divergence. Divergence could be due to the actual state unintentionally drifting from the desired state declarations, or a new desired state declaration version having been changed intentionally. Actions are taken based on policies around feedback from the system and previous reconciliation attempts, in order to reduce deviation over time.
  • Software System: A software system managed by GitOps includes:
    1. One or more runtime environments consisting of resources under management
    2. The management agents within each runtime
    3. Policies for controlling access and management of repositories, deployments, runtimes
  • State Store: A system for storing immutable versions of desired state declarations. This state store should provide access control and auditing on the changes to the desired state. Git, from which GitOps derives its name, is the canonical example used as this state store but any other system that meets these criteria may be used. In all cases, these state stores must be properly configured and precautions must be taken to comply with requirements set out in the GitOps Principles.
  • Feedback: Open GitOps follows control-theory and operates in a closed-loop. In control theory, feedback represents how previous attempts to apply a desired state have affected the actual state. For example if the desired state requires more resources than exist in a system, the software agent may make attempts to add resources, to automatically rollback to a previous version, or to send alerts to human operators.

The GitOps principles and these terms are licensed under the CC BY 4.0 license by OpenGitOps.

Now that “What’s GitOps” was covered, let’s explore some of its use cases and take a closer look at some of the most common and impactful GitOps use cases.

GitOps Use Cases

The benefits of GitOps are numerous, including increased security, better auditability, and improved collaboration. In fact, GitOps has become so popular that it is now being used in a variety of different use cases such as the following.

  • Infrastructure management: GitOps allows you to manage infrastructure as code (IaC), making it easy to version control, review, and apply infrastructure changes. This approach reduces human error.
  • Application deployment: GitOps simplifies application deployment by automating the process and maintaining the desired state in a state store. This enables easy rollbacks, and version control.
  • Multi-environment and multi-tenant management: GitOps makes it easy to manage multiple environments (e.g., dev, staging, prod) and multiple tenants with different configurations.
  • Compliance and auditing: GitOps provides a transparent history of all changes, making it easy to track and audit infrastructure and application changes. This is particularly useful for organizations that need to adhere to strict compliance regulations.
  • Disaster recovery: GitOps helps in disaster recovery by maintaining the desired state in a state store. In case of a disaster, you can easily restore the system to its last known good state by applying the configurations from the state store.

Leveraging the Platform Team through the implementation of these use cases can greatly enhance the effectiveness of GitOps.

Mainly, GitOps provides a streamlined approach to platform teams, enabling organizations to increase their efficiency. With the capability to centrally manage infrastructure and application deployment, developers can concentrate on their core responsibilities of software creation and improvement.

Photo by Tekton on Unsplash

GitOps tools

To implement GitOps and leverage its benefits, organizations need the right tools and technologies. Fortunately, there are many GitOps tools available that can help teams implement this approach quickly and effectively.

Before proceeding, let’s review the GitOps principles through tooling.

Declarative Configuration

Just because something is declarative, it doesn’t mean that it must use YAML configuration files. It could also use other configuration files or software languages that compile, transform, or generate configurations. As long as the configuration can be stored and used as a source of trust in the right way, it can be considered declarative.

So, the possible solutions for defining a declarative configuration are:
- YAML
- ytt (from Carvel projet)
- Pulumi
- Cue Lang
- Starlark
- HCL (HCL is the HashiCorp configuration language)
- CDK for Terraform

Versioned and Immutable

Git’s versioning and immutability features are great, but not everyone enjoys the user experience using Git CLI. To address this, graphical user interfaces, automation, and other tools can be leveraged to improve the workflow. Possible UIs are the following:

Then, most of GitOps tools rely on Git for the state store. In contrast, the Flux Source Controllers can use a Git repository, a storage bucket, or an OCI registry (see dedicated section below).

Pulled Automatically and Continuously Reconciled

GitOps tools use a periodic timer as a trigger for pulling configurations from the state store. This trigger can usually be paused, enabled or forced manually.

The reconciliation does not mean that it must necessarily use Kubernetes operators, but other reconciliations can be used: for instance, the drift detection for Terraform Cloud enables reconciliation using the Terraform state.

GitOps Toolsets

Combining GitOps principles, the following tools are possible GitOps toolsets with some useful extensions such as event notifications and deployment policy management:

OCI registry as state store

OCI registry is a core component for the cloud native ecosystem.

More or more tools can relying on OCI registries to store their configurations. The following tools can leverage OCI registry as its source of trust.

OCI registry can be used by Flux as a state store. The image tag immutability must be enforce to comply with the GitOps principles.

How to configure image tag immutability for the main registry providers.
- Amazon ECR (using image tag immutability)
- Azure Container Registry (using tag locking policy)
- Google Container Registry (feature request for image tag immutability)
- Harbor (using tag immutability rules)

GitOps with GIt only — Flux source controller clones Git to get desired state
GitOps with OCI Registry — Flux pulls desired state from OCI Registry

Using an OCI registry as a state store for GitOps provides several advantages over using Git alone.

  • Pulling an OCI image is a much less resource-intensive process compared to cloning a Git repository, making it more efficient for deployment purposes.
  • High available registries are easier to deploy than Git servers, making it easy to access and deploy container images from any location.
  • OCI repositories can scale out similarly to a Content Delivery Network, enabling platform team to distribute configurations globally with ease.
  • Flux uses Kubernetes workload identity and IAM to pull OCI artifacts from managed registries. This eliminates the need for key management, SSH key generation, and proprietary API usage for token generation. Instead, the same mechanism used for pulling container images for applications is used.
  • OCI images can be signed and verified during container deployment. Flux supports these verifications, out of the box.

Overall, leveraging Flux with the use of an OCI registry as a state store can enhance the scalability, reliability, and security of GitOps deployments.

Moreover, using OCI instead of Git can be particularly useful when the Git repository doesn’t contain the final configuration files. If you’re using ytt, CDK for Terraform, Pulumi, Cue Lang, Starlark, or any other tool that generates configuration files, the generators can be run in a build pipeline and the resulting configuration can be published as OCI artifacts using the Flux CLI for instance.

Conclusion

Flux GitOps Toolkit, in addition to its composability, fully relies on the standard Kubernetes API and can extract the desired state configuration from OCI registries, which is a significant advantage over ArgoCD or Rancher Fleet when targeting multiple Kubernetes clusters.

On the other hand, Terraform Cloud can be used to implement GitOps principles to manage everything as code, such as infrastructure, networking, applications codification…

Resources

Gitops
Kubernetes
Flux Cd
Terraform Cloud
Containers

--

--

Stéphane Este-Gracias
Stéphane Este-Gracias

Written by Stéphane Este-Gracias

54 Followers
20 Following

Open Source & Cloud Native Innovation Lead @ ITQ

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams