Best Practices for Building GitOps with FluxCD

Introduction

In many teams, GitOps is designed as a simple sync setup: manifests are stored in Git, a GitOps tool applies them to Kubernetes, and the work is considered done.

This works for a while. Then manifests start being copied across environments, changes become harder to review, and bootstrapping a new cluster still requires manual fixes.

In this article, I will show how to improve your GitOps setup.

About Flux

I chose FluxCD because it fits the way I want to manage clusters. It runs inside the cluster with several controllers and continuously reconciles the desired state from Git without turning deployment into a separate platform. Simple and clear.

You can list its controllers with:

kubectl get deploy -n flux-system

Controller	What it does
`source-controller`	Fetches sources such as Git repositories, Helm repositories, OCI repositories, and buckets, then produces artifacts.
`kustomize-controller`	Uses artifacts produced by `source-controller` and applies manifests from configured paths. The sync path starts from `flux-system/gotk-sync.yaml` under the cluster directory.
`helm-controller`	Consumes artifacts produced by `source-controller` and works with Helm charts through `HelmRelease` resources.
`notification-controller`	Handles events around the reconciliation flow. It can receive inbound webhooks and send alerts to external systems such as Slack.

The basic flow looks like this:

flowchart TD A[Git] --> B[source-controller] B --> C[kustomize-controller / helm-controller] C --> D[Kubernetes] C --> E[notification-controller] E --> F[Slack / Teams / Webhook]

Flux also has optional image automation controllers: image-reflector-controller and image-automation-controller. I do not use them in this setup.

You can read more about each component in the Flux component documentation and the Flux core concepts.

Repository

A GitOps repository should be easy to trace. I use this structure:

clusters/ # Cluster entry points. Defines what should be installed on a specific cluster and in which order.
  prod-cluster-1/
  prod-cluster-2/
  dev-cluster/
apps/ # Business applications.
  base/
  overlays/
infrastructure/ # Platform components.
  apps/ # Infrastructure applications such as Alertmanager, Fluent Bit, GitLab Runner, etc.
    base/
    overlays/
  resources/ # Supporting resources such as ConfigMaps, Secrets, custom resources, policies, etc.
    base/
    overlays/
  crds/ # CustomResourceDefinitions, operators, and other required CRD-level components.
    base/
    overlays/
  essentials/ # Basic building blocks such as namespaces, Flux sources, and shared prerequisites.
    base/
    overlays/

The base directories contain reusable manifests. The overlays directories contain environment-specific changes, such as replica counts, resource limits, and ingress hostnames.

Order

I order infrastructure with dependsOn between Flux Kustomization resources and treat infrastructure as layers:

flowchart TD infraEssentials["infra-essentials"] --> infraCrds["infra-crds"] infraCrds --> infraResources["infra-resources"] infraResources --> infraApps["infra-apps"] infraApps --> targetCluster["cluster"] apps["apps"] --> targetCluster style infraEssentials fill:#e8f1ff,stroke:#2f5eed,stroke-width:1px,color:#1e1e1e style infraCrds fill:#e8f1ff,stroke:#2f5eed,stroke-width:1px,color:#1e1e1e style infraResources fill:#e8f1ff,stroke:#2f5eed,stroke-width:1px,color:#1e1e1e style infraApps fill:#e8f1ff,stroke:#2f5eed,stroke-width:1px,color:#1e1e1e style apps fill:#fff4e6,stroke:#a88b5a,stroke-width:1px,color:#1e1e1e style targetCluster fill:#eaf7ea,stroke:#5c8a5c,stroke-width:1px,color:#1e1e1e

This prevents common bootstrap failures, such as creating a Certificate resource before cert-manager is installed.

Business apps keep reconciling independently. They may stay not-ready until the required infrastructure is ready, then automatically become ready.

Workflow

I use a main branch that represents the desired state of the platform, and short-lived feature branches.

Every change starts in a feature branch, goes through a pull request, passes CI, and is merged after review.

For CI, run checks such as kustomize build, yamllint, and policy checks. flux diff can also help show what will change before the pull request is merged.

Secrets

Secrets are one of the hardest parts of GitOps.

Tools such as SOPS and Sealed Secrets solve this by storing encrypted secrets in Git. But they add complexity around key management, access control, and encryption/decryption steps in the workflow.

My preferred approach is HashiCorp Vault with Vault Secrets Operator. Vault keeps secrets outside the GitOps repository, while the operator runs inside the cluster and syncs selected Vault data into Kubernetes Secrets. The GitOps repository stores only references, not the secret values themselves.

Applications then consume normal Kubernetes Secrets:

envFrom:
  - secretRef:
      name: my-app-secret

To make sure workloads receive updated secrets, I use Reloader. It restarts pods when synced Kubernetes Secrets change.

The flow looks like this:

flowchart TD A[Vault] --> B[Vault Secrets Operator] B --> C[Kubernetes Secret] C --> D[Application Pod]

Before bootstrapping a cluster, the cluster must be configured to authenticate to Vault. If Vault or the operator becomes unavailable, existing Kubernetes Secrets remain, but refresh and rotation can be affected. Treat Vault availability as part of the platform design.

Troubleshooting

When something is not applied as expected, follow the reconciliation chain: source, Kustomization or HelmRelease, then the applied resources.

flux get all -A | awk 'NR==1 || $0 !~ /True/'

For a failed Kustomization or HelmRelease, describe the resource:

kubectl describe kustomization <name> -n flux-system
kubectl describe helmrelease <name> -n <namespace>

To trigger reconciliation manually:

flux reconcile kustomization <name> -n flux-system
flux reconcile helmrelease <name> -n <namespace>

If reconciliation does not recover a stuck resource, you can sometimes delete the affected Kustomization or HelmRelease and let Flux recreate it. Do this only when the resource is stateless or safe to recreate.

Conclusion

GitOps works best when it is treated as an operating model, not just a YAML sync mechanism. With the approach described here, you can build a simple, scalable and predictable way to manage Kubernetes clusters.

For a practical example, check my GitHub repository, where I show the setup discussed in this article.