Skip to main content

Cluster preparation

Configure cluster add-ons, secrets, identity, and edge routing on AKS before installing the BigHammer Helm chart (bighammer). This section covers everything the application chart depends on but does not install itself.

Before you start

Complete Infrastructure provisioning (AKS, PostgreSQL, Key Vault, DNS, ACR) and the Prerequisites checklist first.

OrderPagePurpose
1Workload IdentityBind KSAs to user-assigned managed identities (ESO + apps)
2External Secrets OperatorInstall ESO to sync Key Vault → Kubernetes
3TLS OptionsSSL/TLS certificates for HTTPS at the edge
4NGINX Ingress DeploymentLegacy edge — single Ingress resource
4bGateway API stackRecommended edge — gateway-crds, Envoy controller, bh-gateway (bh-helm-charts/incubator/gateway/INSTALL_AZURE_AKS.md)
5RabbitMQ brokerDeploy bhrabbitmq (OCI), Bitnami, or use an existing broker
6Key Vault SecretsKey Vault name mapping and chart secret reference
7CD deployment (Gateway mode)GitHub Actions runner RBAC for HTTPRoute (if using CI/CD)

Pick one edge mode in step 4 — nginx or Gateway API. When Gateway mode is enabled, complete step 7 before the first automated bighammer deploy from GitHub Actions.

Database bootstrap and Key Vault timing

Run create-dbs-azure.sh and Key Vault bootstrap scripts from Scripts Reference after PostgreSQL is provisioned and before Helm install. Platform Key Vault secrets must exist before the chart syncs ExternalSecrets.

Install Workload Identity and ESO (steps 1–2 above) before the platform chart syncs secrets.

Cluster add-ons (summary)​

From Prerequisites — install before the platform chart:

Add-onHelm chart
External Secrets Operatorexternal-secrets-0.18.2
Stakater Reloaderreloader-1.1.0
RabbitMQbhrabbitmq (OCI) or existing broker
Edgeingress-nginx Helm 4.15.1 or Envoy Gateway + bh-gateway

CD deployment requirements (Gateway mode)​

When global.gateway.enabled: true on the bighammer chart (and on standalone charts such as airflow-azure, grafana, superset-multicloud), Helm deploys HTTPRoute resources in API group gateway.networking.k8s.io instead of (or in addition to) Ingress objects.

GitHub Actions CD workflows (cd-deploy-bighammer-aks-azure-dev, cd-deploy-airflow-aks-azure-dev, etc.) run helm upgrade using the in-cluster runner identity:

system:serviceaccount:github-runner:github-runner-sa

That service account must be allowed to get, create, update, and delete Gateway API resources. Permission for networking.k8s.io/ingresses does not cover gateway.networking.k8s.io/httproutes.

Symptom if RBAC is missing​

httproutes.gateway.networking.k8s.io "bhui-route" is forbidden:
User "system:serviceaccount:github-runner:github-runner-sa" cannot get resource "httproutes"

Manual helm from a bastion with admin kubeconfig may still work; only the pipeline fails.

Fix (cluster admin — once per cluster)​

Apply the runner ClusterRole from bh-helm-charts (includes Gateway API rules):

kubectl apply -f scripts/github-actions-runner-aks/runner-role.yaml
kubectl apply -f scripts/github-actions-runner-aks/runner-rolebinding.yaml

Repository path: bh-helm-charts/scripts/github-actions-runner-aks/.

Minimum additional rules (if patching an existing role):

- apiGroups: ["gateway.networking.k8s.io"]
  resources: ["httproutes", "gateways", "referencegrants"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["gateway.networking.k8s.io"]
  resources: ["httproutes/status", "gateways/status"]
  verbs: ["get", "update", "patch"]

Verify​

kubectl auth can-i get httproutes \
  --as=system:serviceaccount:github-runner:github-runner-sa \
  -n bh-control-plane
# expected: yes

kubectl auth can-i create httproutes \
  --as=system:serviceaccount:github-runner:github-runner-sa \
  -n bh-control-plane
# expected: yes

When this step is required​

Edge modeRunner needs Gateway API RBAC?
nginx (global.ingress.enabled: true)No — ingresses only
Gateway (global.gateway.enabled: true)Yes — before first CD deploy
Nginx-only CD

If you stay on nginx Ingress and do not enable gateway mode, the existing runner ClusterRole is sufficient; skip this section.

What to do next​

When TLS, secrets, Workload Identity, and edge routing are in place:

  1. Platform Deployment — helm upgrade the bighammer chart
  2. Deployment Checklist — final verification
  3. Troubleshooting — if something fails

For customer artifact access and release pins, see Release Manifest & Distribution.