Skip to main content

v1.0.3 Release

This release introduces the dedicated AI Migrate worker subchart (bhaimigrateworker), a shared PersistentVolumeClaim template for Migrate Validate sandboxes, zero-downtime rolling update deployment strategy configurations across core services, and runtime configuration enhancements for Catalog API and AI Agent services in Azure environments.

Summary of Changes​

  • AI Migrate Worker Subchart (bhaimigrateworker): Added a dedicated Celery worker subchart to handle AI migration workloads asynchronously via the migrate-queue, complete with horizontal pod autoscaling (HPA) and KEDA queue-length scaling.
  • Shared Storage for Migrate Validate (validateSandbox): Introduced a top-level shared ReadWriteMany (RWX) PersistentVolumeClaim template for Migrate Validate sandboxes, mounted into bhaiagent, bhaiagentworker, and bhaimigrateworker.
  • Zero-Downtime Deployment Strategy: Configured rolling update strategies with maxUnavailable: 0 and maxSurge: 1 across 10 service deployments to ensure zero downtime during upgrades, and adjusted bhcelerybeat to maxSurge: 0 to prevent duplicate scheduler processes.
  • Catalog API Runtime Parameters Configuration: Injected RUNTIME_PARAMS_BASE_PATH (default: /Workspace/Shared/runtime_params) into bhcatalogapi ConfigMap and Deployment environment variables.
  • AI Agent Communication & Synthetic Evaluation Settings: Added TEST_RUN_HOST, TEST_RUN_PORT, and TEST_RUN_USE_SECURE configuration, as well as synthetic assessment and validation toggles (ALLOW_DATABRICK_SYNTHETIC_ASSESSMENT, ALLOW_DATABRICK_SYNTHETIC_VALIDATE) in bhaiagent and bhaiagentworker.
  • Azure Environment Values Overlays: Updated values-release.yaml, values-client.yaml, and values-dev.yaml to include configurations for bhaimigrateworker, validateSandbox, and isolated queue subscriptions.

Technical Details​

Chart Updates​

  • Release tag: v1.0.3-bighammer-azure.
  • Added dependency bhaimigrateworker (version 1.0.0, repository file://charts/bhaimigrateworker) under dependencies in Chart.yaml, controlled by global.bhaimigrateworker.enabled.
  • Updated Chart.lock to pin subchart dependencies.

AI Migrate Worker Subchart (bhaimigrateworker)​

  • Introduced subchart under charts/bhaimigrateworker/:
    • Deployment (templates/deployment.yaml): Configured with rolling update (maxUnavailable: 0, maxSurge: 1), health checks (liveness and readiness probes pointing to /api/v1/health on port 8090), security context, and optional volume mount for validate-sandbox.
    • ConfigMap (templates/configmap.yaml): Injects PostgreSQL, Redis, RabbitMQ, and Celery settings (CELERY_WORKER_QUEUES: "migrate-queue", concurrency, max memory, task limits), Spark home (SPARK_HOME: "/opt/spark"), test run parameters, and validate sandbox directory paths.
    • Secrets (templates/secret.yaml): ExternalSecret mapping Azure Key Vault secrets (POSTGRES_PASSWORD, POSTGRES_SSL_CERT, LANGCHAIN_API_KEY, REDIS_PASSWORD, DECRYPTION_KEY, RABBITMQ_PASSWORD, AZURE_CLIENT_SECRET, and BH_APP_USER_PASSWORD).
    • Service (templates/service.yaml): ClusterIP service with port names shortened to <= 15 characters (http on port 80, https on port 443; targeting container port 8090).
    • ServiceAccount (templates/serviceaccount.yaml): Configured with Azure Workload Identity annotations (azure.workload.identity/client-id, azure.workload.identity/use).
    • Autoscaling (templates/hpa.yaml, templates/scaledobject.yaml): Added both standard CPU/memory HPA and KEDA ScaledObject triggering on RabbitMQ migrate-queue length.

Shared Validate Sandbox PVC (templates/validate-sandbox-pvc.yaml)​

  • Added top-level template templates/validate-sandbox-pvc.yaml creating a ReadWriteMany (RWX) PersistentVolumeClaim (default name: bh-migrate2-validate, size: 50Gi, storage class: azurefile-csi) when .Values.global.validateSandbox.enabled is true.
  • Conditionally mounts into /var/bh-migrate2-validate on bhaiagent, bhaiagentworker, and bhaimigrateworker containers.

Zero-Downtime Deployment Strategy​

  • Configured strategy.rollingUpdate.maxUnavailable: 0 and maxSurge: 1 in:
    • charts/bhadminui/templates/deployment.yaml
    • charts/bhaiagent/templates/deployment.yaml
    • charts/bhaiagentworker/templates/deployment.yaml
    • charts/bhauditapi/templates/deployment.yaml
    • charts/bhauditworker/templates/deployment.yaml
    • charts/bhcatalogapi/templates/deployment.yaml
    • charts/bhcatalogworker/templates/deployment.yaml
    • charts/bhkeycloak/templates/deployment.yaml
    • charts/bhkeycloakapi/templates/deployment.yaml
    • charts/bhtransformationutils/templates/deployment.yaml
    • charts/bhui/templates/deployment.yaml
  • Configured strategy.rollingUpdate.maxSurge: 0 and maxUnavailable: 1 in charts/bhcelerybeat/templates/deployment.yaml to guarantee a single active Celery Beat scheduler during deployments.

Catalog API Service (bhcatalogapi)​

  • Added RUNTIME_PARAMS_BASE_PATH to charts/bhcatalogapi/templates/configmap.yaml (default: "/Workspace/Shared/runtime_params").
  • Injected RUNTIME_PARAMS_BASE_PATH as an environment variable in charts/bhcatalogapi/templates/deployment.yaml.

AI Agent & Worker Services (bhaiagent / bhaiagentworker)​

  • Added TEST_RUN_HOST (default: "bhtransformationutils"), TEST_RUN_PORT (default: "80"), and TEST_RUN_USE_SECURE (default: "false") into ConfigMaps and Deployments.
  • Added ALLOW_DATABRICK_SYNTHETIC_ASSESSMENT and ALLOW_DATABRICK_SYNTHETIC_VALIDATE environment variables.
  • Added validate-sandbox volume mount and environment paths (BH_MIGRATE2_VALIDATE_ROOT, BH_VALIDATE_ENV_STORE, BH_VALIDATE_CONN_STORE, BH_VALIDATE_SANDBOX_BACKEND) enabled via global.validateSandbox.enabled.

Values Files Configuration​

  • values-release.yaml:
    • Added global.validateSandbox configuration (enabled: true, size: 50Gi, accessMode: ReadWriteMany, storageClassName: azurefile-csi).
    • Added global.bhaimigrateworker entry (enabled: false, name: bhaimigrateworker).
  • values-client.yaml:
    • Set global.bhcatalogapi.env.RUNTIME_PARAMS_BASE_PATH: "/Workspace/Shared/runtime_params".
    • Added global.bhaimigrateworker: { enabled: false }.
  • values-dev.yaml:
    • Enabled and configured global.bhaimigrateworker (enabled: true, queue migrate-queue, Azure Workload Identity annotations, CPU/memory limits).
    • Updated bhaiagentworker.env.CELERY_WORKER_QUEUES to "agent-queue,observability-queue,ops-queue" to isolate queues.
    • Added global.validateSandbox configuration.
    • Added RUNTIME_PARAMS_BASE_PATH: "/Workspace/Shared/runtime_params" to bhcatalogapi.
    • Added TEST_RUN_HOST, TEST_RUN_PORT, TEST_RUN_USE_SECURE, and synthetic assessment/validation toggles.
    • Set VITE_IS_DASHBOARD_MENU_ENABLE: "false".