ArgoCD Deployment

Overview

ArgoCD renders Helm templates client-side before applying them to the cluster. Helm’s lookup function — which the chart uses to persist auto-generated secrets across upgrades — always returns an empty result in client-side rendering. This means every ArgoCD sync generates new random values for auto-generated secrets, breaking Rails database encryption and session continuity.

Affected Secrets

Secret Key	Purpose	Generation	Conditional	Format
`SECRET_KEY_BASE`	Rails session signing and encryption	`randAlphaNum 64`	Always	Single-line alphanumeric string
`ENCRYPTION_KEY`	Application-level data encryption	Hex (64 chars via sha256sum)	Always	Single-line alphanumeric string
`ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY`	Rails Active Record encryption	`randAlphaNum 64`	Always	Single-line alphanumeric string
`ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY`	Rails deterministic encryption	`randAlphaNum 64`	Always	Single-line alphanumeric string
`ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT`	Rails encryption key derivation	`randAlphaNum 64`	Always	Single-line alphanumeric string
`REGISTRY_HTTP_SECRET`	Internal container registry auth	`randAlphaNum 32`	Always	Single-line alphanumeric string
`CREWAI_PLUS_INTERNAL_API_KEY`	Service-to-service authentication	`randAlphaNum 64`	Always	Single-line alphanumeric string
`WHARF_JWT_SECRET`	Wharf trace collector authentication	`randAlphaNum 64`	Always (wharf.enabled defaults to true)	Single-line alphanumeric string
`DEPLOYMENT_INSTANCE_JWT_SECRET`	Signs JWTs for deployed crew instances	`randAlphaNum 64`	Always	Single-line alphanumeric string
`CUBE_JWT_SECRET`	Cube analytics service authentication	`randAlphaNum 64`	Only when `cube.enabled: true`	Single-line alphanumeric string
`FACTORY_DEBUG_TOKEN`	Authorizes `GET /health/debug` requests	`randAlphaNum 64`	Always	Single-line alphanumeric string
`OIDC_PRIVATE_KEY`	RSA signing key for the OIDC IdP	`genPrivateKey "rsa"` (PEM)	Always — multi-line, requires block scalar	Multi-line RSA PEM — requires YAML block scalar (pipe character `\|`)
`OIDC_KEY_ID`	JWKS `kid` for the OIDC IdP	`crewai-oidc-<randAlphaNum 16>`	Always — must rotate with OIDC_PRIVATE_KEY	Single-line alphanumeric string
`OAUTH_COOKIE_SECRET`	OAuth service session encryption	`randAlphaNum 64`	Only when `oauth.enabled: true`	Single-line alphanumeric string
`OAUTH_DB_ENCRYPTION_KEY`	OAuth database encryption	Hex (64 chars via sha256sum)	Only when `oauth.enabled: true`	Single-line alphanumeric string
`OAUTH_INTERNAL_API_KEY`	OAuth service authentication	`randAlphaNum 64`	Only when `oauth.enabled: true`	Single-line alphanumeric string
`UV_DEFAULT_INDEX`	PyPI registry URL for enterprise crew builds	Not auto-generated — must be set manually from customer portal license credentials	Always for ArgoCD/direct Helm	URL string

OIDC_PRIVATE_KEY is the only secret requiring block scalar YAML syntax because it is a multi-line RSA PEM string. All other secrets are single-line alphanumeric values. See the Configure in Values example — the | character on the OIDC_PRIVATE_KEY line is required.OIDC_KEY_ID and OIDC_PRIVATE_KEY must be rotated together. Changing one without the other breaks Workload Identity trust relationships (AWS IAM OIDC, GCP Workload Identity, Azure Federated Credentials) that pin to the key ID.

If any Active Record encryption key changes, all previously encrypted data becomes unreadable. This includes encrypted database columns that Rails cannot decrypt with new keys. Always pre-set these values before your first deployment.

UV_DEFAULT_INDEX is not auto-generated — it is auto-populated from your Replicated license in KOTS deployments only. When deploying via ArgoCD (direct Helm/OCI install), UV_DEFAULT_INDEX is never set by the chart. Where you provide it depends on which path below you choose:

Pre-Set All Secrets path (no ESO): add it under secrets: in your values file.
External Secrets Operator path (manageAutoGenerated: true): put the URL in your secret store and reference it via externalSecret.autoGeneratedPropertyMap — the secrets: block is ignored when ESO is enabled, so a value placed there will silently have no effect.

See the Secrets Reference for how to obtain the value.

Recommended: Pre-Set All Secrets

Generate stable values and set them explicitly in your Helm values file. This bypasses lookup and randAlphaNum entirely.

Generate Secret Values

The easiest approach is to let the chart generate the values for you. Run helm template with your values file — since it renders client-side (just like ArgoCD), the chart’s auto-generation logic produces random secrets that you can extract and pin:

helm template crewai-platform <chart> --values my-values.yaml \
  | yq 'select(.kind == "Secret" and (.metadata.name | contains("-secrets"))) .data
        | to_entries | .[]
        | .key + ": " + (.value | @base64d)' \
  | grep -E '^(SECRET_KEY_BASE|ENCRYPTION_KEY|ACTIVE_RECORD_ENCRYPTION_|REGISTRY_HTTP_SECRET|CREWAI_PLUS_INTERNAL_API_KEY|WHARF_JWT_SECRET|DEPLOYMENT_INSTANCE_JWT_SECRET|CUBE_JWT_SECRET|FACTORY_DEBUG_TOKEN|OIDC_PRIVATE_KEY|OIDC_KEY_ID)'

Replace <chart> with the path to the chart (e.g., ./helm or an OCI reference). This requires yq — install with brew install yq, snap install yq, or see the yq docs. This prints the auto-generated values in plain text. Copy them into your values file:

Configure in Values

secrets:
  SECRET_KEY_BASE: "<from output above>"
  ENCRYPTION_KEY: "<from output above>"
  ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY: "<from output above>"
  ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY: "<from output above>"
  ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT: "<from output above>"
  REGISTRY_HTTP_SECRET: "<from output above>"
  CREWAI_PLUS_INTERNAL_API_KEY: "<from output above>"
  WHARF_JWT_SECRET: "<from output above>"
  DEPLOYMENT_INSTANCE_JWT_SECRET: "<from output above>"
  CUBE_JWT_SECRET: "<from output above>"
  FACTORY_DEBUG_TOKEN: "<from output above>"
  OIDC_PRIVATE_KEY: |
    <from output above — multi-line PEM>
  OIDC_KEY_ID: "<from output above>"

# OAuth secrets (only when oauth.enabled: true)
oauth:
  secrets:
    cookieSecret: "<from output above>"
    dbEncryptionKey: "<from output above>"
    internalApiKey: "<from output above>"

Store these values in a sealed secret, SOPS-encrypted file, or your CI/CD platform’s secret management — never commit them to version control in plaintext.

Alternative: generate with openssl

If you prefer to generate values independently of the chart. The | tr -dc 'A-Za-z0-9' | head -c N filter mirrors the chart’s randAlphaNum output exactly — pure alphanumeric, no +/= from base64 padding that would otherwise drift from the values a non-ESO chart install produces.

# Rails secret key base (64 chars alphanumeric)
openssl rand -base64 64 | tr -dc 'A-Za-z0-9' | head -c 64

# Encryption key (64 hex chars)
openssl rand -hex 32

# Active Record encryption keys (run 3 times, one for each key)
openssl rand -base64 64 | tr -dc 'A-Za-z0-9' | head -c 64

# Registry HTTP secret (32 chars alphanumeric)
openssl rand -base64 32 | tr -dc 'A-Za-z0-9' | head -c 32

# Internal API key (64 chars alphanumeric)
openssl rand -base64 64 | tr -dc 'A-Za-z0-9' | head -c 64

# Deployment instance JWT secret (64 chars alphanumeric)
openssl rand -base64 64 | tr -dc 'A-Za-z0-9' | head -c 64

# OIDC signing key (multi-line RSA PEM)
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048

# OIDC key id
echo "crewai-oidc-$(openssl rand -hex 8)"

# OAuth cookie secret (when oauth.enabled: true)
openssl rand -base64 64 | tr -dc 'A-Za-z0-9' | head -c 64

# OAuth DB encryption key (64 hex chars, when oauth.enabled: true)
openssl rand -hex 32

# OAuth internal API key (64 chars alphanumeric, when oauth.enabled: true)
openssl rand -base64 64 | tr -dc 'A-Za-z0-9' | head -c 64

Using ESO with ArgoCD

For deployments that want the external store (AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, Vault) as the canonical source of truth, see the dedicated External Secrets Operator guide. It covers fresh installs, migration paths from inline secrets or partial ESO setups, the backend key reference, and the pre-upgrade wait Job. When using ESO with ArgoCD specifically, the pre-upgrade hook sequencing the chart relies on works the same way under ArgoCD’s sync — ArgoCD respects Helm hook annotations and applies resources in the correct phase order.

Alternative: ArgoCD `ignoreDifferences`

You can configure ArgoCD to ignore changes to the Secret resource so that auto-generated values from the initial install are preserved:

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  ignoreDifferences:
    - group: ""
      kind: Secret
      name: crewai-platform-secrets
      jsonPointers:
        - /data

This only works if the Secret already exists from a prior install. On a fresh deployment through ArgoCD, the first sync will still generate random values — and those values will persist. However, any intentional secret changes in your values file will also be ignored. Pre-setting secrets is the more reliable approach.

Self-Signed TLS Certificates

The chart can auto-generate self-signed TLS certificates (web.tls.autoGenerate: true). These also use lookup for persistence and will regenerate on every ArgoCD sync. If you use application-level TLS with ArgoCD, provide your own certificates:

secrets:
  SSL_PRIVATE_KEY: |
    -----BEGIN PRIVATE KEY-----
    ...
    -----END PRIVATE KEY-----
  SSL_CERTIFICATE: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

Or use cert-manager or your cloud provider’s certificate management instead of web.tls.autoGenerate.

Cloud Provider Workload Identity: crewai-crews Namespace

GCP, Azure, and AWS (IRSA) all require the default ServiceAccount in the crewai-crews namespace to be annotated for Workload Identity (GKE), Azure Workload Identity (AKS), or IRSA (EKS). The chart creates this namespace via a Helm Job, so the namespace does not exist until after the first sync completes. Two-Application pattern (recommended): Create a separate ArgoCD Application that runs after the main CrewAI Application and applies only the SA annotation:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crewai-crews-sa-annotation
  annotations:
    argocd.argoproj.io/sync-wave: "2"  # Run after main CrewAI app (wave 1)
spec:
  source:
    repoURL: <your-repo>
    path: manifests/crewai-crews-sa  # Contains only the annotated ServiceAccount
  syncPolicy:
    automated:
      prune: false  # Never delete the SA

Create manifests/crewai-crews-sa/sa.yaml: GCP (GKE Workload Identity):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: crewai-crews
  annotations:
    iam.gke.io/gcp-service-account: crewai-platform@your-project.iam.gserviceaccount.com

Azure (Workload Identity):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: crewai-crews
  annotations:
    azure.workload.identity/client-id: "<IDENTITY_CLIENT_ID>"

AWS (IRSA):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: crewai-crews
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/CrewAIIRSARole

For AWS IRSA, the IAM role trust policy must also include system:serviceaccount:crewai-crews:default in addition to the annotation. See Post-Install: IRSA Trust Policy for Crew Build Pods.

The first sync will fail to apply the SA annotation (namespace does not yet exist). After the main CrewAI Application syncs successfully and the namespace exists, re-sync the annotation Application.

Dependency Ordering with Sync Waves

Do NOT add argocd.argoproj.io/sync-wave annotations to your Helm values.yaml. Helm has no podAnnotations top-level key — this is a silently-ignored unknown value. Sync wave annotations belong on ArgoCD Application manifests in your Git repository, not in chart values.

For deployments that require infrastructure dependencies (NGINX Ingress Controller, External Secrets Operator) to be ready before CrewAI starts, use multiple ArgoCD Applications with sync waves:

# Application 1: Infrastructure (wave 0 — default, runs first)
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ingress-nginx
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  source:
    repoURL: https://kubernetes.github.io/ingress-nginx
    chart: ingress-nginx
    targetRevision: "4.x.x"
  ...

# Application 2: CrewAI Platform (wave 1 — runs after infrastructure)
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crewai-platform
  annotations:
    argocd.argoproj.io/sync-wave: "1"
spec:
  source:
    repoURL: oci://registry.crewai.com/crewai/stable
    chart: crewai-platform
    targetRevision: "0.x.x"
  ...

The chart’s built-in Helm hooks (pre-install, post-install) already sequence the DB migration job before the main web/worker deployments — no additional sync wave configuration within the chart is needed for internal ordering.

OIDC_PRIVATE_KEY in the secrets table requires block scalar YAML syntax because it is a multi-line RSA PEM string, unlike all other secrets in the table which are single-line alphanumeric strings. See the Configure in Values example above for the correct | block scalar syntax.

Core Configuration

Authentication

Built-In Integrations

Advanced

Overview

Affected Secrets

Recommended: Pre-Set All Secrets

Generate Secret Values

Configure in Values

Using ESO with ArgoCD

Alternative: ArgoCD `ignoreDifferences`

Self-Signed TLS Certificates

Cloud Provider Workload Identity: crewai-crews Namespace

Dependency Ordering with Sync Waves

​Overview

​Affected Secrets

​Recommended: Pre-Set All Secrets

​Generate Secret Values

​Configure in Values

​Using ESO with ArgoCD

​Alternative: ArgoCD ignoreDifferences

​Self-Signed TLS Certificates

​Cloud Provider Workload Identity: crewai-crews Namespace

​Dependency Ordering with Sync Waves

Overview

Affected Secrets

Recommended: Pre-Set All Secrets

Generate Secret Values

Configure in Values

Using ESO with ArgoCD

Alternative: ArgoCD `ignoreDifferences`

Self-Signed TLS Certificates

Cloud Provider Workload Identity: crewai-crews Namespace

Dependency Ordering with Sync Waves