Overview
This guide covers AWS-specific integration for CrewAI Platform deployments on Amazon EKS. It focuses on how CrewAI uses AWS services and the platform-specific configuration required, rather than general AWS setup.This guide assumes you have:
- An EKS cluster running Kubernetes 1.32.0+
- AWS CLI and kubectl configured
- Helm 3.10+ installed
- Basic familiarity with AWS services (RDS, S3, ALB)
Prerequisites
Before configuring CrewAI Platform, ensure these AWS components are in place: CrewAI Platform supports AMD64 (x86_64) Kubernetes worker nodes. ARM64 (aarch64) worker nodes are not currently supported. For full platform requirements, see the Requirements Guide.Required AWS Infrastructure
Amazon Aurora for PostgreSQL
CrewAI Platform requires PostgreSQL 16.8+ for production deployments. This section covers RDS-specific requirements for CrewAI.Aurora Instance Sizing
Minimum recommended specifications based on CrewAI workload characteristics:CrewAI’s Rails-based architecture benefits from memory-optimized instances (R6g family). Use gp3 storage with minimum 3000 IOPS for production workloads.
Network Connectivity
CrewAI pods must reach your RDS instance. Two options: Option 1: RDS in Private Subnet (Recommended)- Place RDS in private subnets within your EKS VPC
- No internet exposure
- Security group allows PostgreSQL (5432) from EKS node security group
- Enable public accessibility on RDS instance
- Configure security group to allow EKS NAT gateway IPs
- Requires SSL/TLS (enforce
sslmode=require)
Database Setup
CrewAI requires four databases — primary, cable, OAuth, and Wharf for OTLP trace storage:When using an external RDS instance, you must manually create these databases before deploying CrewAI Platform. The Helm chart does not automatically create databases when
postgres.enabled: false.Helm Configuration
Amazon S3 for Object Storage
CrewAI Platform uses S3 for storing crew artifacts, tool outputs, and user uploads. This section covers S3 integration and authentication.S3 Bucket Configuration
S3 Authentication Options
CrewAI supports three authentication methods for S3. Choose based on your security requirements:Option 1: Pod Identity (Recommended - Newest)
Best for: New EKS deployments (EKS 1.24+), highest security Pod Identity provides credentials without OIDC configuration or static keys. Benefits:- No long-lived credentials
- Simplified setup vs IRSA — does not require an OIDC provider on the EKS cluster
- Automatic credential rotation
- Native EKS integration
- Create IAM policy for S3 access:
- Create IAM role and associate with Pod Identity:
When
rbac.create: true (the default), the chart automatically creates a ServiceAccount named crewai-sa. The Pod Identity association must reference this name. You do not need to set serviceAccount.name explicitly.Option 2: Static Access Keys
Best for: Development environments, non-EKS Kubernetes clustersOption 3: IAM Roles for Service Accounts (IRSA)
Best for: Existing EKS clusters with OIDC provider already configured IRSA binds an IAM role to a Kubernetes ServiceAccount via an OIDC trust relationship. Unlike Pod Identity, IRSA requires an OIDC provider on the EKS cluster. Prerequisites:- EKS cluster with OIDC provider configured (AWS IRSA Setup)
- IAM role with the S3 policy from Option 1 attached
- Get your OIDC provider URL:
- Create IAM role with OIDC trust relationship:
- Configure Helm values with the IRSA annotation:
The same IAM role can grant both S3 and ECR permissions. See Combined IAM Policy for a single role covering all AWS services.
Application Load Balancer (ALB)
CrewAI Platform requires specific ALB configuration to support long-running crew executions and WebSocket connections.CrewAI-Specific ALB Requirements
CrewAI’s architecture has specific needs:- Long-running requests: Crew executions can take 5+ minutes
- WebSocket support: ActionCable requires persistent connections
- Session affinity: Not required (stateless application)
ALB Security Group Configuration
The ALB security group should allow:- Inbound: HTTPS (443) from your allowed CIDR ranges (e.g.,
0.0.0.0/0for public access) - Outbound: HTTP to EKS worker node security group on NodePort range
- Inbound: HTTP from ALB security group
ACM Certificate
CrewAI requires a valid SSL certificate:Internet-Facing vs Internal ALB
Thescheme setting controls whether the ALB is publicly accessible or restricted to your internal network.
Internet-facing ALB (default):
ALB Subnet Selection
The AWS Load Balancer Controller uses subnet tags to automatically discover which subnets to place the ALB in. If the wrong subnets are tagged, the ALB will be created in unintended subnets. Required subnet tags:
Ensure only the subnets where you want the ALB placed have the appropriate tag. Remove the tag from any subnets that should not host the ALB.
Verify current subnet tags:
Explicit Subnet Selection
If you cannot modify subnet tags (e.g., shared VPC environments), you can explicitly specify which subnets the ALB should use via theweb.ingress.annotations field:
When using explicit subnet annotations, the controller bypasses tag-based auto-discovery entirely for that ingress resource.
ALB Security Features
You can enable additional security features on the ALB usingweb.ingress.annotations. These annotations are passed directly to the AWS Load Balancer Controller.
Deletion Protection
Prevents accidental deletion of the ALB:Access Logs
Enables logging of all requests to an S3 bucket:WAF Integration
Attach an AWS WAFv2 Web ACL to protect the ALB:Combined Example
To enable all three features together, combine theload-balancer-attributes values in a single annotation:
Pre-existing Security Groups
By default, the AWS Load Balancer Controller automatically creates a security group for the ALB. In environments with restrictive AWS IAM policies that forbid automatic security group creation, you can use a pre-existing security group instead.Using a Pre-existing Security Group
Specify your existing security group ID via annotation:Disabling Backend Security Group Management
By default, the AWS Load Balancer Controller also modifies the backend (node/pod) security groups to allow inbound traffic from the ALB. If your AWS policies also forbid modifying existing security groups, disable this behavior:Pre-existing Security Group Requirements
Your pre-existing security group must allow the following traffic:Complete Example for Restrictive Environments
For environments where both security group creation and modification are forbidden:Amazon ECR for Container Images
CrewAI Platform requires Amazon ECR for storing crew automation container images. When users create and deploy crews, CrewAI builds container images and pushes them to ECR.ECR Repository Requirements
Create ECR Repository
- ✅
123456789012.dkr.ecr.us-east-1.amazonaws.com/crewai-enterprise - ✅
123456789012.dkr.ecr.us-east-1.amazonaws.com/my-org/crewai-enterprise - ✅
123456789012.dkr.ecr.us-east-1.amazonaws.com/prod/crewai-enterprise - ❌
123456789012.dkr.ecr.us-east-1.amazonaws.com/crewai(must end in/crewai-enterprise) - ❌
123456789012.dkr.ecr.us-east-1.amazonaws.com/crewai-platform(wrong suffix)
ECR Authentication with Pod Identity
CrewAI pods require ECR push and pull permissions for building and deploying crew images. Create IAM policy for ECR access:Combined IAM Policy (S3 + ECR)
For production deployments using Pod Identity, combine S3 and ECR permissions:Helm Configuration for ECR
ECR Authentication: credHelper vs Pod Identity / IRSA
When mirroring platform images to ECR and usingimage.registries[].credHelper: "ecr-login", be aware of an important distinction:
For crew image builds and pushes (via BuildKit), Pod Identity or IRSA on the crewai-sa ServiceAccount is the recommended approach. BuildKit inherits the pod’s IAM credentials and can authenticate to ECR without static tokens.
Verifying ECR Access
Test ECR authentication from CrewAI pods:Post-Install: IRSA Trust Policy for Crew Build Pods
When a crew is deployed, BuildKit spawns per-build child pods in thecrewai-crews namespace. These child pods use the default ServiceAccount in that namespace — not the crewai-sa ServiceAccount used by the main platform pods. If you are using IRSA for ECR authentication, the trust policy must include both ServiceAccounts.
Update the IAM role trust policy to include the crews namespace:
The trust policy uses
StringEquals with an array to allow multiple ServiceAccounts to assume the same role. Replace crewai with your platform namespace and crewai-crews with the value of global.crewNamespace (default: crewai-crews).crewai-crews namespace automatically), annotate the default ServiceAccount so build pods inherit IRSA credentials:
ArgoCD users: This step requires a two-Application pattern because the
crewai-crews namespace does not exist until after the first sync completes. See the ArgoCD Deployment Guide.k8s_namespace_isolation is enabled, crew pods run in per-org namespaces (e.g., crewai-crews-org-1, crewai-crews-org-2). Each org namespace’s default ServiceAccount must also be included in the IRSA trust policy and annotated. Use a wildcard condition with StringLike instead:
AWS Secrets Manager Integration
AWS Secrets Manager provides centralized secret management with automatic rotation for CrewAI Platform.Which Secrets to Store
Store in AWS Secrets Manager (sensitive, need rotation):DB_PASSWORD- Database credentialsSECRET_KEY_BASE- Rails secret keyENTRA_ID_CLIENT_SECRET/OKTA_CLIENT_SECRET- OAuth secretsAWS_SECRET_ACCESS_KEY- If using static S3 credentialsGITHUB_TOKEN- For private repository access
DB_HOST,DB_PORT,DB_USER,POSTGRES_DB,POSTGRES_CABLE_DBAWS_REGION,AWS_BUCKETAPPLICATION_HOSTAUTH_PROVIDER
Secret Structure in Secrets Manager
CrewAI expects secrets in specific formats. Two options: Option 1: Single Secret with Multiple Keys Create one secretcrewai/platform with JSON structure:
crewai/db-passwordcrewai/secret-key-basecrewai/entra-id-credentials(JSON with client_id, client_secret, tenant_id)
External Secrets Operator Setup
CrewAI uses External Secrets Operator (ESO) to sync secrets from AWS Secrets Manager to Kubernetes. Install ESO (if not already installed):Secret Mapping Example
If using single secret with JSON structure:Complete AWS Deployment Example
If using WorkOS authentication, place
WORKOS_API_KEY under envVars: — NOT under secrets:. See the WorkOS SSO guide for the full explanation and the correct YAML placement. This is a known chart limitation.Troubleshooting AWS-Specific Issues
ALB Not Provisioning
Symptoms: Ingress shows no ADDRESS after several minutes- Incorrect
schemecasing — Theschemevalue is case-sensitive. Use"internal"or"internet-facing"(all lowercase). Values like"Internal"or"Internet-Facing"are silently rejected. See Internet-Facing vs Internal ALB. - AWS Load Balancer Controller not installed or not running
- Insufficient IAM permissions for LBC
- Subnet tags missing for ALB discovery
- Public subnets:
kubernetes.io/role/elb=1 - Private subnets:
kubernetes.io/role/internal-elb=1
ALB Created in Wrong Subnets
Symptoms: ALB is provisioned but placed in subnets outside the EKS cluster VPC or in unintended subnets. Common causes:- Multiple subnets tagged with
kubernetes.io/role/elborkubernetes.io/role/internal-elbacross different VPCs or availability zones - Shared VPC where non-EKS subnets also carry the discovery tag
- Identify which subnets the EKS cluster uses:
- Remove discovery tags from non-EKS subnets and ensure only the correct subnets are tagged. See ALB Subnet Selection for details.
- Alternatively, use the explicit subnet annotation to bypass auto-discovery:
- After fixing tags or adding the annotation, delete the ingress and re-deploy to force the LBC to re-discover subnets.
RDS Connection Timeout
Symptoms: Pods showcould not connect to server: Connection timed out
Check security groups:
S3 Access Denied
Symptoms: Logs showAccess Denied or 403 errors for S3 operations
Verify authentication method:
For Pod Identity:
Secrets Manager Access Denied
Symptoms: ExternalSecret showsSecretSyncedError
ECR Builds Fail After Running Successfully
Symptoms: Crew builds worked initially but start failing withunauthorized or denied errors after several hours
Common causes:
- ECR auth token expiration — ECR tokens obtained via
aws ecr get-login-passwordexpire after 12 hours. If you configuredimage.registrieswith staticusername/password(using an ECR token), builds will fail once the token expires. credHelperbinary not present — If usingcredHelper: "ecr-login"inimage.registries, theecr-loginbinary must be available in the BuildKit image. Without it, the docker config references a helper that doesn’t exist, causing silent auth failures.
BuildKit Cannot Push to ECR
Symptoms: BuildKit logs showfailed to push or unauthorized: authentication required when building crew images
Diagnosis:
- Missing ECR permissions — The IAM role must include
ecr:PutImage,ecr:InitiateLayerUpload,ecr:UploadLayerPart, andecr:CompleteLayerUpload. See Combined IAM Policy. - ECR repository doesn’t exist — The repository path ending in
/crewai-enterprisemust be pre-created in ECR. CrewAI does not auto-create ECR repositories. - Pod Identity / IRSA not reaching BuildKit — Ensure the Pod Identity association or IRSA trust relationship references the correct ServiceAccount (
crewai-sa) and namespace.
ServiceAccount Mismatch with Pod Identity or IRSA
Symptoms: Pods showAccessDeniedException, ExpiredTokenException, or WebIdentityErr despite correct IAM policies
Common causes:
- Wrong ServiceAccount name in association/trust — Pod Identity associations and IRSA trust relationships must reference the exact ServiceAccount name (
crewai-saby default) and namespace. A mismatch means pods get no IAM credentials. - Namespace mismatch — The namespace in the Pod Identity association or IRSA trust policy must match the Helm release namespace.
- Missing crew build pod ServiceAccount — The IRSA trust policy only includes
crewai-sabut notcrewai-crews:default. Platform pods work fine, but crew image builds fail withUnauthorizedwhen pushing to ECR. See Post-Install: IRSA Trust Policy for Crew Build Pods.
