Managed AWS resources

By default the Enterprise stack runs Postgres and MinIO inside the deploy boundary. For any production AWS deployment we recommend backing these onto RDS Postgres, real S3, and DynamoDB instead — better durability, point-in-time recovery, IAM-driven credentials, and operational ownership by AWS. This page covers the AWS-side setup and the config knobs. The same managed-resource path works on both the Compose and EKS deploys.

RDS Postgres

Instance setup

Engine: PostgreSQL 16 (Aurora PostgreSQL 16 also works)
Database: create the nebula logical database before application bootstrap with nebula-enterprise postgres provision, unless your platform team already manages database/user creation through its own workflow. The application role does not need CREATEDB.
Parameter group: vector, pg_partman, and pg_cron must be available to the master user; shared_preload_libraries must include pg_cron; cron.database_name must match the Nebula database name. Changing these settings may require a cluster reboot before bootstrap.
Network: private subnet in the same VPC as the compose host or EKS cluster. The host’s / cluster’s security group must be allowed inbound on :5432 from the application security group.
Storage: gp3, 100 GB minimum to start. Enable autoscaling up to ~1 TB; Nebula’s working set scales linearly with the number of collections + total ingested document volume.
Backups: RDS automatic backups, 7-day retention minimum. PITR is enabled by default.

AWS Terraform / OpenTofu

The Enterprise bundle includes a guided AWS workspace generator plus a focused RDS module at infra/aws-rds-postgres. The module creates the physical RDS PostgreSQL 16 instance, subnet group, security group, parameter group, RDS-managed master password, backups, CloudWatch Postgres log export, encryption, and Multi-AZ storage shape. It does not create the Nebula role or database; run the logical bootstrap below after the instance is available. For first-time EKS installs, generate a workspace and run the emitted phase scripts:

./nebula-enterprise init aws \
  --output-dir nebula-prod-install \
  --eks-cluster-name <cluster-name> \
  --ecr-registry <account-id>.dkr.ecr.us-east-1.amazonaws.com \
  --vpc-id <vpc-id> \
  --subnet-id <private-subnet-a> \
  --subnet-id <private-subnet-b> \
  --allowed-security-group <eks-node-or-pod-sg> \
  --s3-bucket <bucket-name> \
  --service-account-role-arn <nebula-irsa-role-arn> \
  --eso-secret-path <secrets-manager-path> \
  --domain nebula.<your-domain> \
  --ingress-certificate-arn <acm-cert-arn>

./nebula-prod-install/scripts/00-seed-app-secret.sh
./nebula-prod-install/scripts/01-provision-rds.sh
./nebula-prod-install/scripts/02-install-nebula.sh
./nebula-prod-install/scripts/03-verify.sh

The scripts keep Terraform state in nebula-prod-install/terraform/aws-rds-postgres, write nebula-install.env with mode 0600, seed the AWS Secrets Manager app secret, fetch the RDS-managed admin password into PGPASSWORD, bootstrap and verify Postgres, run Helm, and verify the rollout. Export provider keys such as OPENAI_API_KEY before 00-seed-app-secret.sh; to update an existing secret, edit nebula-prod-install/app-secret.json and rerun with UPDATE_APP_SECRET=1. If you want to run Terraform manually instead, use the bundled module directly:

cd infra/aws-rds-postgres
cp terraform.tfvars.example terraform.tfvars
# Fill in aws_region, vpc_id, private subnet_ids, and the EKS/host security group allowed to reach RDS.
terraform init
terraform apply

terraform output -raw nebula_install_env >> ../../nebula-install.env
# Requires AWS CLI + jq.
export PGPASSWORD="$(
  aws secretsmanager get-secret-value \
    --secret-id "$(terraform output -raw master_user_secret_arn)" \
    --query SecretString \
    --output text | jq -r .password
)"

Keep the admin password out of nebula-install.env; prefer PGPASSWORD or .pgpass. If you do put any password value in the env file, run chmod 600 nebula-install.env before writing it.

Database / user provisioning

Use the bundle helper as the canonical external Postgres bootstrap. It creates the Nebula role, database, required extensions, and Kubernetes credential Secret with the shape the Helm chart expects:

./nebula-enterprise postgres provision \
  --namespace nebula \
  --admin-url "postgresql://postgres@<rds-endpoint>:5432/postgres?sslmode=require" \
  --nebula-database nebula \
  --nebula-user nebula \
  --nebula-secret nebula-postgres-credentials

Set PGPASSWORD for the admin role. If you do not pass NEBULA_POSTGRES_PASSWORD in the environment, the helper generates a stable random password, reusing the existing Kubernetes Secret on rerun. Pass --nebula-host only when application pods should connect through a hostname different from the --admin-url host. To assert an existing setup without changing Postgres or Kubernetes, run the verifier with the same arguments. It checks the database objects, required extensions, Secret shapes, and that the Secret credentials can authenticate with SELECT 1:

./nebula-enterprise postgres verify \
  --namespace nebula \
  --admin-url "postgresql://postgres@<rds-endpoint>:5432/postgres?sslmode=require" \
  --nebula-database nebula \
  --nebula-user nebula \
  --nebula-secret nebula-postgres-credentials

For Compose installs, run the same helper with --skip-k8s-secrets and an explicit NEBULA_POSTGRES_PASSWORD value in the environment, then place that same password in env/.env.enterprise.

NEBULA_POSTGRES_PASSWORD=<generate-32-bytes-random> \
PGPASSWORD=<admin-password> \
./nebula-enterprise postgres provision \
  --skip-k8s-secrets \
  --admin-url "postgresql://postgres@<rds-endpoint>:5432/postgres?sslmode=require" \
  --nebula-database nebula \
  --nebula-user nebula

Then assert the database-side contract:

PGPASSWORD=<admin-password> \
./nebula-enterprise postgres verify \
  --skip-k8s-secrets \
  --admin-url "postgresql://postgres@<rds-endpoint>:5432/postgres?sslmode=require" \
  --nebula-database nebula \
  --nebula-user nebula

For the EKS installer, set these values in nebula-install.env and run the normal install:

PROVISION_POSTGRES=1
POSTGRES_ADMIN_URL=postgresql://postgres@<rds-endpoint>:5432/postgres?sslmode=require
POSTGRES_HOST=<rds-endpoint>

If you used infra/aws-rds-postgres, terraform output -raw nebula_install_env writes the Postgres values for this block. Fetch the admin password from the RDS-managed Secrets Manager secret as shown above. Before running the installer, set PGPASSWORD for the admin role or use .pgpass. If you set any password value in nebula-install.env, protect that file with chmod 600 before writing it. If your platform team provisions Postgres outside the bundle helper, mirror the same contract: a Nebula user/database, vector / pg_partman / pg_cron installed in the Nebula database, and a Kubernetes Secret with username and password keys. Run nebula-enterprise postgres verify before Helm install to check that contract without granting the helper permission to mutate resources.

Compose: wire up via `.env.enterprise`

Append to env/.env.enterprise:

NEBULA_POSTGRES_HOST=<rds-endpoint>.us-east-1.rds.amazonaws.com
NEBULA_POSTGRES_PORT=5432
NEBULA_POSTGRES_USER=nebula
NEBULA_POSTGRES_DBNAME=nebula
NEBULA_POSTGRES_SSL_MODE=require
NEBULA_POSTGRES_PASSWORD=<same NEBULA_POSTGRES_PASSWORD passed to the helper>

bootstrap.sh auto-detects the override and skips the in-stack postgres container via compose profiles. No other changes required.

EKS: wire up via Helm values

In your your-values.yaml (copied from helm/examples/eks/values.yaml):

postgres:
  mode: external
  host: <rds-endpoint>.us-east-1.rds.amazonaws.com
  port: 5432
  database: nebula
  credentialsSecret: nebula-postgres-credentials

The provisioner creates this Secret for Kubernetes installs. If your platform workflow creates it instead, preserve the same shape:

postgres.credentialsSecret (Nebula application DB): Kubernetes Secret with username and password keys (those exact lowercase key names — the chart reads them via secretKeyRef.key: username / .key: password).

S3 (object storage)

Bucket setup

Region: same as the compose host / EKS cluster (cross-region adds latency to every snapshot read)
Versioning: enabled (recommended; protects against accidental deletes)
Encryption: SSE-S3 or SSE-KMS
Public access: blocked at the account level
Lifecycle: optional — Nebula doesn’t expire its own objects, but you can set a policy on the incomplete-multipart-uploads prefix to clean up failed ingests after 7 days

IAM policy

The principal accessing S3 (instance profile / ECS task role / EKS IRSA role / IAM user) needs:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<your-bucket>",
        "arn:aws:s3:::<your-bucket>/*"
      ]
    }
  ]
}

If you’re using SSE-KMS, also grant kms:Encrypt, kms:Decrypt, kms:GenerateDataKey on the KMS key ARN.

Compose: wire up via `.env.enterprise`

Append to env/.env.enterprise:

NEBULA_USE_EXTERNAL_S3=1

# Graph-engine (Rust) S3 config:
NEBULA_S3_BUCKET=<your-bucket>
NEBULA_S3_ENDPOINT_URL=
NEBULA_S3_REGION=us-east-1
NEBULA_LOCAL_WRITE_REGION=us-east-1
NEBULA_DEFAULT_WORKSPACE_HOME_REGION=us-east-1

# Python service S3 config (must reference the same bucket):
S3_BUCKET_NAME=<your-bucket>
S3_GRAPH_STORAGE_BUCKET=<your-bucket>
S3_ENDPOINT_URL=
S3_REGION=us-east-1

# Empty AWS_*/S3_* credentials → SDK falls through to instance/task role:
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
S3_ACCESS_KEY_ID=
S3_SECRET_ACCESS_KEY=

The = with no value (e.g. NEBULA_S3_ENDPOINT_URL=) is intentional — it tells the AWS SDK to resolve the regional endpoint instead of falling back to the in-stack MinIO URL, and tells boto3 to use the default credential chain (instance profile, ECS task role) instead of static keys. bootstrap.sh auto-detects NEBULA_USE_EXTERNAL_S3=1 and skips the in-stack minio + minio-init containers.

EKS: wire up via Helm values

The example values file at helm/examples/eks/values.yaml has this pre-wired:

objectStorage:
  endpoint: ""              # empty → AWS SDK regional default
  bucket: <your-bucket>
  region: us-east-1
  forcePathStyle: false     # MUST be false for real AWS S3
  credentialsSecret: ""     # empty → SDK uses IRSA role on the SA

serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/nebula-irsa

DynamoDB orchestration state

Nebula stores durable orchestration state in four DynamoDB tables. Create or verify those tables before Helm install. The helper creates pk / sk string-key tables with on-demand billing and bootstraps the writer-authority records required by the runtime:

./nebula-enterprise orchestration dynamodb ensure \
  --project-name nebula_default \
  --aws-region us-east-1 \
  --writer-region us-east-1 \
  --state-table nebula-orchestration-state \
  --runtime-table nebula-orchestration-runtime \
  --events-table nebula-orchestration-events \
  --transitions-table nebula-orchestration-transitions

For DynamoDB-compatible endpoints, add --endpoint-url <endpoint> and use the same endpoint in Helm values. Grant the Nebula runtime principal access to the four tables for GetItem, PutItem, DeleteItem, Query, BatchGetItem, BatchWriteItem, UpdateItem, ConditionCheckItem, and DescribeTable. DynamoDB transaction writes are authorized through those underlying item permissions plus ConditionCheckItem; the endpoint must still support the TransactWriteItems API. Then wire the table names into Helm values:

secrets:
  values:
    NEBULA_ORCHESTRATION_DYNAMODB_STATE_TABLE: nebula-orchestration-state
    NEBULA_ORCHESTRATION_DYNAMODB_RUNTIME_TABLE: nebula-orchestration-runtime
    NEBULA_ORCHESTRATION_DYNAMODB_EVENTS_TABLE: nebula-orchestration-events
    NEBULA_ORCHESTRATION_DYNAMODB_TRANSITIONS_TABLE: nebula-orchestration-transitions
    NEBULA_ORCHESTRATION_DYNAMODB_ENDPOINT_URL: ""

Leave NEBULA_ORCHESTRATION_DYNAMODB_ENDPOINT_URL empty for AWS DynamoDB so boto3 uses the regional endpoint.

Sanity checks

After re-bootstrapping with managed resources, verify:

Check	Command	Expected
App reaches RDS	`curl -fsS http://localhost:7272/v1/health`	Health endpoint returns OK
Migrations applied	Alembic head matches bundle	Tables `collections`, `memories`, `entities` exist in the `nebula` DB
Postgres extensions loaded	`\dx vector`, `\dx pg_partman`, `\dx pg_cron` in psql	all three extensions are present
S3 writes succeed	Ingest a small doc	New objects appear under `<your-bucket>/nebula-graphs/`
Orchestration state is available	`aws dynamodb describe-table --table-name "${NEBULA_ORCHESTRATION_DYNAMODB_STATE_TABLE:-nebula-orchestration-state}"`	Table status is `ACTIVE`; add `--endpoint-url "$NEBULA_ORCHESTRATION_DYNAMODB_ENDPOINT_URL"` for DynamoDB-compatible endpoints

Migrating an existing in-stack deploy to managed resources

If you’ve been running with in-stack Postgres + MinIO and want to move to RDS + S3 without losing data:

Dump in-stack Postgres — docker exec -t <postgres-container> pg_dumpall -U postgres > nebula-dump.sql
Restore into RDS — psql -h <rds-endpoint> -U postgres < nebula-dump.sql
Copy MinIO contents to S3 — aws s3 sync s3://nebula-files/ s3://<your-bucket>/ --source-region us-east-1 --region us-east-1 (with appropriate MinIO/S3 credentials)
Update .env.enterprise with the managed-resource overrides above
Restart: ./enterprise/bootstrap.sh — the script will pick up the overrides and skip the in-stack services

Test on a staging deployment before doing this in production; the cutover involves a brief downtime window between the in-stack stop and the managed-resource start.

Get Started

Kubernetes

Docker Compose

Connectors

Reference

Managed AWS resources

RDS Postgres

Instance setup

AWS Terraform / OpenTofu

Database / user provisioning

Compose: wire up via `.env.enterprise`

EKS: wire up via Helm values

S3 (object storage)

Bucket setup

IAM policy

Compose: wire up via `.env.enterprise`

EKS: wire up via Helm values

DynamoDB orchestration state

Sanity checks

Migrating an existing in-stack deploy to managed resources

​RDS Postgres

​Instance setup

​AWS Terraform / OpenTofu

​Database / user provisioning

​Compose: wire up via .env.enterprise

​EKS: wire up via Helm values

​S3 (object storage)

​Bucket setup

​IAM policy

​Compose: wire up via .env.enterprise

​EKS: wire up via Helm values

​DynamoDB orchestration state

​Sanity checks

​Migrating an existing in-stack deploy to managed resources

RDS Postgres

Instance setup

AWS Terraform / OpenTofu

Database / user provisioning

Compose: wire up via `.env.enterprise`

EKS: wire up via Helm values

S3 (object storage)

Bucket setup

IAM policy

Compose: wire up via `.env.enterprise`

EKS: wire up via Helm values

DynamoDB orchestration state

Sanity checks

Migrating an existing in-stack deploy to managed resources