Prerequisites
- AWS CLI installed and configured with appropriate credentials
- AWS Copilot CLI installed (installation guide)
- Docker installed and running
- An AWS account with appropriate permissions
- Inbox Zero repository cloned locally (run all commands from the repo root)
CLI Setup
The CLI automates Copilot setup, addons (RDS + ElastiCache), secrets, and deployment. Run from the cloned repo root:The CLI will updateIf you use the CLI, you can skip the manual steps below.copilot/environments/addons/addons.parameters.yml, configure SSM secrets, deploy the environment, and then deploy the service. It also handles the webhook gateway if enabled. Note: The CLI now writesDATABASE_URL,DIRECT_URL, andREDIS_URLafter the environment deploy, because creating those SSM parameters inside addon templates can trigger EarlyValidation failures.
Manual Copilot Setup
Use this section if you prefer to drive Copilot directly.1. Initialize the Copilot Application
First, initialize a new Copilot application with your domain:<YOUR DOMAIN HERE> with your actual domain (without the http:// or https:// prefix), for example: example.com.
This creates the Copilot application structure and sets up your domain.
Note: The--domainflag only works if your domain is hosted on AWS Route53. If your domain is managed elsewhere, omit the--domainflag and remove thehttpsection fromcopilot/inbox-zero-ecs/manifest.yml(thealiasandhosted_zonefields). You’ll need to configure your domain’s DNS separately to point to the load balancer.
2. Configure the Service Manifest
Before initializing the service, configure the environment variables in the manifest file. The service manifest (copilot/inbox-zero-ecs/manifest.yml) is already included in the repository.
Edit copilot/inbox-zero-ecs/manifest.yml to add your environment variables in the variables section.
Required environment variables include:
DATABASE_URL- Your PostgreSQL connection stringDIRECT_URL- Direct database connection (for migrations)AUTH_SECRET- Authentication secretGOOGLE_CLIENT_ID- Google OAuth client IDGOOGLE_CLIENT_SECRET- Google OAuth client secretNEXT_PUBLIC_BASE_URL- Your application URL- And other required variables (see
apps/web/env.ts)
secrets section instead of variables (see Managing Secrets below).
3. Initialize the Production Environment
Create a production environment:- AWS profile/region (if not already configured)
- Other infrastructure options
4. Initialize the Service
Initialize the Load Balanced Web Service:5. Deploy the Environment
Deploy the production environment infrastructure:6. Deploy the Service
Deploy your application service:- Use the pre-built Docker image from GitHub Container Registry (
ghcr.io/elie222/inbox-zero:latest), or - Build your Docker image using
docker/Dockerfile.prodif you prefer to build from source - Push the image to Amazon ECR (if building)
- Deploy the service to ECS/Fargate
- Set up the load balancer and domain
image.location line in copilot/inbox-zero-ecs/manifest.yml and Copilot will build using the image.build configuration.
Post-Deployment
The following sections apply whether you used the CLI or manual setup.Updating Your Deployment
To update your application after making changes:- Pull the latest pre-built image from GitHub Container Registry (if using the default configuration), or
- Rebuild and redeploy your service with the latest changes (if building from source)
ElastiCache Redis (Optional)
Redis is deployed as an environment addon. You can enable or change its size by editingcopilot/environments/addons/addons.parameters.yml:
Managing Secrets
For sensitive values, use AWS Systems Manager Parameter Store:-
Store secrets in Parameter Store:
-
Reference them in
manifest.yml:
Viewing Logs
View your application logs:Checking Service Status
Check the status of your service:Database Migrations
Database migrations run automatically on container startup via thedocker/scripts/start.sh script. The script uses prisma migrate deploy to apply any pending migrations.
Important: The service manifest includes a grace_period of 320 seconds in the healthcheck configuration to ensure the container is not killed before migrations complete. This is especially important for the initial deployment when all migrations need to be applied. If you have a large number of migrations, you may need to increase this value in copilot/inbox-zero-ecs/manifest.yml.
If you need to manually run migrations:
Troubleshooting
Service Won’t Start
- Check logs:
copilot svc logs - Verify environment variables are set correctly
- Ensure database is accessible from the ECS task
- Check that the Docker image builds successfully
Migration Issues
If migrations fail:- Check database connectivity
- Verify
DATABASE_URLandDIRECT_URLare correct - Check the container logs for specific error messages
- You may need to manually resolve failed migrations using
prisma migrate resolve
Addons Change Set EarlyValidation
Ifcopilot env deploy fails with AWS::EarlyValidation::PropertyValidation, make sure addon
templates do not create SSM parameters that include dynamic Secrets Manager references. The CLI
setup flow creates DATABASE_URL, DIRECT_URL, and REDIS_URL after the environment deploy.
Domain Not Working
- Verify DNS settings for your domain
- Check that the load balancer is properly configured
- Ensure SSL certificate is provisioned (Copilot handles this automatically)
Firewalled Deployments (Webhook Gateway)
For deployments where the main application is behind a firewall or private network (e.g., only accessible to employees via VPN), you need a way for Google Pub/Sub to deliver Gmail webhook notifications. The webhook gateway addon solves this by creating a public API Gateway endpoint that validates Google’s OIDC tokens before forwarding to your private infrastructure.Prerequisites
- IAM User (not root): AWS Copilot requires IAM role assumption, which doesn’t work with root account credentials. Create an IAM user with
AdministratorAccesspolicy. - AWS CLI Profile: Configure an AWS CLI profile for your deployment:
- Set environment variables before running Copilot commands:
Architecture
- API Gateway: Public endpoint that Google Pub/Sub can reach
- JWT Authorizer: Validates Google’s OIDC tokens cryptographically
- VPC Link: Connects API Gateway to your private VPC
- Internal ALB: Your Copilot-managed load balancer
How It Works
- Google Pub/Sub sends webhook requests with a signed JWT in the
Authorizationheader - API Gateway validates the JWT:
- Verifies signature using Google’s public keys
- Checks issuer is
https://accounts.google.com - Validates audience matches your configured endpoint
- Ensures token is not expired
- Valid requests are forwarded to your internal ALB via VPC Link
- Invalid requests are rejected with 401 (never reach your app)
Deployment
The webhook gateway is an environment addon. However, it requires the ALB’s HTTPS listener which is only created when a Load Balanced Web Service is deployed. Follow this specific order:
Important: The addon references HTTPSListenerArn which only exists after a service is deployed. If you try to deploy the environment addon before the service, it will fail.
First-time Setup (New Deployment)
Keep the webhook gateway template incopilot/templates/ until the service is deployed.
-
Deploy the environment (without the addon):
-
Deploy the service (this creates the ALB and HTTPS listener):
-
Add and deploy the addon:
Existing Deployment (Service Already Running)
If you already have a deployed service with an ALB, add the addon then deploy the environment:Get the Webhook Endpoint URL
After the addon is deployed, get the webhook URL from the addon stack outputs:https://abc123xyz.execute-api.us-east-1.amazonaws.com/api/google/webhook
Google Cloud Configuration
Configure your Google Cloud Pub/Sub push subscription to use OIDC authentication:-
Create or update the push subscription:
Or update an existing subscription:
-
Grant token creation permissions:
Custom Domain (Optional)
If you want to use a custom domain for the webhook endpoint:-
Edit
copilot/environments/addons/addons.parameters.yml: - Set up a custom domain in API Gateway (via AWS Console or additional CloudFormation)
- Update the Google Pub/Sub subscription with the custom domain URL
Verification
Test that the endpoint correctly rejects unauthenticated requests:Security Notes
| Aspect | Details |
|---|---|
| Authentication | Cryptographic JWT validation using Google’s public keys |
| Issuer | Fixed to https://accounts.google.com |
| Audience | Must match exactly between AWS and Google configurations |
| Token lifetime | Google tokens are valid for up to 1 hour |
| Throttling | API Gateway applies rate limiting (50 req/sec, 100 burst) |
Troubleshooting
401 Unauthorized from API Gateway:- Verify the audience in Google Pub/Sub matches the AWS configuration exactly
- Check that the service account has
iam.serviceAccountTokenCreatorpermissions - Ensure the push subscription has OIDC authentication enabled
- The VPC Link may not have connectivity to the ALB
- Check security group rules allow traffic from API Gateway to ALB
- Verify the ALB listener is healthy
Additional Resources
- AWS Copilot Documentation
- Copilot Manifest Reference
- Self-Hosting Guide - For local Docker setup
- Google Pub/Sub Push Authentication