Deployment Guide

Note: Production infrastructure (VPC, ECS, Aurora, S3, IAM, Prefect API, etc.) is now defined and deployed from the separate blog_infra repository (see its infra/envs/prod stack). Use that repo to apply Terraform; this document focuses on the application-side pieces for blog_data.

How deployment works

Application deployment for blog_data is now handled entirely by CircleCI and a Prefect deployer ECS task defined in the blog_infra repository:

1. You push to main, pipeline, or ecs in rlhatcher/blog_data.
2. CircleCI builds the blog_data Docker image and pushes it to the blog-data ECR repository tagged with the git commit SHA.
3. CircleCI runs the deploy-to-prefect job, which:
   • Assumes an AWS IAM role using STS.
   • Starts a one-off Prefect deployer ECS task (from blog_infra).
   • That task clones blog_data, installs it, and runs prefect deploy --all against the self-hosted Prefect API at https://pipelines.rocketclub.online.
4. The Prefect deployer script ensures the blog-data-pool ECS work pool exists with the correct base job template and updates all deployments using the new image tag.

Infrastructure (VPC, ECS services, ALB/CloudFront, Aurora, etc.) is provisioned and updated from the blog_infra repo; see that repo’s docs for Terraform commands and infra rollbacks.

Visual overview

For a high-level view of how the Prefect worker pool and CI/CD fit together, see the generated diagrams under docs/diagrams/:

• Prefect runtime:

  

• CI & Prefect deployer:

  

These diagrams are generated from code using generate_architecture_diagram.py. See ARCHITECTURE.md for generation instructions.

Day-to-day deployment

For practical deployment tasks, use the following documents:

• CircleCI Setup Guide – how CircleCI connects to AWS and Prefect.
• CircleCI Deployment Guide – detailed behavior of the build-and-push-image and deploy-to-prefect jobs.
• Quick Start Deployment – 5-minute path from zero to automated deployments.
• Deployment Checklist – end-to-end checklist for verifying a deployment.
• Prefect Work Pool Configuration – details of the blog-data-pool ECS work pool and how it is managed by the Prefect deployer.

This DEPLOYMENT.md file is intentionally high-level – treat code and prefect.yaml as the source of truth for deployments, and use blog_infra for all production infrastructure changes.