Skip to content

Latest commit

 

History

History
528 lines (433 loc) · 15.9 KB

File metadata and controls

528 lines (433 loc) · 15.9 KB

CloudForge Backend

FastAPI backend for CloudForge — AWS resource management platform.

Base URL: http://localhost:8000 Interactive docs: http://localhost:8000/docs


Table of Contents


Getting Started

# Copy and configure environment
cp .env.example .env
# Edit .env — at minimum set SECRET_KEY

# Install dependencies
pip install -r requirements.txt

# Run (development)
uvicorn app.main:app --reload

# Or via Docker Compose (recommended)
docker-compose up

Environment Variables

Variable Required Default Description
SECRET_KEY Yes JWT signing key (32+ random chars)
DATABASE_URL Yes Async PostgreSQL URL (postgresql+asyncpg://...)
DATABASE_URL_SYNC Yes Sync PostgreSQL URL for Celery workers
REDIS_URL Yes redis://localhost:6379/0 Redis connection
CELERY_BROKER_URL No same as REDIS_URL Celery broker
CELERY_RESULT_BACKEND No redis://localhost:6379/1 Celery results
TERRAFORM_WORKSPACES_DIR No /app/terraform/workspaces Where workspace HCL files are written
TERRAFORM_BIN No terraform Path to terraform binary
ALLOWED_ORIGINS No http://localhost:3000 CORS origins (comma-separated)
ENVIRONMENT No development development auto-creates tables

Authentication

All endpoints (except /api/v1/auth/login and /api/v1/auth/register) require a Bearer token.

# Login
curl -X POST /api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"admin@example.com","password":"yourpassword"}'

# Response
{
  "access_token": "eyJ...",
  "token_type": "bearer",
  "role_name": "admin"
}

# Use token
curl -H "Authorization: Bearer eyJ..." /api/v1/auth/me

Role Management

Creating a Role

The complete workflow for setting up a new role with scoped AWS access:

Step 1 — Check available services

GET /api/v1/admin/services

Returns the list of all 19 supported AWS service types:

["ALB", "ASG", "CLOUDFRONT", "DYNAMODB", "EBS", "EC2", "ECS",
 "EKS", "ELASTICACHE", "KMS", "LAMBDA", "NAT_GATEWAY", "NLB",
 "RDS", "ROUTE53", "S3", "SECURITY_GROUP", "SNS", "SQS"]

Step 2 — Preview the IAM policy (optional)

Before creating the role, preview what IAM policy will be needed:

POST /api/v1/admin/roles/preview-iam-policy
Authorization: Bearer <admin-token>
Content-Type: application/json

["EC2", "RDS", "S3"]

Response:

{
  "services": ["EC2", "RDS", "S3"],
  "policy_document": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "CloudForgeEc2Access",
        "Effect": "Allow",
        "Action": ["ec2:RunInstances", "ec2:TerminateInstances", ...],
        "Resource": "*"
      },
      ...
    ]
  },
  "policy_json": "{\n  \"Version\": ...",
  "total_actions_granted": 47
}

Step 3 — Create the role

POST /api/v1/admin/roles
Authorization: Bearer <admin-token>
Content-Type: application/json

{
  "name": "web-team",
  "description": "Web team — can deploy EC2, RDS, and S3",
  "aws_region": "us-east-1",
  "aws_account_id": "123456789012",
  "allowed_services": ["EC2", "RDS", "S3", "ALB", "SECURITY_GROUP"],
  "permissions": {
    "can_deploy": true,
    "can_view_cost": true,
    "can_manage_roles": false
  }
}

Response includes the role ID you'll need for subsequent steps.

IAM Policy Generation

Step 4 — Get generated IAM policy + AWS setup guide

GET /api/v1/admin/roles/{role_id}/iam-policy
Authorization: Bearer <admin-token>

Response:

{
  "role_id": "...",
  "role_name": "web-team",
  "allowed_services": ["EC2", "RDS", "S3", "ALB", "SECURITY_GROUP"],
  "policy_document": { ... },
  "policy_json": "{ formatted JSON to copy-paste }",
  "setup_steps": [
    {
      "step": "1",
      "title": "Sign in to AWS Console",
      "description": "Open https://console.aws.amazon.com/iam/ in account 123456789012.",
      "method": "console"
    },
    {
      "step": "2",
      "title": "Create a new IAM Policy",
      "description": "Go to IAM → Policies → Create policy. Select the JSON tab and paste the generated policy document below. Name the policy: CloudForge-web-team-Policy",
      "method": "console",
      "action": "paste_policy_json"
    },
    {
      "step": "3",
      "title": "Create a new IAM User (for Terraform)",
      "description": "Go to IAM → Users → Create user. Username: cloudforge-web-team-terraform ...",
      "method": "console"
    },
    {
      "step": "4",
      "title": "Generate Access Keys",
      "description": "In the user's Security credentials tab, click Create access key ...",
      "method": "console"
    },
    {
      "step": "5",
      "title": "Enter credentials in CloudForge",
      "description": "Return to CloudForge → Admin → Roles → select this role → Credentials tab.",
      "method": "cloudforge"
    },
    {
      "step": "CLI Alternative",
      "title": "Create everything via AWS CLI",
      "description": "# Create policy\naws iam create-policy \\\n  --policy-name CloudForge-web-team-Policy \\\n  --policy-document file://cloudforge-policy.json\n\n# Create user\naws iam create-user --user-name cloudforge-web-team-terraform\n ...",
      "method": "cli"
    }
  ],
  "summary": {
    "services_covered": 5,
    "total_actions_granted": 73,
    "policy_name_suggestion": "CloudForge-web-team-Policy",
    "user_name_suggestion": "cloudforge-web-team-terraform"
  }
}

Setting AWS Credentials

Step 5 — Save the Terraform credentials

After creating the IAM user and access keys in AWS, save them to the role:

PUT /api/v1/admin/roles/{role_id}/credentials
Authorization: Bearer <admin-token>
Content-Type: application/json

{
  "aws_access_key": "AKIAIOSFODNN7EXAMPLE",
  "aws_secret_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "aws_account_id": "123456789012",
  "aws_region": "us-east-1"
}

Validation rules:

  • aws_access_key must be 20 characters and start with AKIA or ASIA
  • aws_secret_key must be at least 40 characters
  • Credentials are never returned in API responses — only has_credentials: true/false is exposed

To remove credentials:

DELETE /api/v1/admin/roles/{role_id}/credentials

Step 6 — Assign users to the role

PATCH /api/v1/admin/users/{user_id}/role?role_id={role_id}
Authorization: Bearer <admin-token>

The user can now log in and use their scoped canvas to deploy the services assigned to their role.


API Reference

Auth Endpoints

Method Path Auth Description
POST /api/v1/auth/login No Login → returns JWT
POST /api/v1/auth/register No Register new user
GET /api/v1/auth/me Yes Current user info

Admin Endpoints

All admin endpoints require the authenticated user's role to have name == "admin" or permissions.can_manage_roles == true.

Method Path Description
GET /api/v1/admin/services List all 19 supported AWS service types
GET /api/v1/admin/roles List all roles (including inactive)
POST /api/v1/admin/roles Create role with allowed_services
PATCH /api/v1/admin/roles/{id} Update role
DELETE /api/v1/admin/roles/{id} Soft-delete role
GET /api/v1/admin/roles/{id}/allowed-services Get allowed services list
PUT /api/v1/admin/roles/{id}/allowed-services Replace allowed services list
GET /api/v1/admin/roles/{id}/iam-policy Generate IAM policy + setup guide
POST /api/v1/admin/roles/preview-iam-policy Preview policy for service list (no save)
PUT /api/v1/admin/roles/{id}/credentials Save AWS credentials for Terraform
DELETE /api/v1/admin/roles/{id}/credentials Remove stored credentials
GET /api/v1/admin/roles/{id}/running-services All canvases + deployed resources
GET /api/v1/admin/roles/{id}/users Users assigned to role
PATCH /api/v1/admin/users/{id}/role Assign/unassign role to user
GET /api/v1/admin/overview Dashboard: all roles + cost + resource summary

Role Endpoints

Basic role CRUD (non-admin users can list/get; only admins can create/update/delete).

Method Path Auth Description
GET /api/v1/roles/ Yes List active roles
POST /api/v1/roles/ Admin Create role
GET /api/v1/roles/{id} Yes Get role detail
PATCH /api/v1/roles/{id} Admin Update role
DELETE /api/v1/roles/{id} Admin Soft-delete role

Canvas Endpoints

Canvases are scoped to the user's role. Users can only see and edit canvases belonging to their own role. Service allowlist is enforced on save.

Method Path Auth Description
GET /api/v1/canvases/ Yes List canvases for current user's role
POST /api/v1/canvases/ Yes Create canvas (validates allowed_services)
GET /api/v1/canvases/{id} Yes Get canvas with full state
PATCH /api/v1/canvases/{id} Yes Update canvas state (validates allowed_services)
DELETE /api/v1/canvases/{id} Yes Delete canvas

Canvas state format (stored as JSONB):

{
  "nodes": [
    {
      "id": "node-1",
      "type": "resource",
      "position": { "x": 200, "y": 150 },
      "data": {
        "resourceType": "EC2",
        "label": "web-server",
        "config": {
          "instanceType": "t3.medium",
          "count": 2,
          "ami": "ami-0c55b159cbfafe1f0"
        }
      }
    }
  ],
  "edges": [
    {
      "id": "edge-1",
      "source": "node-1",
      "target": "node-2"
    }
  ],
  "viewport": { "x": 0, "y": 0, "zoom": 1 }
}

Cost Endpoints

Method Path Query Params Description
GET /api/v1/cost/canvas/{id} currency=USD|INR|EUR|GBP Estimate total canvas cost
POST /api/v1/cost/diff Cost diff between two canvas states
GET /api/v1/cost/exchange-rates Current exchange rates from USD

Cost estimate response:

{
  "canvas_id": "...",
  "region": "us-east-1",
  "resources": [
    {
      "resource_type": "EC2",
      "resource_name": "web-server",
      "hourly_usd": 0.0832,
      "monthly_usd": 60.74,
      "yearly_usd": 728.83,
      "pricing_note": "t3.medium × 2 on-demand"
    }
  ],
  "total_hourly_usd": 0.1748,
  "total_monthly_usd": 127.60,
  "total_yearly_usd": 1531.25,
  "currency": "INR",
  "exchange_rate": 83.5,
  "total_monthly_converted": 10654.60,
  "total_yearly_converted": 127859.38,
  "estimated_at": "2024-01-15T10:30:00Z"
}

Cost diff request:

{
  "old_canvas_state": { "nodes": [...] },
  "new_canvas_state": { "nodes": [...] },
  "region": "us-east-1",
  "currency": "INR"
}

Deploy Endpoints

Method Path Description
POST /api/v1/deploy/{canvas_id} Trigger terraform plan/apply/destroy
GET /api/v1/deploy/{canvas_id}/preview-hcl Get generated Terraform HCL
GET /api/v1/deploy/{canvas_id}/history Last 20 deployment logs
WS /api/v1/deploy/ws/{canvas_id} Live log stream (WebSocket)

Deploy request:

{ "action": "plan" }
{ "action": "apply" }
{ "action": "destroy" }

Deploy response:

{
  "deployment_id": "...",
  "canvas_id": "...",
  "action": "apply",
  "status": "pending",
  "celery_task_id": "abc123"
}

WebSocket messages (connect to /api/v1/deploy/ws/{canvas_id} before triggering deploy):

{ "type": "log",    "line": "aws_instance.web-server: Creating...", "ts": "..." }
{ "type": "status", "status": "success", "action": "apply", "ts": "..." }
{ "type": "status", "status": "failed",  "action": "apply", "error": "...", "ts": "..." }

Terraform Service

Supported Resources

All 19 resource types have Jinja2 HCL templates in terraform/templates/:

Resource Type Template AWS Resources Created
EC2 ec2.tf.j2 aws_instance
RDS rds.tf.j2 aws_db_instance, aws_db_subnet_group
LAMBDA lambda.tf.j2 aws_lambda_function, IAM exec role
ECS ecs.tf.j2 aws_ecs_cluster, task def, service, IAM roles
EKS eks.tf.j2 aws_eks_cluster, node group, IAM roles
ASG asg.tf.j2 aws_autoscaling_group, launch template
S3 s3.tf.j2 aws_s3_bucket, versioning, public access block
EBS ebs.tf.j2 aws_ebs_volume
DYNAMODB dynamodb.tf.j2 aws_dynamodb_table
ELASTICACHE elasticache.tf.j2 aws_elasticache_cluster, subnet group
ALB alb.tf.j2 aws_lb, target group, listener
NLB alb.tf.j2 aws_lb (type=network)
NAT_GATEWAY nat_gateway.tf.j2 aws_nat_gateway, EIP, route table
CLOUDFRONT cloudfront.tf.j2 aws_cloudfront_distribution
ROUTE53 route53.tf.j2 aws_route53_zone, records
SQS sqs.tf.j2 aws_sqs_queue
SNS sns.tf.j2 aws_sns_topic, subscriptions
KMS kms.tf.j2 aws_kms_key, alias
SECURITY_GROUP security_group.tf.j2 aws_security_group

VPC, subnets, internet gateway, and route tables are always generated as the base network layer.

HCL Preview

Before deploying, get the generated HCL:

GET /api/v1/deploy/{canvas_id}/preview-hcl
Authorization: Bearer <token>

Cost Estimation

Pricing data is static on-demand rates sourced from AWS public pricing (us-east-1, Q4 2024). The engine covers:

Service Pricing Model
EC2 Per-instance per-hour × count (40+ instance types)
RDS / Aurora Per-instance per-hour + storage GB (×2 for Multi-AZ)
Lambda Per-request + per GB-second
ECS Fargate Per vCPU-hour + per GB-hour
EKS Control plane ($0.10/hr) + node EC2 costs
S3 Per GB-month (Standard/IA/Glacier) + request costs
DynamoDB Provisioned WCU/RCU or on-demand per million ops
ElastiCache Per node per-hour
ALB / NLB Per-hour
NAT Gateway Per-hour + per GB processed
CloudFront Per GB out + per 10K requests
SQS / SNS Per million messages
KMS Per key per-month + per 10K API calls
Route53 Per hosted zone + per million queries

Currency support: USD, INR, EUR, GBP. Exchange rates are static (update _STATIC_USD_TO_INR in cost_service.py or integrate a live FX API).


Running Tests

cd backend

# Unit tests only (no DB required)
pytest tests/test_cost_service.py tests/test_terraform_service.py tests/test_iam_policy_service.py -v

# All tests (requires test PostgreSQL DB)
pytest

# With coverage report
pytest --cov=app --cov-report=html
# Open htmlcov/index.html

Test files:

File What it tests
tests/test_auth.py Login, register, JWT, password validation
tests/test_roles.py Role CRUD, permissions
tests/test_admin.py IAM policy generation, credentials, running services, overview
tests/test_iam_policy_service.py Policy generation for all 19 services
tests/test_cost_service.py Per-resource cost calculations, canvas totals
tests/test_terraform_service.py HCL generation, subnet derivation, resource templates