Skip to main content

Cloud Storage Component

The Cloud Storage component creates GCS buckets for application data, static assets, backups, and observability data.

Buckets Created

BucketPurposeAccess
{project}-app-dataApplication filesPrivate
{project}-static-assetsPublic static filesPublic
{project}-backupsDatabase backupsPrivate
{project}-langfuseObservability dataPrivate

Architecture

Configuration

# All environments
config:
serko-northsky:langfuseEnabled: "true"

# Pulumi.dev.yaml
config:
serko-northsky:retentionDays: "30"

# Pulumi.prod.yaml
config:
serko-northsky:retentionDays: "90"

Bucket Details

App Data Bucket

Stores application-generated files with versioning enabled:

{
name: `${projectId}-app-data`,
location: region,
versioning: { enabled: true },
uniformBucketLevelAccess: true,
}

Features:

  • Object versioning for data recovery
  • Lifecycle rules for cost management
  • Uniform bucket-level access

Static Assets Bucket

Public bucket for static files (images, CSS, JS):

{
name: `${projectId}-static-assets`,
location: region,
uniformBucketLevelAccess: true,
// Public access configured via IAM
}

Features:

  • Public read access
  • Optimized for CDN integration
  • CORS configuration for web access

Backups Bucket

Secure storage for database backups:

{
name: `${projectId}-backups`,
location: region,
versioning: { enabled: true },
lifecycleRule: [{
action: { type: 'Delete' },
condition: { age: retentionDays },
}],
}

Features:

  • Automatic cleanup of old backups
  • Versioning for backup history
  • Retention: 30 days (dev/test) or 90 days (prod)

Langfuse Bucket

Storage for AI observability data:

{
name: `${projectId}-langfuse`,
location: region,
uniformBucketLevelAccess: true,
}

Features:

  • Stores traces, spans, and metrics
  • Used by Langfuse for LLM observability

Outputs

interface GcsOutputs {
appDataBucketName: string;
staticAssetsBucketName: string;
backupsBucketName: string;
langfuseBucketName?: string;
}

Usage Examples

Upload Files (Backend)

from google.cloud import storage

client = storage.Client()
bucket = client.bucket('project-app-data')
blob = bucket.blob('uploads/file.pdf')
blob.upload_from_filename('/tmp/file.pdf')

Access Static Assets

https://storage.googleapis.com/project-static-assets/images/logo.png

Backup Database

# Export database to GCS
pg_dump $DATABASE_URL | gcloud storage cp - gs://project-backups/db-$(date +%Y%m%d).sql.gz

IAM Permissions

Service AccountBucketsRole
app@project.iamapp-data, static-assetsstorage.objectAdmin
cicd@project.iamstatic-assetsstorage.objectAdmin
langfuse@project.iamlangfusestorage.objectAdmin

Lifecycle Management

Old objects are automatically deleted based on retention policy:

EnvironmentRetention
Development30 days
Testing30 days
Production90 days