at-container-registry/pkg/appview/db/migrations

Database Migrations

This directory contains database migrations for the ATCR AppView database.

Schema vs Migrations

schema.sql (in parent directory) contains the complete base schema for fresh database installations. It includes all tables, indexes, and constraints.

Migrations (this directory) handle changes to existing databases. They are only for:

• ALTER TABLE statements (add/modify/drop columns)
• UPDATE statements (data transformations)
• DELETE statements (data cleanup)
• Creating/modifying indexes on existing tables

NEW TABLES go in schema.sql, NOT in migrations.

Migration Format

Each migration is a YAML file with the following structure:

description: Optional human-readable description of what this migration does
query: |
  SQL commands to apply the migration

Version and name are parsed from the filename, so you don't need to specify them in the YAML.

Naming Convention

Migration files must be named: {version:04d}_{migration_name}.yaml

The filename determines:

• Version: Numeric prefix (e.g., 0001 → version 1)
• Name: Everything after first underscore (e.g., add_repository_labels → "add repository labels")

Examples:

• 0001_remove_star_count_from_repository_stats.yaml → version 1, name "remove star count from repository stats"
• 0002_add_repository_labels.yaml → version 2, name "add repository labels"
• 0003_create_webhooks_table.yaml → version 3, name "create webhooks table"

Creating a New Migration

1. Choose the next version number - Look at existing migrations and increment by 1
2. Create a new YAML file with format 000N_descriptive_name.yaml
3. Add description (optional) - Explain what the migration does
4. Write your SQL in query - Use the | block scalar for clean multi-line SQL
5. Use IF EXISTS / IF NOT EXISTS where possible for idempotency

Examples

Adding a column to existing table:

Filename: 0007_add_readme_url_to_manifests.yaml

description: Add readme_url column to manifests table for storing io.atcr.readme annotation
query: |
  ALTER TABLE manifests ADD COLUMN readme_url TEXT;

IMPORTANT: After creating this migration, also add the column to schema.sql so fresh installations include it!

Data transformation migration:

Filename: 0005_normalize_hold_endpoint_to_did.yaml

description: Normalize hold_endpoint column to store DIDs instead of URLs
query: |
  -- Convert HTTPS URLs to did:web: format
  UPDATE manifests
  SET hold_endpoint = 'did:web:' || substr(hold_endpoint, 9)
  WHERE hold_endpoint LIKE 'https://%';

  -- Convert HTTP URLs to did:web: format
  UPDATE manifests
  SET hold_endpoint = 'did:web:' || substr(hold_endpoint, 8)
  WHERE hold_endpoint LIKE 'http://%';

Adding an index to existing table:

Filename: 0008_add_repository_description_index.yaml

description: Add index on manifests description field for faster searches
query: |
  CREATE INDEX IF NOT EXISTS idx_manifests_description ON manifests(description);

How Migrations Run

1. Migrations are loaded from this directory on startup
2. Sorted by version number (ascending)
3. Each migration is checked against the schema_migrations table
4. Only unapplied migrations are executed
5. After successful execution, the version is recorded in schema_migrations

Important Notes

• Never modify existing migrations - Once applied, they're immutable
• Test migrations before committing - Ensure they work on existing databases
• Version numbers must be unique - The migration system will fail if duplicates exist
• Migrations run automatically on InitDB() - Schema first, then migrations
• CRITICAL: Update schema.sql for structural changes - When you ALTER a table or add columns, update both the migration AND schema.sql so fresh installations have the same structure
• New tables go in schema.sql only - Don't create migration files for new tables