mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-26 04:04:15 +00:00
108 lines
3.9 KiB
Markdown
108 lines
3.9 KiB
Markdown
# Database Migrations
|
|
|
|
This directory contains database migrations for the ATCR AppView database.
|
|
|
|
## Schema vs Migrations
|
|
|
|
**`schema.sql`** (in parent directory) contains the **complete base schema** for fresh database installations. It includes all tables, indexes, and constraints.
|
|
|
|
**Migrations** (this directory) handle **changes to existing databases**. They are only for:
|
|
- `ALTER TABLE` statements (add/modify/drop columns)
|
|
- `UPDATE` statements (data transformations)
|
|
- `DELETE` statements (data cleanup)
|
|
- Creating/modifying indexes on existing tables
|
|
|
|
**NEW TABLES go in `schema.sql`, NOT in migrations.**
|
|
|
|
## Migration Format
|
|
|
|
Each migration is a YAML file with the following structure:
|
|
|
|
```yaml
|
|
description: Optional human-readable description of what this migration does
|
|
query: |
|
|
SQL commands to apply the migration
|
|
```
|
|
|
|
**Version and name are parsed from the filename**, so you don't need to specify them in the YAML.
|
|
|
|
## Naming Convention
|
|
|
|
Migration files **must** be named: `{version:04d}_{migration_name}.yaml`
|
|
|
|
The filename determines:
|
|
- **Version**: Numeric prefix (e.g., `0001` → version 1)
|
|
- **Name**: Everything after first underscore (e.g., `add_repository_labels` → "add repository labels")
|
|
|
|
Examples:
|
|
- `0001_remove_star_count_from_repository_stats.yaml` → version 1, name "remove star count from repository stats"
|
|
- `0002_add_repository_labels.yaml` → version 2, name "add repository labels"
|
|
- `0003_create_webhooks_table.yaml` → version 3, name "create webhooks table"
|
|
|
|
## Creating a New Migration
|
|
|
|
1. **Choose the next version number** - Look at existing migrations and increment by 1
|
|
2. **Create a new YAML file** with format `000N_descriptive_name.yaml`
|
|
3. **Add description** (optional) - Explain what the migration does
|
|
4. **Write your SQL in `query`** - Use the `|` block scalar for clean multi-line SQL
|
|
5. **Use `IF EXISTS` / `IF NOT EXISTS`** where possible for idempotency
|
|
|
|
## Examples
|
|
|
|
### Adding a column to existing table:
|
|
|
|
Filename: `0007_add_readme_url_to_manifests.yaml`
|
|
|
|
```yaml
|
|
description: Add readme_url column to manifests table for storing io.atcr.readme annotation
|
|
query: |
|
|
ALTER TABLE manifests ADD COLUMN readme_url TEXT;
|
|
```
|
|
|
|
**IMPORTANT:** After creating this migration, also add the column to `schema.sql` so fresh installations include it!
|
|
|
|
### Data transformation migration:
|
|
|
|
Filename: `0005_normalize_hold_endpoint_to_did.yaml`
|
|
|
|
```yaml
|
|
description: Normalize hold_endpoint column to store DIDs instead of URLs
|
|
query: |
|
|
-- Convert HTTPS URLs to did:web: format
|
|
UPDATE manifests
|
|
SET hold_endpoint = 'did:web:' || substr(hold_endpoint, 9)
|
|
WHERE hold_endpoint LIKE 'https://%';
|
|
|
|
-- Convert HTTP URLs to did:web: format
|
|
UPDATE manifests
|
|
SET hold_endpoint = 'did:web:' || substr(hold_endpoint, 8)
|
|
WHERE hold_endpoint LIKE 'http://%';
|
|
```
|
|
|
|
### Adding an index to existing table:
|
|
|
|
Filename: `0008_add_repository_description_index.yaml`
|
|
|
|
```yaml
|
|
description: Add index on manifests description field for faster searches
|
|
query: |
|
|
CREATE INDEX IF NOT EXISTS idx_manifests_description ON manifests(description);
|
|
```
|
|
|
|
## How Migrations Run
|
|
|
|
1. Migrations are loaded from this directory on startup
|
|
2. Sorted by version number (ascending)
|
|
3. Each migration is checked against the `schema_migrations` table
|
|
4. Only unapplied migrations are executed
|
|
5. After successful execution, the version is recorded in `schema_migrations`
|
|
|
|
## Important Notes
|
|
|
|
- **Never modify existing migrations** - Once applied, they're immutable
|
|
- **Test migrations** before committing - Ensure they work on existing databases
|
|
- **Version numbers must be unique** - The migration system will fail if duplicates exist
|
|
- **Migrations run automatically** on `InitDB()` - Schema first, then migrations
|
|
- **CRITICAL: Update `schema.sql` for structural changes** - When you ALTER a table or add columns, update both the migration AND `schema.sql` so fresh installations have the same structure
|
|
- **New tables go in `schema.sql` only** - Don't create migration files for new tables
|