implement hold discovery dropdown in settings. implement a data privacy export feature

2026-08-29 04:06:58 +00:00 · 2026-01-07 22:41:14 -06:00
parent d4b88b5105
commit 3409af6c67
39 changed files with 4124 additions and 159 deletions
@@ -0,0 +1,304 @@
+# Accessing Hold Data Without AppView
+
+This document explains how to retrieve your data directly from a hold service without going through the ATCR AppView. This is useful for:
+- GDPR data export requests
+- Backup and migration
+- Debugging and development
+- Building alternative clients
+
+## Quick Start: App Passwords (Recommended)
+
+The simplest way to authenticate is using an ATProto app password. This avoids the complexity of OAuth + DPoP.
+
+### Step 1: Create an App Password
+
+1. Go to your Bluesky settings: https://bsky.app/settings/app-passwords
+2. Create a new app password
+3. Save it securely (you'll only see it once)
+
+### Step 2: Get a Session Token
+
+```bash
+# Replace with your handle and app password
+HANDLE="yourhandle.bsky.social"
+APP_PASSWORD="xxxx-xxxx-xxxx-xxxx"
+
+# Create session with your PDS
+SESSION=$(curl -s -X POST "https://bsky.social/xrpc/com.atproto.server.createSession" \
+  -H "Content-Type: application/json" \
+  -d "{\"identifier\": \"$HANDLE\", \"password\": \"$APP_PASSWORD\"}")
+
+# Extract tokens
+ACCESS_JWT=$(echo "$SESSION" | jq -r '.accessJwt')
+DID=$(echo "$SESSION" | jq -r '.did')
+PDS=$(echo "$SESSION" | jq -r '.didDoc.service[0].serviceEndpoint')
+
+echo "DID: $DID"
+echo "PDS: $PDS"
+```
+
+### Step 3: Get a Service Token for the Hold
+
+```bash
+# The hold DID you want to access (e.g., did:web:hold01.atcr.io)
+HOLD_DID="did:web:hold01.atcr.io"
+
+# Get a service token from your PDS
+SERVICE_TOKEN=$(curl -s -X GET "$PDS/xrpc/com.atproto.server.getServiceAuth?aud=$HOLD_DID" \
+  -H "Authorization: Bearer $ACCESS_JWT" | jq -r '.token')
+
+echo "Service Token: $SERVICE_TOKEN"
+```
+
+### Step 4: Call Hold Endpoints
+
+Now you can call any authenticated hold endpoint with the service token:
+
+```bash
+# Export your data from the hold
+curl -s "https://hold01.atcr.io/xrpc/io.atcr.hold.exportUserData" \
+  -H "Authorization: Bearer $SERVICE_TOKEN" | jq .
+```
+
+### Complete Script
+
+Here's a complete script that does all the above:
+
+```bash
+#!/bin/bash
+# export-hold-data.sh - Export your data from an ATCR hold
+
+set -e
+
+# Configuration
+HANDLE="${1:-yourhandle.bsky.social}"
+APP_PASSWORD="${2:-xxxx-xxxx-xxxx-xxxx}"
+HOLD_DID="${3:-did:web:hold01.atcr.io}"
+
+# Default PDS (Bluesky's main PDS)
+DEFAULT_PDS="https://bsky.social"
+
+echo "Authenticating as $HANDLE..."
+
+# Step 1: Create session
+SESSION=$(curl -s -X POST "$DEFAULT_PDS/xrpc/com.atproto.server.createSession" \
+  -H "Content-Type: application/json" \
+  -d "{\"identifier\": \"$HANDLE\", \"password\": \"$APP_PASSWORD\"}")
+
+# Check for errors
+if echo "$SESSION" | jq -e '.error' > /dev/null 2>&1; then
+  echo "Error: $(echo "$SESSION" | jq -r '.message')"
+  exit 1
+fi
+
+ACCESS_JWT=$(echo "$SESSION" | jq -r '.accessJwt')
+DID=$(echo "$SESSION" | jq -r '.did')
+
+# Try to get PDS from didDoc, fall back to default
+PDS=$(echo "$SESSION" | jq -r '.didDoc.service[] | select(.id == "#atproto_pds") | .serviceEndpoint' 2>/dev/null || echo "$DEFAULT_PDS")
+if [ "$PDS" = "null" ] || [ -z "$PDS" ]; then
+  PDS="$DEFAULT_PDS"
+fi
+
+echo "Authenticated as $DID"
+echo "PDS: $PDS"
+
+# Step 2: Get service token for the hold
+echo "Getting service token for $HOLD_DID..."
+SERVICE_RESPONSE=$(curl -s -X GET "$PDS/xrpc/com.atproto.server.getServiceAuth?aud=$HOLD_DID" \
+  -H "Authorization: Bearer $ACCESS_JWT")
+
+if echo "$SERVICE_RESPONSE" | jq -e '.error' > /dev/null 2>&1; then
+  echo "Error getting service token: $(echo "$SERVICE_RESPONSE" | jq -r '.message')"
+  exit 1
+fi
+
+SERVICE_TOKEN=$(echo "$SERVICE_RESPONSE" | jq -r '.token')
+
+# Step 3: Resolve hold DID to URL
+if [[ "$HOLD_DID" == did:web:* ]]; then
+  # did:web:example.com -> https://example.com
+  HOLD_HOST="${HOLD_DID#did:web:}"
+  HOLD_URL="https://$HOLD_HOST"
+else
+  echo "Error: Only did:web holds are currently supported for direct resolution"
+  exit 1
+fi
+
+echo "Hold URL: $HOLD_URL"
+
+# Step 4: Export data
+echo "Exporting data from $HOLD_URL..."
+curl -s "$HOLD_URL/xrpc/io.atcr.hold.exportUserData" \
+  -H "Authorization: Bearer $SERVICE_TOKEN" | jq .
+```
+
+Usage:
+```bash
+chmod +x export-hold-data.sh
+./export-hold-data.sh yourhandle.bsky.social xxxx-xxxx-xxxx-xxxx did:web:hold01.atcr.io
+```
+
+---
+
+## Available Hold Endpoints
+
+Once you have a service token, you can call these endpoints:
+
+### Data Export (GDPR)
+```bash
+GET /xrpc/io.atcr.hold.exportUserData
+Authorization: Bearer {service_token}
+```
+
+Returns all your data stored on that hold:
+- Layer records (blobs you've pushed)
+- Crew membership status
+- Usage statistics
+- Whether you're the hold captain
+
+### Quota Information
+```bash
+GET /xrpc/io.atcr.hold.getQuota?userDid={your_did}
+# No auth required - just needs your DID
+```
+
+### Blob Download (if you have read access)
+```bash
+GET /xrpc/com.atproto.sync.getBlob?did={owner_did}&cid={blob_digest}
+Authorization: Bearer {service_token}
+```
+
+Returns a presigned URL to download the blob directly from storage.
+
+---
+
+## OAuth + DPoP (Advanced)
+
+App passwords are the simplest option, but OAuth with DPoP is the "proper" way to authenticate in ATProto. However, it's significantly more complex because:
+
+1. **DPoP (Demonstrating Proof of Possession)** - Every request requires a cryptographically signed JWT proving you control a specific key
+2. **PAR (Pushed Authorization Requests)** - Authorization parameters are sent server-to-server
+3. **PKCE (Proof Key for Code Exchange)** - Prevents authorization code interception
+
+### Why DPoP Makes Curl Impractical
+
+Each request requires a fresh DPoP proof JWT with:
+- Unique `jti` (request ID)
+- Current `iat` timestamp
+- HTTP method and URL bound to the request
+- Server-provided `nonce`
+- Signature using your P-256 private key
+
+Example DPoP proof structure:
+```json
+{
+  "alg": "ES256",
+  "typ": "dpop+jwt",
+  "jwk": { "kty": "EC", "crv": "P-256", "x": "...", "y": "..." }
+}
+{
+  "htm": "GET",
+  "htu": "https://bsky.social/xrpc/com.atproto.server.getServiceAuth",
+  "jti": "550e8400-e29b-41d4-a716-446655440000",
+  "iat": 1735689100,
+  "nonce": "server-provided-nonce"
+}
+```
+
+### If You Need OAuth
+
+If you need OAuth (e.g., for a production application), you'll want to use a library:
+
+**Go:**
+```go
+import "github.com/bluesky-social/indigo/atproto/auth/oauth"
+```
+
+**TypeScript/JavaScript:**
+```bash
+npm install @atproto/oauth-client-node
+```
+
+**Python:**
+```bash
+pip install atproto
+```
+
+These libraries handle all the DPoP complexity for you.
+
+### High-Level OAuth Flow
+
+For documentation purposes, here's what the flow looks like:
+
+1. **Resolve identity**: `handle` → `DID` → `PDS endpoint`
+2. **Discover OAuth server**: `GET {pds}/.well-known/oauth-authorization-server`
+3. **Generate DPoP key**: Create P-256 key pair
+4. **PAR request**: Send authorization parameters (with DPoP proof)
+5. **User authorization**: Browser-based login
+6. **Token exchange**: Exchange code for tokens (with DPoP proof)
+7. **Use tokens**: All subsequent requests include DPoP proofs
+
+Each step after #3 requires generating a fresh DPoP proof JWT, which is why libraries are essential.
+
+---
+
+## Troubleshooting
+
+### "Invalid token" or "Token expired"
+
+Service tokens are only valid for ~60 seconds. Get a fresh one:
+```bash
+SERVICE_TOKEN=$(curl -s "$PDS/xrpc/com.atproto.server.getServiceAuth?aud=$HOLD_DID" \
+  -H "Authorization: Bearer $ACCESS_JWT" | jq -r '.token')
+```
+
+### "Session expired"
+
+Your access JWT from `createSession` has expired. Create a new session:
+```bash
+SESSION=$(curl -s -X POST "$PDS/xrpc/com.atproto.server.createSession" ...)
+ACCESS_JWT=$(echo "$SESSION" | jq -r '.accessJwt')
+```
+
+### "Audience mismatch"
+
+The service token is scoped to a specific hold. Make sure `HOLD_DID` matches exactly what's in the `aud` claim of your token.
+
+### "Access denied: user is not a crew member"
+
+You don't have access to this hold. You need to either:
+- Be the hold captain (owner)
+- Be a crew member with appropriate permissions
+
+### Finding Your Hold DID
+
+Check your sailor profile to find your default hold:
+```bash
+curl -s "https://bsky.social/xrpc/com.atproto.repo.getRecord?repo=$DID&collection=io.atcr.sailor.profile&rkey=self" \
+  -H "Authorization: Bearer $ACCESS_JWT" | jq -r '.value.defaultHold'
+```
+
+Or check your manifest records for the hold where your images are stored:
+```bash
+curl -s "https://bsky.social/xrpc/com.atproto.repo.listRecords?repo=$DID&collection=io.atcr.manifest&limit=1" \
+  -H "Authorization: Bearer $ACCESS_JWT" | jq -r '.records[0].value.holdDid'
+```
+
+---
+
+## Security Notes
+
+- **App passwords** are scoped tokens that can be revoked without changing your main password
+- **Service tokens** are short-lived (60 seconds) and scoped to a specific hold
+- **Never share** your app password or access tokens
+- Service tokens can only be used for the specific hold they were requested for (`aud` claim)
+
+---
+
+## References
+
+- [ATProto OAuth Specification](https://atproto.com/specs/oauth)
+- [DPoP RFC 9449](https://datatracker.ietf.org/doc/html/rfc9449)
+- [Bluesky OAuth Guide](https://docs.bsky.app/docs/advanced-guides/oauth-client)
+- [ATCR BYOS Documentation](./BYOS.md)
@@ -37,11 +37,12 @@ This document lists all XRPC endpoints implemented in the Hold service (`pkg/hol
 | `/xrpc/com.atproto.repo.deleteRecord` | POST | Delete a record |
 | `/xrpc/com.atproto.repo.uploadBlob` | POST | Upload ATProto blob |

-### DPoP Auth Required
+### Auth Required (Service Token or DPoP)

 | Endpoint | Method | Description |
 |----------|--------|-------------|
 | `/xrpc/io.atcr.hold.requestCrew` | POST | Request crew membership |
+| `/xrpc/io.atcr.hold.exportUserData` | GET | GDPR data export (returns user's records) |

 ---

@@ -60,6 +61,22 @@ All require `blob:write` permission via service token:

 ---

+## ATCR Hold-Specific Endpoints (`io.atcr.hold.*`)
+
+| Endpoint | Method | Auth | Description |
+|----------|--------|------|-------------|
+| `/xrpc/io.atcr.hold.initiateUpload` | POST | blob:write | Start multipart upload |
+| `/xrpc/io.atcr.hold.getPartUploadUrl` | POST | blob:write | Get presigned URL for part |
+| `/xrpc/io.atcr.hold.uploadPart` | PUT | blob:write | Direct buffered part upload |
+| `/xrpc/io.atcr.hold.completeUpload` | POST | blob:write | Finalize multipart upload |
+| `/xrpc/io.atcr.hold.abortUpload` | POST | blob:write | Cancel multipart upload |
+| `/xrpc/io.atcr.hold.notifyManifest` | POST | blob:write | Notify manifest push |
+| `/xrpc/io.atcr.hold.requestCrew` | POST | auth | Request crew membership |
+| `/xrpc/io.atcr.hold.exportUserData` | GET | auth | GDPR data export |
+| `/xrpc/io.atcr.hold.getQuota` | GET | none | Get user quota info |
+
+---
+
 ## Standard ATProto Endpoints (excluding io.atcr.hold.*)

 | Endpoint |
@@ -82,3 +99,11 @@ All require `blob:write` permission via service token:
 | /xrpc/app.bsky.actor.getProfiles |
 | /.well-known/did.json |
 | /.well-known/atproto-did |
+
+---
+
+## See Also
+
+- [DIRECT_HOLD_ACCESS.md](./DIRECT_HOLD_ACCESS.md) - How to call hold endpoints directly without AppView (app passwords, curl examples)
+- [BYOS.md](./BYOS.md) - Bring Your Own Storage architecture
+- [OAUTH.md](./OAUTH.md) - OAuth + DPoP authentication details