usda-dash-config/NIX_FLAKE_MIGRATION.md

t pu# Nix Flake Migration Summary

## What Changed

The USDA Vision repository has been migrated from inline Nix derivations to a self-contained flake with integrated secrets management.

### Before
```
usda-dash-config/
├── default.nix           # Contains all build logic inline
└── usda-vision/          # Git submodule
    ├── .env              # Plaintext secrets (git-ignored)
    └── .env.azure        # Plaintext secrets (git-ignored)
```

### After
```
usda-dash-config/
├── default.nix           # Now references usda-vision flake
└── usda-vision/          # Git submodule with its own flake
    ├── flake.nix         # Flake definition
    ├── package.nix       # Application build
    ├── camera-sdk.nix    # Camera SDK build
    ├── secrets.nix       # ragenix config
    └── secrets/
        ├── secrets.nix   # Public keys
        ├── env.age       # Encrypted secrets (safe to commit)
        └── env.azure.age # Encrypted secrets (safe to commit)
```

## Key Improvements

### 1. Self-Contained Flake
- **Standalone Development**: The `usda-vision` directory now has its own complete build and development environment
- **Independent Versioning**: Can pin dependencies independently from parent project
- **Reusable**: Can be used in other projects via flake inputs

### 2. Development Shell
```bash
cd usda-vision
nix develop
```

Provides:
- Docker & Docker Compose
- Node.js 20 (npm, pnpm)
- Python 3.11 (pip, virtualenv)
- Supabase CLI
- Camera SDK (pre-configured)
- ragenix for secrets
- All utilities (jq, yq, rsync, etc.)

### 3. Encrypted Secrets with ragenix
- **Git-Safe**: Encrypted `.age` files can be committed safely
- **Key-Based Access**: Only users with the right age/SSH keys can decrypt
- **No Manual Distribution**: Secrets are in the repo (encrypted)
- **Audit Trail**: Changes to secrets tracked in git history

### 4. NixOS Module
The flake exports a NixOS module for easy deployment:

```nix
{
  imports = [ inputs.usda-vision.nixosModules.default ];
  
  services.usda-vision = {
    enable = true;
    secretsFile = config.age.secrets.usda-vision-env.path;
  };
}
```

## Files Created

### In `usda-vision/`:
- [`flake.nix`](usda-vision/flake.nix) - Main flake definition
- [`package.nix`](usda-vision/package.nix) - Application package
- [`camera-sdk.nix`](usda-vision/camera-sdk.nix) - Camera SDK package
- [`secrets.nix`](usda-vision/secrets.nix) - ragenix configuration
- [`FLAKE_SETUP.md`](usda-vision/FLAKE_SETUP.md) - Complete documentation
- [`setup-dev.sh`](usda-vision/setup-dev.sh) - Quick setup script

### In `usda-vision/secrets/`:
- [`secrets.nix`](usda-vision/secrets/secrets.nix) - Public key list
- [`README.md`](usda-vision/secrets/README.md) - Secrets documentation
- [`.gitignore`](usda-vision/secrets/.gitignore) - Protect unencrypted files

## Files Modified

### [`default.nix`](default.nix)
- Removed inline build derivations (200+ lines)
- Now imports packages from usda-vision flake
- Cleaner, more maintainable

### [`usda-vision/.gitignore`](usda-vision/.gitignore)
- Added Nix-specific ignores (`result`, `.direnv/`)

## Migration Guide

### For Developers

1. **Setup age key** (one-time):
   ```bash
   cd usda-vision
   ./setup-dev.sh  # Interactive setup
   ```

2. **Enter dev environment**:
   ```bash
   nix develop
   ```

3. **Encrypt secrets** (if you have them):
   ```bash
   ragenix -e secrets/env.age
   # Paste your .env contents, save, exit
   ```

### For System Administrators

1. **Generate age key** on the deployment server:
   ```bash
   mkdir -p /root/.config/age
   age-keygen -o /root/.config/age/keys.txt
   ```

2. **Add public key** to `usda-vision/secrets/secrets.nix`

3. **Re-encrypt secrets** (from your dev machine):
   ```bash
   cd usda-vision
   nix develop
   ragenix -r  # Re-key all secrets
   git commit secrets/ -m "Add server key"
   git push
   ```

4. **Configure NixOS module** (see FLAKE_SETUP.md)

## Testing

### Build the package:
```bash
cd usda-vision
nix build
```

### Test development shell:
```bash
nix develop
# Should see welcome message with available commands
```

### Test from parent directory:
```bash
cd usda-dash-config
nix build .#usda-vision  # If default.nix is updated
```

## Rollback Plan

If needed, revert to the previous setup:

```bash
git revert <commit-hash>
```

The old inline derivations are preserved in git history.

## Security Considerations

### Secrets in Git
- ✅ Encrypted files (`.age`) are safe to commit
- ❌ Never commit `.env` or unencrypted secrets
- ✅ `.gitignore` is configured to prevent accidents
- ✅ Age encryption is secure (modern crypto)

### Key Management
- Private keys stay on developer machines
- Public keys are in the repo
- Server keys generated on-server (never transmitted)
- Multiple keys can decrypt the same secrets (team access)

### Migration of Existing Secrets
- Old `.env` files should be deleted after encryption
- Verify `.gitignore` is working: `git status`
- Consider rotating sensitive credentials after migration

## Benefits Summary

| Aspect | Before | After |
|--------|--------|-------|
| **Build** | Inline in parent | Self-contained flake |
| **Dev Environment** | Manual setup | `nix develop` |
| **Secrets** | Plaintext, git-ignored | Encrypted, in git |
| **Dependencies** | Parent's nixpkgs | Pinned in flake.lock |
| **Reusability** | Coupled to parent | Standalone |
| **Documentation** | Scattered | Centralized |

## Next Steps

1. **Encrypt existing secrets**
   - Run `setup-dev.sh` in usda-vision
   - Encrypt `.env` files with ragenix
   - Delete plaintext versions

2. **Test locally**
   - `nix develop` in usda-vision
   - Verify all tools available
   - Test docker-compose workflow

3. **Update CI/CD** (if applicable)
   - Add age keys to CI secrets
   - Update build commands to use flake

4. **Team onboarding**
   - Share FLAKE_SETUP.md
   - Collect public keys
   - Add to secrets.nix

## Questions?

See [FLAKE_SETUP.md](usda-vision/FLAKE_SETUP.md) for detailed documentation.