feat: Add flake and ragenix package generation and dev environment
This commit is contained in:
223
NIX_FLAKE_MIGRATION.md
Normal file
223
NIX_FLAKE_MIGRATION.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Nix Flake Migration Summary
|
||||
|
||||
## What Changed
|
||||
|
||||
The USDA Vision repository has been migrated from inline Nix derivations to a self-contained flake with integrated secrets management.
|
||||
|
||||
### Before
|
||||
```
|
||||
usda-dash-config/
|
||||
├── default.nix # Contains all build logic inline
|
||||
└── usda-vision/ # Git submodule
|
||||
├── .env # Plaintext secrets (git-ignored)
|
||||
└── .env.azure # Plaintext secrets (git-ignored)
|
||||
```
|
||||
|
||||
### After
|
||||
```
|
||||
usda-dash-config/
|
||||
├── default.nix # Now references usda-vision flake
|
||||
└── usda-vision/ # Git submodule with its own flake
|
||||
├── flake.nix # Flake definition
|
||||
├── package.nix # Application build
|
||||
├── camera-sdk.nix # Camera SDK build
|
||||
├── secrets.nix # ragenix config
|
||||
└── secrets/
|
||||
├── secrets.nix # Public keys
|
||||
├── env.age # Encrypted secrets (safe to commit)
|
||||
└── env.azure.age # Encrypted secrets (safe to commit)
|
||||
```
|
||||
|
||||
## Key Improvements
|
||||
|
||||
### 1. Self-Contained Flake
|
||||
- **Standalone Development**: The `usda-vision` directory now has its own complete build and development environment
|
||||
- **Independent Versioning**: Can pin dependencies independently from parent project
|
||||
- **Reusable**: Can be used in other projects via flake inputs
|
||||
|
||||
### 2. Development Shell
|
||||
```bash
|
||||
cd usda-vision
|
||||
nix develop
|
||||
```
|
||||
|
||||
Provides:
|
||||
- Docker & Docker Compose
|
||||
- Node.js 20 (npm, pnpm)
|
||||
- Python 3.11 (pip, virtualenv)
|
||||
- Supabase CLI
|
||||
- Camera SDK (pre-configured)
|
||||
- ragenix for secrets
|
||||
- All utilities (jq, yq, rsync, etc.)
|
||||
|
||||
### 3. Encrypted Secrets with ragenix
|
||||
- **Git-Safe**: Encrypted `.age` files can be committed safely
|
||||
- **Key-Based Access**: Only users with the right age/SSH keys can decrypt
|
||||
- **No Manual Distribution**: Secrets are in the repo (encrypted)
|
||||
- **Audit Trail**: Changes to secrets tracked in git history
|
||||
|
||||
### 4. NixOS Module
|
||||
The flake exports a NixOS module for easy deployment:
|
||||
|
||||
```nix
|
||||
{
|
||||
imports = [ inputs.usda-vision.nixosModules.default ];
|
||||
|
||||
services.usda-vision = {
|
||||
enable = true;
|
||||
secretsFile = config.age.secrets.usda-vision-env.path;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Files Created
|
||||
|
||||
### In `usda-vision/`:
|
||||
- [`flake.nix`](usda-vision/flake.nix) - Main flake definition
|
||||
- [`package.nix`](usda-vision/package.nix) - Application package
|
||||
- [`camera-sdk.nix`](usda-vision/camera-sdk.nix) - Camera SDK package
|
||||
- [`secrets.nix`](usda-vision/secrets.nix) - ragenix configuration
|
||||
- [`FLAKE_SETUP.md`](usda-vision/FLAKE_SETUP.md) - Complete documentation
|
||||
- [`setup-dev.sh`](usda-vision/setup-dev.sh) - Quick setup script
|
||||
|
||||
### In `usda-vision/secrets/`:
|
||||
- [`secrets.nix`](usda-vision/secrets/secrets.nix) - Public key list
|
||||
- [`README.md`](usda-vision/secrets/README.md) - Secrets documentation
|
||||
- [`.gitignore`](usda-vision/secrets/.gitignore) - Protect unencrypted files
|
||||
|
||||
## Files Modified
|
||||
|
||||
### [`default.nix`](default.nix)
|
||||
- Removed inline build derivations (200+ lines)
|
||||
- Now imports packages from usda-vision flake
|
||||
- Cleaner, more maintainable
|
||||
|
||||
### [`usda-vision/.gitignore`](usda-vision/.gitignore)
|
||||
- Added Nix-specific ignores (`result`, `.direnv/`)
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### For Developers
|
||||
|
||||
1. **Setup age key** (one-time):
|
||||
```bash
|
||||
cd usda-vision
|
||||
./setup-dev.sh # Interactive setup
|
||||
```
|
||||
|
||||
2. **Enter dev environment**:
|
||||
```bash
|
||||
nix develop
|
||||
```
|
||||
|
||||
3. **Encrypt secrets** (if you have them):
|
||||
```bash
|
||||
ragenix -e secrets/env.age
|
||||
# Paste your .env contents, save, exit
|
||||
```
|
||||
|
||||
### For System Administrators
|
||||
|
||||
1. **Generate age key** on the deployment server:
|
||||
```bash
|
||||
mkdir -p /root/.config/age
|
||||
age-keygen -o /root/.config/age/keys.txt
|
||||
```
|
||||
|
||||
2. **Add public key** to `usda-vision/secrets/secrets.nix`
|
||||
|
||||
3. **Re-encrypt secrets** (from your dev machine):
|
||||
```bash
|
||||
cd usda-vision
|
||||
nix develop
|
||||
ragenix -r # Re-key all secrets
|
||||
git commit secrets/ -m "Add server key"
|
||||
git push
|
||||
```
|
||||
|
||||
4. **Configure NixOS module** (see FLAKE_SETUP.md)
|
||||
|
||||
## Testing
|
||||
|
||||
### Build the package:
|
||||
```bash
|
||||
cd usda-vision
|
||||
nix build
|
||||
```
|
||||
|
||||
### Test development shell:
|
||||
```bash
|
||||
nix develop
|
||||
# Should see welcome message with available commands
|
||||
```
|
||||
|
||||
### Test from parent directory:
|
||||
```bash
|
||||
cd usda-dash-config
|
||||
nix build .#usda-vision # If default.nix is updated
|
||||
```
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If needed, revert to the previous setup:
|
||||
|
||||
```bash
|
||||
git revert <commit-hash>
|
||||
```
|
||||
|
||||
The old inline derivations are preserved in git history.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Secrets in Git
|
||||
- ✅ Encrypted files (`.age`) are safe to commit
|
||||
- ❌ Never commit `.env` or unencrypted secrets
|
||||
- ✅ `.gitignore` is configured to prevent accidents
|
||||
- ✅ Age encryption is secure (modern crypto)
|
||||
|
||||
### Key Management
|
||||
- Private keys stay on developer machines
|
||||
- Public keys are in the repo
|
||||
- Server keys generated on-server (never transmitted)
|
||||
- Multiple keys can decrypt the same secrets (team access)
|
||||
|
||||
### Migration of Existing Secrets
|
||||
- Old `.env` files should be deleted after encryption
|
||||
- Verify `.gitignore` is working: `git status`
|
||||
- Consider rotating sensitive credentials after migration
|
||||
|
||||
## Benefits Summary
|
||||
|
||||
| Aspect | Before | After |
|
||||
|--------|--------|-------|
|
||||
| **Build** | Inline in parent | Self-contained flake |
|
||||
| **Dev Environment** | Manual setup | `nix develop` |
|
||||
| **Secrets** | Plaintext, git-ignored | Encrypted, in git |
|
||||
| **Dependencies** | Parent's nixpkgs | Pinned in flake.lock |
|
||||
| **Reusability** | Coupled to parent | Standalone |
|
||||
| **Documentation** | Scattered | Centralized |
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Encrypt existing secrets**
|
||||
- Run `setup-dev.sh` in usda-vision
|
||||
- Encrypt `.env` files with ragenix
|
||||
- Delete plaintext versions
|
||||
|
||||
2. **Test locally**
|
||||
- `nix develop` in usda-vision
|
||||
- Verify all tools available
|
||||
- Test docker-compose workflow
|
||||
|
||||
3. **Update CI/CD** (if applicable)
|
||||
- Add age keys to CI secrets
|
||||
- Update build commands to use flake
|
||||
|
||||
4. **Team onboarding**
|
||||
- Share FLAKE_SETUP.md
|
||||
- Collect public keys
|
||||
- Add to secrets.nix
|
||||
|
||||
## Questions?
|
||||
|
||||
See [FLAKE_SETUP.md](usda-vision/FLAKE_SETUP.md) for detailed documentation.
|
||||
Reference in New Issue
Block a user