fix: Convert ssh keys to age keys

secrets/DESIGN.md

@@ -0,0 +1,174 @@
+# Athenix Secrets System Design
+
+## Overview
+
+The Athenix secrets management system integrates ragenix (agenix) with automatic host discovery based on the repository's fleet inventory structure. It provides a seamless workflow for managing encrypted secrets across all systems.
+
+## Architecture
+
+### Auto-Discovery Module (`sw/secrets.nix`)
+
+**Purpose**: Automatically load and configure secrets at system deployment time.
+
+**Features**:
+- Discovers `.age` encrypted files from `secrets/` directories
+- Loads global secrets from `secrets/global/` on ALL systems
+- Loads host-specific secrets from `secrets/{hostname}/` on matching hosts
+- Auto-configures decryption keys based on `.pub` files in directories
+- Supports custom secret configuration via `default.nix` in each directory
+
+**Key Behaviors**:
+- Secrets are decrypted to `/run/agenix/{name}` at boot
+- Identity paths include: system SSH keys + global keys + host-specific keys
+- Host-specific secrets override global secrets with the same name
+
+### Dynamic Recipients Configuration (`secrets/secrets.nix`)
+
+**Purpose**: Generate ragenix recipient configuration from directory structure.
+
+**Features**:
+- Automatically discovers hosts from `secrets/` subdirectories
+- Reads age public keys from `.age.pub` files (converted from SSH keys)
+- Generates recipient lists based on secret location:
+  - `secrets/global/*.age` → ALL hosts + admins
+  - `secrets/{hostname}/*.age` → that host + global keys + admins
+- Supports admin keys in `secrets/admins/` for secret editing
+
+**Key Behaviors**:
+- No manual recipient list maintenance required
+- Adding a new host = create directory + add .pub key + run `update-age-keys.sh`
+- Works with ragenix CLI: `ragenix -e`, `ragenix -r`
+
+## Workflow
+
+### Adding a New Host
+
+1. **Capture SSH host key**:
+   ```bash
+   # From the running system
+   cat /etc/ssh/ssh_host_ed25519_key.pub > secrets/new-host/ssh_host_ed25519_key.pub
+   ```
+
+2. **Convert to age format**:
+   ```bash
+   cd secrets/
+   ./update-age-keys.sh
+   ```
+
+3. **Re-key existing secrets** (if needed):
+   ```bash
+   ragenix -r
+   ```
+
+### Creating a New Secret
+
+1. **Choose location**:
+   - `secrets/global/` → all systems can decrypt
+   - `secrets/{hostname}/` → only that host can decrypt
+
+2. **Create/edit secret**:
+   ```bash
+   ragenix -e secrets/global/my-secret.age
+   ```
+   
+3. **Recipients are auto-determined** from `secrets.nix`:
+   - Global secrets: all host keys + admin keys
+   - Host-specific: that host + global keys + admin keys
+
+### Cross-Host Secret Management
+
+Any Athenix host can manage secrets for other hosts because:
+- All public keys are in the repository (`*.age.pub` files)
+- `secrets/secrets.nix` auto-generates recipient lists
+- Hosts decrypt using their own private keys (not shared)
+
+Example: From `nix-builder`, create a secret for `usda-dash`:
+```bash
+ragenix -e secrets/usda-dash/database-password.age
+# Encrypted for usda-dash's public key + admins
+# usda-dash will decrypt using its private key at /etc/ssh/ssh_host_ed25519_key
+```
+
+## Directory Structure
+
+```
+secrets/
+├── secrets.nix              # Auto-generated recipient config
+├── update-age-keys.sh       # Helper to convert SSH → age keys
+├── README.md                # User documentation
+├── DESIGN.md                # This file
+│
+├── global/                  # Secrets for ALL hosts
+│   ├── *.pub                # SSH public keys
+│   ├── *.age.pub            # Age public keys (generated)
+│   ├── *.age                # Encrypted secrets
+│   └── default.nix          # Optional: custom secret config
+│
+├── {hostname}/              # Host-specific secrets
+│   ├── *.pub
+│   ├── *.age.pub
+│   ├── *.age
+│   └── default.nix
+│
+└── admins/                  # Admin keys for editing
+    └── *.age.pub
+```
+
+## Security Model
+
+1. **Public keys in git**: Safe to commit (only public keys, `.age.pub` and `.pub`)
+2. **Private keys on hosts**: Never leave the system (`/etc/ssh/ssh_host_*_key`, `/etc/age/identity.key`)
+3. **Encrypted secrets in git**: Safe to commit (`.age` files)
+4. **Decrypted secrets**: Only in memory/tmpfs (`/run/agenix/*`)
+
+## Integration Points
+
+### With NixOS Configuration
+
+```nix
+# Access decrypted secrets in any NixOS module
+config.age.secrets.my-secret.path  # => /run/agenix/my-secret
+
+# Example usage
+services.myapp.passwordFile = config.age.secrets.database-password.path;
+```
+
+### With Inventory System
+
+The system automatically matches `secrets/{hostname}/` to hostnames from `inventory.nix`. No manual configuration needed.
+
+### With External Modules
+
+External user/system modules can reference secrets:
+```nix
+# In external module
+{ config, ... }:
+{
+  programs.git.extraConfig.credential.helper = 
+    "store --file ${config.age.secrets.git-credentials.path}";
+}
+```
+
+## Advantages
+
+1. **Zero manual recipient management**: Just add directories and keys
+2. **Cross-host secret creation**: Any host can manage secrets for others
+3. **Automatic host discovery**: Syncs with inventory structure
+4. **Flexible permission model**: Global vs host-specific + custom configs
+5. **Version controlled**: All public data in git, auditable history
+6. **Secure by default**: Private keys never shared, secrets encrypted at rest
+
+## Limitations
+
+1. **Requires age key conversion**: SSH keys must be converted to age format (automated by script)
+2. **Bootstrap chicken-egg**: Need initial host key before encrypting secrets (capture from first boot or generate locally)
+3. **No secret rotation automation**: Must manually re-key with `ragenix -r`
+4. **Git history contains old encrypted versions**: Rotating keys doesn't remove old ciphertexts from history
+
+## Future Enhancements
+
+- Auto-run `update-age-keys.sh` in pre-commit hook
+- Integrate with inventory.nix to auto-generate host directories
+- Support for multiple identity types per host
+- Automated secret rotation scheduling
+- Integration with hardware security modules (YubiKey, etc.)

secrets/README.md

@@ -55,6 +55,14 @@ Or from SSH host key:
 cat /etc/ssh/ssh_host_ed25519_key.pub > secrets/nix-builder/ssh_host_ed25519_key.pub
 ```
 
+**Then convert SSH keys to age format:**
+```bash
+cd secrets/
+./update-age-keys.sh
+```
+
+This creates `.age.pub` files that `secrets.nix` uses for ragenix recipient configuration.
+
 ### 3. Encrypt Secrets
 
 Encrypt a secret for specific hosts:
@@ -72,37 +80,35 @@ age -R secrets/nix-builder/ssh_host_ed25519_key.pub \
 
 ### 4. Using ragenix CLI (Recommended)
 
-The `ragenix` CLI tool simplifies secret management:
+The `ragenix` CLI tool simplifies secret management. The `secrets/secrets.nix` file **automatically discovers** hosts and their keys from the directory structure:
 
 ```bash
 # Install ragenix
 nix shell github:yaxitech/ragenix
 
 # Edit a secret (creates if doesn't exist)
+# Recipients are automatically determined based on the path:
+# - secrets/global/*.age -> encrypted for ALL hosts + admins
+# - secrets/{hostname}/*.age -> encrypted for that host + global keys + admins
 ragenix -e secrets/global/example.age
 
-# Re-key secrets after adding/removing hosts
+# Re-key all secrets after adding/removing hosts
 ragenix -r
 ```
 
-Create a `secrets.nix` file in the repository root to define recipients:
-```nix
-# secrets.nix
-let
-  # System public keys (age format)
-  nix-builder = "age1...";
-  usda-dash = "age1...";
-  
-  # User keys for editing secrets
-  admin = "age1...";
-  
-  allHosts = [ nix-builder usda-dash ];
-in
-{
-  "secrets/global/example.age".publicKeys = allHosts ++ [ admin ];
-  "secrets/nix-builder/ssh_host_key.age".publicKeys = [ nix-builder admin ];
-  "secrets/usda-dash/ssh_host_key.age".publicKeys = [ usda-dash admin ];
-}
+The `secrets.nix` file automatically:
+- **Discovers hosts** from directory names in `secrets/`
+- **Reads age public keys** from `.age.pub` files in each directory
+- **Generates recipient lists** based on secret location (global vs host-specific)
+- **Includes admin keys** from `secrets/admins/*.age.pub` for editing
+
+To add admin keys for editing secrets:
+```bash
+# Generate personal age key
+age-keygen -o ~/.config/age/personal.key
+
+# Extract public key and add to secrets
+grep "public key:" ~/.config/age/personal.key | cut -d: -f2 | tr -d ' ' > secrets/admins/your-name.age.pub
 ```
 
 ## Using Secrets in Configuration

secrets/global/default.nix

@@ -1,10 +0,0 @@
-# Custom configuration for global secrets
-# This file defines additional options for secrets beyond the auto-discovered .age files
-{
-  # Example: Custom permissions for the example secret
-  example = {
-    mode = "0440";
-    owner = "root";
-    group = "root";
-  };
-}

secrets/global/example.age

@@ -1,6 +0,0 @@
-age-encryption.org/v1
--> X25519 2lgS/8zamn6WD6eKM+Mcms+TY/PlzLr43WoWk6EZ7BU
-/H6DRcTul3ZtlfyYRFS6v+YZIYYi/sNdkaqZYdKbBIE
---- NSMzaAQArlhVdSuRICYD6lAiq/nOd2c5n/wcye4FXao
-D]t=<3D>m<EFBFBD>
-<EFBFBD><EFBFBD><EFBFBD>@<40><1A>

secrets/global/hunter_halloran_key.age.pub

@@ -0,0 +1 @@
+age1udmpqkedupd33gyut85ud3nvppydzeg04kkuneymkvxcjjej244s4v8xjc

secrets/global/hunter_halloran_key.pub

@@ -1 +0,0 @@
-ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBC7xzHxY2BfFUybMvG4wHSF9oEAGzRiLTFEndLvWV/X hdh20267@engr733847d.engr.uga.edu

secrets/nix-builder/ssh_host_ed25519_key.age.pub

@@ -0,0 +1 @@
+age1u5tczg2sx90n03uuz9h549f4h3h7sq5uehhqpampzs7vj8ew7y6s2mjwz0

secrets/nix-builder/ssh_host_ed25519_key.pub

@@ -1 +0,0 @@
-ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHW0Hxldo3EkniotitzJ2XiZbIq9Rfo27yI1+sBrgG39 root@nix-builder

secrets/secrets.nix

@@ -1,52 +1,193 @@
 # ============================================================================
-# Agenix Secret Recipients Configuration
+# Agenix Secret Recipients Configuration (Auto-Generated)
 # ============================================================================
-# This file defines which age public keys can decrypt which secrets.
-# Used by the ragenix CLI tool for encrypting/editing secrets.
+# This file automatically discovers hosts and their public keys from the
+# secrets/ directory structure and generates recipient configurations.
+#
+# Directory structure:
+#   secrets/{hostname}/*.pub  -> SSH/age public keys for that host
+#   secrets/global/*.pub      -> Keys accessible to all hosts
 #
 # Usage:
 #   ragenix -e secrets/global/example.age     # Edit/create secret
 #   ragenix -r                                # Re-key all secrets
+#
+# To add admin keys for editing secrets, create secrets/admins/*.pub files
+# with your personal age public keys (generated with: age-keygen)
 
 let
-  # ========== System Public Keys (Age Format) ==========
-  # Convert SSH host keys to age format:
-  #   ssh-to-age < secrets/{hostname}/ssh_host_ed25519_key.pub
+  lib = builtins;
 
-  # Example (replace with actual age keys):
-  # nix-builder = "age1...";
-  # usda-dash = "age1...";
+  # Helper functions not in builtins
+  filterAttrs =
+    pred: set:
+    lib.listToAttrs (
+      lib.filter (item: pred item.name item.value) (
+        lib.map (name: {
+          inherit name;
+          value = set.${name};
+        }) (lib.attrNames set)
+      )
+    );
 
-  # ========== User Public Keys (for editing secrets) ==========
-  # These are personal age keys for administrators who need to edit secrets
-  # Generate with: age-keygen
+  concatLists = lists: lib.foldl' (acc: list: acc ++ list) [ ] lists;
 
-  # Example:
-  # admin1 = "age1...";
-  # admin2 = "age1...";
+  unique =
+    list:
+    let
+      go =
+        acc: remaining:
+        if remaining == [ ] then
+          acc
+        else if lib.elem (lib.head remaining) acc then
+          go acc (lib.tail remaining)
+        else
+          go (acc ++ [ (lib.head remaining) ]) (lib.tail remaining);
+    in
+    go [ ] list;
 
-  # ========== Host Groups ==========
-  allHosts = [
-    # Add all system keys here
-    # nix-builder
-    # usda-dash
-  ];
+  hasSuffix =
+    suffix: str:
+    let
+      lenStr = lib.stringLength str;
+      lenSuffix = lib.stringLength suffix;
+    in
+    lenStr >= lenSuffix && lib.substring (lenStr - lenSuffix) lenSuffix str == suffix;
 
-  admins = [
-    # Add all admin user keys here
-    # admin1
-    # admin2
-  ];
+  hasPrefix =
+    prefix: str:
+    let
+      lenPrefix = lib.stringLength prefix;
+    in
+    lib.stringLength str >= lenPrefix && lib.substring 0 lenPrefix str == prefix;
+
+  nameValuePair = name: value: { inherit name value; };
+
+  secretsPath = ./.;
+
+  # Helper to convert SSH public key content to age public key
+  sshToAge =
+    sshPubKey:
+    let
+      # This is a simple check - in practice, use ssh-to-age tool
+      # For now, we'll just use the keys as-is if they look like age keys
+      trimmed = lib.replaceStrings [ "\n" ] [ "" ] sshPubKey;
+    in
+    if lib.substring 0 4 trimmed == "age1" then trimmed else null; # Will need manual conversion with ssh-to-age
+
+  # Read all directories in secrets/
+  secretDirs = if lib.pathExists secretsPath then lib.readDir secretsPath else { };
+
+  # Filter to only directories (excludes files)
+  isDirectory = name: type: type == "directory";
+  directories = lib.filter (name: isDirectory name secretDirs.${name}) (lib.attrNames secretDirs);
+
+  # Read public keys from a directory and convert to age format
+  readHostKeys =
+    dirName:
+    let
+      dirPath = secretsPath + "/${dirName}";
+      files = if lib.pathExists dirPath then lib.readDir dirPath else { };
+
+      # Prefer .age.pub files (pre-converted), fall back to .pub files
+      agePubFiles = filterAttrs (name: type: type == "regular" && hasSuffix ".age.pub" name) files;
+
+      sshPubFiles = filterAttrs (
+        name: type: type == "regular" && hasSuffix ".pub" name && !(hasSuffix ".age.pub" name)
+      ) files;
+
+      # Read age public keys (already in correct format)
+      ageKeys = lib.map (
+        name:
+        let
+          content = lib.readFile (dirPath + "/${name}");
+          # Trim whitespace/newlines
+          trimmed = lib.replaceStrings [ "\n" " " "\r" "\t" ] [ "" "" "" "" ] content;
+        in
+        trimmed
+      ) (lib.attrNames agePubFiles);
+
+      # For SSH keys, just include them as-is (user needs to convert with ssh-to-age)
+      # Or they can run the update-age-keys.sh script
+      sshKeys =
+        if (lib.length (lib.attrNames sshPubFiles)) > 0 then
+          lib.trace "Warning: ${dirName} has unconverted SSH keys. Run secrets/update-age-keys.sh" [ ]
+        else
+          [ ];
+    in
+    lib.filter (k: k != null && k != "") (ageKeys ++ sshKeys);
+
+  # Build host key mappings: { hostname = [ "age1..." "age2..." ]; }
+  hostKeys = lib.listToAttrs (
+    lib.map (dir: nameValuePair dir (readHostKeys dir)) (
+      lib.filter (d: d != "global" && d != "admins") directories
+    )
+  );
+
+  # Global keys that all hosts can use
+  globalKeys = if lib.elem "global" directories then readHostKeys "global" else [ ];
+
+  # Admin keys for editing secrets
+  adminKeys = if lib.elem "admins" directories then readHostKeys "admins" else [ ];
+
+  # All host keys combined
+  allHostKeys = concatLists (lib.attrValues hostKeys);
+
+  # Find all .age files in the secrets directory
+  findSecrets =
+    dir:
+    let
+      dirPath = secretsPath + "/${dir}";
+      files = if lib.pathExists dirPath then lib.readDir dirPath else { };
+      ageFiles = filterAttrs (name: type: type == "regular" && hasSuffix ".age" name) files;
+    in
+    lib.map (name: "secrets/${dir}/${name}") (lib.attrNames ageFiles);
+
+  # Generate recipient list for a secret based on its location
+  getRecipients =
+    secretPath:
+    let
+      # Extract directory name from path: "secrets/nix-builder/foo.age" -> "nix-builder"
+      pathParts = lib.split "/" secretPath;
+      dirName = lib.elemAt pathParts 2;
+    in
+    if dirName == "global" then
+      # Global secrets: all hosts + admins
+      allHostKeys ++ globalKeys ++ adminKeys
+    else if hostKeys ? ${dirName} then
+      # Host-specific secrets: that host + global keys + admins
+      hostKeys.${dirName} ++ globalKeys ++ adminKeys
+    else
+      # Fallback: just admins
+      adminKeys;
+
+  # Find all secrets across all directories
+  allSecrets = concatLists (lib.map findSecrets directories);
+
+  # Generate the configuration
+  secretsConfig = lib.listToAttrs (
+    lib.map (
+      secretPath:
+      let
+        recipients = getRecipients secretPath;
+        # Remove duplicates and empty keys
+        uniqueRecipients = unique (lib.filter (k: k != null && k != "") recipients);
+      in
+      nameValuePair secretPath {
+        publicKeys = uniqueRecipients;
+      }
+    ) allSecrets
+  );
 
 in
-{
-  # Global secrets (encrypted for all hosts + admins)
-  # "secrets/global/example.age".publicKeys = allHosts ++ admins;
-
-  # Host-specific secrets (encrypted for specific host + admins)
-  # "secrets/nix-builder/ssh_host_ed25519_key.age".publicKeys = [ nix-builder ] ++ admins;
-  # "secrets/usda-dash/ssh_host_ed25519_key.age".publicKeys = [ usda-dash ] ++ admins;
-
-  # NOTE: Until you populate the keys above, you can create secrets with:
-  #   age -r <public-key> -o secrets/path/to/secret.age <<< "secret content"
+secretsConfig
+// {
+  # Export helper information for debugging
+  _meta = {
+    hostKeys = hostKeys;
+    globalKeys = globalKeys;
+    adminKeys = adminKeys;
+    allHostKeys = allHostKeys;
+    discoveredSecrets = allSecrets;
+  };
 }

secrets/update-age-keys.sh

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+# ============================================================================
+# Update Age Keys from SSH Public Keys
+# ============================================================================
+# This script converts SSH public keys to age format for use with ragenix.
+# Run this after adding new SSH .pub files to create corresponding .age.pub files.
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$SCRIPT_DIR"
+
+echo "Converting SSH public keys to age format..."
+
+# Find all .pub files that are SSH keys (not already .age.pub)
+find . -name "*.pub" -not -name "*.age.pub" -type f | while read -r pubkey; do
+    # Check if it's an SSH key
+    if grep -q "^ssh-" "$pubkey" 2>/dev/null || grep -q "^ecdsa-" "$pubkey" 2>/dev/null; then
+        age_key=$(nix shell nixpkgs#ssh-to-age -c ssh-to-age < "$pubkey" 2>/dev/null || true)
+        
+        if [ -n "$age_key" ]; then
+            # Create .age.pub file with the age key
+            age_file="${pubkey%.pub}.age.pub"
+            echo "$age_key" > "$age_file"
+            echo "✓ Converted: $pubkey -> $age_file"
+        else
+            echo "⚠ Skipped: $pubkey (conversion failed)"
+        fi
+    fi
+done
+
+echo ""
+echo "Done! Age public keys have been generated."
+echo "You can now use ragenix to manage secrets:"
+echo "  ragenix -e secrets/global/my-secret.age"
+echo "  ragenix -r  # Re-key all secrets with updated keys"

secrets/usda-dash/ssh_host_ed25519_key.age.pub

@@ -0,0 +1 @@
+age1lr24yvk7rdfh5wkle7h32jpxqxm2e8vk85mc4plv370u2sh4yfmszaaejx

secrets/usda-dash/ssh_host_ed25519_key.pub

@@ -1 +0,0 @@
-ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHI73LOEK2RgfjhZWpryntlLbx0LouHrhQ6v0vZu4Etr root@usda-dash