From 94e618bf9132c8da7c87e857565467894a023f82 Mon Sep 17 00:00:00 2001 From: Hunter Halloran Date: Fri, 9 Jan 2026 12:17:00 -0500 Subject: [PATCH 1/6] feat: Begin support for OIDC login --- docs/MICROSOFT_ENTRA_QUICKSTART.md | 109 ++++++ docs/MICROSOFT_ENTRA_SETUP.md | 342 ++++++++++++++++++ management-dashboard-web-app/.env.example | 7 + management-dashboard-web-app/src/App.tsx | 2 +- .../src/components/Login.tsx | 55 +++ .../src/components/MicrosoftIcon.tsx | 11 + .../src/vite-env.d.ts | 1 + package-lock.json | 61 ++++ package.json | 9 + 9 files changed, 596 insertions(+), 1 deletion(-) create mode 100644 docs/MICROSOFT_ENTRA_QUICKSTART.md create mode 100644 docs/MICROSOFT_ENTRA_SETUP.md create mode 100644 management-dashboard-web-app/src/components/MicrosoftIcon.tsx create mode 100644 package-lock.json create mode 100644 package.json diff --git a/docs/MICROSOFT_ENTRA_QUICKSTART.md b/docs/MICROSOFT_ENTRA_QUICKSTART.md new file mode 100644 index 0000000..9b343cf --- /dev/null +++ b/docs/MICROSOFT_ENTRA_QUICKSTART.md @@ -0,0 +1,109 @@ +# Microsoft Entra OpenID Connect - Quick Start + +## What Was Implemented + +The Login flow now supports Microsoft Entra ID (Azure AD) authentication via OpenID Connect using Supabase's Azure OAuth provider. + +## Files Modified + +1. **[Login.tsx](management-dashboard-web-app/src/components/Login.tsx)**: Added Microsoft login button and OAuth flow +2. **[MicrosoftIcon.tsx](management-dashboard-web-app/src/components/MicrosoftIcon.tsx)**: Created Microsoft logo component +3. **[vite-env.d.ts](management-dashboard-web-app/src/vite-env.d.ts)**: Added TypeScript type for new environment variable +4. **[.env.example](management-dashboard-web-app/.env.example)**: Added Microsoft login configuration + +## How It Works + +### User Experience +1. User visits login page and sees two options: + - Traditional email/password login (existing) + - "Sign in with Microsoft" button (new) +2. Clicking Microsoft button redirects to Microsoft login +3. User authenticates with Microsoft credentials +4. Microsoft redirects back to app with authenticated session + +### Technical Flow +``` +User clicks "Sign in with Microsoft" + ↓ +Supabase signInWithOAuth() with provider: 'azure' + ↓ +Redirect to Microsoft Entra login page + ↓ +User authenticates with Microsoft + ↓ +Microsoft redirects to Supabase callback URL + ↓ +Supabase validates and creates session + ↓ +User redirected back to application (authenticated) +``` + +## Configuration Required + +### 1. Azure Portal Setup +- Register application in Microsoft Entra ID +- Configure redirect URI: `https://.supabase.co/auth/v1/callback` +- Generate client ID and client secret +- Set API permissions (openid, profile, email) + +### 2. Supabase Configuration +Navigate to Authentication > Providers > Azure and configure: +- **Azure Client ID**: From Azure app registration +- **Azure Secret**: From Azure client secrets +- **Azure Tenant**: Use `common` for multi-tenant or specific tenant ID + +### 3. Application Environment +Set in `.env` file: +```bash +VITE_ENABLE_MICROSOFT_LOGIN=true +``` + +## Testing + +### Enable Microsoft Login +```bash +# In .env file +VITE_ENABLE_MICROSOFT_LOGIN=true +``` + +### Disable Microsoft Login +```bash +# In .env file +VITE_ENABLE_MICROSOFT_LOGIN=false +# Or simply remove the variable +``` + +## Features + +✅ **Hybrid Authentication**: Supports both email/password and Microsoft login +✅ **Feature Flag**: Microsoft login can be enabled/disabled via environment variable +✅ **Dark Mode Support**: Microsoft button matches the existing theme +✅ **Error Handling**: Displays authentication errors to users +✅ **Loading States**: Shows loading indicator during authentication +✅ **Type Safety**: Full TypeScript support with proper types + +## Next Steps + +1. **Complete Azure Setup**: Follow [MICROSOFT_ENTRA_SETUP.md](./MICROSOFT_ENTRA_SETUP.md) for detailed configuration +2. **Configure Supabase**: Enable Azure provider in Supabase dashboard +3. **Test Authentication**: Verify the complete login flow +4. **Deploy**: Update production environment variables + +## Documentation + +- **Full Setup Guide**: [MICROSOFT_ENTRA_SETUP.md](./MICROSOFT_ENTRA_SETUP.md) - Complete Azure and Supabase configuration +- **Supabase Docs**: https://supabase.com/docs/guides/auth/social-login/auth-azure +- **Microsoft Identity Platform**: https://docs.microsoft.com/azure/active-directory/develop/ + +## Support + +For issues or questions: +- Check [MICROSOFT_ENTRA_SETUP.md](./MICROSOFT_ENTRA_SETUP.md) troubleshooting section +- Review Supabase Auth logs +- Check Azure sign-in logs in Azure Portal +- Verify redirect URIs match exactly + +--- + +**Implementation Date**: January 9, 2026 +**Status**: Ready for configuration and testing diff --git a/docs/MICROSOFT_ENTRA_SETUP.md b/docs/MICROSOFT_ENTRA_SETUP.md new file mode 100644 index 0000000..38278bf --- /dev/null +++ b/docs/MICROSOFT_ENTRA_SETUP.md @@ -0,0 +1,342 @@ +# Microsoft Entra (Azure AD) OpenID Connect Setup Guide + +## Overview + +This guide walks you through configuring Microsoft Entra ID (formerly Azure Active Directory) authentication for the USDA Vision Management Dashboard using Supabase's Azure OAuth provider. + +## Prerequisites + +- Access to Azure Portal (https://portal.azure.com) +- Admin permissions to register applications in Azure AD +- Access to your Supabase project dashboard +- The USDA Vision application deployed and accessible via URL + +## Step 1: Register Application in Microsoft Entra ID + +### 1.1 Navigate to Azure Portal + +1. Log in to [Azure Portal](https://portal.azure.com) +2. Navigate to **Microsoft Entra ID** (or **Azure Active Directory**) +3. Select **App registrations** from the left sidebar +4. Click **+ New registration** + +### 1.2 Configure Application Registration + +Fill in the following details: + +- **Name**: `USDA Vision Management Dashboard` (or your preferred name) +- **Supported account types**: Choose one of: + - **Single tenant**: Only users in your organization can sign in (most restrictive) + - **Multitenant**: Users in any Azure AD tenant can sign in + - **Multitenant + personal Microsoft accounts**: Broadest support +- **Redirect URI**: + - Platform: **Web** + - URI: `https://.supabase.co/auth/v1/callback` + - Example: `https://abcdefghij.supabase.co/auth/v1/callback` + +Click **Register** to create the application. + +### 1.3 Note Application (Client) ID + +After registration, you'll be taken to the app overview page. Copy and save: +- **Application (client) ID**: This is your `AZURE_CLIENT_ID` +- **Directory (tenant) ID**: This is your `AZURE_TENANT_ID` + +## Step 2: Configure Client Secret + +### 2.1 Create a Client Secret + +1. In your app registration, navigate to **Certificates & secrets** +2. Click **+ New client secret** +3. Add a description: `Supabase Auth` +4. Choose an expiration period (recommendation: 12-24 months) +5. Click **Add** + +### 2.2 Save the Secret Value + +**IMPORTANT**: Copy the **Value** immediately - it will only be shown once! +- This is your `AZURE_CLIENT_SECRET` +- Store it securely (password manager, secure vault, etc.) + +## Step 3: Configure API Permissions + +### 3.1 Add Required Permissions + +1. Navigate to **API permissions** in your app registration +2. Click **+ Add a permission** +3. Select **Microsoft Graph** +4. Choose **Delegated permissions** +5. Add the following permissions: + - `openid` (Sign users in) + - `profile` (View users' basic profile) + - `email` (View users' email address) + - `User.Read` (Sign in and read user profile) + +6. Click **Add permissions** + +### 3.2 Grant Admin Consent (Optional but Recommended) + +If you have admin privileges: +1. Click **Grant admin consent for [Your Organization]** +2. Confirm the action + +This prevents users from seeing a consent prompt on first login. + +## Step 4: Configure Authentication Settings + +### 4.1 Set Token Configuration + +1. Navigate to **Token configuration** in your app registration +2. Click **+ Add optional claim** +3. Choose **ID** token type +4. Add the following claims: + - `email` + - `family_name` + - `given_name` + - `upn` (User Principal Name) + +5. Check **Turn on the Microsoft Graph email, profile permission** if prompted + +### 4.2 Configure Authentication Flow + +1. Navigate to **Authentication** in your app registration +2. Under **Implicit grant and hybrid flows**, ensure: + - ✅ **ID tokens** (used for implicit and hybrid flows) is checked +3. Under **Allow public client flows**: + - Select **No** (keep it secure) + +## Step 5: Configure Supabase + +### 5.1 Navigate to Supabase Auth Settings + +1. Log in to your [Supabase Dashboard](https://app.supabase.com) +2. Select your project +3. Navigate to **Authentication** > **Providers** +4. Find **Azure** in the provider list + +### 5.2 Enable and Configure Azure Provider + +1. Toggle **Enable Sign in with Azure** to ON +2. Fill in the configuration: + + - **Azure Client ID**: Paste your Application (client) ID from Step 1.3 + - **Azure Secret**: Paste your client secret value from Step 2.2 + - **Azure Tenant**: You have two options: + - Use `common` for multi-tenant applications + - Use your specific **Directory (tenant) ID** for single-tenant + - Format: Just the GUID (e.g., `12345678-1234-1234-1234-123456789012`) + +3. Click **Save** + +### 5.3 Note the Callback URL + +Supabase provides the callback URL in the format: +``` +https://.supabase.co/auth/v1/callback +``` + +Verify this matches what you configured in Azure (Step 1.2). + +## Step 6: Configure Application Environment + +### 6.1 Update Environment Variables + +In your application's `.env` file, add or update: + +```bash +# Supabase Configuration (if not already present) +VITE_SUPABASE_URL=https://.supabase.co +VITE_SUPABASE_ANON_KEY= + +# Enable Microsoft Login +VITE_ENABLE_MICROSOFT_LOGIN=true +``` + +### 6.2 Restart Development Server + +If running locally: +```bash +npm run dev +``` + +## Step 7: Test the Integration + +### 7.1 Test Login Flow + +1. Navigate to your application's login page +2. You should see a "Sign in with Microsoft" button +3. Click the button +4. You should be redirected to Microsoft's login page +5. Sign in with your Microsoft account +6. Grant consent if prompted +7. You should be redirected back to your application, logged in + +### 7.2 Verify User Data + +After successful login, check that: +- User profile information is available +- Email address is correctly populated +- User roles/permissions are properly assigned (if using RBAC) + +### 7.3 Common Issues and Troubleshooting + +#### Redirect URI Mismatch +**Error**: `AADSTS50011: The redirect URI ... does not match the redirect URIs configured` + +**Solution**: Ensure the redirect URI in Azure matches exactly: +``` +https://.supabase.co/auth/v1/callback +``` + +#### Invalid Client Secret +**Error**: `Invalid client secret provided` + +**Solution**: +- Verify you copied the secret **Value**, not the Secret ID +- Generate a new client secret if the old one expired + +#### Missing Permissions +**Error**: User consent required or permission denied + +**Solution**: +- Add the required API permissions in Azure (Step 3) +- Grant admin consent if available + +#### CORS Errors +**Error**: CORS policy blocking requests + +**Solution**: +- Ensure your application URL is properly configured in Supabase +- Check that you're using the correct Supabase URL + +## Step 8: Production Deployment + +### 8.1 Update Redirect URI for Production + +1. Go back to Azure Portal > App registrations +2. Navigate to **Authentication** +3. Add production redirect URI: + ``` + https://your-production-domain.com/ + ``` +4. Ensure Supabase callback URI is still present + +### 8.2 Set Production Environment Variables + +Update your production environment with: +```bash +VITE_SUPABASE_URL=https://.supabase.co +VITE_SUPABASE_ANON_KEY= +VITE_ENABLE_MICROSOFT_LOGIN=true +``` + +### 8.3 Security Best Practices + +1. **Rotate Secrets Regularly**: Set calendar reminders before client secret expiration +2. **Use Azure Key Vault**: Store secrets in Azure Key Vault for enhanced security +3. **Monitor Sign-ins**: Use Azure AD sign-in logs to monitor authentication activity +4. **Implement MFA**: Require multi-factor authentication for sensitive accounts +5. **Review Permissions**: Regularly audit API permissions and remove unnecessary ones + +## Advanced Configuration + +### Multi-Tenant Support + +If you want to support users from multiple Azure AD tenants: + +1. In Azure App Registration > **Authentication**: + - Set **Supported account types** to **Multitenant** + +2. In Supabase Azure provider configuration: + - Set **Azure Tenant** to `common` + +### Custom Domain Configuration + +If using a custom domain with Supabase: + +1. Configure custom domain in Supabase dashboard +2. Update redirect URI in Azure to use custom domain +3. Update application environment variables + +### User Attribute Mapping + +To map Azure AD attributes to Supabase user metadata: + +1. Use Supabase database triggers or functions +2. Extract attributes from JWT token +3. Update user metadata in `auth.users` table + +Example attributes available: +- `sub`: User's unique ID +- `email`: Email address +- `name`: Full name +- `given_name`: First name +- `family_name`: Last name +- `preferred_username`: Username + +## Integration with RBAC System + +To integrate Microsoft Entra authentication with your existing RBAC system: + +### Option 1: Manual Role Assignment + +After user signs in via Microsoft: +1. Admin assigns roles in the management dashboard +2. Roles stored in your `user_roles` table +3. User gets appropriate permissions on next login + +### Option 2: Azure AD Group Mapping + +Map Azure AD groups to application roles: +1. Configure group claims in Azure token configuration +2. Read groups from JWT token in Supabase +3. Automatically assign roles based on group membership + +### Option 3: Hybrid Approach + +Support both Microsoft and email/password login: +- Keep existing email/password authentication +- Add Microsoft as an additional option +- Users can link accounts or use either method + +## Monitoring and Maintenance + +### Azure AD Monitoring + +- **Sign-in logs**: Monitor authentication attempts +- **Audit logs**: Track configuration changes +- **Usage analytics**: Understand authentication patterns + +### Supabase Monitoring + +- **Auth logs**: View authentication events +- **User activity**: Track active users +- **Error logs**: Identify authentication issues + +### Regular Maintenance Tasks + +- [ ] Rotate client secrets before expiration (set reminders) +- [ ] Review and update API permissions quarterly +- [ ] Audit user access and remove inactive users +- [ ] Update token configuration as needed +- [ ] Test authentication flow after any infrastructure changes + +## Support and Resources + +### Microsoft Documentation +- [Microsoft identity platform documentation](https://docs.microsoft.com/azure/active-directory/develop/) +- [Azure AD app registration](https://docs.microsoft.com/azure/active-directory/develop/quickstart-register-app) +- [OpenID Connect on Microsoft identity platform](https://docs.microsoft.com/azure/active-directory/develop/v2-protocols-oidc) + +### Supabase Documentation +- [Supabase Auth with Azure](https://supabase.com/docs/guides/auth/social-login/auth-azure) +- [Supabase Auth API reference](https://supabase.com/docs/reference/javascript/auth-signinwithoauth) + +### Troubleshooting Resources +- Azure AD Error Codes: https://docs.microsoft.com/azure/active-directory/develop/reference-aadsts-error-codes +- Supabase Discord Community: https://discord.supabase.com + +--- + +**Last Updated**: January 2026 +**Maintained By**: USDA Vision System Team diff --git a/management-dashboard-web-app/.env.example b/management-dashboard-web-app/.env.example index 91a7ca1..f292910 100755 --- a/management-dashboard-web-app/.env.example +++ b/management-dashboard-web-app/.env.example @@ -10,3 +10,10 @@ VITE_VISION_SYSTEM_REMOTE_URL=http://exp-dash:3002/assets/remoteEntry.js?v=$(dat # API URLs VITE_VISION_API_URL=http://exp-dash:8000 VITE_MEDIA_API_URL=http://exp-dash:8090 + +# Supabase Configuration +VITE_SUPABASE_URL=https://your-project-url.supabase.co +VITE_SUPABASE_ANON_KEY=your-anon-key + +# Microsoft Entra (Azure AD) OAuth Configuration +VITE_ENABLE_MICROSOFT_LOGIN=true diff --git a/management-dashboard-web-app/src/App.tsx b/management-dashboard-web-app/src/App.tsx index f7f3786..2cae9fd 100755 --- a/management-dashboard-web-app/src/App.tsx +++ b/management-dashboard-web-app/src/App.tsx @@ -14,7 +14,7 @@ function App() { checkAuthState() // Listen for auth changes - const { data: { subscription } } = supabase.auth.onAuthStateChange((event, session) => { + const { data: { subscription } } = supabase.auth.onAuthStateChange((event: string, session: any) => { console.log('Auth state changed:', event, !!session) setIsAuthenticated(!!session) setLoading(false) diff --git a/management-dashboard-web-app/src/components/Login.tsx b/management-dashboard-web-app/src/components/Login.tsx index 8e4929f..26a89c1 100755 --- a/management-dashboard-web-app/src/components/Login.tsx +++ b/management-dashboard-web-app/src/components/Login.tsx @@ -1,5 +1,6 @@ import { useState } from 'react' import { supabase } from '../lib/supabase' +import { MicrosoftIcon } from './MicrosoftIcon' interface LoginProps { onLoginSuccess: () => void @@ -10,6 +11,7 @@ export function Login({ onLoginSuccess }: LoginProps) { const [password, setPassword] = useState('') const [loading, setLoading] = useState(false) const [error, setError] = useState(null) + const enableMicrosoftLogin = import.meta.env.VITE_ENABLE_MICROSOFT_LOGIN === 'true' const handleLogin = async (e: React.FormEvent) => { e.preventDefault() @@ -35,6 +37,32 @@ export function Login({ onLoginSuccess }: LoginProps) { } } + const handleMicrosoftLogin = async () => { + setLoading(true) + setError(null) + + try { + const { error } = await supabase.auth.signInWithOAuth({ + provider: 'azure', + options: { + scopes: 'email openid profile', + redirectTo: `${window.location.origin}/`, + }, + }) + + if (error) { + setError(error.message) + setLoading(false) + } + // If successful, user will be redirected to Microsoft login + // and then back to the app, so we don't stop loading here + } catch (err) { + setError('An unexpected error occurred during Microsoft login') + console.error('Microsoft login error:', err) + setLoading(false) + } + } + return (
@@ -108,6 +136,33 @@ export function Login({ onLoginSuccess }: LoginProps) {
+ + {enableMicrosoftLogin && ( + <> +
+
+
+
+
+ + Or continue with + +
+
+ +
+ +
+ + )}
) diff --git a/management-dashboard-web-app/src/components/MicrosoftIcon.tsx b/management-dashboard-web-app/src/components/MicrosoftIcon.tsx new file mode 100644 index 0000000..8676966 --- /dev/null +++ b/management-dashboard-web-app/src/components/MicrosoftIcon.tsx @@ -0,0 +1,11 @@ +export function MicrosoftIcon({ className = "w-5 h-5" }: { className?: string }) { + return ( + + + + + + + + ) +} diff --git a/management-dashboard-web-app/src/vite-env.d.ts b/management-dashboard-web-app/src/vite-env.d.ts index 46479a0..14266d5 100755 --- a/management-dashboard-web-app/src/vite-env.d.ts +++ b/management-dashboard-web-app/src/vite-env.d.ts @@ -4,6 +4,7 @@ interface ImportMetaEnv { readonly VITE_SUPABASE_URL: string; readonly VITE_SUPABASE_ANON_KEY: string; readonly VITE_VISION_API_URL?: string; // optional; defaults to "/api" via vite proxy + readonly VITE_ENABLE_MICROSOFT_LOGIN?: string; // optional; enable Microsoft Entra authentication } interface ImportMeta { diff --git a/package-lock.json b/package-lock.json new file mode 100644 index 0000000..4d8cb8b --- /dev/null +++ b/package-lock.json @@ -0,0 +1,61 @@ +{ + "name": "usda-vision", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "dependencies": { + "react": "^19.2.3", + "react-dom": "^19.2.3" + }, + "devDependencies": { + "@types/react": "^19.2.7" + } + }, + "node_modules/@types/react": { + "version": "19.2.7", + "resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.7.tgz", + "integrity": "sha512-MWtvHrGZLFttgeEj28VXHxpmwYbor/ATPYbBfSFZEIRK0ecCFLl2Qo55z52Hss+UV9CRN7trSeq1zbgx7YDWWg==", + "dev": true, + "license": "MIT", + "dependencies": { + "csstype": "^3.2.2" + } + }, + "node_modules/csstype": { + "version": "3.2.3", + "resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.3.tgz", + "integrity": "sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/react": { + "version": "19.2.3", + "resolved": "https://registry.npmjs.org/react/-/react-19.2.3.tgz", + "integrity": "sha512-Ku/hhYbVjOQnXDZFv2+RibmLFGwFdeeKHFcOTlrt7xplBnya5OGn/hIRDsqDiSUcfORsDC7MPxwork8jBwsIWA==", + "license": "MIT", + "peer": true, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/react-dom": { + "version": "19.2.3", + "resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.2.3.tgz", + "integrity": "sha512-yELu4WmLPw5Mr/lmeEpox5rw3RETacE++JgHqQzd2dg+YbJuat3jH4ingc+WPZhxaoFzdv9y33G+F7Nl5O0GBg==", + "license": "MIT", + "dependencies": { + "scheduler": "^0.27.0" + }, + "peerDependencies": { + "react": "^19.2.3" + } + }, + "node_modules/scheduler": { + "version": "0.27.0", + "resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.27.0.tgz", + "integrity": "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==", + "license": "MIT" + } + } +} diff --git a/package.json b/package.json new file mode 100644 index 0000000..7ad4e9b --- /dev/null +++ b/package.json @@ -0,0 +1,9 @@ +{ + "dependencies": { + "react": "^19.2.3", + "react-dom": "^19.2.3" + }, + "devDependencies": { + "@types/react": "^19.2.7" + } +} From 32504f7196c150f3c617f432ee112908157fc6a0 Mon Sep 17 00:00:00 2001 From: Hunter Halloran Date: Fri, 9 Jan 2026 12:52:42 -0500 Subject: [PATCH 2/6] feat: Add Azure external auth provider --- .env.azure.example | 24 +++++ docs/MICROSOFT_ENTRA_QUICKSTART.md | 31 +++++- docs/MICROSOFT_ENTRA_SETUP.md | 91 ++++++++++++++++- docs/SELF_HOSTED_AZURE_SETUP.md | 158 +++++++++++++++++++++++++++++ supabase/config.toml | 12 +++ 5 files changed, 311 insertions(+), 5 deletions(-) create mode 100644 .env.azure.example create mode 100644 docs/SELF_HOSTED_AZURE_SETUP.md diff --git a/.env.azure.example b/.env.azure.example new file mode 100644 index 0000000..4c7b6c6 --- /dev/null +++ b/.env.azure.example @@ -0,0 +1,24 @@ +# Microsoft Entra (Azure AD) OAuth Configuration for Self-Hosted Supabase +# Copy this file to your actual environment configuration and fill in the values + +# Azure Application (Client) ID +# Get this from Azure Portal > App registrations > Your app > Overview +AZURE_CLIENT_ID=your-application-client-id-here + +# Azure Client Secret +# Get this from Azure Portal > App registrations > Your app > Certificates & secrets +AZURE_CLIENT_SECRET=your-client-secret-value-here + +# Azure Tenant ID or 'common' +# Options: +# - 'common': Multi-tenant (any Azure AD organization) +# - 'organizations': Any Azure AD organization (excludes personal accounts) +# - 'consumers': Personal Microsoft accounts only +# - Your specific tenant ID: Single-tenant (e.g., '12345678-1234-1234-1234-123456789012') +# Get tenant ID from Azure Portal > App registrations > Your app > Overview +AZURE_TENANT_ID=common + +# Notes: +# 1. These variables are used in supabase/config.toml via env() substitution +# 2. Never commit this file with real secrets to git +# 3. After setting these, restart your Supabase services: docker-compose restart diff --git a/docs/MICROSOFT_ENTRA_QUICKSTART.md b/docs/MICROSOFT_ENTRA_QUICKSTART.md index 9b343cf..9f79d7a 100644 --- a/docs/MICROSOFT_ENTRA_QUICKSTART.md +++ b/docs/MICROSOFT_ENTRA_QUICKSTART.md @@ -42,16 +42,45 @@ User redirected back to application (authenticated) ### 1. Azure Portal Setup - Register application in Microsoft Entra ID -- Configure redirect URI: `https://.supabase.co/auth/v1/callback` +- Configure redirect URI: + - **Supabase Cloud**: `https://.supabase.co/auth/v1/callback` + - **Self-hosted**: `http://:/auth/v1/callback` - Generate client ID and client secret - Set API permissions (openid, profile, email) ### 2. Supabase Configuration + +#### For Supabase Cloud: Navigate to Authentication > Providers > Azure and configure: - **Azure Client ID**: From Azure app registration - **Azure Secret**: From Azure client secrets - **Azure Tenant**: Use `common` for multi-tenant or specific tenant ID +#### For Self-Hosted Supabase: + +Edit `supabase/config.toml`: +```toml +[auth.external.azure] +enabled = true +client_id = "env(AZURE_CLIENT_ID)" +secret = "env(AZURE_CLIENT_SECRET)" +redirect_uri = "" +url = "https://login.microsoftonline.com/env(AZURE_TENANT_ID)/v2.0" +skip_nonce_check = false +``` + +Set environment variables: +```bash +AZURE_CLIENT_ID="your-application-client-id" +AZURE_CLIENT_SECRET="your-client-secret" +AZURE_TENANT_ID="common" # or specific tenant ID +``` + +Restart Supabase: +```bash +docker-compose down && docker-compose up -d +``` + ### 3. Application Environment Set in `.env` file: ```bash diff --git a/docs/MICROSOFT_ENTRA_SETUP.md b/docs/MICROSOFT_ENTRA_SETUP.md index 38278bf..d532815 100644 --- a/docs/MICROSOFT_ENTRA_SETUP.md +++ b/docs/MICROSOFT_ENTRA_SETUP.md @@ -4,11 +4,13 @@ This guide walks you through configuring Microsoft Entra ID (formerly Azure Active Directory) authentication for the USDA Vision Management Dashboard using Supabase's Azure OAuth provider. +> **📌 Self-Hosted Supabase Users**: If you're using a self-hosted Supabase instance, see the simplified guide: [SELF_HOSTED_AZURE_SETUP.md](SELF_HOSTED_AZURE_SETUP.md). Self-hosted instances configure OAuth providers via `config.toml` and environment variables, not through the UI. + ## Prerequisites - Access to Azure Portal (https://portal.azure.com) - Admin permissions to register applications in Azure AD -- Access to your Supabase project dashboard +- Access to your Supabase project (Cloud dashboard or self-hosted instance) - The USDA Vision application deployed and accessible via URL ## Step 1: Register Application in Microsoft Entra ID @@ -107,14 +109,16 @@ This prevents users from seeing a consent prompt on first login. ## Step 5: Configure Supabase -### 5.1 Navigate to Supabase Auth Settings +### For Supabase Cloud (Hosted) + +#### 5.1 Navigate to Supabase Auth Settings 1. Log in to your [Supabase Dashboard](https://app.supabase.com) 2. Select your project 3. Navigate to **Authentication** > **Providers** 4. Find **Azure** in the provider list -### 5.2 Enable and Configure Azure Provider +#### 5.2 Enable and Configure Azure Provider 1. Toggle **Enable Sign in with Azure** to ON 2. Fill in the configuration: @@ -128,7 +132,7 @@ This prevents users from seeing a consent prompt on first login. 3. Click **Save** -### 5.3 Note the Callback URL +#### 5.3 Note the Callback URL Supabase provides the callback URL in the format: ``` @@ -137,6 +141,85 @@ https://.supabase.co/auth/v1/callback Verify this matches what you configured in Azure (Step 1.2). +### For Self-Hosted Supabase + +If you're running a self-hosted Supabase instance, OAuth providers are configured via the `config.toml` file and environment variables rather than through the UI. + +#### 5.1 Edit config.toml + +1. Open your `supabase/config.toml` file +2. Find or add the `[auth.external.azure]` section: + +```toml +[auth.external.azure] +enabled = true +client_id = "env(AZURE_CLIENT_ID)" +secret = "env(AZURE_CLIENT_SECRET)" +redirect_uri = "" +url = "https://login.microsoftonline.com/env(AZURE_TENANT_ID)/v2.0" +skip_nonce_check = false +``` + +3. Set `enabled = true` to activate Azure authentication + +#### 5.2 Set Environment Variables + +Create or update your environment file (`.env` or set in your deployment): + +```bash +# Azure AD OAuth Configuration +AZURE_CLIENT_ID="your-application-client-id-from-azure" +AZURE_CLIENT_SECRET="your-client-secret-from-azure" +AZURE_TENANT_ID="common" # or your specific tenant ID +``` + +**Important**: +- Use `common` for multi-tenant (any Azure AD organization) +- Use `organizations` for any Azure AD organization (excludes personal Microsoft accounts) +- Use `consumers` for personal Microsoft accounts only +- Use your specific tenant ID (GUID) for single-tenant applications + +#### 5.3 Update Azure Redirect URI + +For self-hosted Supabase, your callback URL will be: +``` +http://:/auth/v1/callback +``` + +For example, if your Supabase API is at `http://192.168.1.100:54321`: +``` +http://192.168.1.100:54321/auth/v1/callback +``` + +**Go back to Azure Portal** (Step 1.2) and add this redirect URI to your app registration. + +#### 5.4 Restart Supabase Services + +After making these changes, restart your Supabase services: + +```bash +# If using docker-compose +docker-compose down +docker-compose up -d + +# Or if using the provided script +./docker-compose.sh restart +``` + +#### 5.5 Verify Configuration + +Check that the auth service picked up your configuration: + +```bash +# View auth service logs +docker-compose logs auth + +# Or for specific service name +docker-compose logs supabase-auth +``` + +Look for log entries indicating Azure provider is enabled. + ## Step 6: Configure Application Environment ### 6.1 Update Environment Variables diff --git a/docs/SELF_HOSTED_AZURE_SETUP.md b/docs/SELF_HOSTED_AZURE_SETUP.md new file mode 100644 index 0000000..188f358 --- /dev/null +++ b/docs/SELF_HOSTED_AZURE_SETUP.md @@ -0,0 +1,158 @@ +# Self-Hosted Supabase - Microsoft Entra Setup + +## Quick Setup Guide + +For self-hosted Supabase instances, OAuth providers like Microsoft Entra (Azure AD) are configured through config files and environment variables, not through the UI. + +### Step 1: Configure Azure Application + +Follow steps 1-4 in [MICROSOFT_ENTRA_SETUP.md](MICROSOFT_ENTRA_SETUP.md) to: +1. Register your app in Azure Portal +2. Get your Client ID and Secret +3. Set up API permissions +4. Configure token claims + +**Important**: Your redirect URI should be: +``` +http://:/auth/v1/callback +``` + +Example: `http://192.168.1.100:54321/auth/v1/callback` + +### Step 2: Configure Supabase + +The Azure provider configuration is already added to `supabase/config.toml`: + +```toml +[auth.external.azure] +enabled = false # Change this to true +client_id = "env(AZURE_CLIENT_ID)" +secret = "env(AZURE_CLIENT_SECRET)" +redirect_uri = "" +url = "https://login.microsoftonline.com/env(AZURE_TENANT_ID)/v2.0" +skip_nonce_check = false +``` + +### Step 3: Set Environment Variables + +1. Copy the example file: + ```bash + cp .env.azure.example .env.azure + ``` + +2. Edit `.env.azure` with your actual values: + ```bash + AZURE_CLIENT_ID=your-application-client-id + AZURE_CLIENT_SECRET=your-client-secret + AZURE_TENANT_ID=common # or your specific tenant ID + ``` + +3. Source the environment file before starting Supabase: + ```bash + source .env.azure + ``` + + Or add it to your docker-compose environment. + +### Step 4: Enable Azure Provider + +Edit `supabase/config.toml` and change: +```toml +[auth.external.azure] +enabled = true # Change from false to true +``` + +### Step 5: Restart Supabase + +```bash +docker-compose down +docker-compose up -d +``` + +Or if using the project script: +```bash +./docker-compose.sh restart +``` + +### Step 6: Enable in Application + +In `management-dashboard-web-app/.env`: +```bash +VITE_ENABLE_MICROSOFT_LOGIN=true +``` + +### Verification + +1. Check auth service logs: + ```bash + docker-compose logs auth | grep -i azure + ``` + +2. You should see the Microsoft login button on your application's login page + +3. Click it and verify you're redirected to Microsoft login + +### Troubleshooting + +#### Azure Provider Not Working + +**Check logs**: +```bash +docker-compose logs auth +``` + +**Verify environment variables are loaded**: +```bash +docker-compose exec auth env | grep AZURE +``` + +#### Redirect URI Mismatch + +Ensure the redirect URI in Azure exactly matches: +``` +http://:/auth/v1/callback +``` + +Common mistake: Using `localhost` instead of the actual IP address. + +#### Environment Variables Not Set + +If you see errors about missing AZURE variables, make sure to: +1. Export them in your shell before running docker-compose +2. Or add them to your docker-compose.yml environment section +3. Or use a .env file that docker-compose automatically loads + +### Docker Compose Environment Variables + +You can also add the variables directly to your `docker-compose.yml`: + +```yaml +services: + auth: + environment: + AZURE_CLIENT_ID: ${AZURE_CLIENT_ID} + AZURE_CLIENT_SECRET: ${AZURE_CLIENT_SECRET} + AZURE_TENANT_ID: ${AZURE_TENANT_ID:-common} +``` + +Then create a `.env` file in the same directory: +```bash +AZURE_CLIENT_ID=your-client-id +AZURE_CLIENT_SECRET=your-secret +AZURE_TENANT_ID=common +``` + +### Security Notes + +- Never commit `.env.azure` or `.env` files with real secrets to git +- Add them to `.gitignore` +- Use environment variable substitution in config.toml +- Rotate client secrets regularly (before expiration) +- Monitor sign-in logs in Azure Portal + +### Additional Resources + +- Full setup guide: [MICROSOFT_ENTRA_SETUP.md](MICROSOFT_ENTRA_SETUP.md) +- Quick reference: [MICROSOFT_ENTRA_QUICKSTART.md](MICROSOFT_ENTRA_QUICKSTART.md) +- Supabase self-hosting docs: https://supabase.com/docs/guides/self-hosting +- Azure OAuth docs: https://docs.microsoft.com/azure/active-directory/develop/v2-oauth2-auth-code-flow diff --git a/supabase/config.toml b/supabase/config.toml index b4e8807..8e5f4ef 100755 --- a/supabase/config.toml +++ b/supabase/config.toml @@ -278,6 +278,18 @@ url = "" # If enabled, the nonce check will be skipped. Required for local sign in with Google auth. skip_nonce_check = false +[auth.external.azure] +enabled = "env(VITE_ENABLE_MICROSOFT_LOGIN)" +client_id = "env(AZURE_CLIENT_ID)" +# DO NOT commit your OAuth provider secret to git. Use environment variable substitution instead: +secret = "env(AZURE_CLIENT_SECRET)" +# Overrides the default auth redirectUrl. +redirect_uri = "" +# Azure tenant ID or 'common' for multi-tenant. Use 'common', 'organizations', 'consumers', or your specific tenant ID. +url = "https://login.microsoftonline.com/env(AZURE_TENANT_ID)/v2.0" +# If enabled, the nonce check will be skipped. +skip_nonce_check = false + # Allow Solana wallet holders to sign in to your project via the Sign in with Solana (SIWS, EIP-4361) standard. # You can configure "web3" rate limit in the [auth.rate_limit] section and set up [auth.captcha] if self-hosting. [auth.web3.solana] From 83084158b54ab8149981b5b9572cd8367a896fa6 Mon Sep 17 00:00:00 2001 From: Hunter Halloran Date: Tue, 13 Jan 2026 13:47:33 -0500 Subject: [PATCH 3/6] feat: Enable UGA SSO with Microsoft Entra --- docs/OAUTH_USER_SYNC_FIX.md | 139 ++++++++++++++++++ management-dashboard-web-app/src/App.tsx | 9 +- .../src/lib/supabase.ts | 54 +++++++ .../migrations/00003_oauth_user_sync.sql | 46 ++++++ supabase/config.toml | 4 +- supabase/migrations/00003_oauth_user_sync.sql | 46 ++++++ 6 files changed, 295 insertions(+), 3 deletions(-) create mode 100644 docs/OAUTH_USER_SYNC_FIX.md create mode 100644 management-dashboard-web-app/supabase/migrations/00003_oauth_user_sync.sql create mode 100644 supabase/migrations/00003_oauth_user_sync.sql diff --git a/docs/OAUTH_USER_SYNC_FIX.md b/docs/OAUTH_USER_SYNC_FIX.md new file mode 100644 index 0000000..8aa63b8 --- /dev/null +++ b/docs/OAUTH_USER_SYNC_FIX.md @@ -0,0 +1,139 @@ +# OAuth User Synchronization Fix + +## Problem +When a user signs on with an OAuth provider (Microsoft Entra/Azure AD) for the first time, the user is added to `auth.users` in Supabase but NOT to the application's `user_profiles` table. This causes the application to fail when trying to load user data, as there's no user profile available. + +## Solution +Implemented automatic user profile creation for OAuth users through a multi-layered approach: + +### 1. Database Trigger (00003_oauth_user_sync.sql) +- **Location**: `supabase/migrations/00003_oauth_user_sync.sql` and `management-dashboard-web-app/supabase/migrations/00003_oauth_user_sync.sql` +- **Function**: `handle_new_oauth_user()` + - Automatically creates a user profile in `public.user_profiles` when a new user is created in `auth.users` + - Handles the synchronization at the database level, ensuring it works regardless of where users are created + - Includes race condition handling with `ON CONFLICT DO NOTHING` + +- **Trigger**: `on_auth_user_created` + - Fires after INSERT on `auth.users` + - Ensures every new OAuth user gets a profile entry + +### 2. Client-Side Utility Function (src/lib/supabase.ts) +- **Function**: `userManagement.syncOAuthUser()` + - Provides a fallback synchronization mechanism for any OAuth users that slip through + - Checks if user profile exists before creating + - Handles race conditions gracefully (duplicate key errors) + - Includes comprehensive error logging for debugging + +**Logic**: +```typescript +1. Get current authenticated user from Supabase Auth +2. Check if user profile already exists in user_profiles table +3. If exists: return (no action needed) +4. If not exists: create user profile with: + - id: user's UUID from auth.users + - email: user's email from auth.users + - status: 'active' (default status) +5. Handle errors gracefully: + - "No rows returned" (PGRST116): Expected, user doesn't exist yet + - "Duplicate key" (23505): Race condition, another process created it first + - Other errors: Log and continue +``` + +### 3. App Integration (src/App.tsx) +- **Updated**: Auth state change listener +- **Triggers**: When `SIGNED_IN` or `INITIAL_SESSION` event occurs +- **Action**: Calls `userManagement.syncOAuthUser()` asynchronously +- **Benefit**: Ensures user profile exists before the rest of the app tries to access it + +## How It Works + +### OAuth Sign-In Flow (New) +``` +1. User clicks "Sign in with Microsoft" + ↓ +2. Redirected to Microsoft login + ↓ +3. Microsoft authenticates and redirects back + ↓ +4. Supabase creates entry in auth.users + ↓ +5. Database trigger fires → user_profiles entry created + ↓ +6. App receives SIGNED_IN event + ↓ +7. App calls syncOAuthUser() as extra safety measure + ↓ +8. User profile is guaranteed to exist + ↓ +9. getUserProfile() and loadData() succeed +``` + +## Backward Compatibility +- **Non-invasive**: The solution uses triggers and utility functions, doesn't modify existing tables +- **Graceful degradation**: If either layer fails, the other provides a fallback +- **No breaking changes**: Existing APIs and components remain unchanged + +## Testing Recommendations + +### Test 1: First-Time OAuth Sign-In +1. Clear browser cookies/session +2. Click "Sign in with Microsoft" +3. Complete OAuth flow +4. Verify: + - User is in Supabase auth + - User is in `user_profiles` table + - App loads user data without errors + - Dashboard displays correctly + +### Test 2: Verify Database Trigger +1. Directly create a user in `auth.users` via SQL +2. Verify that `user_profiles` entry is automatically created +3. Check timestamp to confirm trigger fired + +### Test 3: Verify Client-Side Fallback +1. Manually delete a user's `user_profiles` entry +2. Reload the app +3. Verify that `syncOAuthUser()` recreates the profile +4. Check browser console for success logs + +## Files Modified +1. **Database Migrations**: + - `/usda-vision/supabase/migrations/00003_oauth_user_sync.sql` (NEW) + - `/usda-vision/management-dashboard-web-app/supabase/migrations/00003_oauth_user_sync.sql` (NEW) + +2. **TypeScript/React**: + - `/usda-vision/management-dashboard-web-app/src/lib/supabase.ts` (MODIFIED) + - Added `syncOAuthUser()` method to `userManagement` object + + - `/usda-vision/management-dashboard-web-app/src/App.tsx` (MODIFIED) + - Import `userManagement` + - Call `syncOAuthUser()` on auth state change + +## Deployment Steps + +1. **Apply Database Migrations**: + ```bash + # Run the new migration + supabase migration up + ``` + +2. **Deploy Application Code**: + - Push the changes to `src/lib/supabase.ts` and `src/App.tsx` + - No environment variable changes needed + - No configuration changes needed + +3. **Test in Staging**: + - Test OAuth sign-in with a fresh account + - Verify user profile is created + - Check app functionality + +4. **Monitor in Production**: + - Watch browser console for any errors from `syncOAuthUser()` + - Check database logs to confirm trigger is firing + - Monitor user creation metrics + +## Future Enhancements +- Assign default roles to new OAuth users (currently requires manual assignment) +- Pre-populate `first_name` and `last_name` from OAuth provider data +- Add user profile completion workflow for new OAuth users +- Auto-disable account creation for users outside organization diff --git a/management-dashboard-web-app/src/App.tsx b/management-dashboard-web-app/src/App.tsx index 2cae9fd..195558a 100755 --- a/management-dashboard-web-app/src/App.tsx +++ b/management-dashboard-web-app/src/App.tsx @@ -1,5 +1,5 @@ import { useState, useEffect } from 'react' -import { supabase } from './lib/supabase' +import { supabase, userManagement } from './lib/supabase' import { Login } from './components/Login' import { Dashboard } from './components/Dashboard' import { CameraRoute } from './components/CameraRoute' @@ -19,6 +19,13 @@ function App() { setIsAuthenticated(!!session) setLoading(false) + // Sync OAuth user on successful sign in (creates user profile if needed) + if ((event === 'SIGNED_IN' || event === 'INITIAL_SESSION') && session) { + userManagement.syncOAuthUser().catch((err) => { + console.error('Failed to sync OAuth user:', err) + }) + } + // Handle signout route if (event === 'SIGNED_OUT') { setCurrentRoute('/') diff --git a/management-dashboard-web-app/src/lib/supabase.ts b/management-dashboard-web-app/src/lib/supabase.ts index 720182b..af5cebc 100755 --- a/management-dashboard-web-app/src/lib/supabase.ts +++ b/management-dashboard-web-app/src/lib/supabase.ts @@ -557,6 +557,60 @@ export const userManagement = { if (error) throw error return data + }, + + // Sync OAuth user - ensures user profile exists for OAuth-authenticated users + async syncOAuthUser(): Promise { + try { + const { data: { user: authUser }, error: authError } = await supabase.auth.getUser() + + if (authError || !authUser) { + console.warn('No authenticated user found for OAuth sync') + return + } + + // Check if user profile already exists + const { data: existingProfile, error: checkError } = await supabase + .from('user_profiles') + .select('id') + .eq('id', authUser.id) + .single() + + // If profile already exists, no need to create it + if (existingProfile && !checkError) { + console.log('User profile already exists for user:', authUser.id) + return + } + + // If error is not "no rows returned", it's a real error + if (checkError && checkError.code !== 'PGRST116') { + console.error('Error checking for existing profile:', checkError) + return + } + + // Create user profile for new OAuth user + const { error: insertError } = await supabase + .from('user_profiles') + .insert({ + id: authUser.id, + email: authUser.email || '', + status: 'active' + }) + + if (insertError) { + // Ignore "duplicate key value" errors in case of race condition + if (insertError.code === '23505') { + console.log('User profile was already created (race condition handled)') + return + } + console.error('Error creating user profile for OAuth user:', insertError) + return + } + + console.log('Successfully created user profile for OAuth user:', authUser.id) + } catch (error) { + console.error('Unexpected error in syncOAuthUser:', error) + } } } diff --git a/management-dashboard-web-app/supabase/migrations/00003_oauth_user_sync.sql b/management-dashboard-web-app/supabase/migrations/00003_oauth_user_sync.sql new file mode 100644 index 0000000..26101fb --- /dev/null +++ b/management-dashboard-web-app/supabase/migrations/00003_oauth_user_sync.sql @@ -0,0 +1,46 @@ +-- OAuth User Synchronization +-- This migration adds functionality to automatically create user profiles when users sign up via OAuth + +-- ============================================= +-- 1. CREATE FUNCTION FOR OAUTH USER AUTO-PROFILE CREATION +-- ============================================= + +CREATE OR REPLACE FUNCTION public.handle_new_oauth_user() +RETURNS TRIGGER AS $$ +BEGIN + -- Check if user profile already exists + IF NOT EXISTS ( + SELECT 1 FROM public.user_profiles WHERE id = NEW.id + ) THEN + -- Create user profile with default active status + INSERT INTO public.user_profiles (id, email, status) + VALUES ( + NEW.id, + NEW.email, + 'active' + ) + ON CONFLICT (id) DO NOTHING; + END IF; + + RETURN NEW; +END; +$$ LANGUAGE plpgsql SECURITY DEFINER; + +-- ============================================= +-- 2. CREATE TRIGGER FOR NEW AUTH USERS +-- ============================================= + +-- Drop the trigger if it exists to avoid conflicts +DROP TRIGGER IF EXISTS on_auth_user_created ON auth.users; + +-- Create trigger that fires after a new user is created in auth.users +CREATE TRIGGER on_auth_user_created + AFTER INSERT ON auth.users + FOR EACH ROW EXECUTE FUNCTION public.handle_new_oauth_user(); + +-- ============================================= +-- 3. COMMENT FOR DOCUMENTATION +-- ============================================= + +COMMENT ON FUNCTION public.handle_new_oauth_user() IS +'Automatically creates a user profile in public.user_profiles when a new user is created via OAuth in auth.users. This ensures OAuth users are immediately accessible in the application without manual provisioning.'; diff --git a/supabase/config.toml b/supabase/config.toml index 8e5f4ef..0cfe5de 100755 --- a/supabase/config.toml +++ b/supabase/config.toml @@ -284,9 +284,9 @@ client_id = "env(AZURE_CLIENT_ID)" # DO NOT commit your OAuth provider secret to git. Use environment variable substitution instead: secret = "env(AZURE_CLIENT_SECRET)" # Overrides the default auth redirectUrl. -redirect_uri = "" +redirect_uri = "env(AZURE_REDIRECT_URI)" # Azure tenant ID or 'common' for multi-tenant. Use 'common', 'organizations', 'consumers', or your specific tenant ID. -url = "https://login.microsoftonline.com/env(AZURE_TENANT_ID)/v2.0" +url = "env(AZURE_TENANT_URL)" # If enabled, the nonce check will be skipped. skip_nonce_check = false diff --git a/supabase/migrations/00003_oauth_user_sync.sql b/supabase/migrations/00003_oauth_user_sync.sql new file mode 100644 index 0000000..26101fb --- /dev/null +++ b/supabase/migrations/00003_oauth_user_sync.sql @@ -0,0 +1,46 @@ +-- OAuth User Synchronization +-- This migration adds functionality to automatically create user profiles when users sign up via OAuth + +-- ============================================= +-- 1. CREATE FUNCTION FOR OAUTH USER AUTO-PROFILE CREATION +-- ============================================= + +CREATE OR REPLACE FUNCTION public.handle_new_oauth_user() +RETURNS TRIGGER AS $$ +BEGIN + -- Check if user profile already exists + IF NOT EXISTS ( + SELECT 1 FROM public.user_profiles WHERE id = NEW.id + ) THEN + -- Create user profile with default active status + INSERT INTO public.user_profiles (id, email, status) + VALUES ( + NEW.id, + NEW.email, + 'active' + ) + ON CONFLICT (id) DO NOTHING; + END IF; + + RETURN NEW; +END; +$$ LANGUAGE plpgsql SECURITY DEFINER; + +-- ============================================= +-- 2. CREATE TRIGGER FOR NEW AUTH USERS +-- ============================================= + +-- Drop the trigger if it exists to avoid conflicts +DROP TRIGGER IF EXISTS on_auth_user_created ON auth.users; + +-- Create trigger that fires after a new user is created in auth.users +CREATE TRIGGER on_auth_user_created + AFTER INSERT ON auth.users + FOR EACH ROW EXECUTE FUNCTION public.handle_new_oauth_user(); + +-- ============================================= +-- 3. COMMENT FOR DOCUMENTATION +-- ============================================= + +COMMENT ON FUNCTION public.handle_new_oauth_user() IS +'Automatically creates a user profile in public.user_profiles when a new user is created via OAuth in auth.users. This ensures OAuth users are immediately accessible in the application without manual provisioning.'; From cfa31347c67a0a6673f9730546401f72f02b600a Mon Sep 17 00:00:00 2001 From: Hunter Halloran Date: Fri, 30 Jan 2026 12:02:13 -0500 Subject: [PATCH 4/6] feat: Add flake and ragenix package generation and dev environment --- .envrc | 3 + .gitignore | 4 + FLAKE_SETUP.md | 264 ++++++++++++++++++++++++++++++++++++++++++++ SETUP_COMPLETE.md | 241 ++++++++++++++++++++++++++++++++++++++++ camera-sdk.nix | 44 ++++++++ flake.lock | 239 +++++++++++++++++++++++++++++++++++++++ flake.nix | 176 +++++++++++++++++++++++++++++ package.nix | 131 ++++++++++++++++++++++ secrets.nix | 14 +++ secrets/.gitignore | 11 ++ secrets/README.md | 75 +++++++++++++ secrets/secrets.nix | 14 +++ setup-dev.sh | 77 +++++++++++++ 13 files changed, 1293 insertions(+) create mode 100644 .envrc create mode 100644 FLAKE_SETUP.md create mode 100644 SETUP_COMPLETE.md create mode 100644 camera-sdk.nix create mode 100644 flake.lock create mode 100644 flake.nix create mode 100644 package.nix create mode 100644 secrets.nix create mode 100644 secrets/.gitignore create mode 100644 secrets/README.md create mode 100644 secrets/secrets.nix create mode 100755 setup-dev.sh diff --git a/.envrc b/.envrc new file mode 100644 index 0000000..d8cac13 --- /dev/null +++ b/.envrc @@ -0,0 +1,3 @@ +# Automatically load the Nix development shell when entering this directory +# Requires direnv: https://direnv.net/ +use flake diff --git a/.gitignore b/.gitignore index 541bba4..e630a8a 100644 --- a/.gitignore +++ b/.gitignore @@ -35,3 +35,7 @@ management-dashboard-web-app/users.txt # Jupyter Notebooks *.ipynb +# Nix +result +result-* +.direnv/ \ No newline at end of file diff --git a/FLAKE_SETUP.md b/FLAKE_SETUP.md new file mode 100644 index 0000000..39bc6c8 --- /dev/null +++ b/FLAKE_SETUP.md @@ -0,0 +1,264 @@ +# USDA Vision - Nix Flake Setup + +This directory now has a Nix flake for building and developing the USDA Vision system, with ragenix for managing secrets. + +## Quick Start + +### Development Environment + +Enter the development shell with all tools: + +```bash +cd usda-vision +nix develop +``` + +This gives you: +- Docker & Docker Compose +- Node.js 20 with npm/pnpm +- Python 3.11 with pip/virtualenv +- Supabase CLI +- Camera SDK (automatically in `LD_LIBRARY_PATH`) +- ragenix for secrets management +- All standard utilities (jq, yq, rsync, etc.) + +### Building + +Build the package: + +```bash +nix build +# Or explicitly: +nix build .#usda-vision +``` + +Build just the camera SDK: + +```bash +nix build .#camera-sdk +``` + +## Secrets Management with ragenix + +### Initial Setup + +1. **Generate or use an age key**: + +```bash +# Option 1: Generate a new age key +mkdir -p ~/.config/age +age-keygen -o ~/.config/age/keys.txt + +# Option 2: Use your SSH key +ssh-to-age < ~/.ssh/id_ed25519.pub +# Copy the output to secrets/secrets.nix +``` + +2. **Add your public key** to [secrets/secrets.nix](secrets/secrets.nix): + +```nix +{ + publicKeys = [ + "age1your_public_key_here" + # or + "ssh-ed25519 AAAA... user@host" + ]; +} +``` + +3. **Create encrypted environment files**: + +```bash +nix develop # Enter dev shell first +ragenix -e secrets/env.age +``` + +This opens your `$EDITOR` to edit the encrypted file. Add your environment variables: + +```bash +# Web environment (Vite) +VITE_SUPABASE_URL=http://exp-dash:54321 +VITE_SUPABASE_ANON_KEY=your-anon-key-here +# ... etc +``` + +For Azure OAuth: + +```bash +ragenix -e secrets/env.azure.age +``` + +### Using Secrets in Development + +In the development shell, you can: + +```bash +# Edit secrets +ragenix -e secrets/env.age + +# View decrypted content (careful in shared screens!) +age -d -i ~/.config/age/keys.txt secrets/env.age + +# Re-encrypt all secrets after adding a new public key +ragenix -r +``` + +### Using Secrets in Production (NixOS) + +The flake includes a NixOS module that handles secrets automatically: + +```nix +# In your NixOS configuration +{ + inputs.usda-vision.url = "path:/path/to/usda-vision"; + + # ... in your module: + imports = [ inputs.usda-vision.nixosModules.default ]; + + services.usda-vision = { + enable = true; + secretsFile = config.age.secrets.usda-vision-env.path; + }; + + # Configure ragenix/agenix to decrypt the secrets + age.secrets.usda-vision-env = { + file = inputs.usda-vision + "/secrets/env.age"; + mode = "0644"; + owner = "root"; + }; +} +``` + +## Project Structure + +``` +usda-vision/ +├── flake.nix # Flake definition with outputs +├── package.nix # Main application build +├── camera-sdk.nix # Camera SDK build +├── secrets.nix # ragenix configuration +├── secrets/ +│ ├── secrets.nix # Public keys +│ ├── env.age # Encrypted .env (safe to commit) +│ ├── env.azure.age # Encrypted Azure config (safe to commit) +│ └── README.md # Secrets documentation +└── ... (rest of the app) +``` + +## Migration from Old Setup + +### Old Workflow +- Manual `.env` file management +- Secrets in plaintext (git-ignored) +- Build defined in parent `default.nix` + +### New Workflow +- Encrypted `.age` files in git +- Secrets managed with ragenix +- Self-contained flake in `usda-vision/` +- Development shell with all tools + +### Migration Steps + +1. **Encrypt existing `.env` files**: + +```bash +cd usda-vision +nix develop + +# Setup your age key first (see above) + +# Encrypt the main .env +ragenix -e secrets/env.age +# Paste contents of old .env file, save and exit + +# Encrypt Azure config +ragenix -e secrets/env.azure.age +# Paste contents of old .env.azure file, save and exit +``` + +2. **Delete unencrypted files** (they're git-ignored but still local): + +```bash +rm .env .env.azure management-dashboard-web-app/.env +``` + +3. **Commit encrypted secrets**: + +```bash +git add secrets/env.age secrets/env.azure.age secrets/secrets.nix +git commit -m "Add encrypted secrets with ragenix" +``` + +## Benefits + +### Security +- ✅ Secrets encrypted at rest +- ✅ Safe to commit to git +- ✅ Key-based access control +- ✅ Audit trail (git history) + +### Development +- ✅ Reproducible environment +- ✅ All tools included +- ✅ No manual setup +- ✅ Version-locked dependencies + +### Deployment +- ✅ Declarative secrets management +- ✅ Automatic decryption on NixOS +- ✅ No manual key distribution +- ✅ Clean integration with existing infrastructure + +## Common Tasks + +### Add a new developer + +1. They generate an age key or use their SSH key +2. They send you their public key +3. You add it to `secrets/secrets.nix` +4. Re-encrypt all secrets: `ragenix -r` +5. Commit and push + +### Rotate a secret + +1. Edit the encrypted file: `ragenix -e secrets/env.age` +2. Update the value +3. Save and exit +4. Commit: `git commit secrets/env.age -m "Rotate API key"` + +### Build without flakes + +If you need to build on a system without flakes enabled: + +```bash +nix-build -E '(import (fetchTarball https://github.com/edolstra/flake-compat/archive/master.tar.gz) { src = ./.; }).defaultNix.default' +``` + +## Troubleshooting + +### "error: getting status of '...': No such file or directory" + +Make sure you're in the `usda-vision` directory when running `nix develop` or `nix build`. + +### "cannot decrypt: no valid identity" + +Your age private key isn't found. Check: +- `~/.config/age/keys.txt` exists +- Your public key is in `secrets/secrets.nix` +- You've run `ragenix -r` after adding your key + +### "experimental feature 'flakes' not enabled" + +Add to `~/.config/nix/nix.conf`: +``` +experimental-features = nix-command flakes +``` + +Or run with: `nix --experimental-features 'nix-command flakes' develop` + +## Further Reading + +- [Nix Flakes](https://nixos.wiki/wiki/Flakes) +- [ragenix](https://github.com/yaxitech/ragenix) +- [age encryption](https://github.com/FiloSottile/age) diff --git a/SETUP_COMPLETE.md b/SETUP_COMPLETE.md new file mode 100644 index 0000000..136c900 --- /dev/null +++ b/SETUP_COMPLETE.md @@ -0,0 +1,241 @@ +# USDA Vision - Flake Migration Complete ✅ + +## Summary + +Your USDA Vision repository now has: + +1. **Self-contained Nix flake** (`flake.nix`) + - Independent build system + - Development environment + - NixOS module for deployment + +2. **Encrypted secrets management** (ragenix) + - `.age` files safe to commit to git + - Key-based access control + - No more plaintext `.env` files + +3. **Modular build** (package.nix, camera-sdk.nix) + - Cleaner organization + - Easier to maintain + - Reusable components + +4. **Updated parent** (../default.nix) + - Now references the flake + - Removed 200+ lines of inline derivations + +## Files Added + +### Core Flake Files +- ✅ `flake.nix` - Main flake definition with outputs +- ✅ `package.nix` - Application build logic +- ✅ `camera-sdk.nix` - Camera SDK build logic +- ✅ `secrets.nix` - ragenix configuration + +### Secrets Infrastructure +- ✅ `secrets/secrets.nix` - Public key list +- ✅ `secrets/README.md` - Secrets documentation +- ✅ `secrets/.gitignore` - Protect plaintext files + +### Documentation & Helpers +- ✅ `FLAKE_SETUP.md` - Complete setup guide +- ✅ `setup-dev.sh` - Interactive setup script +- ✅ `.envrc` - direnv integration (optional) + +### Parent Directory +- ✅ `NIX_FLAKE_MIGRATION.md` - Migration summary + +## Next Steps + +### 1. Commit the Flake Files + +The flake needs to be in git to work: + +```bash +cd /home/engr-ugaif/usda-dash-config/usda-vision + +# Add all new flake files +git add flake.nix package.nix camera-sdk.nix secrets.nix +git add secrets/secrets.nix secrets/README.md secrets/.gitignore +git add FLAKE_SETUP.md setup-dev.sh .envrc .gitignore + +# Commit +git commit -m "Add Nix flake with ragenix secrets management + +- Self-contained flake build system +- Development shell with all tools +- ragenix for encrypted secrets +- Modular package definitions +" +``` + +### 2. Set Up Your Age Key + +```bash +cd /home/engr-ugaif/usda-dash-config/usda-vision + +# Option A: Use the interactive setup script +./setup-dev.sh + +# Option B: Manual setup +mkdir -p ~/.config/age +age-keygen -o ~/.config/age/keys.txt +# Then add your public key to secrets/secrets.nix +``` + +### 3. Encrypt Your Secrets + +```bash +# Enter the development environment +nix develop + +# Encrypt main .env file +ragenix -e secrets/env.age +# Paste your current .env contents, save, exit + +# Encrypt Azure config +ragenix -e secrets/env.azure.age +# Paste your current .env.azure contents, save, exit + +# Commit encrypted secrets +git add secrets/env.age secrets/env.azure.age +git commit -m "Add encrypted environment configuration" +``` + +### 4. Test the Setup + +```bash +# Test that the build works +nix build + +# Test the development shell +nix develop +# You should see a welcome message + +# Inside the dev shell, verify tools +docker-compose --version +supabase --version +ragenix --help +``` + +### 5. Update the Parent Repository + +```bash +cd /home/engr-ugaif/usda-dash-config + +# Commit the updated default.nix +git add default.nix NIX_FLAKE_MIGRATION.md +git commit -m "Update default.nix to use usda-vision flake + +- Removed inline derivations +- Now references usda-vision flake packages +- Cleaner, more maintainable code +" +``` + +### 6. Clean Up Old Files (Optional) + +After verifying everything works, you can delete the old plaintext secrets: + +```bash +cd /home/engr-ugaif/usda-dash-config/usda-vision + +# These are already git-ignored, but remove them locally +rm -f .env .env.azure management-dashboard-web-app/.env + +echo "✅ Old plaintext secrets removed" +``` + +## Verification Checklist + +- [ ] Flake files committed to git +- [ ] Age key generated at `~/.config/age/keys.txt` +- [ ] Public key added to `secrets/secrets.nix` +- [ ] Secrets encrypted and committed +- [ ] `nix build` succeeds +- [ ] `nix develop` works +- [ ] Parent `default.nix` updated and committed +- [ ] Old `.env` files deleted + +## Usage Quick Reference + +### Development + +```bash +# Enter dev environment (one-time per session) +cd usda-vision +nix develop + +# Edit secrets +ragenix -e secrets/env.age + +# Normal docker-compose workflow +docker-compose up -d +docker-compose logs -f +``` + +### Building + +```bash +# Build everything +nix build + +# Build specific packages +nix build .#usda-vision +nix build .#camera-sdk +``` + +### Secrets Management + +```bash +# Edit encrypted secret +ragenix -e secrets/env.age + +# Re-key after adding a new public key +ragenix -r + +# View decrypted (careful!) +age -d -i ~/.config/age/keys.txt secrets/env.age +``` + +## Troubleshooting + +### "cannot decrypt: no valid identity" + +Your age key isn't configured. Run: +```bash +./setup-dev.sh +``` + +### "error: flake.nix is not in git" + +Commit the flake files: +```bash +git add flake.nix package.nix camera-sdk.nix secrets.nix +git commit -m "Add flake files" +``` + +### "experimental feature 'flakes' not enabled" + +Add to `~/.config/nix/nix.conf`: +``` +experimental-features = nix-command flakes +``` + +## Documentation + +- **Full Setup Guide**: [FLAKE_SETUP.md](FLAKE_SETUP.md) +- **Secrets Guide**: [secrets/README.md](secrets/README.md) +- **Migration Summary**: [../NIX_FLAKE_MIGRATION.md](../NIX_FLAKE_MIGRATION.md) + +## Questions? + +Refer to [FLAKE_SETUP.md](FLAKE_SETUP.md) for detailed documentation, or run: + +```bash +./setup-dev.sh # Interactive setup +``` + +--- + +**Migration completed on**: 2026-01-30 +**Created by**: GitHub Copilot diff --git a/camera-sdk.nix b/camera-sdk.nix new file mode 100644 index 0000000..0d84c40 --- /dev/null +++ b/camera-sdk.nix @@ -0,0 +1,44 @@ +{ stdenv +, lib +, makeWrapper +, libusb1 +}: + +stdenv.mkDerivation { + pname = "mindvision-camera-sdk"; + version = "2.1.0.49"; + + # Use the camera_sdk directory as source + src = ./camera-management-api/camera_sdk; + + nativeBuildInputs = [ makeWrapper ]; + buildInputs = [ libusb1 ]; + + unpackPhase = '' + cp -r $src/* . + tar xzf "linuxSDK_V2.1.0.49(250108).tar.gz" + cd "linuxSDK_V2.1.0.49(250108)" + ''; + + installPhase = '' + mkdir -p $out/lib $out/include + + # Copy x64 library files (SDK has arch-specific subdirs) + if [ -d lib/x64 ]; then + cp -r lib/x64/* $out/lib/ || true + fi + + # Copy header files + if [ -d include ]; then + cp -r include/* $out/include/ || true + fi + + # Make libraries executable + chmod +x $out/lib/*.so* 2>/dev/null || true + ''; + + meta = with lib; { + description = "MindVision Camera SDK"; + platforms = platforms.linux; + }; +} diff --git a/flake.lock b/flake.lock new file mode 100644 index 0000000..7ed4d7e --- /dev/null +++ b/flake.lock @@ -0,0 +1,239 @@ +{ + "nodes": { + "agenix": { + "inputs": { + "darwin": "darwin", + "home-manager": "home-manager", + "nixpkgs": [ + "ragenix", + "nixpkgs" + ], + "systems": "systems_2" + }, + "locked": { + "lastModified": 1761656077, + "narHash": "sha256-lsNWuj4Z+pE7s0bd2OKicOFq9bK86JE0ZGeKJbNqb94=", + "owner": "ryantm", + "repo": "agenix", + "rev": "9ba0d85de3eaa7afeab493fed622008b6e4924f5", + "type": "github" + }, + "original": { + "owner": "ryantm", + "repo": "agenix", + "type": "github" + } + }, + "crane": { + "locked": { + "lastModified": 1760924934, + "narHash": "sha256-tuuqY5aU7cUkR71sO2TraVKK2boYrdW3gCSXUkF4i44=", + "owner": "ipetkov", + "repo": "crane", + "rev": "c6b4d5308293d0d04fcfeee92705017537cad02f", + "type": "github" + }, + "original": { + "owner": "ipetkov", + "repo": "crane", + "type": "github" + } + }, + "darwin": { + "inputs": { + "nixpkgs": [ + "ragenix", + "agenix", + "nixpkgs" + ] + }, + "locked": { + "lastModified": 1744478979, + "narHash": "sha256-dyN+teG9G82G+m+PX/aSAagkC+vUv0SgUw3XkPhQodQ=", + "owner": "lnl7", + "repo": "nix-darwin", + "rev": "43975d782b418ebf4969e9ccba82466728c2851b", + "type": "github" + }, + "original": { + "owner": "lnl7", + "ref": "master", + "repo": "nix-darwin", + "type": "github" + } + }, + "flake-utils": { + "inputs": { + "systems": "systems" + }, + "locked": { + "lastModified": 1731533236, + "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=", + "owner": "numtide", + "repo": "flake-utils", + "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b", + "type": "github" + }, + "original": { + "owner": "numtide", + "repo": "flake-utils", + "type": "github" + } + }, + "flake-utils_2": { + "inputs": { + "systems": "systems_3" + }, + "locked": { + "lastModified": 1731533236, + "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=", + "owner": "numtide", + "repo": "flake-utils", + "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b", + "type": "github" + }, + "original": { + "owner": "numtide", + "repo": "flake-utils", + "type": "github" + } + }, + "home-manager": { + "inputs": { + "nixpkgs": [ + "ragenix", + "agenix", + "nixpkgs" + ] + }, + "locked": { + "lastModified": 1745494811, + "narHash": "sha256-YZCh2o9Ua1n9uCvrvi5pRxtuVNml8X2a03qIFfRKpFs=", + "owner": "nix-community", + "repo": "home-manager", + "rev": "abfad3d2958c9e6300a883bd443512c55dfeb1be", + "type": "github" + }, + "original": { + "owner": "nix-community", + "repo": "home-manager", + "type": "github" + } + }, + "nixpkgs": { + "locked": { + "lastModified": 1769461804, + "narHash": "sha256-msG8SU5WsBUfVVa/9RPLaymvi5bI8edTavbIq3vRlhI=", + "owner": "NixOS", + "repo": "nixpkgs", + "rev": "bfc1b8a4574108ceef22f02bafcf6611380c100d", + "type": "github" + }, + "original": { + "owner": "NixOS", + "ref": "nixos-unstable", + "repo": "nixpkgs", + "type": "github" + } + }, + "ragenix": { + "inputs": { + "agenix": "agenix", + "crane": "crane", + "flake-utils": "flake-utils_2", + "nixpkgs": [ + "nixpkgs" + ], + "rust-overlay": "rust-overlay" + }, + "locked": { + "lastModified": 1761832913, + "narHash": "sha256-VCNVjjuRvrKPiYYwqhE3BAKIaReiKXGpxGp27lZ0MFM=", + "owner": "yaxitech", + "repo": "ragenix", + "rev": "83bccfdea758241999f32869fb6b36f7ac72f1ac", + "type": "github" + }, + "original": { + "owner": "yaxitech", + "repo": "ragenix", + "type": "github" + } + }, + "root": { + "inputs": { + "flake-utils": "flake-utils", + "nixpkgs": "nixpkgs", + "ragenix": "ragenix" + } + }, + "rust-overlay": { + "inputs": { + "nixpkgs": [ + "ragenix", + "nixpkgs" + ] + }, + "locked": { + "lastModified": 1761791894, + "narHash": "sha256-myRIDh+PxaREz+z9LzbqBJF+SnTFJwkthKDX9zMyddY=", + "owner": "oxalica", + "repo": "rust-overlay", + "rev": "59c45eb69d9222a4362673141e00ff77842cd219", + "type": "github" + }, + "original": { + "owner": "oxalica", + "repo": "rust-overlay", + "type": "github" + } + }, + "systems": { + "locked": { + "lastModified": 1681028828, + "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", + "owner": "nix-systems", + "repo": "default", + "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", + "type": "github" + }, + "original": { + "owner": "nix-systems", + "repo": "default", + "type": "github" + } + }, + "systems_2": { + "locked": { + "lastModified": 1681028828, + "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", + "owner": "nix-systems", + "repo": "default", + "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", + "type": "github" + }, + "original": { + "owner": "nix-systems", + "repo": "default", + "type": "github" + } + }, + "systems_3": { + "locked": { + "lastModified": 1681028828, + "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", + "owner": "nix-systems", + "repo": "default", + "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", + "type": "github" + }, + "original": { + "owner": "nix-systems", + "repo": "default", + "type": "github" + } + } + }, + "root": "root", + "version": 7 +} diff --git a/flake.nix b/flake.nix new file mode 100644 index 0000000..d6dd1fc --- /dev/null +++ b/flake.nix @@ -0,0 +1,176 @@ +{ + description = "USDA Vision camera management system"; + + inputs = { + nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; + flake-utils.url = "github:numtide/flake-utils"; + + # For secrets management + ragenix = { + url = "github:yaxitech/ragenix"; + inputs.nixpkgs.follows = "nixpkgs"; + }; + }; + + outputs = { self, nixpkgs, flake-utils, ragenix }: + flake-utils.lib.eachDefaultSystem (system: + let + pkgs = import nixpkgs { + inherit system; + config.allowUnfree = true; + }; + + # Import our package definition + usda-vision-package = pkgs.callPackage ./package.nix { }; + camera-sdk = pkgs.callPackage ./camera-sdk.nix { }; + + in + { + packages = { + default = usda-vision-package; + usda-vision = usda-vision-package; + camera-sdk = camera-sdk; + }; + + devShells.default = pkgs.mkShell { + name = "usda-vision-dev"; + + # Input packages for the development shell + buildInputs = with pkgs; [ + # Core development tools + git + vim + curl + wget + + # Docker for local development + docker + docker-compose + + # Supabase CLI + supabase-cli + + # Node.js for web app development + nodejs_20 + nodePackages.npm + nodePackages.pnpm + + # Python for camera API + python311 + python311Packages.pip + python311Packages.virtualenv + + # Camera SDK + camera-sdk + + # Secrets management + ragenix.packages.${system}.default + age + ssh-to-age + + # Utilities + jq + yq + rsync + gnused + gawk + ]; + + # Environment variables for development + shellHook = '' + export LD_LIBRARY_PATH="${camera-sdk}/lib:$LD_LIBRARY_PATH" + export CAMERA_SDK_PATH="${camera-sdk}" + + # Set up Python virtual environment + if [ ! -d .venv ]; then + echo "Creating Python virtual environment..." + python -m venv .venv + fi + + echo "USDA Vision Development Environment" + echo "====================================" + echo "Camera SDK: ${camera-sdk}" + echo "" + echo "Available commands:" + echo " - docker-compose: Manage containers" + echo " - supabase: Supabase CLI" + echo " - ragenix: Manage encrypted secrets" + echo " - age: Encrypt/decrypt files" + echo "" + echo "To activate Python venv: source .venv/bin/activate" + echo "To edit secrets: ragenix -e secrets/env.age" + echo "" + ''; + + # Additional environment configuration + DOCKER_BUILDKIT = "1"; + COMPOSE_DOCKER_CLI_BUILD = "1"; + }; + + # NixOS module for easy integration + nixosModules.default = { config, lib, ... }: { + options.services.usda-vision = { + enable = lib.mkEnableOption "USDA Vision camera management system"; + + secretsFile = lib.mkOption { + type = lib.types.path; + description = "Path to the ragenix-managed secrets file"; + }; + + dataDir = lib.mkOption { + type = lib.types.str; + default = "/var/lib/usda-vision"; + description = "Directory for USDA Vision application data"; + }; + }; + + config = lib.mkIf config.services.usda-vision.enable { + environment.systemPackages = [ + usda-vision-package + camera-sdk + pkgs.docker-compose + ]; + + environment.variables.LD_LIBRARY_PATH = "${camera-sdk}/lib"; + + virtualisation.docker = { + enable = true; + autoPrune.enable = true; + }; + + systemd.services.usda-vision = { + description = "USDA Vision Docker Compose Stack"; + after = [ "docker.service" "network-online.target" ]; + wants = [ "network-online.target" ]; + wantedBy = [ "multi-user.target" ]; + + preStart = '' + # Sync application code + ${pkgs.rsync}/bin/rsync -av --delete \ + --checksum \ + --exclude='node_modules' \ + --exclude='.env' \ + --exclude='__pycache__' \ + --exclude='.venv' \ + ${usda-vision-package}/opt/usda-vision/ ${config.services.usda-vision.dataDir}/ + + # Copy secrets if managed by ragenix + if [ -f "${config.services.usda-vision.secretsFile}" ]; then + cp "${config.services.usda-vision.secretsFile}" ${config.services.usda-vision.dataDir}/.env + fi + ''; + + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + WorkingDirectory = config.services.usda-vision.dataDir; + ExecStart = "${pkgs.docker-compose}/bin/docker-compose up -d --build"; + ExecStop = "${pkgs.docker-compose}/bin/docker-compose down"; + TimeoutStartSec = 300; + }; + }; + }; + }; + } + ); +} diff --git a/package.nix b/package.nix new file mode 100644 index 0000000..b0e9730 --- /dev/null +++ b/package.nix @@ -0,0 +1,131 @@ +{ lib +, stdenv +, makeWrapper +, rsync +, gnused +, gawk +, docker-compose +}: + +stdenv.mkDerivation { + pname = "usda-vision"; + version = "1.0.0"; + + # Use the directory from this repository with explicit source filtering + src = lib.cleanSourceWith { + src = ./.; + filter = path: type: + let + baseName = baseNameOf path; + in + # Exclude git, but include everything else + baseName != ".git" && + baseName != ".cursor" && + baseName != "__pycache__" && + baseName != "node_modules" && + baseName != ".venv" && + baseName != ".age" && + baseName != "flake.nix" && + baseName != "flake.lock" && + baseName != "package.nix" && + baseName != "camera-sdk.nix"; + }; + + nativeBuildInputs = [ makeWrapper rsync ]; + + # Don't run these phases, we'll do everything in installPhase + dontBuild = true; + dontConfigure = true; + + installPhase = '' + mkdir -p $out/opt/usda-vision + + # Debug: show what's in source + echo "Source directory contents:" + ls -la $src/ || true + + # Process docker-compose.yml - replace paths and configure SDK from Nix + if [ -f $src/docker-compose.yml ]; then + # Basic path replacements with sed + ${gnused}/bin/sed \ + -e 's|env_file:.*management-dashboard-web-app/\.env|env_file: /var/lib/usda-vision/.env|g' \ + -e 's|\./management-dashboard-web-app/\.env|/var/lib/usda-vision/.env|g' \ + -e 's|\./management-dashboard-web-app|/var/lib/usda-vision/management-dashboard-web-app|g' \ + -e 's|\./media-api|/var/lib/usda-vision/media-api|g' \ + -e 's|\./video-remote|/var/lib/usda-vision/video-remote|g' \ + -e 's|\./scheduling-remote|/var/lib/usda-vision/scheduling-remote|g' \ + -e 's|\./vision-system-remote|/var/lib/usda-vision/vision-system-remote|g' \ + -e 's|\./camera-management-api|/var/lib/usda-vision/camera-management-api|g' \ + $src/docker-compose.yml > $TMPDIR/docker-compose-step1.yml + + # Remove SDK installation blocks using awk for better multi-line handling + ${gawk}/bin/awk ' + /# Only install system packages if not already installed/ { skip=1 } + skip && /^ fi$/ { skip=0; next } + /# Install camera SDK if not already installed/ { skip_sdk=1 } + skip_sdk && /^ fi;$/ { skip_sdk=0; next } + !skip && !skip_sdk { print } + ' $TMPDIR/docker-compose-step1.yml > $TMPDIR/docker-compose.yml + + rm -f $TMPDIR/docker-compose-step1.yml + fi + + # Copy all application files using rsync with chmod + ${rsync}/bin/rsync -av \ + --chmod=Du+w \ + --exclude='.git' \ + --exclude='docker-compose.yml' \ + --exclude='.env' \ + --exclude='*.age' \ + --exclude='flake.nix' \ + --exclude='flake.lock' \ + --exclude='package.nix' \ + --exclude='camera-sdk.nix' \ + --exclude='management-dashboard-web-app/.env' \ + $src/ $out/opt/usda-vision/ + + # Copy the processed docker-compose.yml + if [ -f $TMPDIR/docker-compose.yml ]; then + cp $TMPDIR/docker-compose.yml $out/opt/usda-vision/docker-compose.yml + fi + + # Verify files were copied + echo "Destination directory contents:" + ls -la $out/opt/usda-vision/ || true + + # Create convenience scripts + mkdir -p $out/bin + + cat > $out/bin/usda-vision-start <<'EOF' +#!/usr/bin/env bash +cd $out/opt/usda-vision +${docker-compose}/bin/docker-compose up -d --build +EOF + + cat > $out/bin/usda-vision-stop <<'EOF' +#!/usr/bin/env bash +cd $out/opt/usda-vision +${docker-compose}/bin/docker-compose down +EOF + + cat > $out/bin/usda-vision-logs <<'EOF' +#!/usr/bin/env bash +cd $out/opt/usda-vision +${docker-compose}/bin/docker-compose logs -f "$@" +EOF + + cat > $out/bin/usda-vision-restart <<'EOF' +#!/usr/bin/env bash +cd $out/opt/usda-vision +${docker-compose}/bin/docker-compose restart "$@" +EOF + + chmod +x $out/bin/usda-vision-* + ''; + + meta = with lib; { + description = "USDA Vision camera management system"; + maintainers = [ "UGA Innovation Factory" ]; + platforms = platforms.linux; + }; +} diff --git a/secrets.nix b/secrets.nix new file mode 100644 index 0000000..efd9940 --- /dev/null +++ b/secrets.nix @@ -0,0 +1,14 @@ +# Ragenix Configuration +# This file defines which secrets to manage and their permissions + +let + # Import public keys from secrets.nix + keys = import ./secrets.nix; +in +{ + # Main environment file + "env.age".publicKeys = keys.publicKeys; + + # Azure OAuth configuration + "env.azure.age".publicKeys = keys.publicKeys; +} diff --git a/secrets/.gitignore b/secrets/.gitignore new file mode 100644 index 0000000..fae6303 --- /dev/null +++ b/secrets/.gitignore @@ -0,0 +1,11 @@ +# Ignore unencrypted secrets +*.env +!*.env.example +.env.* +!.env.*.example + +# Ignore age private keys (if accidentally placed here) +*.txt + +# Keep encrypted files +!*.age diff --git a/secrets/README.md b/secrets/README.md new file mode 100644 index 0000000..4608822 --- /dev/null +++ b/secrets/README.md @@ -0,0 +1,75 @@ +# USDA Vision Secrets Management + +This directory contains encrypted secrets managed by [ragenix](https://github.com/yaxitech/ragenix). + +## Setup + +1. **Generate an age key** (if you don't have one): + ```bash + # Generate a new age key + age-keygen -o ~/.config/age/keys.txt + + # Or convert your SSH key + ssh-to-age < ~/.ssh/id_ed25519.pub + ``` + +2. **Add your public key to `secrets.nix`**: + ```nix + { + publicKeys = [ + "age1..." # Your age public key + "ssh-ed25519 ..." # Or your SSH public key + ]; + } + ``` + +3. **Create and encrypt environment files**: + ```bash + # Create the encrypted .env file + ragenix -e secrets/env.age + + # Create the encrypted .env.azure file + ragenix -e secrets/env.azure.age + ``` + +## Usage in Development + +In the development shell: +```bash +# Edit encrypted secrets +ragenix -e secrets/env.age + +# Re-key secrets after adding a new public key +ragenix -r +``` + +## Usage in NixOS + +The flake's NixOS module automatically handles decryption: + +```nix +{ + services.usda-vision = { + enable = true; + secretsFile = config.age.secrets.usda-vision-env.path; + }; + + age.secrets.usda-vision-env = { + file = ./usda-vision/secrets/env.age; + mode = "0644"; + }; +} +``` + +## Files + +- `secrets.nix` - Public keys configuration +- `env.age` - Encrypted main .env file +- `env.azure.age` - Encrypted Azure OAuth configuration +- `README.md` - This file + +## Security Notes + +- Never commit unencrypted `.env` files +- Keep your age private key secure (`~/.config/age/keys.txt`) +- The `.age` encrypted files are safe to commit to git diff --git a/secrets/secrets.nix b/secrets/secrets.nix new file mode 100644 index 0000000..a0421d2 --- /dev/null +++ b/secrets/secrets.nix @@ -0,0 +1,14 @@ +# Public keys for secret encryption +# Add your age or SSH public keys here +{ + publicKeys = [ + # Example age public key: + # "age1qyqszqgpqyqszqgpqyqszqgpqyqszqgpqyqszqgpqyqszqgpqyqs3ekg8p" + + # Example SSH public key (ed25519): + # "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... user@host" + + # Add your keys below: + # TODO: Add your age or SSH public keys + ]; +} diff --git a/setup-dev.sh b/setup-dev.sh new file mode 100755 index 0000000..13f0350 --- /dev/null +++ b/setup-dev.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# Quick setup script for USDA Vision development + +set -e + +echo "======================================" +echo "USDA Vision - Quick Setup" +echo "======================================" +echo "" + +# Check if we're in the right directory +if [ ! -f "flake.nix" ]; then + echo "❌ Error: Must run from usda-vision directory" + echo " cd to the directory containing flake.nix" + exit 1 +fi + +# Check for age key +if [ ! -f "$HOME/.config/age/keys.txt" ]; then + echo "📝 No age key found at ~/.config/age/keys.txt" + echo "" + read -p "Would you like to generate one? (y/n) " -n 1 -r + echo "" + if [[ $REPLY =~ ^[Yy]$ ]]; then + mkdir -p "$HOME/.config/age" + age-keygen -o "$HOME/.config/age/keys.txt" + echo "✅ Age key generated!" + echo "" + else + echo "❌ Cannot proceed without an age key" + exit 1 + fi +fi + +# Get public key +AGE_PUBLIC_KEY=$(grep "public key:" "$HOME/.config/age/keys.txt" | cut -d: -f2 | xargs) + +echo "Your age public key is:" +echo " $AGE_PUBLIC_KEY" +echo "" + +# Check if key is already in secrets.nix +if grep -q "$AGE_PUBLIC_KEY" secrets/secrets.nix 2>/dev/null; then + echo "✅ Your key is already in secrets/secrets.nix" +else + echo "⚠️ Your key is NOT in secrets/secrets.nix" + echo "" + read -p "Would you like to add it now? (y/n) " -n 1 -r + echo "" + if [[ $REPLY =~ ^[Yy]$ ]]; then + # Backup original + cp secrets/secrets.nix secrets/secrets.nix.backup + + # Add the key + sed -i "/publicKeys = \[/a\ \"$AGE_PUBLIC_KEY\"" secrets/secrets.nix + + echo "✅ Key added to secrets/secrets.nix" + echo "" + fi +fi + +echo "======================================" +echo "Setup complete! Next steps:" +echo "======================================" +echo "" +echo "1. Enter development environment:" +echo " $ nix develop" +echo "" +echo "2. Create/edit encrypted secrets:" +echo " $ ragenix -e secrets/env.age" +echo " $ ragenix -e secrets/env.azure.age" +echo "" +echo "3. Start development:" +echo " $ docker-compose up -d" +echo "" +echo "For more information, see FLAKE_SETUP.md" +echo "" From 065e5f368f439dfb8321cd8fcb640d3d7437a435 Mon Sep 17 00:00:00 2001 From: Hunter Halloran Date: Fri, 30 Jan 2026 12:48:48 -0500 Subject: [PATCH 5/6] fix: Move ragenix to externally managed, and ask for env file references --- .gitignore | 4 +- flake.lock | 180 +--------------------------------------------------- flake.nix | 82 +----------------------- package.nix | 2 - 4 files changed, 6 insertions(+), 262 deletions(-) diff --git a/.gitignore b/.gitignore index e630a8a..293689c 100644 --- a/.gitignore +++ b/.gitignore @@ -4,7 +4,7 @@ __pycache__/ *.egg-info/ .venv/ .uv/ -.env +*.env .env.*.local .pytest_cache/ .mypy_cache/ @@ -38,4 +38,4 @@ management-dashboard-web-app/users.txt # Nix result result-* -.direnv/ \ No newline at end of file +.direnv/ diff --git a/flake.lock b/flake.lock index 7ed4d7e..6bae7c7 100644 --- a/flake.lock +++ b/flake.lock @@ -1,67 +1,5 @@ { "nodes": { - "agenix": { - "inputs": { - "darwin": "darwin", - "home-manager": "home-manager", - "nixpkgs": [ - "ragenix", - "nixpkgs" - ], - "systems": "systems_2" - }, - "locked": { - "lastModified": 1761656077, - "narHash": "sha256-lsNWuj4Z+pE7s0bd2OKicOFq9bK86JE0ZGeKJbNqb94=", - "owner": "ryantm", - "repo": "agenix", - "rev": "9ba0d85de3eaa7afeab493fed622008b6e4924f5", - "type": "github" - }, - "original": { - "owner": "ryantm", - "repo": "agenix", - "type": "github" - } - }, - "crane": { - "locked": { - "lastModified": 1760924934, - "narHash": "sha256-tuuqY5aU7cUkR71sO2TraVKK2boYrdW3gCSXUkF4i44=", - "owner": "ipetkov", - "repo": "crane", - "rev": "c6b4d5308293d0d04fcfeee92705017537cad02f", - "type": "github" - }, - "original": { - "owner": "ipetkov", - "repo": "crane", - "type": "github" - } - }, - "darwin": { - "inputs": { - "nixpkgs": [ - "ragenix", - "agenix", - "nixpkgs" - ] - }, - "locked": { - "lastModified": 1744478979, - "narHash": "sha256-dyN+teG9G82G+m+PX/aSAagkC+vUv0SgUw3XkPhQodQ=", - "owner": "lnl7", - "repo": "nix-darwin", - "rev": "43975d782b418ebf4969e9ccba82466728c2851b", - "type": "github" - }, - "original": { - "owner": "lnl7", - "ref": "master", - "repo": "nix-darwin", - "type": "github" - } - }, "flake-utils": { "inputs": { "systems": "systems" @@ -80,46 +18,6 @@ "type": "github" } }, - "flake-utils_2": { - "inputs": { - "systems": "systems_3" - }, - "locked": { - "lastModified": 1731533236, - "narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=", - "owner": "numtide", - "repo": "flake-utils", - "rev": "11707dc2f618dd54ca8739b309ec4fc024de578b", - "type": "github" - }, - "original": { - "owner": "numtide", - "repo": "flake-utils", - "type": "github" - } - }, - "home-manager": { - "inputs": { - "nixpkgs": [ - "ragenix", - "agenix", - "nixpkgs" - ] - }, - "locked": { - "lastModified": 1745494811, - "narHash": "sha256-YZCh2o9Ua1n9uCvrvi5pRxtuVNml8X2a03qIFfRKpFs=", - "owner": "nix-community", - "repo": "home-manager", - "rev": "abfad3d2958c9e6300a883bd443512c55dfeb1be", - "type": "github" - }, - "original": { - "owner": "nix-community", - "repo": "home-manager", - "type": "github" - } - }, "nixpkgs": { "locked": { "lastModified": 1769461804, @@ -136,56 +34,10 @@ "type": "github" } }, - "ragenix": { - "inputs": { - "agenix": "agenix", - "crane": "crane", - "flake-utils": "flake-utils_2", - "nixpkgs": [ - "nixpkgs" - ], - "rust-overlay": "rust-overlay" - }, - "locked": { - "lastModified": 1761832913, - "narHash": "sha256-VCNVjjuRvrKPiYYwqhE3BAKIaReiKXGpxGp27lZ0MFM=", - "owner": "yaxitech", - "repo": "ragenix", - "rev": "83bccfdea758241999f32869fb6b36f7ac72f1ac", - "type": "github" - }, - "original": { - "owner": "yaxitech", - "repo": "ragenix", - "type": "github" - } - }, "root": { "inputs": { "flake-utils": "flake-utils", - "nixpkgs": "nixpkgs", - "ragenix": "ragenix" - } - }, - "rust-overlay": { - "inputs": { - "nixpkgs": [ - "ragenix", - "nixpkgs" - ] - }, - "locked": { - "lastModified": 1761791894, - "narHash": "sha256-myRIDh+PxaREz+z9LzbqBJF+SnTFJwkthKDX9zMyddY=", - "owner": "oxalica", - "repo": "rust-overlay", - "rev": "59c45eb69d9222a4362673141e00ff77842cd219", - "type": "github" - }, - "original": { - "owner": "oxalica", - "repo": "rust-overlay", - "type": "github" + "nixpkgs": "nixpkgs" } }, "systems": { @@ -202,36 +54,6 @@ "repo": "default", "type": "github" } - }, - "systems_2": { - "locked": { - "lastModified": 1681028828, - "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", - "owner": "nix-systems", - "repo": "default", - "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", - "type": "github" - }, - "original": { - "owner": "nix-systems", - "repo": "default", - "type": "github" - } - }, - "systems_3": { - "locked": { - "lastModified": 1681028828, - "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=", - "owner": "nix-systems", - "repo": "default", - "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e", - "type": "github" - }, - "original": { - "owner": "nix-systems", - "repo": "default", - "type": "github" - } } }, "root": "root", diff --git a/flake.nix b/flake.nix index d6dd1fc..9711703 100644 --- a/flake.nix +++ b/flake.nix @@ -4,15 +4,9 @@ inputs = { nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; flake-utils.url = "github:numtide/flake-utils"; - - # For secrets management - ragenix = { - url = "github:yaxitech/ragenix"; - inputs.nixpkgs.follows = "nixpkgs"; - }; }; - outputs = { self, nixpkgs, flake-utils, ragenix }: + outputs = { self, nixpkgs, flake-utils }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { @@ -63,11 +57,6 @@ # Camera SDK camera-sdk - # Secrets management - ragenix.packages.${system}.default - age - ssh-to-age - # Utilities jq yq @@ -94,83 +83,18 @@ echo "Available commands:" echo " - docker-compose: Manage containers" echo " - supabase: Supabase CLI" - echo " - ragenix: Manage encrypted secrets" - echo " - age: Encrypt/decrypt files" echo "" echo "To activate Python venv: source .venv/bin/activate" echo "To edit secrets: ragenix -e secrets/env.age" echo "" + echo "NOTE: Secrets should be managed by ragenix in athenix for production deployments" + echo "" ''; # Additional environment configuration DOCKER_BUILDKIT = "1"; COMPOSE_DOCKER_CLI_BUILD = "1"; }; - - # NixOS module for easy integration - nixosModules.default = { config, lib, ... }: { - options.services.usda-vision = { - enable = lib.mkEnableOption "USDA Vision camera management system"; - - secretsFile = lib.mkOption { - type = lib.types.path; - description = "Path to the ragenix-managed secrets file"; - }; - - dataDir = lib.mkOption { - type = lib.types.str; - default = "/var/lib/usda-vision"; - description = "Directory for USDA Vision application data"; - }; - }; - - config = lib.mkIf config.services.usda-vision.enable { - environment.systemPackages = [ - usda-vision-package - camera-sdk - pkgs.docker-compose - ]; - - environment.variables.LD_LIBRARY_PATH = "${camera-sdk}/lib"; - - virtualisation.docker = { - enable = true; - autoPrune.enable = true; - }; - - systemd.services.usda-vision = { - description = "USDA Vision Docker Compose Stack"; - after = [ "docker.service" "network-online.target" ]; - wants = [ "network-online.target" ]; - wantedBy = [ "multi-user.target" ]; - - preStart = '' - # Sync application code - ${pkgs.rsync}/bin/rsync -av --delete \ - --checksum \ - --exclude='node_modules' \ - --exclude='.env' \ - --exclude='__pycache__' \ - --exclude='.venv' \ - ${usda-vision-package}/opt/usda-vision/ ${config.services.usda-vision.dataDir}/ - - # Copy secrets if managed by ragenix - if [ -f "${config.services.usda-vision.secretsFile}" ]; then - cp "${config.services.usda-vision.secretsFile}" ${config.services.usda-vision.dataDir}/.env - fi - ''; - - serviceConfig = { - Type = "oneshot"; - RemainAfterExit = true; - WorkingDirectory = config.services.usda-vision.dataDir; - ExecStart = "${pkgs.docker-compose}/bin/docker-compose up -d --build"; - ExecStop = "${pkgs.docker-compose}/bin/docker-compose down"; - TimeoutStartSec = 300; - }; - }; - }; - }; } ); } diff --git a/package.nix b/package.nix index b0e9730..d83cdf7 100644 --- a/package.nix +++ b/package.nix @@ -48,8 +48,6 @@ stdenv.mkDerivation { if [ -f $src/docker-compose.yml ]; then # Basic path replacements with sed ${gnused}/bin/sed \ - -e 's|env_file:.*management-dashboard-web-app/\.env|env_file: /var/lib/usda-vision/.env|g' \ - -e 's|\./management-dashboard-web-app/\.env|/var/lib/usda-vision/.env|g' \ -e 's|\./management-dashboard-web-app|/var/lib/usda-vision/management-dashboard-web-app|g' \ -e 's|\./media-api|/var/lib/usda-vision/media-api|g' \ -e 's|\./video-remote|/var/lib/usda-vision/video-remote|g' \ From c8cd95361ab196f81403dba1af5ab1f3ed374921 Mon Sep 17 00:00:00 2001 From: salirezav Date: Mon, 2 Feb 2026 12:34:59 -0500 Subject: [PATCH 6/6] Update README.md to reflect new services and development setup - Added descriptions for new services: media-api, video-remote, vision-system-remote, and scheduling-remote. - Updated development instructions to use docker-compose for starting the stack. - Changed web port from 5173 to 8080 and clarified development commands. - Removed MQTT broker details and updated service sections accordingly. --- README.md | 74 +++++++++++++++++++++++++++++++++---------------------- 1 file changed, 44 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 8ae816f..a7a2b06 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,13 @@ A unified monorepo combining the camera API service and the web dashboard for US - `camera-management-api/` - Python API service for camera management (USDA-Vision-Cameras) - `management-dashboard-web-app/` - React web dashboard for experiment management (pecan_experiments) - `supabase/` - Database configuration, migrations, and seed data (shared infrastructure) +- `media-api/` - Python service for video/thumbnail serving (port 8090) +- `video-remote/` - Frontend for video browsing (port 3001) +- `vision-system-remote/` - Camera/vision UI (port 3002) +- `scheduling-remote/` - Scheduling/availability UI (port 3003) +- `scripts/` - Host IP, RTSP checks, env helpers (see [scripts/README.md](scripts/README.md)) +- `docs/` - Setup, Supabase, RTSP, design docs +- `mediamtx.yml` - RTSP/WebRTC config for MediaMTX streaming ## Quick Start @@ -28,15 +35,15 @@ The wrapper script automatically: For more details, see [Docker Compose Environment Setup](docs/DOCKER_COMPOSE_ENV_SETUP.md). -- Web: +- Web: - API: -- MQTT broker: localhost:1883 +- MQTT is optional; configure in API config if used (see `.env.example`). To stop: `docker compose down` ### Development Mode (Recommended for Development) -For development with live logging, debugging, and hot reloading: +For development, use the same Docker Compose stack as production. The web app runs with the Vite dev server on port 8080 (hot reload); the API runs on port 8000. 1) Copy env template and set values (for web/Supabase): @@ -45,36 +52,25 @@ cp .env.example .env # set VITE_SUPABASE_URL and VITE_SUPABASE_ANON_KEY in .env ``` -2) Start the development environment: +2) Start the stack (with logs in the foreground, or add `-d` for detached): ```bash -./dev-start.sh +./docker-compose.sh up --build ``` -This will: -- Start containers with debug logging enabled -- Enable hot reloading for both API and web services -- Show all logs in real-time -- Keep containers running for debugging - **Development URLs:** -- Web: (with hot reloading) -- API: (with debug logging) +- Web: (Vite dev server with hot reload) +- API: **Development Commands:** -- `./dev-start.sh` - Start development environment -- `./dev-stop.sh` - Stop development environment -- `./dev-logs.sh` - View logs (use `-f` to follow, `-t N` for last N lines) -- `./dev-logs.sh -f api` - Follow API logs only -- `./dev-logs.sh -f web` - Follow web logs only -- `./dev-shell.sh` - Open shell in API container -- `./dev-shell.sh web` - Open shell in web container - -**Debug Features:** -- API runs with `--debug --verbose` flags for maximum logging -- Web runs with Vite dev server for hot reloading -- All containers have `stdin_open: true` and `tty: true` for debugging -- Environment variables set for development mode +- `./docker-compose.sh up --build` - Start stack (omit `-d` to see logs) +- `./docker-compose.sh up --build -d` - Start stack in background +- `docker compose down` - Stop all services +- `docker compose logs -f` - Follow all logs +- `docker compose logs -f api` - Follow API logs only +- `docker compose logs -f web` - Follow web logs only +- `docker compose exec api sh` - Open shell in API container +- `docker compose exec web sh` - Open shell in web container ## Services @@ -84,16 +80,34 @@ This will: - Video recording controls - File management -### Web Dashboard (Port 5173) +### Web Dashboard (Port 8080) - User authentication via Supabase - Experiment definition and management - Camera control interface - Video playback and analysis -### MQTT Broker (Port 1883) +### Media API (Port 8090) -- Local Mosquitto broker for development and integration testing +- Video listing, thumbnails, transcoding + +### Video Remote (Port 3001) + +- Video browser UI + +### Vision System Remote (Port 3002) + +- Camera/vision control UI + +### Scheduling Remote (Port 3003) + +- Scheduling/availability UI + +### MediaMTX (Ports 8554, 8889, 8189) + +- RTSP and WebRTC streaming (config: [mediamtx.yml](mediamtx.yml)) + +Supabase services are currently commented out in `docker-compose.yml` and can be run via Supabase CLI (e.g. from `management-dashboard-web-app`). See [docs](docs/) for setup. ## Git Subtree Workflow @@ -138,7 +152,7 @@ Notes: - Storage (recordings) is mapped to `camera-management-api/storage/` and ignored by git. - Web - - Code lives under `management-dashboard-web-app/` with a Vite dev server on port 5173. + - Code lives under `management-dashboard-web-app/` with a Vite dev server on port 8080 when run via Docker. - Environment: set `VITE_SUPABASE_URL` and `VITE_SUPABASE_ANON_KEY` in `.env` (not committed). - Common scripts: `npm run dev`, `npm run build` (executed inside the container by compose).