From 81828f61cf893039b89d3cf1861555f31167c37d Mon Sep 17 00:00:00 2001 From: Alireza Vaezi Date: Wed, 6 Aug 2025 11:46:25 -0400 Subject: [PATCH] feat(video-streaming): add ApiStatusIndicator, PerformanceDashboard, VideoDebugger, and VideoErrorBoundary components - Implemented ApiStatusIndicator to monitor video API connection status with health check functionality. - Created PerformanceDashboard for monitoring video streaming performance metrics in development mode. - Developed VideoDebugger for diagnosing video streaming issues with direct access to test video URLs. - Added VideoErrorBoundary to handle errors in video streaming components with user-friendly messages and recovery options. - Introduced utility functions for performance monitoring and thumbnail caching to optimize video streaming operations. - Added comprehensive tests for video streaming API connectivity and functionality. --- .env.example | 14 + .../docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md | 415 +++++++++++++++++ .../docs/API_CHANGES_SUMMARY.md | 49 +-- API Documentations/docs/API_DOCUMENTATION.md | 310 ++++++++----- .../docs/API_QUICK_REFERENCE.md | 48 +- API Documentations/docs/MP4_FORMAT_UPDATE.md | 39 +- API Documentations/docs/PROJECT_COMPLETE.md | 14 +- API Documentations/docs/README.md | 14 + API Documentations/docs/VIDEO_STREAMING.md | 416 ++++++++++++++++-- .../docs/WEB_AI_AGENT_VIDEO_INTEGRATION.md | 302 +++++++++++++ .../docs/api/CAMERA_CONFIG_API.md | 10 +- .../docs/guides/CAMERA_RECOVERY_GUIDE.md | 10 +- .../docs/guides/MQTT_LOGGING_GUIDE.md | 12 +- .../docs/guides/STREAMING_GUIDE.md | 14 +- .../docs/legacy/IMPLEMENTATION_SUMMARY.md | 16 +- .../docs/legacy/README_SYSTEM.md | 8 +- .../docs/legacy/TIMEZONE_SETUP_SUMMARY.md | 2 +- docs/VIDEO_STREAMING_INTEGRATION_COMPLETE.md | 175 ++++++++ .../video-streaming/VideoStreamingPage.tsx | 224 +++++----- .../components/ApiStatusIndicator.tsx | 133 ++++++ .../video-streaming/components/Pagination.tsx | 6 +- .../components/PerformanceDashboard.tsx | 167 +++++++ .../video-streaming/components/VideoCard.tsx | 11 +- .../components/VideoDebugger.tsx | 196 +++++++++ .../components/VideoErrorBoundary.tsx | 146 ++++++ .../video-streaming/components/VideoList.tsx | 44 +- .../video-streaming/components/VideoModal.tsx | 10 +- .../components/VideoPlayer.tsx | 21 +- .../components/VideoThumbnail.tsx | 26 +- .../video-streaming/components/index.ts | 4 + .../video-streaming/hooks/useVideoList.ts | 8 +- .../video-streaming/hooks/useVideoPlayer.ts | 25 ++ .../video-streaming/services/videoApi.ts | 78 ++-- src/features/video-streaming/utils/index.ts | 9 + .../utils/performanceMonitor.ts | 197 +++++++++ .../video-streaming/utils/thumbnailCache.ts | 224 ++++++++++ src/lib/visionApi.ts | 5 +- src/test/videoStreamingTest.ts | 156 +++++++ 38 files changed, 3117 insertions(+), 441 deletions(-) create mode 100644 .env.example create mode 100644 API Documentations/docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md create mode 100644 API Documentations/docs/WEB_AI_AGENT_VIDEO_INTEGRATION.md create mode 100644 docs/VIDEO_STREAMING_INTEGRATION_COMPLETE.md create mode 100644 src/features/video-streaming/components/ApiStatusIndicator.tsx create mode 100644 src/features/video-streaming/components/PerformanceDashboard.tsx create mode 100644 src/features/video-streaming/components/VideoDebugger.tsx create mode 100644 src/features/video-streaming/components/VideoErrorBoundary.tsx create mode 100644 src/features/video-streaming/utils/index.ts create mode 100644 src/features/video-streaming/utils/performanceMonitor.ts create mode 100644 src/features/video-streaming/utils/thumbnailCache.ts create mode 100644 src/test/videoStreamingTest.ts diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..f0a9612 --- /dev/null +++ b/.env.example @@ -0,0 +1,14 @@ +# Environment Configuration for Pecan Experiments Application + +# USDA Vision Camera System API Configuration +# Default: http://vision:8000 (current working setup) +# For localhost setup, use: http://localhost:8000 +# For remote systems, use: http://192.168.1.100:8000 (replace with actual IP) +VITE_VISION_API_URL=http://vision:8000 + +# Supabase Configuration (if needed for production) +# VITE_SUPABASE_URL=your_supabase_url +# VITE_SUPABASE_ANON_KEY=your_supabase_anon_key + +# Development Configuration +# VITE_DEV_MODE=true diff --git a/API Documentations/docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md b/API Documentations/docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md new file mode 100644 index 0000000..8901049 --- /dev/null +++ b/API Documentations/docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md @@ -0,0 +1,415 @@ +# ๐Ÿค– AI Agent Video Integration Guide + +This guide provides comprehensive step-by-step instructions for AI agents and external systems to successfully integrate with the USDA Vision Camera System's video streaming functionality. + +## ๐ŸŽฏ Overview + +The USDA Vision Camera System provides a complete video streaming API that allows AI agents to: +- Browse and select videos from multiple cameras +- Stream videos with seeking capabilities +- Generate thumbnails for preview +- Access video metadata and technical information + +## ๐Ÿ”— API Base Configuration + +### Connection Details +```bash +# Default API Base URL +API_BASE_URL="http://localhost:8000" + +# For remote access, replace with actual server IP/hostname +API_BASE_URL="http://192.168.1.100:8000" +``` + +### Authentication +**โš ๏ธ IMPORTANT: No authentication is currently required.** +- All endpoints are publicly accessible +- No API keys or tokens needed +- CORS is enabled for web browser integration + +## ๐Ÿ“‹ Step-by-Step Integration Workflow + +### Step 1: Verify System Connectivity +```bash +# Test basic connectivity +curl -f "${API_BASE_URL}/health" || echo "โŒ System not accessible" + +# Check system status +curl "${API_BASE_URL}/system/status" +``` + +**Expected Response:** +```json +{ + "status": "healthy", + "timestamp": "2025-08-05T10:30:00Z" +} +``` + +### Step 2: List Available Videos +```bash +# Get all videos with metadata +curl "${API_BASE_URL}/videos/?include_metadata=true&limit=50" + +# Filter by specific camera +curl "${API_BASE_URL}/videos/?camera_name=camera1&include_metadata=true" + +# Filter by date range +curl "${API_BASE_URL}/videos/?start_date=2025-08-04T00:00:00&end_date=2025-08-05T23:59:59" +``` + +**Response Structure:** +```json +{ + "videos": [ + { + "file_id": "camera1_auto_blower_separator_20250804_143022.mp4", + "camera_name": "camera1", + "filename": "camera1_auto_blower_separator_20250804_143022.mp4", + "file_size_bytes": 31457280, + "format": "mp4", + "status": "completed", + "created_at": "2025-08-04T14:30:22", + "start_time": "2025-08-04T14:30:22", + "end_time": "2025-08-04T14:32:22", + "machine_trigger": "blower_separator", + "is_streamable": true, + "needs_conversion": false, + "metadata": { + "duration_seconds": 120.5, + "width": 1920, + "height": 1080, + "fps": 30.0, + "codec": "mp4v", + "bitrate": 5000000, + "aspect_ratio": 1.777 + } + } + ], + "total_count": 1 +} +``` + +### Step 3: Select and Validate Video +```bash +# Get detailed video information +FILE_ID="camera1_auto_blower_separator_20250804_143022.mp4" +curl "${API_BASE_URL}/videos/${FILE_ID}" + +# Validate video is playable +curl -X POST "${API_BASE_URL}/videos/${FILE_ID}/validate" + +# Get streaming technical details +curl "${API_BASE_URL}/videos/${FILE_ID}/info" +``` + +### Step 4: Generate Video Thumbnail +```bash +# Generate thumbnail at 5 seconds, 320x240 resolution +curl "${API_BASE_URL}/videos/${FILE_ID}/thumbnail?timestamp=5.0&width=320&height=240" \ + --output "thumbnail_${FILE_ID}.jpg" + +# Generate multiple thumbnails for preview +for timestamp in 1 30 60 90; do + curl "${API_BASE_URL}/videos/${FILE_ID}/thumbnail?timestamp=${timestamp}&width=160&height=120" \ + --output "preview_${timestamp}s.jpg" +done +``` + +### Step 5: Stream Video Content +```bash +# Stream entire video +curl "${API_BASE_URL}/videos/${FILE_ID}/stream" --output "video.mp4" + +# Stream specific byte range (for seeking) +curl -H "Range: bytes=0-1048575" \ + "${API_BASE_URL}/videos/${FILE_ID}/stream" \ + --output "video_chunk.mp4" + +# Test range request support +curl -I -H "Range: bytes=0-1023" \ + "${API_BASE_URL}/videos/${FILE_ID}/stream" +``` + +## ๐Ÿ”ง Programming Language Examples + +### Python Integration +```python +import requests +import json +from typing import List, Dict, Optional + +class USDAVideoClient: + def __init__(self, base_url: str = "http://localhost:8000"): + self.base_url = base_url.rstrip('/') + self.session = requests.Session() + + def list_videos(self, camera_name: Optional[str] = None, + include_metadata: bool = True, limit: int = 50) -> Dict: + """List available videos with optional filtering.""" + params = { + 'include_metadata': include_metadata, + 'limit': limit + } + if camera_name: + params['camera_name'] = camera_name + + response = self.session.get(f"{self.base_url}/videos/", params=params) + response.raise_for_status() + return response.json() + + def get_video_info(self, file_id: str) -> Dict: + """Get detailed video information.""" + response = self.session.get(f"{self.base_url}/videos/{file_id}") + response.raise_for_status() + return response.json() + + def get_thumbnail(self, file_id: str, timestamp: float = 1.0, + width: int = 320, height: int = 240) -> bytes: + """Generate and download video thumbnail.""" + params = { + 'timestamp': timestamp, + 'width': width, + 'height': height + } + response = self.session.get( + f"{self.base_url}/videos/{file_id}/thumbnail", + params=params + ) + response.raise_for_status() + return response.content + + def stream_video_range(self, file_id: str, start_byte: int, + end_byte: int) -> bytes: + """Stream specific byte range of video.""" + headers = {'Range': f'bytes={start_byte}-{end_byte}'} + response = self.session.get( + f"{self.base_url}/videos/{file_id}/stream", + headers=headers + ) + response.raise_for_status() + return response.content + + def validate_video(self, file_id: str) -> bool: + """Validate that video is accessible and playable.""" + response = self.session.post(f"{self.base_url}/videos/{file_id}/validate") + response.raise_for_status() + return response.json().get('is_valid', False) + +# Usage example +client = USDAVideoClient("http://192.168.1.100:8000") + +# List videos from camera1 +videos = client.list_videos(camera_name="camera1") +print(f"Found {videos['total_count']} videos") + +# Select first video +if videos['videos']: + video = videos['videos'][0] + file_id = video['file_id'] + + # Validate video + if client.validate_video(file_id): + print(f"โœ… Video {file_id} is valid") + + # Get thumbnail + thumbnail = client.get_thumbnail(file_id, timestamp=5.0) + with open(f"thumbnail_{file_id}.jpg", "wb") as f: + f.write(thumbnail) + + # Stream first 1MB + chunk = client.stream_video_range(file_id, 0, 1048575) + print(f"Downloaded {len(chunk)} bytes") +``` + +### JavaScript/Node.js Integration +```javascript +class USDAVideoClient { + constructor(baseUrl = 'http://localhost:8000') { + this.baseUrl = baseUrl.replace(/\/$/, ''); + } + + async listVideos(options = {}) { + const params = new URLSearchParams({ + include_metadata: options.includeMetadata || true, + limit: options.limit || 50 + }); + + if (options.cameraName) { + params.append('camera_name', options.cameraName); + } + + const response = await fetch(`${this.baseUrl}/videos/?${params}`); + if (!response.ok) throw new Error(`HTTP ${response.status}`); + return response.json(); + } + + async getVideoInfo(fileId) { + const response = await fetch(`${this.baseUrl}/videos/${fileId}`); + if (!response.ok) throw new Error(`HTTP ${response.status}`); + return response.json(); + } + + async getThumbnail(fileId, options = {}) { + const params = new URLSearchParams({ + timestamp: options.timestamp || 1.0, + width: options.width || 320, + height: options.height || 240 + }); + + const response = await fetch( + `${this.baseUrl}/videos/${fileId}/thumbnail?${params}` + ); + if (!response.ok) throw new Error(`HTTP ${response.status}`); + return response.blob(); + } + + async validateVideo(fileId) { + const response = await fetch( + `${this.baseUrl}/videos/${fileId}/validate`, + { method: 'POST' } + ); + if (!response.ok) throw new Error(`HTTP ${response.status}`); + const result = await response.json(); + return result.is_valid; + } + + getStreamUrl(fileId) { + return `${this.baseUrl}/videos/${fileId}/stream`; + } +} + +// Usage example +const client = new USDAVideoClient('http://192.168.1.100:8000'); + +async function integrateWithVideos() { + try { + // List videos + const videos = await client.listVideos({ cameraName: 'camera1' }); + console.log(`Found ${videos.total_count} videos`); + + if (videos.videos.length > 0) { + const video = videos.videos[0]; + const fileId = video.file_id; + + // Validate video + const isValid = await client.validateVideo(fileId); + if (isValid) { + console.log(`โœ… Video ${fileId} is valid`); + + // Get thumbnail + const thumbnail = await client.getThumbnail(fileId, { + timestamp: 5.0, + width: 320, + height: 240 + }); + + // Create video element for playback + const videoElement = document.createElement('video'); + videoElement.controls = true; + videoElement.src = client.getStreamUrl(fileId); + document.body.appendChild(videoElement); + } + } + } catch (error) { + console.error('Integration error:', error); + } +} +``` + +## ๐Ÿšจ Error Handling + +### Common HTTP Status Codes +```bash +# Success responses +200 # OK - Request successful +206 # Partial Content - Range request successful + +# Client error responses +400 # Bad Request - Invalid parameters +404 # Not Found - Video file doesn't exist +416 # Range Not Satisfiable - Invalid range request + +# Server error responses +500 # Internal Server Error - Failed to process video +503 # Service Unavailable - Video module not available +``` + +### Error Response Format +```json +{ + "detail": "Video camera1_recording_20250804_143022.avi not found" +} +``` + +### Robust Error Handling Example +```python +def safe_video_operation(client, file_id): + try: + # Validate video first + if not client.validate_video(file_id): + return {"error": "Video is not valid or accessible"} + + # Get video info + video_info = client.get_video_info(file_id) + + # Check if streamable + if not video_info.get('is_streamable', False): + return {"error": "Video is not streamable"} + + return {"success": True, "video_info": video_info} + + except requests.exceptions.HTTPError as e: + if e.response.status_code == 404: + return {"error": "Video not found"} + elif e.response.status_code == 416: + return {"error": "Invalid range request"} + else: + return {"error": f"HTTP error: {e.response.status_code}"} + except requests.exceptions.ConnectionError: + return {"error": "Cannot connect to video server"} + except Exception as e: + return {"error": f"Unexpected error: {str(e)}"} +``` + +## โœ… Integration Checklist + +### Pre-Integration +- [ ] Verify network connectivity to USDA Vision Camera System +- [ ] Test basic API endpoints (`/health`, `/system/status`) +- [ ] Understand video file naming conventions +- [ ] Plan error handling strategy + +### Video Selection +- [ ] Implement video listing with appropriate filters +- [ ] Add video validation before processing +- [ ] Handle pagination for large video collections +- [ ] Implement caching for video metadata + +### Video Playback +- [ ] Test video streaming with range requests +- [ ] Implement thumbnail generation for previews +- [ ] Add progress tracking for video playback +- [ ] Handle different video formats (MP4, AVI) + +### Error Handling +- [ ] Handle network connectivity issues +- [ ] Manage video not found scenarios +- [ ] Deal with invalid range requests +- [ ] Implement retry logic for transient failures + +### Performance +- [ ] Use range requests for efficient seeking +- [ ] Implement client-side caching where appropriate +- [ ] Monitor bandwidth usage for video streaming +- [ ] Consider thumbnail caching for better UX + +## ๐ŸŽฏ Next Steps + +1. **Test Integration**: Use the provided examples to test basic connectivity +2. **Implement Error Handling**: Add robust error handling for production use +3. **Optimize Performance**: Implement caching and efficient streaming +4. **Monitor Usage**: Track API usage and performance metrics +5. **Security Review**: Consider authentication if exposing externally + +This guide provides everything needed for successful integration with the USDA Vision Camera System's video streaming functionality. The system is designed to be simple and reliable for AI agents and external systems to consume video content efficiently. diff --git a/API Documentations/docs/API_CHANGES_SUMMARY.md b/API Documentations/docs/API_CHANGES_SUMMARY.md index a23b324..d7af414 100644 --- a/API Documentations/docs/API_CHANGES_SUMMARY.md +++ b/API Documentations/docs/API_CHANGES_SUMMARY.md @@ -1,32 +1,27 @@ # API Changes Summary: Camera Settings and Video Format Updates ## Overview - This document tracks major API changes including camera settings enhancements and the MP4 video format update. ## ๐ŸŽฅ Latest Update: MP4 Video Format (v2.1) - **Date**: August 2025 **Major Changes**: - -- **Video Format**: Changed from AVI/XVID to MP4/H.264 format +- **Video Format**: Changed from AVI/XVID to MP4/MPEG-4 format - **File Extensions**: New recordings use `.mp4` instead of `.avi` - **File Size**: ~40% reduction in file sizes - **Streaming**: Better web browser compatibility **New Configuration Fields**: - ```json { "video_format": "mp4", // File format: "mp4" or "avi" - "video_codec": "h264", // Video codec: "h264", "mp4v", "XVID", "MJPG" + "video_codec": "mp4v", // Video codec: "mp4v", "XVID", "MJPG" "video_quality": 95 // Quality: 0-100 (higher = better) } ``` **Frontend Impact**: - - โœ… Better streaming performance and browser support - โœ… Smaller file sizes for faster transfers - โœ… Universal HTML5 video player compatibility @@ -43,14 +38,12 @@ Enhanced the `POST /cameras/{camera_name}/start-recording` API endpoint to accep ## Changes Made ### 1. API Models (`usda_vision_system/api/models.py`) - - **Enhanced `StartRecordingRequest`** to include optional parameters: - `exposure_ms: Optional[float]` - Exposure time in milliseconds - `gain: Optional[float]` - Camera gain value - `fps: Optional[float]` - Target frames per second ### 2. Camera Recorder (`usda_vision_system/camera/recorder.py`) - - **Added `update_camera_settings()` method** to dynamically update camera settings: - Updates exposure time using `mvsdk.CameraSetExposureTime()` - Updates gain using `mvsdk.CameraSetAnalogGain()` @@ -59,23 +52,20 @@ Enhanced the `POST /cameras/{camera_name}/start-recording` API endpoint to accep - Returns boolean indicating success/failure ### 3. Camera Manager (`usda_vision_system/camera/manager.py`) - - **Enhanced `manual_start_recording()` method** to accept new parameters: - Added optional `exposure_ms`, `gain`, and `fps` parameters - Calls `update_camera_settings()` if any settings are provided - **Automatic datetime prefix**: Always prepends timestamp to filename - If custom filename provided: `{timestamp}_{custom_filename}` - - If no filename provided: `{camera_name}_manual_{timestamp}.mp4` + - If no filename provided: `{camera_name}_manual_{timestamp}.avi` ### 4. API Server (`usda_vision_system/api/server.py`) - - **Updated start-recording endpoint** to: - Pass new camera settings to camera manager - Handle filename response with datetime prefix - Maintain backward compatibility with existing requests ### 5. API Tests (`api-tests.http`) - - **Added comprehensive test examples**: - Basic recording (existing functionality) - Recording with camera settings @@ -85,9 +75,8 @@ Enhanced the `POST /cameras/{camera_name}/start-recording` API endpoint to accep ## Usage Examples ### Basic Recording (unchanged) - ```http -POST http://vision:8000/cameras/camera1/start-recording +POST http://localhost:8000/cameras/camera1/start-recording Content-Type: application/json { @@ -95,13 +84,11 @@ Content-Type: application/json "filename": "test.avi" } ``` - **Result**: File saved as `20241223_143022_test.avi` ### Recording with Camera Settings - ```http -POST http://vision:8000/cameras/camera1/start-recording +POST http://localhost:8000/cameras/camera1/start-recording Content-Type: application/json { @@ -112,16 +99,13 @@ Content-Type: application/json "fps": 5.0 } ``` - **Result**: - - Camera settings updated before recording - File saved as `20241223_143022_high_quality.avi` ### Maximum FPS Recording - ```http -POST http://vision:8000/cameras/camera1/start-recording +POST http://localhost:8000/cameras/camera1/start-recording Content-Type: application/json { @@ -132,17 +116,14 @@ Content-Type: application/json "fps": 0 } ``` - **Result**: - - Camera captures at maximum possible speed (no delay between frames) - Video file saved with 30 FPS metadata for proper playback - Actual capture rate depends on camera hardware and exposure settings ### Settings Only (no filename) - ```http -POST http://vision:8000/cameras/camera1/start-recording +POST http://localhost:8000/cameras/camera1/start-recording Content-Type: application/json { @@ -152,41 +133,34 @@ Content-Type: application/json "fps": 7.0 } ``` - -**Result**: - +**Result**: - Camera settings updated - File saved as `camera1_manual_20241223_143022.avi` ## Key Features ### 1. **Backward Compatibility** - - All existing API calls continue to work unchanged - New parameters are optional - Default behavior preserved when no settings provided ### 2. **Automatic Datetime Prefix** - - **ALL filenames now have datetime prefix** regardless of what's sent - Format: `YYYYMMDD_HHMMSS_` (Atlanta timezone) - Ensures unique filenames and chronological ordering ### 3. **Dynamic Camera Settings** - - Settings can be changed per recording without restarting system - Based on proven implementation from `old tests/camera_video_recorder.py` - Proper error handling and logging ### 4. **Maximum FPS Capture** - - **`fps: 0`** = Capture at maximum possible speed (no delay between frames) - **`fps > 0`** = Capture at specified frame rate with controlled timing - **`fps` omitted** = Uses camera config default (usually 3.0 fps) - Video files saved with 30 FPS metadata when fps=0 for proper playback ### 5. **Parameter Validation** - - Uses Pydantic models for automatic validation - Optional parameters with proper type checking - Descriptive field documentation @@ -194,7 +168,6 @@ Content-Type: application/json ## Testing Run the test script to verify functionality: - ```bash # Start the system first python main.py @@ -204,7 +177,6 @@ python test_api_changes.py ``` The test script verifies: - - Basic recording functionality - Camera settings application - Filename datetime prefix handling @@ -213,27 +185,22 @@ The test script verifies: ## Implementation Notes ### Camera Settings Mapping - - **Exposure**: Converted from milliseconds to microseconds for SDK - **Gain**: Converted to camera units (multiplied by 100) - **FPS**: Stored in camera config, used by recording loop ### Error Handling - - Settings update failures are logged but don't prevent recording - Invalid camera names return appropriate HTTP errors - Camera initialization failures are handled gracefully ### Filename Generation - - Uses `format_filename_timestamp()` from timezone utilities - Ensures Atlanta timezone consistency - Handles both custom and auto-generated filenames ## Similar to Old Implementation - The camera settings functionality mirrors the proven approach in `old tests/camera_video_recorder.py`: - - Same parameter names and ranges - Same SDK function calls - Same conversion factors diff --git a/API Documentations/docs/API_DOCUMENTATION.md b/API Documentations/docs/API_DOCUMENTATION.md index 065413d..81ac03f 100644 --- a/API Documentations/docs/API_DOCUMENTATION.md +++ b/API Documentations/docs/API_DOCUMENTATION.md @@ -13,18 +13,16 @@ This document provides comprehensive documentation for all API endpoints in the - [๐Ÿ’พ Storage & File Management](#-storage--file-management) - [๐Ÿ”„ Camera Recovery & Diagnostics](#-camera-recovery--diagnostics) - [๐Ÿ“บ Live Streaming](#-live-streaming) +- [๐ŸŽฌ Video Streaming & Playback](#-video-streaming--playback) - [๐ŸŒ WebSocket Real-time Updates](#-websocket-real-time-updates) ## ๐Ÿ”ง System Status & Health ### Get System Status - ```http GET /system/status ``` - **Response**: `SystemStatusResponse` - ```json { "system_started": true, @@ -52,13 +50,10 @@ GET /system/status ``` ### Health Check - ```http GET /health ``` - **Response**: Simple health status - ```json { "status": "healthy", @@ -69,21 +64,16 @@ GET /health ## ๐Ÿ“ท Camera Management ### Get All Cameras - ```http GET /cameras ``` - **Response**: `Dict[str, CameraStatusResponse]` ### Get Specific Camera Status - ```http GET /cameras/{camera_name}/status ``` - **Response**: `CameraStatusResponse` - ```json { "name": "camera1", @@ -108,13 +98,12 @@ GET /cameras/{camera_name}/status ## ๐ŸŽฅ Recording Control ### Start Recording - ```http POST /cameras/{camera_name}/start-recording Content-Type: application/json { - "filename": "test_recording.mp4", + "filename": "test_recording.avi", "exposure_ms": 2.0, "gain": 4.0, "fps": 5.0 @@ -122,36 +111,30 @@ Content-Type: application/json ``` **Request Model**: `StartRecordingRequest` - - `filename` (optional): Custom filename (datetime prefix will be added automatically) - `exposure_ms` (optional): Exposure time in milliseconds - `gain` (optional): Camera gain value - `fps` (optional): Target frames per second **Response**: `StartRecordingResponse` - ```json { "success": true, "message": "Recording started for camera1", - "filename": "20240115_103000_test_recording.mp4" + "filename": "20240115_103000_test_recording.avi" } ``` **Key Features**: - - โœ… **Automatic datetime prefix**: All filenames get `YYYYMMDD_HHMMSS_` prefix - โœ… **Dynamic camera settings**: Adjust exposure, gain, and FPS per recording - โœ… **Backward compatibility**: All existing API calls work unchanged ### Stop Recording - ```http POST /cameras/{camera_name}/stop-recording ``` - **Response**: `StopRecordingResponse` - ```json { "success": true, @@ -163,13 +146,10 @@ POST /cameras/{camera_name}/stop-recording ## ๐Ÿค– Auto-Recording Management ### Enable Auto-Recording for Camera - ```http POST /cameras/{camera_name}/auto-recording/enable ``` - **Response**: `AutoRecordingConfigResponse` - ```json { "success": true, @@ -180,21 +160,16 @@ POST /cameras/{camera_name}/auto-recording/enable ``` ### Disable Auto-Recording for Camera - ```http POST /cameras/{camera_name}/auto-recording/disable ``` - **Response**: `AutoRecordingConfigResponse` ### Get Auto-Recording Status - ```http GET /auto-recording/status ``` - **Response**: `AutoRecordingStatusResponse` - ```json { "running": true, @@ -205,7 +180,6 @@ GET /auto-recording/status ``` **Auto-Recording Features**: - - ๐Ÿค– **MQTT-triggered recording**: Automatically starts/stops based on machine state - ๐Ÿ”„ **Retry logic**: Failed recordings are retried with configurable delays - ๐Ÿ“Š **Per-camera control**: Enable/disable auto-recording individually @@ -214,13 +188,10 @@ GET /auto-recording/status ## ๐ŸŽ›๏ธ Camera Configuration ### Get Camera Configuration - ```http GET /cameras/{camera_name}/config ``` - **Response**: `CameraConfigResponse` - ```json { "name": "camera1", @@ -255,7 +226,6 @@ GET /cameras/{camera_name}/config ``` ### Update Camera Configuration - ```http PUT /cameras/{camera_name}/config Content-Type: application/json @@ -269,13 +239,11 @@ Content-Type: application/json ``` ### Apply Configuration (Restart Required) - ```http POST /cameras/{camera_name}/apply-config ``` **Configuration Categories**: - - โœ… **Real-time**: `exposure_ms`, `gain`, `target_fps`, `sharpness`, `contrast`, etc. - โš ๏ธ **Restart required**: `noise_filter_enabled`, `denoise_3d_enabled`, `bit_depth`, `video_format`, `video_codec`, `video_quality` @@ -284,21 +252,16 @@ For detailed configuration options, see [Camera Configuration API Guide](api/CAM ## ๐Ÿ“ก MQTT & Machine Status ### Get All Machines - ```http GET /machines ``` - **Response**: `Dict[str, MachineStatusResponse]` ### Get MQTT Status - ```http GET /mqtt/status ``` - **Response**: `MQTTStatusResponse` - ```json { "connected": true, @@ -313,13 +276,10 @@ GET /mqtt/status ``` ### Get MQTT Events History - ```http GET /mqtt/events?limit=10 ``` - **Response**: `MQTTEventsHistoryResponse` - ```json { "events": [ @@ -340,13 +300,10 @@ GET /mqtt/events?limit=10 ## ๐Ÿ’พ Storage & File Management ### Get Storage Statistics - ```http GET /storage/stats ``` - **Response**: `StorageStatsResponse` - ```json { "base_path": "/storage", @@ -372,7 +329,6 @@ GET /storage/stats ``` ### Get File List - ```http POST /storage/files Content-Type: application/json @@ -384,9 +340,7 @@ Content-Type: application/json "limit": 50 } ``` - **Response**: `FileListResponse` - ```json { "files": [ @@ -403,7 +357,6 @@ Content-Type: application/json ``` ### Cleanup Old Files - ```http POST /storage/cleanup Content-Type: application/json @@ -412,9 +365,7 @@ Content-Type: application/json "max_age_days": 30 } ``` - **Response**: `CleanupResponse` - ```json { "files_removed": 25, @@ -426,55 +377,42 @@ Content-Type: application/json ## ๐Ÿ”„ Camera Recovery & Diagnostics ### Test Camera Connection - ```http POST /cameras/{camera_name}/test-connection ``` - **Response**: `CameraTestResponse` ### Reconnect Camera - ```http POST /cameras/{camera_name}/reconnect ``` - **Response**: `CameraRecoveryResponse` ### Restart Camera Grab Process - ```http POST /cameras/{camera_name}/restart-grab ``` - **Response**: `CameraRecoveryResponse` ### Reset Camera Timestamp - ```http POST /cameras/{camera_name}/reset-timestamp ``` - **Response**: `CameraRecoveryResponse` ### Full Camera Reset - ```http POST /cameras/{camera_name}/full-reset ``` - **Response**: `CameraRecoveryResponse` ### Reinitialize Camera - ```http POST /cameras/{camera_name}/reinitialize ``` - **Response**: `CameraRecoveryResponse` **Recovery Response Example**: - ```json { "success": true, @@ -488,39 +426,176 @@ POST /cameras/{camera_name}/reinitialize ## ๐Ÿ“บ Live Streaming ### Get Live MJPEG Stream - ```http GET /cameras/{camera_name}/stream ``` - **Response**: MJPEG video stream (multipart/x-mixed-replace) ### Start Camera Stream - ```http POST /cameras/{camera_name}/start-stream ``` ### Stop Camera Stream - ```http POST /cameras/{camera_name}/stop-stream ``` **Streaming Features**: - - ๐Ÿ“บ **MJPEG format**: Compatible with web browsers and React apps - ๐Ÿ”„ **Concurrent operation**: Stream while recording simultaneously - โšก **Low latency**: Real-time preview for monitoring For detailed streaming integration, see [Streaming Guide](guides/STREAMING_GUIDE.md). +## ๐ŸŽฌ Video Streaming & Playback + +The system includes a comprehensive video streaming module that provides YouTube-like video playback capabilities with HTTP range request support, thumbnail generation, and intelligent caching. + +### List Videos +```http +GET /videos/ +``` +**Query Parameters:** +- `camera_name` (optional): Filter by camera name +- `start_date` (optional): Filter videos created after this date (ISO format) +- `end_date` (optional): Filter videos created before this date (ISO format) +- `limit` (optional): Maximum number of results (default: 50, max: 1000) +- `include_metadata` (optional): Include video metadata (default: false) + +**Response**: `VideoListResponse` +```json +{ + "videos": [ + { + "file_id": "camera1_auto_blower_separator_20250804_143022.mp4", + "camera_name": "camera1", + "filename": "camera1_auto_blower_separator_20250804_143022.mp4", + "file_size_bytes": 31457280, + "format": "mp4", + "status": "completed", + "created_at": "2025-08-04T14:30:22", + "start_time": "2025-08-04T14:30:22", + "end_time": "2025-08-04T14:32:22", + "machine_trigger": "blower_separator", + "is_streamable": true, + "needs_conversion": false, + "metadata": { + "duration_seconds": 120.5, + "width": 1920, + "height": 1080, + "fps": 30.0, + "codec": "mp4v", + "bitrate": 5000000, + "aspect_ratio": 1.777 + } + } + ], + "total_count": 1 +} +``` + +### Get Video Information +```http +GET /videos/{file_id} +``` +**Response**: `VideoInfoResponse` with detailed video information including metadata. + +### Stream Video +```http +GET /videos/{file_id}/stream +``` +**Headers:** +- `Range: bytes=0-1023` (optional): Request specific byte range for seeking + +**Features:** +- โœ… **HTTP Range Requests**: Enables video seeking and progressive download +- โœ… **Partial Content**: Returns 206 status for range requests +- โœ… **Format Conversion**: Automatic AVI to MP4 conversion for web compatibility +- โœ… **Intelligent Caching**: Optimized performance with byte-range caching +- โœ… **CORS Enabled**: Ready for web browser integration + +**Response Headers:** +- `Accept-Ranges: bytes` +- `Content-Length: {size}` +- `Content-Range: bytes {start}-{end}/{total}` (for range requests) +- `Cache-Control: public, max-age=3600` + +### Get Video Thumbnail +```http +GET /videos/{file_id}/thumbnail?timestamp=5.0&width=320&height=240 +``` +**Query Parameters:** +- `timestamp` (optional): Time position in seconds (default: 1.0) +- `width` (optional): Thumbnail width in pixels (default: 320) +- `height` (optional): Thumbnail height in pixels (default: 240) + +**Response**: JPEG image data with caching headers + +### Get Streaming Information +```http +GET /videos/{file_id}/info +``` +**Response**: `StreamingInfoResponse` +```json +{ + "file_id": "camera1_recording_20250804_143022.avi", + "file_size_bytes": 52428800, + "content_type": "video/mp4", + "supports_range_requests": true, + "chunk_size_bytes": 262144 +} +``` + +### Video Validation +```http +POST /videos/{file_id}/validate +``` +**Response**: Validation status and accessibility check +```json +{ + "file_id": "camera1_recording_20250804_143022.avi", + "is_valid": true +} +``` + +### Cache Management +```http +POST /videos/{file_id}/cache/invalidate +``` +**Response**: Cache invalidation status +```json +{ + "file_id": "camera1_recording_20250804_143022.avi", + "cache_invalidated": true +} +``` + +### Admin: Cache Cleanup +```http +POST /admin/videos/cache/cleanup?max_size_mb=100 +``` +**Response**: Cache cleanup results +```json +{ + "cache_cleaned": true, + "entries_removed": 15, + "max_size_mb": 100 +} +``` + +**Video Streaming Features**: +- ๐ŸŽฅ **Multiple Formats**: Native MP4 support with AVI conversion +- ๐Ÿ“ฑ **Web Compatible**: Direct integration with HTML5 video elements +- โšก **High Performance**: Intelligent caching and adaptive chunking +- ๐Ÿ–ผ๏ธ **Thumbnail Generation**: Extract preview images at any timestamp +- ๐Ÿ”„ **Range Requests**: Efficient seeking and progressive download + ## ๐ŸŒ WebSocket Real-time Updates ### Connect to WebSocket - ```javascript -const ws = new WebSocket('ws://vision:8000/ws'); +const ws = new WebSocket('ws://localhost:8000/ws'); ws.onmessage = (event) => { const update = JSON.parse(event.data); @@ -529,7 +604,6 @@ ws.onmessage = (event) => { ``` **WebSocket Message Types**: - - `system_status`: System status changes - `camera_status`: Camera status updates - `recording_started`: Recording start events @@ -538,7 +612,6 @@ ws.onmessage = (event) => { - `auto_recording_event`: Auto-recording status changes **Example WebSocket Message**: - ```json { "type": "recording_started", @@ -554,28 +627,26 @@ ws.onmessage = (event) => { ## ๐Ÿš€ Quick Start Examples ### Basic System Monitoring - ```bash # Check system health -curl http://vision:8000/health +curl http://localhost:8000/health # Get overall system status -curl http://vision:8000/system/status +curl http://localhost:8000/system/status # Get all camera statuses -curl http://vision:8000/cameras +curl http://localhost:8000/cameras ``` ### Manual Recording Control - ```bash # Start recording with default settings -curl -X POST http://vision:8000/cameras/camera1/start-recording \ +curl -X POST http://localhost:8000/cameras/camera1/start-recording \ -H "Content-Type: application/json" \ -d '{"filename": "manual_test.avi"}' # Start recording with custom camera settings -curl -X POST http://vision:8000/cameras/camera1/start-recording \ +curl -X POST http://localhost:8000/cameras/camera1/start-recording \ -H "Content-Type: application/json" \ -d '{ "filename": "high_quality.avi", @@ -585,30 +656,57 @@ curl -X POST http://vision:8000/cameras/camera1/start-recording \ }' # Stop recording -curl -X POST http://vision:8000/cameras/camera1/stop-recording +curl -X POST http://localhost:8000/cameras/camera1/stop-recording ``` ### Auto-Recording Management - ```bash # Enable auto-recording for camera1 -curl -X POST http://vision:8000/cameras/camera1/auto-recording/enable +curl -X POST http://localhost:8000/cameras/camera1/auto-recording/enable # Check auto-recording status -curl http://vision:8000/auto-recording/status +curl http://localhost:8000/auto-recording/status # Disable auto-recording for camera1 -curl -X POST http://vision:8000/cameras/camera1/auto-recording/disable +curl -X POST http://localhost:8000/cameras/camera1/auto-recording/disable +``` + +### Video Streaming Operations +```bash +# List all videos +curl http://localhost:8000/videos/ + +# List videos from specific camera with metadata +curl "http://localhost:8000/videos/?camera_name=camera1&include_metadata=true" + +# Get video information +curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi + +# Get video thumbnail +curl "http://localhost:8000/videos/camera1_recording_20250804_143022.avi/thumbnail?timestamp=5.0&width=320&height=240" \ + --output thumbnail.jpg + +# Get streaming info +curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi/info + +# Stream video with range request +curl -H "Range: bytes=0-1023" \ + http://localhost:8000/videos/camera1_recording_20250804_143022.avi/stream + +# Validate video file +curl -X POST http://localhost:8000/videos/camera1_recording_20250804_143022.avi/validate + +# Clean up video cache (admin) +curl -X POST "http://localhost:8000/admin/videos/cache/cleanup?max_size_mb=100" ``` ### Camera Configuration - ```bash # Get current camera configuration -curl http://vision:8000/cameras/camera1/config +curl http://localhost:8000/cameras/camera1/config # Update camera settings (real-time) -curl -X PUT http://vision:8000/cameras/camera1/config \ +curl -X PUT http://localhost:8000/cameras/camera1/config \ -H "Content-Type: application/json" \ -d '{ "exposure_ms": 1.5, @@ -623,47 +721,47 @@ curl -X PUT http://vision:8000/cameras/camera1/config \ ### โœจ New in Latest Version #### 1. Enhanced Recording API - - **Dynamic camera settings**: Set exposure, gain, and FPS per recording - **Automatic datetime prefixes**: All filenames get timestamp prefixes - **Backward compatibility**: Existing API calls work unchanged #### 2. Auto-Recording Feature - - **Per-camera control**: Enable/disable auto-recording individually - **MQTT integration**: Automatic recording based on machine states - **Retry logic**: Failed recordings are automatically retried - **Status tracking**: Monitor auto-recording attempts and failures #### 3. Advanced Camera Configuration - - **Real-time settings**: Update exposure, gain, image quality without restart - **Image enhancement**: Sharpness, contrast, saturation, gamma controls - **Noise reduction**: Configurable noise filtering and 3D denoising - **HDR support**: High Dynamic Range imaging capabilities #### 4. Live Streaming - - **MJPEG streaming**: Real-time camera preview - **Concurrent operation**: Stream while recording simultaneously - **Web-compatible**: Direct integration with React/HTML video elements #### 5. Enhanced Monitoring - - **MQTT event history**: Track machine state changes over time - **Storage statistics**: Monitor disk usage and file counts - **WebSocket updates**: Real-time system status notifications +#### 6. Video Streaming Module +- **HTTP Range Requests**: Efficient video seeking and progressive download +- **Thumbnail Generation**: Extract preview images from videos at any timestamp +- **Format Conversion**: Automatic AVI to MP4 conversion for web compatibility +- **Intelligent Caching**: Byte-range caching for optimal streaming performance +- **Admin Tools**: Cache management and video validation endpoints + ### ๐Ÿ”„ Migration Notes #### From Previous Versions - 1. **Recording API**: All existing calls work, but now return filenames with datetime prefixes 2. **Configuration**: New camera settings are optional and backward compatible 3. **Auto-recording**: New feature, requires enabling in `config.json` and per camera #### Configuration Updates - ```json { "cameras": [ @@ -689,34 +787,38 @@ curl -X PUT http://vision:8000/cameras/camera1/config \ - [๐Ÿ“ท Camera Configuration API Guide](api/CAMERA_CONFIG_API.md) - Detailed camera settings - [๐Ÿค– Auto-Recording Feature Guide](features/AUTO_RECORDING_FEATURE_GUIDE.md) - React integration - [๐Ÿ“บ Streaming Guide](guides/STREAMING_GUIDE.md) - Live video streaming +- [๐ŸŽฌ Video Streaming Guide](VIDEO_STREAMING.md) - Video playback and streaming +- [๐Ÿค– AI Agent Video Integration Guide](AI_AGENT_VIDEO_INTEGRATION_GUIDE.md) - Complete integration guide for AI agents - [๐Ÿ”ง Camera Recovery Guide](guides/CAMERA_RECOVERY_GUIDE.md) - Troubleshooting - [๐Ÿ“ก MQTT Logging Guide](guides/MQTT_LOGGING_GUIDE.md) - MQTT configuration ## ๐Ÿ“ž Support & Integration ### API Base URL - -- **Development**: `http://vision:8000` +- **Development**: `http://localhost:8000` - **Production**: Configure in `config.json` under `system.api_host` and `system.api_port` ### Error Handling - All endpoints return standard HTTP status codes: - - `200`: Success -- `404`: Resource not found (camera, file, etc.) +- `206`: Partial Content (for video range requests) +- `400`: Bad Request (invalid parameters) +- `404`: Resource not found (camera, file, video, etc.) +- `416`: Range Not Satisfiable (invalid video range request) - `500`: Internal server error - `503`: Service unavailable (camera manager, MQTT, etc.) -### Rate Limiting +**Video Streaming Specific Errors:** +- `404`: Video file not found or not streamable +- `416`: Invalid range request (malformed Range header) +- `500`: Failed to read video data or generate thumbnail +### Rate Limiting - No rate limiting currently implemented - WebSocket connections are limited to reasonable concurrent connections ### CORS Support - - CORS is enabled for web dashboard integration - Configure allowed origins in the API server settings - ``` ``` diff --git a/API Documentations/docs/API_QUICK_REFERENCE.md b/API Documentations/docs/API_QUICK_REFERENCE.md index 0c267bf..1ec7a54 100644 --- a/API Documentations/docs/API_QUICK_REFERENCE.md +++ b/API Documentations/docs/API_QUICK_REFERENCE.md @@ -6,30 +6,30 @@ Quick reference for the most commonly used API endpoints. For complete documenta ```bash # Health check -curl http://vision:8000/health +curl http://localhost:8000/health # System overview -curl http://vision:8000/system/status +curl http://localhost:8000/system/status # All cameras -curl http://vision:8000/cameras +curl http://localhost:8000/cameras # All machines -curl http://vision:8000/machines +curl http://localhost:8000/machines ``` ## ๐ŸŽฅ Recording Control ### Start Recording (Basic) ```bash -curl -X POST http://vision:8000/cameras/camera1/start-recording \ +curl -X POST http://localhost:8000/cameras/camera1/start-recording \ -H "Content-Type: application/json" \ -d '{"filename": "test.avi"}' ``` ### Start Recording (With Settings) ```bash -curl -X POST http://vision:8000/cameras/camera1/start-recording \ +curl -X POST http://localhost:8000/cameras/camera1/start-recording \ -H "Content-Type: application/json" \ -d '{ "filename": "high_quality.avi", @@ -41,30 +41,30 @@ curl -X POST http://vision:8000/cameras/camera1/start-recording \ ### Stop Recording ```bash -curl -X POST http://vision:8000/cameras/camera1/stop-recording +curl -X POST http://localhost:8000/cameras/camera1/stop-recording ``` ## ๐Ÿค– Auto-Recording ```bash # Enable auto-recording -curl -X POST http://vision:8000/cameras/camera1/auto-recording/enable +curl -X POST http://localhost:8000/cameras/camera1/auto-recording/enable # Disable auto-recording -curl -X POST http://vision:8000/cameras/camera1/auto-recording/disable +curl -X POST http://localhost:8000/cameras/camera1/auto-recording/disable # Check auto-recording status -curl http://vision:8000/auto-recording/status +curl http://localhost:8000/auto-recording/status ``` ## ๐ŸŽ›๏ธ Camera Configuration ```bash # Get camera config -curl http://vision:8000/cameras/camera1/config +curl http://localhost:8000/cameras/camera1/config # Update camera settings -curl -X PUT http://vision:8000/cameras/camera1/config \ +curl -X PUT http://localhost:8000/cameras/camera1/config \ -H "Content-Type: application/json" \ -d '{ "exposure_ms": 1.5, @@ -77,41 +77,41 @@ curl -X PUT http://vision:8000/cameras/camera1/config \ ```bash # Start streaming -curl -X POST http://vision:8000/cameras/camera1/start-stream +curl -X POST http://localhost:8000/cameras/camera1/start-stream # Get MJPEG stream (use in browser/video element) -# http://vision:8000/cameras/camera1/stream +# http://localhost:8000/cameras/camera1/stream # Stop streaming -curl -X POST http://vision:8000/cameras/camera1/stop-stream +curl -X POST http://localhost:8000/cameras/camera1/stop-stream ``` ## ๐Ÿ”„ Camera Recovery ```bash # Test connection -curl -X POST http://vision:8000/cameras/camera1/test-connection +curl -X POST http://localhost:8000/cameras/camera1/test-connection # Reconnect camera -curl -X POST http://vision:8000/cameras/camera1/reconnect +curl -X POST http://localhost:8000/cameras/camera1/reconnect # Full reset -curl -X POST http://vision:8000/cameras/camera1/full-reset +curl -X POST http://localhost:8000/cameras/camera1/full-reset ``` ## ๐Ÿ’พ Storage Management ```bash # Storage statistics -curl http://vision:8000/storage/stats +curl http://localhost:8000/storage/stats # List files -curl -X POST http://vision:8000/storage/files \ +curl -X POST http://localhost:8000/storage/files \ -H "Content-Type: application/json" \ -d '{"camera_name": "camera1", "limit": 10}' # Cleanup old files -curl -X POST http://vision:8000/storage/cleanup \ +curl -X POST http://localhost:8000/storage/cleanup \ -H "Content-Type: application/json" \ -d '{"max_age_days": 30}' ``` @@ -120,17 +120,17 @@ curl -X POST http://vision:8000/storage/cleanup \ ```bash # MQTT status -curl http://vision:8000/mqtt/status +curl http://localhost:8000/mqtt/status # Recent MQTT events -curl http://vision:8000/mqtt/events?limit=10 +curl http://localhost:8000/mqtt/events?limit=10 ``` ## ๐ŸŒ WebSocket Connection ```javascript // Connect to real-time updates -const ws = new WebSocket('ws://vision:8000/ws'); +const ws = new WebSocket('ws://localhost:8000/ws'); ws.onmessage = (event) => { const update = JSON.parse(event.data); diff --git a/API Documentations/docs/MP4_FORMAT_UPDATE.md b/API Documentations/docs/MP4_FORMAT_UPDATE.md index a6f2dcc..ecae663 100644 --- a/API Documentations/docs/MP4_FORMAT_UPDATE.md +++ b/API Documentations/docs/MP4_FORMAT_UPDATE.md @@ -1,24 +1,20 @@ # ๐ŸŽฅ MP4 Video Format Update - Frontend Integration Guide ## Overview - The USDA Vision Camera System has been updated to record videos in **MP4 format** instead of AVI format for better streaming compatibility and smaller file sizes. ## ๐Ÿ”„ What Changed ### Video Format - - **Before**: AVI files with XVID codec (`.avi` extension) -- **After**: MP4 files with H.264 codec (`.mp4` extension) +- **After**: MP4 files with MPEG-4 codec (`.mp4` extension) ### File Extensions - - All new video recordings now use `.mp4` extension - Existing `.avi` files remain accessible and functional - File size reduction: ~40% smaller than equivalent AVI files ### API Response Updates - New fields added to camera configuration responses: ```json @@ -32,17 +28,13 @@ New fields added to camera configuration responses: ## ๐ŸŒ Frontend Impact ### 1. Video Player Compatibility - **โœ… Better Browser Support** - - MP4 format has native support in all modern browsers - No need for additional codecs or plugins - Better mobile device compatibility (iOS/Android) ### 2. File Handling Updates - **File Extension Handling** - ```javascript // Update file extension checks const isVideoFile = (filename) => { @@ -58,9 +50,7 @@ const getVideoMimeType = (filename) => { ``` ### 3. Video Streaming - **Improved Streaming Performance** - ```javascript // MP4 files can be streamed directly without conversion const videoUrl = `/api/videos/${videoId}/stream`; @@ -73,9 +63,7 @@ const videoUrl = `/api/videos/${videoId}/stream`; ``` ### 4. File Size Display - **Updated Size Expectations** - - MP4 files are ~40% smaller than equivalent AVI files - Update any file size warnings or storage calculations - Better compression means faster downloads and uploads @@ -83,11 +71,9 @@ const videoUrl = `/api/videos/${videoId}/stream`; ## ๐Ÿ“ก API Changes ### Camera Configuration Endpoint - **GET** `/cameras/{camera_name}/config` **New Response Fields:** - ```json { "name": "camera1", @@ -109,9 +95,7 @@ const videoUrl = `/api/videos/${videoId}/stream`; ``` ### Video Listing Endpoints - **File Extension Updates** - - Video files in responses will now have `.mp4` extensions - Existing `.avi` files will still appear in listings - Filter by both extensions when needed @@ -119,49 +103,42 @@ const videoUrl = `/api/videos/${videoId}/stream`; ## ๐Ÿ”ง Configuration Options ### Video Format Settings - ```json { "video_format": "mp4", // Options: "mp4", "avi" - "video_codec": "h264", // Options: "h264", "mp4v", "XVID", "MJPG" + "video_codec": "mp4v", // Options: "mp4v", "XVID", "MJPG" "video_quality": 95 // Range: 0-100 (higher = better quality) } ``` ### Recommended Settings - -- **Production**: `"mp4"` format, `"h264"` codec, `95` quality -- **Storage Optimized**: `"mp4"` format, `"h264"` codec, `85` quality +- **Production**: `"mp4"` format, `"mp4v"` codec, `95` quality +- **Storage Optimized**: `"mp4"` format, `"mp4v"` codec, `85` quality - **Legacy Mode**: `"avi"` format, `"XVID"` codec, `95` quality ## ๐ŸŽฏ Frontend Implementation Checklist ### โœ… Video Player Updates - - [ ] Verify HTML5 video player works with MP4 files - [ ] Update video MIME type handling - [ ] Test streaming performance with new format ### โœ… File Management - - [ ] Update file extension filters to include `.mp4` - [ ] Modify file type detection logic - [ ] Update download/upload handling for MP4 files ### โœ… UI/UX Updates - - [ ] Update file size expectations in UI - [ ] Modify any format-specific icons or indicators - [ ] Update help text or tooltips mentioning video formats ### โœ… Configuration Interface - - [ ] Add video format settings to camera config UI - [ ] Include video quality slider/selector - [ ] Add restart warning for video format changes ### โœ… Testing - - [ ] Test video playback with new MP4 files - [ ] Verify backward compatibility with existing AVI files - [ ] Test streaming performance and loading times @@ -169,13 +146,11 @@ const videoUrl = `/api/videos/${videoId}/stream`; ## ๐Ÿ”„ Backward Compatibility ### Existing AVI Files - - All existing `.avi` files remain fully functional - No conversion or migration required - Video player should handle both formats ### API Compatibility - - All existing API endpoints continue to work - New fields are additive (won't break existing code) - Default values provided for new configuration fields @@ -183,7 +158,6 @@ const videoUrl = `/api/videos/${videoId}/stream`; ## ๐Ÿ“Š Performance Benefits ### File Size Reduction - ``` Example 5-minute recording at 1280x1024: - AVI/XVID: ~180 MB @@ -191,14 +165,12 @@ Example 5-minute recording at 1280x1024: ``` ### Streaming Improvements - - Faster initial load times - Better progressive download support - Reduced bandwidth usage - Native browser optimization ### Storage Efficiency - - More recordings fit in same storage space - Faster backup and transfer operations - Reduced storage costs over time @@ -206,19 +178,16 @@ Example 5-minute recording at 1280x1024: ## ๐Ÿšจ Important Notes ### Restart Required - - Video format changes require camera service restart - Mark video format settings as "restart required" in UI - Provide clear user feedback about restart necessity ### Browser Compatibility - - MP4 format supported in all modern browsers - Better mobile device support than AVI - No additional plugins or codecs needed ### Quality Assurance - - Video quality maintained at 95/100 setting - No visual degradation compared to AVI - High bitrate ensures professional quality diff --git a/API Documentations/docs/PROJECT_COMPLETE.md b/API Documentations/docs/PROJECT_COMPLETE.md index 7f240d6..0f4df48 100644 --- a/API Documentations/docs/PROJECT_COMPLETE.md +++ b/API Documentations/docs/PROJECT_COMPLETE.md @@ -97,11 +97,11 @@ python test_system.py ### Dashboard Integration ```javascript // React component example -const systemStatus = await fetch('http://vision:8000/system/status'); -const cameras = await fetch('http://vision:8000/cameras'); +const systemStatus = await fetch('http://localhost:8000/system/status'); +const cameras = await fetch('http://localhost:8000/cameras'); // WebSocket for real-time updates -const ws = new WebSocket('ws://vision:8000/ws'); +const ws = new WebSocket('ws://localhost:8000/ws'); ws.onmessage = (event) => { const update = JSON.parse(event.data); // Handle real-time system updates @@ -111,13 +111,13 @@ ws.onmessage = (event) => { ### Manual Control ```bash # Start recording manually -curl -X POST http://vision:8000/cameras/camera1/start-recording +curl -X POST http://localhost:8000/cameras/camera1/start-recording # Stop recording manually -curl -X POST http://vision:8000/cameras/camera1/stop-recording +curl -X POST http://localhost:8000/cameras/camera1/stop-recording # Get system status -curl http://vision:8000/system/status +curl http://localhost:8000/system/status ``` ## ๐Ÿ“Š System Capabilities @@ -151,7 +151,7 @@ curl http://vision:8000/system/status ### Troubleshooting - **Test Suite**: `python test_system.py` - **Time Check**: `python check_time.py` -- **API Health**: `curl http://vision:8000/health` +- **API Health**: `curl http://localhost:8000/health` - **Debug Mode**: `python main.py --log-level DEBUG` ## ๐ŸŽฏ Production Readiness diff --git a/API Documentations/docs/README.md b/API Documentations/docs/README.md index daccd3d..5ba7b70 100644 --- a/API Documentations/docs/README.md +++ b/API Documentations/docs/README.md @@ -48,6 +48,20 @@ Complete project overview and final status documentation. Contains: - Camera-specific settings comparison - MQTT topics and machine mappings +### ๐ŸŽฌ [VIDEO_STREAMING.md](VIDEO_STREAMING.md) **โญ UPDATED** +**Complete video streaming module documentation**: +- Comprehensive API endpoint documentation +- Authentication and security information +- Error handling and troubleshooting +- Performance optimization guidelines + +### ๐Ÿค– [AI_AGENT_VIDEO_INTEGRATION_GUIDE.md](AI_AGENT_VIDEO_INTEGRATION_GUIDE.md) **โญ NEW** +**Complete integration guide for AI agents and external systems**: +- Step-by-step integration workflow +- Programming language examples (Python, JavaScript) +- Error handling and debugging strategies +- Performance optimization recommendations + ### ๐Ÿ”ง [API_CHANGES_SUMMARY.md](API_CHANGES_SUMMARY.md) Summary of API changes and enhancements made to the system. diff --git a/API Documentations/docs/VIDEO_STREAMING.md b/API Documentations/docs/VIDEO_STREAMING.md index 8cbed70..69b9d6e 100644 --- a/API Documentations/docs/VIDEO_STREAMING.md +++ b/API Documentations/docs/VIDEO_STREAMING.md @@ -4,11 +4,16 @@ The USDA Vision Camera System now includes a modular video streaming system that ## ๐ŸŒŸ Features -- **HTTP Range Request Support** - Enables seeking and progressive download -- **Native MP4 Support** - Direct streaming of MP4 files with automatic AVI conversion -- **Intelligent Caching** - Optimized streaming performance +- **Progressive Streaming** - True chunked streaming for web browsers (no download required) +- **HTTP Range Request Support** - Enables seeking and progressive download with 206 Partial Content +- **Native MP4 Support** - Direct streaming of MP4 files optimized for web playback +- **Memory Efficient** - 8KB chunked delivery, no large file loading into memory +- **Browser Compatible** - Works with HTML5 `