Add comprehensive video streaming module and AI agent integration guide

- Introduced a new video streaming module with endpoints for listing, streaming, and managing video files. - Added detailed API documentation for video streaming, including features like HTTP range requests, thumbnail generation, and caching. - Created a new AI Agent Video Integration Guide with step-by-step instructions for integrating with the video streaming API. - Implemented a video reindexing script to update video statuses from "unknown" to "completed". - Enhanced error handling and logging throughout the video streaming and reindexing processes.
2025-08-05 14:44:31 -04:00
parent 14757807aa
commit 07e8e52503
5 changed files with 1144 additions and 23 deletions
--- a/docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md
+++ b/docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md
@@ -0,0 +1,415 @@
+# 🤖 AI Agent Video Integration Guide
+
+This guide provides comprehensive step-by-step instructions for AI agents and external systems to successfully integrate with the USDA Vision Camera System's video streaming functionality.
+
+## 🎯 Overview
+
+The USDA Vision Camera System provides a complete video streaming API that allows AI agents to:
+- Browse and select videos from multiple cameras
+- Stream videos with seeking capabilities
+- Generate thumbnails for preview
+- Access video metadata and technical information
+
+## 🔗 API Base Configuration
+
+### Connection Details
+```bash
+# Default API Base URL
+API_BASE_URL="http://localhost:8000"
+
+# For remote access, replace with actual server IP/hostname
+API_BASE_URL="http://192.168.1.100:8000"
+```
+
+### Authentication
+**⚠️ IMPORTANT: No authentication is currently required.**
+- All endpoints are publicly accessible
+- No API keys or tokens needed
+- CORS is enabled for web browser integration
+
+## 📋 Step-by-Step Integration Workflow
+
+### Step 1: Verify System Connectivity
+```bash
+# Test basic connectivity
+curl -f "${API_BASE_URL}/health" || echo "❌ System not accessible"
+
+# Check system status
+curl "${API_BASE_URL}/system/status"
+```
+
+**Expected Response:**
+```json
+{
+  "status": "healthy",
+  "timestamp": "2025-08-05T10:30:00Z"
+}
+```
+
+### Step 2: List Available Videos
+```bash
+# Get all videos with metadata
+curl "${API_BASE_URL}/videos/?include_metadata=true&limit=50"
+
+# Filter by specific camera
+curl "${API_BASE_URL}/videos/?camera_name=camera1&include_metadata=true"
+
+# Filter by date range
+curl "${API_BASE_URL}/videos/?start_date=2025-08-04T00:00:00&end_date=2025-08-05T23:59:59"
+```
+
+**Response Structure:**
+```json
+{
+  "videos": [
+    {
+      "file_id": "camera1_auto_blower_separator_20250804_143022.mp4",
+      "camera_name": "camera1",
+      "filename": "camera1_auto_blower_separator_20250804_143022.mp4",
+      "file_size_bytes": 31457280,
+      "format": "mp4",
+      "status": "completed",
+      "created_at": "2025-08-04T14:30:22",
+      "start_time": "2025-08-04T14:30:22",
+      "end_time": "2025-08-04T14:32:22",
+      "machine_trigger": "blower_separator",
+      "is_streamable": true,
+      "needs_conversion": false,
+      "metadata": {
+        "duration_seconds": 120.5,
+        "width": 1920,
+        "height": 1080,
+        "fps": 30.0,
+        "codec": "mp4v",
+        "bitrate": 5000000,
+        "aspect_ratio": 1.777
+      }
+    }
+  ],
+  "total_count": 1
+}
+```
+
+### Step 3: Select and Validate Video
+```bash
+# Get detailed video information
+FILE_ID="camera1_auto_blower_separator_20250804_143022.mp4"
+curl "${API_BASE_URL}/videos/${FILE_ID}"
+
+# Validate video is playable
+curl -X POST "${API_BASE_URL}/videos/${FILE_ID}/validate"
+
+# Get streaming technical details
+curl "${API_BASE_URL}/videos/${FILE_ID}/info"
+```
+
+### Step 4: Generate Video Thumbnail
+```bash
+# Generate thumbnail at 5 seconds, 320x240 resolution
+curl "${API_BASE_URL}/videos/${FILE_ID}/thumbnail?timestamp=5.0&width=320&height=240" \
+  --output "thumbnail_${FILE_ID}.jpg"
+
+# Generate multiple thumbnails for preview
+for timestamp in 1 30 60 90; do
+  curl "${API_BASE_URL}/videos/${FILE_ID}/thumbnail?timestamp=${timestamp}&width=160&height=120" \
+    --output "preview_${timestamp}s.jpg"
+done
+```
+
+### Step 5: Stream Video Content
+```bash
+# Stream entire video
+curl "${API_BASE_URL}/videos/${FILE_ID}/stream" --output "video.mp4"
+
+# Stream specific byte range (for seeking)
+curl -H "Range: bytes=0-1048575" \
+  "${API_BASE_URL}/videos/${FILE_ID}/stream" \
+  --output "video_chunk.mp4"
+
+# Test range request support
+curl -I -H "Range: bytes=0-1023" \
+  "${API_BASE_URL}/videos/${FILE_ID}/stream"
+```
+
+## 🔧 Programming Language Examples
+
+### Python Integration
+```python
+import requests
+import json
+from typing import List, Dict, Optional
+
+class USDAVideoClient:
+    def __init__(self, base_url: str = "http://localhost:8000"):
+        self.base_url = base_url.rstrip('/')
+        self.session = requests.Session()
+    
+    def list_videos(self, camera_name: Optional[str] = None, 
+                   include_metadata: bool = True, limit: int = 50) -> Dict:
+        """List available videos with optional filtering."""
+        params = {
+            'include_metadata': include_metadata,
+            'limit': limit
+        }
+        if camera_name:
+            params['camera_name'] = camera_name
+        
+        response = self.session.get(f"{self.base_url}/videos/", params=params)
+        response.raise_for_status()
+        return response.json()
+    
+    def get_video_info(self, file_id: str) -> Dict:
+        """Get detailed video information."""
+        response = self.session.get(f"{self.base_url}/videos/{file_id}")
+        response.raise_for_status()
+        return response.json()
+    
+    def get_thumbnail(self, file_id: str, timestamp: float = 1.0, 
+                     width: int = 320, height: int = 240) -> bytes:
+        """Generate and download video thumbnail."""
+        params = {
+            'timestamp': timestamp,
+            'width': width,
+            'height': height
+        }
+        response = self.session.get(
+            f"{self.base_url}/videos/{file_id}/thumbnail", 
+            params=params
+        )
+        response.raise_for_status()
+        return response.content
+    
+    def stream_video_range(self, file_id: str, start_byte: int, 
+                          end_byte: int) -> bytes:
+        """Stream specific byte range of video."""
+        headers = {'Range': f'bytes={start_byte}-{end_byte}'}
+        response = self.session.get(
+            f"{self.base_url}/videos/{file_id}/stream",
+            headers=headers
+        )
+        response.raise_for_status()
+        return response.content
+    
+    def validate_video(self, file_id: str) -> bool:
+        """Validate that video is accessible and playable."""
+        response = self.session.post(f"{self.base_url}/videos/{file_id}/validate")
+        response.raise_for_status()
+        return response.json().get('is_valid', False)
+
+# Usage example
+client = USDAVideoClient("http://192.168.1.100:8000")
+
+# List videos from camera1
+videos = client.list_videos(camera_name="camera1")
+print(f"Found {videos['total_count']} videos")
+
+# Select first video
+if videos['videos']:
+    video = videos['videos'][0]
+    file_id = video['file_id']
+    
+    # Validate video
+    if client.validate_video(file_id):
+        print(f"✅ Video {file_id} is valid")
+        
+        # Get thumbnail
+        thumbnail = client.get_thumbnail(file_id, timestamp=5.0)
+        with open(f"thumbnail_{file_id}.jpg", "wb") as f:
+            f.write(thumbnail)
+        
+        # Stream first 1MB
+        chunk = client.stream_video_range(file_id, 0, 1048575)
+        print(f"Downloaded {len(chunk)} bytes")
+```
+
+### JavaScript/Node.js Integration
+```javascript
+class USDAVideoClient {
+    constructor(baseUrl = 'http://localhost:8000') {
+        this.baseUrl = baseUrl.replace(/\/$/, '');
+    }
+    
+    async listVideos(options = {}) {
+        const params = new URLSearchParams({
+            include_metadata: options.includeMetadata || true,
+            limit: options.limit || 50
+        });
+        
+        if (options.cameraName) {
+            params.append('camera_name', options.cameraName);
+        }
+        
+        const response = await fetch(`${this.baseUrl}/videos/?${params}`);
+        if (!response.ok) throw new Error(`HTTP ${response.status}`);
+        return response.json();
+    }
+    
+    async getVideoInfo(fileId) {
+        const response = await fetch(`${this.baseUrl}/videos/${fileId}`);
+        if (!response.ok) throw new Error(`HTTP ${response.status}`);
+        return response.json();
+    }
+    
+    async getThumbnail(fileId, options = {}) {
+        const params = new URLSearchParams({
+            timestamp: options.timestamp || 1.0,
+            width: options.width || 320,
+            height: options.height || 240
+        });
+        
+        const response = await fetch(
+            `${this.baseUrl}/videos/${fileId}/thumbnail?${params}`
+        );
+        if (!response.ok) throw new Error(`HTTP ${response.status}`);
+        return response.blob();
+    }
+    
+    async validateVideo(fileId) {
+        const response = await fetch(
+            `${this.baseUrl}/videos/${fileId}/validate`,
+            { method: 'POST' }
+        );
+        if (!response.ok) throw new Error(`HTTP ${response.status}`);
+        const result = await response.json();
+        return result.is_valid;
+    }
+    
+    getStreamUrl(fileId) {
+        return `${this.baseUrl}/videos/${fileId}/stream`;
+    }
+}
+
+// Usage example
+const client = new USDAVideoClient('http://192.168.1.100:8000');
+
+async function integrateWithVideos() {
+    try {
+        // List videos
+        const videos = await client.listVideos({ cameraName: 'camera1' });
+        console.log(`Found ${videos.total_count} videos`);
+        
+        if (videos.videos.length > 0) {
+            const video = videos.videos[0];
+            const fileId = video.file_id;
+            
+            // Validate video
+            const isValid = await client.validateVideo(fileId);
+            if (isValid) {
+                console.log(`✅ Video ${fileId} is valid`);
+                
+                // Get thumbnail
+                const thumbnail = await client.getThumbnail(fileId, {
+                    timestamp: 5.0,
+                    width: 320,
+                    height: 240
+                });
+                
+                // Create video element for playback
+                const videoElement = document.createElement('video');
+                videoElement.controls = true;
+                videoElement.src = client.getStreamUrl(fileId);
+                document.body.appendChild(videoElement);
+            }
+        }
+    } catch (error) {
+        console.error('Integration error:', error);
+    }
+}
+```
+
+## 🚨 Error Handling
+
+### Common HTTP Status Codes
+```bash
+# Success responses
+200  # OK - Request successful
+206  # Partial Content - Range request successful
+
+# Client error responses  
+400  # Bad Request - Invalid parameters
+404  # Not Found - Video file doesn't exist
+416  # Range Not Satisfiable - Invalid range request
+
+# Server error responses
+500  # Internal Server Error - Failed to process video
+503  # Service Unavailable - Video module not available
+```
+
+### Error Response Format
+```json
+{
+  "detail": "Video camera1_recording_20250804_143022.avi not found"
+}
+```
+
+### Robust Error Handling Example
+```python
+def safe_video_operation(client, file_id):
+    try:
+        # Validate video first
+        if not client.validate_video(file_id):
+            return {"error": "Video is not valid or accessible"}
+        
+        # Get video info
+        video_info = client.get_video_info(file_id)
+        
+        # Check if streamable
+        if not video_info.get('is_streamable', False):
+            return {"error": "Video is not streamable"}
+        
+        return {"success": True, "video_info": video_info}
+        
+    except requests.exceptions.HTTPError as e:
+        if e.response.status_code == 404:
+            return {"error": "Video not found"}
+        elif e.response.status_code == 416:
+            return {"error": "Invalid range request"}
+        else:
+            return {"error": f"HTTP error: {e.response.status_code}"}
+    except requests.exceptions.ConnectionError:
+        return {"error": "Cannot connect to video server"}
+    except Exception as e:
+        return {"error": f"Unexpected error: {str(e)}"}
+```
+
+## ✅ Integration Checklist
+
+### Pre-Integration
+- [ ] Verify network connectivity to USDA Vision Camera System
+- [ ] Test basic API endpoints (`/health`, `/system/status`)
+- [ ] Understand video file naming conventions
+- [ ] Plan error handling strategy
+
+### Video Selection
+- [ ] Implement video listing with appropriate filters
+- [ ] Add video validation before processing
+- [ ] Handle pagination for large video collections
+- [ ] Implement caching for video metadata
+
+### Video Playback
+- [ ] Test video streaming with range requests
+- [ ] Implement thumbnail generation for previews
+- [ ] Add progress tracking for video playback
+- [ ] Handle different video formats (MP4, AVI)
+
+### Error Handling
+- [ ] Handle network connectivity issues
+- [ ] Manage video not found scenarios
+- [ ] Deal with invalid range requests
+- [ ] Implement retry logic for transient failures
+
+### Performance
+- [ ] Use range requests for efficient seeking
+- [ ] Implement client-side caching where appropriate
+- [ ] Monitor bandwidth usage for video streaming
+- [ ] Consider thumbnail caching for better UX
+
+## 🎯 Next Steps
+
+1. **Test Integration**: Use the provided examples to test basic connectivity
+2. **Implement Error Handling**: Add robust error handling for production use
+3. **Optimize Performance**: Implement caching and efficient streaming
+4. **Monitor Usage**: Track API usage and performance metrics
+5. **Security Review**: Consider authentication if exposing externally
+
+This guide provides everything needed for successful integration with the USDA Vision Camera System's video streaming functionality. The system is designed to be simple and reliable for AI agents and external systems to consume video content efficiently.
--- a/docs/API_DOCUMENTATION.md
+++ b/docs/API_DOCUMENTATION.md
@@ -13,6 +13,7 @@ This document provides comprehensive documentation for all API endpoints in the
 - [💾 Storage & File Management](#-storage--file-management)
 - [🔄 Camera Recovery & Diagnostics](#-camera-recovery--diagnostics)
 - [📺 Live Streaming](#-live-streaming)
+- [🎬 Video Streaming & Playback](#-video-streaming--playback)
 - [🌐 WebSocket Real-time Updates](#-websocket-real-time-updates)

 ## 🔧 System Status & Health
@@ -447,6 +448,149 @@ POST /cameras/{camera_name}/stop-stream

 For detailed streaming integration, see [Streaming Guide](guides/STREAMING_GUIDE.md).

+## 🎬 Video Streaming & Playback
+
+The system includes a comprehensive video streaming module that provides YouTube-like video playback capabilities with HTTP range request support, thumbnail generation, and intelligent caching.
+
+### List Videos
+```http
+GET /videos/
+```
+**Query Parameters:**
+- `camera_name` (optional): Filter by camera name
+- `start_date` (optional): Filter videos created after this date (ISO format)
+- `end_date` (optional): Filter videos created before this date (ISO format)
+- `limit` (optional): Maximum number of results (default: 50, max: 1000)
+- `include_metadata` (optional): Include video metadata (default: false)
+
+**Response**: `VideoListResponse`
+```json
+{
+  "videos": [
+    {
+      "file_id": "camera1_auto_blower_separator_20250804_143022.mp4",
+      "camera_name": "camera1",
+      "filename": "camera1_auto_blower_separator_20250804_143022.mp4",
+      "file_size_bytes": 31457280,
+      "format": "mp4",
+      "status": "completed",
+      "created_at": "2025-08-04T14:30:22",
+      "start_time": "2025-08-04T14:30:22",
+      "end_time": "2025-08-04T14:32:22",
+      "machine_trigger": "blower_separator",
+      "is_streamable": true,
+      "needs_conversion": false,
+      "metadata": {
+        "duration_seconds": 120.5,
+        "width": 1920,
+        "height": 1080,
+        "fps": 30.0,
+        "codec": "mp4v",
+        "bitrate": 5000000,
+        "aspect_ratio": 1.777
+      }
+    }
+  ],
+  "total_count": 1
+}
+```
+
+### Get Video Information
+```http
+GET /videos/{file_id}
+```
+**Response**: `VideoInfoResponse` with detailed video information including metadata.
+
+### Stream Video
+```http
+GET /videos/{file_id}/stream
+```
+**Headers:**
+- `Range: bytes=0-1023` (optional): Request specific byte range for seeking
+
+**Features:**
+- ✅ **HTTP Range Requests**: Enables video seeking and progressive download
+- ✅ **Partial Content**: Returns 206 status for range requests
+- ✅ **Format Conversion**: Automatic AVI to MP4 conversion for web compatibility
+- ✅ **Intelligent Caching**: Optimized performance with byte-range caching
+- ✅ **CORS Enabled**: Ready for web browser integration
+
+**Response Headers:**
+- `Accept-Ranges: bytes`
+- `Content-Length: {size}`
+- `Content-Range: bytes {start}-{end}/{total}` (for range requests)
+- `Cache-Control: public, max-age=3600`
+
+### Get Video Thumbnail
+```http
+GET /videos/{file_id}/thumbnail?timestamp=5.0&width=320&height=240
+```
+**Query Parameters:**
+- `timestamp` (optional): Time position in seconds (default: 1.0)
+- `width` (optional): Thumbnail width in pixels (default: 320)
+- `height` (optional): Thumbnail height in pixels (default: 240)
+
+**Response**: JPEG image data with caching headers
+
+### Get Streaming Information
+```http
+GET /videos/{file_id}/info
+```
+**Response**: `StreamingInfoResponse`
+```json
+{
+  "file_id": "camera1_recording_20250804_143022.avi",
+  "file_size_bytes": 52428800,
+  "content_type": "video/mp4",
+  "supports_range_requests": true,
+  "chunk_size_bytes": 262144
+}
+```
+
+### Video Validation
+```http
+POST /videos/{file_id}/validate
+```
+**Response**: Validation status and accessibility check
+```json
+{
+  "file_id": "camera1_recording_20250804_143022.avi",
+  "is_valid": true
+}
+```
+
+### Cache Management
+```http
+POST /videos/{file_id}/cache/invalidate
+```
+**Response**: Cache invalidation status
+```json
+{
+  "file_id": "camera1_recording_20250804_143022.avi",
+  "cache_invalidated": true
+}
+```
+
+### Admin: Cache Cleanup
+```http
+POST /admin/videos/cache/cleanup?max_size_mb=100
+```
+**Response**: Cache cleanup results
+```json
+{
+  "cache_cleaned": true,
+  "entries_removed": 15,
+  "max_size_mb": 100
+}
+```
+
+**Video Streaming Features**:
+- 🎥 **Multiple Formats**: Native MP4 support with AVI conversion
+- 📱 **Web Compatible**: Direct integration with HTML5 video elements
+- ⚡ **High Performance**: Intelligent caching and adaptive chunking
+- 🖼️ **Thumbnail Generation**: Extract preview images at any timestamp
+- 🔄 **Range Requests**: Efficient seeking and progressive download
+
 ## 🌐 WebSocket Real-time Updates

 ### Connect to WebSocket
@@ -527,6 +671,35 @@ curl http://localhost:8000/auto-recording/status
 curl -X POST http://localhost:8000/cameras/camera1/auto-recording/disable
 ```

+### Video Streaming Operations
+```bash
+# List all videos
+curl http://localhost:8000/videos/
+
+# List videos from specific camera with metadata
+curl "http://localhost:8000/videos/?camera_name=camera1&include_metadata=true"
+
+# Get video information
+curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi
+
+# Get video thumbnail
+curl "http://localhost:8000/videos/camera1_recording_20250804_143022.avi/thumbnail?timestamp=5.0&width=320&height=240" \
+  --output thumbnail.jpg
+
+# Get streaming info
+curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi/info
+
+# Stream video with range request
+curl -H "Range: bytes=0-1023" \
+  http://localhost:8000/videos/camera1_recording_20250804_143022.avi/stream
+
+# Validate video file
+curl -X POST http://localhost:8000/videos/camera1_recording_20250804_143022.avi/validate
+
+# Clean up video cache (admin)
+curl -X POST "http://localhost:8000/admin/videos/cache/cleanup?max_size_mb=100"
+```
+
 ### Camera Configuration
 ```bash
 # Get current camera configuration
@@ -574,6 +747,13 @@ curl -X PUT http://localhost:8000/cameras/camera1/config \
 - **Storage statistics**: Monitor disk usage and file counts
 - **WebSocket updates**: Real-time system status notifications

+#### 6. Video Streaming Module
+- **HTTP Range Requests**: Efficient video seeking and progressive download
+- **Thumbnail Generation**: Extract preview images from videos at any timestamp
+- **Format Conversion**: Automatic AVI to MP4 conversion for web compatibility
+- **Intelligent Caching**: Byte-range caching for optimal streaming performance
+- **Admin Tools**: Cache management and video validation endpoints
+
 ### 🔄 Migration Notes

 #### From Previous Versions
@@ -607,6 +787,8 @@ curl -X PUT http://localhost:8000/cameras/camera1/config \
 - [📷 Camera Configuration API Guide](api/CAMERA_CONFIG_API.md) - Detailed camera settings
 - [🤖 Auto-Recording Feature Guide](features/AUTO_RECORDING_FEATURE_GUIDE.md) - React integration
 - [📺 Streaming Guide](guides/STREAMING_GUIDE.md) - Live video streaming
+- [🎬 Video Streaming Guide](VIDEO_STREAMING.md) - Video playback and streaming
+- [🤖 AI Agent Video Integration Guide](AI_AGENT_VIDEO_INTEGRATION_GUIDE.md) - Complete integration guide for AI agents
 - [🔧 Camera Recovery Guide](guides/CAMERA_RECOVERY_GUIDE.md) - Troubleshooting
 - [📡 MQTT Logging Guide](guides/MQTT_LOGGING_GUIDE.md) - MQTT configuration

@@ -619,10 +801,18 @@ curl -X PUT http://localhost:8000/cameras/camera1/config \
 ### Error Handling
 All endpoints return standard HTTP status codes:
 - `200`: Success
- `404`: Resource not found (camera, file, etc.)
+- `206`: Partial Content (for video range requests)
+- `400`: Bad Request (invalid parameters)
+- `404`: Resource not found (camera, file, video, etc.)
+- `416`: Range Not Satisfiable (invalid video range request)
 - `500`: Internal server error
 - `503`: Service unavailable (camera manager, MQTT, etc.)

+**Video Streaming Specific Errors:**
+- `404`: Video file not found or not streamable
+- `416`: Invalid range request (malformed Range header)
+- `500`: Failed to read video data or generate thumbnail
+
 ### Rate Limiting
 - No rate limiting currently implemented
 - WebSocket connections are limited to reasonable concurrent connections
--- a/docs/README.md
+++ b/docs/README.md
@@ -48,6 +48,20 @@ Complete project overview and final status documentation. Contains:
 - Camera-specific settings comparison
 - MQTT topics and machine mappings

+### 🎬 [VIDEO_STREAMING.md](VIDEO_STREAMING.md) **⭐ UPDATED**
+**Complete video streaming module documentation**:
+- Comprehensive API endpoint documentation
+- Authentication and security information
+- Error handling and troubleshooting
+- Performance optimization guidelines
+
+### 🤖 [AI_AGENT_VIDEO_INTEGRATION_GUIDE.md](AI_AGENT_VIDEO_INTEGRATION_GUIDE.md) **⭐ NEW**
+**Complete integration guide for AI agents and external systems**:
+- Step-by-step integration workflow
+- Programming language examples (Python, JavaScript)
+- Error handling and debugging strategies
+- Performance optimization recommendations
+
 ### 🔧 [API_CHANGES_SUMMARY.md](API_CHANGES_SUMMARY.md)
 Summary of API changes and enhancements made to the system.

--- a/docs/VIDEO_STREAMING.md
+++ b/docs/VIDEO_STREAMING.md
@@ -9,6 +9,8 @@ The USDA Vision Camera System now includes a modular video streaming system that
 - **Intelligent Caching** - Optimized streaming performance
 - **Thumbnail Generation** - Extract preview images from videos
 - **Modular Architecture** - Clean separation of concerns
+- **No Authentication Required** - Open access for internal network use
+- **CORS Enabled** - Ready for web browser integration

 ## 🏗️ Architecture

@@ -30,11 +32,16 @@ usda_vision_system/video/
 GET /videos/
 ```
 **Query Parameters:**
- `camera_name` - Filter by camera
- `start_date` - Filter by date range
- `end_date` - Filter by date range  
- `limit` - Maximum results (default: 50)
- `include_metadata` - Include video metadata
+- `camera_name` (optional): Filter by camera name
+- `start_date` (optional): Filter videos created after this date (ISO format: 2025-08-04T14:30:22)
+- `end_date` (optional): Filter videos created before this date (ISO format: 2025-08-04T14:30:22)
+- `limit` (optional): Maximum results (default: 50, max: 1000)
+- `include_metadata` (optional): Include video metadata (default: false)
+
+**Example Request:**
+```bash
+curl "http://localhost:8000/videos/?camera_name=camera1&include_metadata=true&limit=10"
+```

 **Response:**
 ```json
@@ -48,8 +55,20 @@ GET /videos/
      "format": "mp4",
      "status": "completed",
      "created_at": "2025-08-04T14:30:22",
+      "start_time": "2025-08-04T14:30:22",
+      "end_time": "2025-08-04T14:32:22",
+      "machine_trigger": "blower_separator",
      "is_streamable": true,
-      "needs_conversion": true
+      "needs_conversion": false,
+      "metadata": {
+        "duration_seconds": 120.5,
+        "width": 1920,
+        "height": 1080,
+        "fps": 30.0,
+        "codec": "mp4v",
+        "bitrate": 5000000,
+        "aspect_ratio": 1.777
+      }
    }
  ],
  "total_count": 1
@@ -61,28 +80,63 @@ GET /videos/
 GET /videos/{file_id}/stream
 ```
 **Headers:**
- `Range: bytes=0-1023` - Request specific byte range
+- `Range: bytes=0-1023` (optional): Request specific byte range for seeking
+
+**Example Requests:**
+```bash
+# Stream entire video
+curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi/stream
+
+# Stream specific byte range (for seeking)
+curl -H "Range: bytes=0-1023" \
+  http://localhost:8000/videos/camera1_recording_20250804_143022.avi/stream
+```
+
+**Response Headers:**
+- `Accept-Ranges: bytes`
+- `Content-Length: {size}`
+- `Content-Range: bytes {start}-{end}/{total}` (for range requests)
+- `Cache-Control: public, max-age=3600`
+- `Content-Type: video/mp4` or `video/x-msvideo`

 **Features:**
- Supports HTTP range requests for seeking
- Returns 206 Partial Content for range requests
- Automatic format conversion for web compatibility
- Intelligent caching for performance
+- ✅ **HTTP Range Requests**: Enables video seeking and progressive download
+- ✅ **Partial Content**: Returns 206 status for range requests
+- ✅ **Format Conversion**: Automatic AVI to MP4 conversion for web compatibility
+- ✅ **Intelligent Caching**: Byte-range caching for optimal performance
+- ✅ **CORS Enabled**: Ready for web browser integration

 ### Get Video Info
 ```http
 GET /videos/{file_id}
 ```
-**Response includes metadata:**
+**Example Request:**
+```bash
+curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi
+```
+
+**Response includes complete metadata:**
 ```json
 {
  "file_id": "camera1_recording_20250804_143022.avi",
+  "camera_name": "camera1",
+  "filename": "camera1_recording_20250804_143022.avi",
+  "file_size_bytes": 52428800,
+  "format": "avi",
+  "status": "completed",
+  "created_at": "2025-08-04T14:30:22",
+  "start_time": "2025-08-04T14:30:22",
+  "end_time": "2025-08-04T14:32:22",
+  "machine_trigger": "vibratory_conveyor",
+  "is_streamable": true,
+  "needs_conversion": true,
  "metadata": {
    "duration_seconds": 120.5,
    "width": 1920,
    "height": 1080,
    "fps": 30.0,
    "codec": "XVID",
+    "bitrate": 5000000,
    "aspect_ratio": 1.777
  }
 }
@@ -92,13 +146,31 @@ GET /videos/{file_id}
 ```http
 GET /videos/{file_id}/thumbnail?timestamp=5.0&width=320&height=240
 ```
-Returns JPEG thumbnail image.
+**Query Parameters:**
+- `timestamp` (optional): Time position in seconds to extract thumbnail from (default: 1.0)
+- `width` (optional): Thumbnail width in pixels (default: 320)
+- `height` (optional): Thumbnail height in pixels (default: 240)
+
+**Example Request:**
+```bash
+curl "http://localhost:8000/videos/camera1_recording_20250804_143022.avi/thumbnail?timestamp=5.0&width=320&height=240" \
+  --output thumbnail.jpg
+```
+
+**Response**: JPEG image data with caching headers
+- `Content-Type: image/jpeg`
+- `Cache-Control: public, max-age=3600`

 ### Streaming Info
 ```http
 GET /videos/{file_id}/info
 ```
-Returns technical streaming details:
+**Example Request:**
+```bash
+curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi/info
+```
+
+**Response**: Technical streaming details
 ```json
 {
  "file_id": "camera1_recording_20250804_143022.avi",
@@ -109,6 +181,58 @@ Returns technical streaming details:
 }
 ```

+### Video Validation
+```http
+POST /videos/{file_id}/validate
+```
+**Example Request:**
+```bash
+curl -X POST http://localhost:8000/videos/camera1_recording_20250804_143022.avi/validate
+```
+
+**Response**: Validation status
+```json
+{
+  "file_id": "camera1_recording_20250804_143022.avi",
+  "is_valid": true
+}
+```
+
+### Cache Management
+```http
+POST /videos/{file_id}/cache/invalidate
+```
+**Example Request:**
+```bash
+curl -X POST http://localhost:8000/videos/camera1_recording_20250804_143022.avi/cache/invalidate
+```
+
+**Response**: Cache invalidation status
+```json
+{
+  "file_id": "camera1_recording_20250804_143022.avi",
+  "cache_invalidated": true
+}
+```
+
+### Admin: Cache Cleanup
+```http
+POST /admin/videos/cache/cleanup?max_size_mb=100
+```
+**Example Request:**
+```bash
+curl -X POST "http://localhost:8000/admin/videos/cache/cleanup?max_size_mb=100"
+```
+
+**Response**: Cache cleanup results
+```json
+{
+  "cache_cleaned": true,
+  "entries_removed": 15,
+  "max_size_mb": 100
+}
+```
+
 ## 🌐 React Integration

 ### Basic Video Player
@@ -187,6 +311,101 @@ video_module = create_video_module(
 )
 ```

+### Configuration Parameters
+- **`enable_caching`**: Enable/disable intelligent byte-range caching (default: True)
+- **`cache_size_mb`**: Maximum cache size in MB (default: 100)
+- **`cache_max_age_minutes`**: Cache entry expiration time (default: 30)
+- **`enable_conversion`**: Enable/disable automatic AVI to MP4 conversion (default: True)
+- **`conversion_quality`**: Video conversion quality: "low", "medium", "high" (default: "medium")
+
+### System Requirements
+- **OpenCV**: Required for thumbnail generation and metadata extraction
+- **FFmpeg**: Optional, for video format conversion (graceful fallback if not available)
+- **Storage**: Sufficient disk space for video files and cache
+- **Memory**: Recommended 2GB+ RAM for caching and video processing
+
+## 🔐 Authentication & Security
+
+### Current Security Model
+**⚠️ IMPORTANT: No authentication is currently implemented.**
+
+- **Open Access**: All video streaming endpoints are publicly accessible
+- **CORS Policy**: Currently set to allow all origins (`allow_origins=["*"]`)
+- **Network Security**: Designed for internal network use only
+- **No API Keys**: No authentication tokens or API keys required
+- **No Rate Limiting**: No request rate limiting currently implemented
+
+### Security Considerations for Production
+
+#### For Internal Network Deployment
+```bash
+# Current configuration is suitable for:
+# - Internal corporate networks
+# - Isolated network segments
+# - Development and testing environments
+```
+
+#### For External Access (Recommendations)
+If you need to expose the video streaming API externally, consider implementing:
+
+1. **Authentication Layer**
+   ```python
+   # Example: Add JWT authentication
+   from fastapi import Depends, HTTPException
+   from fastapi.security import HTTPBearer
+
+   security = HTTPBearer()
+
+   async def verify_token(token: str = Depends(security)):
+       # Implement token verification logic
+       pass
+   ```
+
+2. **CORS Configuration**
+   ```python
+   # Restrict CORS to specific domains
+   app.add_middleware(
+       CORSMiddleware,
+       allow_origins=["https://yourdomain.com"],
+       allow_credentials=True,
+       allow_methods=["GET", "POST"],
+       allow_headers=["*"]
+   )
+   ```
+
+3. **Rate Limiting**
+   ```python
+   # Example: Add rate limiting
+   from slowapi import Limiter
+
+   limiter = Limiter(key_func=get_remote_address)
+
+   @app.get("/videos/")
+   @limiter.limit("10/minute")
+   async def list_videos():
+       pass
+   ```
+
+4. **Network Security**
+   - Use HTTPS/TLS for encrypted communication
+   - Implement firewall rules to restrict access
+   - Consider VPN access for remote users
+   - Use reverse proxy (nginx) for additional security
+
+### Access Control Summary
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Current Access Model                     │
+├─────────────────────────────────────────────────────────────┤
+│ Authentication:     ❌ None                                 │
+│ Authorization:      ❌ None                                 │
+│ CORS:              ✅ Enabled (all origins)                │
+│ Rate Limiting:      ❌ None                                 │
+│ HTTPS:             ⚠️  Depends on deployment               │
+│ Network Security:   ⚠️  Firewall/VPN recommended           │
+└─────────────────────────────────────────────────────────────┘
+```
+
 ## 📊 Performance

 - **Caching**: Intelligent byte-range caching reduces disk I/O
@@ -226,18 +445,136 @@ PYTHONPATH=/home/alireza/USDA-vision-cameras python tests/test_video_module.py
 ## 🔍 Troubleshooting

 ### Video Not Playing
-1. Check if file exists: `GET /videos/{file_id}`
-2. Verify streaming info: `GET /videos/{file_id}/info`
-3. Test direct stream: `GET /videos/{file_id}/stream`
+1. **Check if file exists**: `GET /videos/{file_id}`
+   ```bash
+   curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi
+   ```
+2. **Verify streaming info**: `GET /videos/{file_id}/info`
+   ```bash
+   curl http://localhost:8000/videos/camera1_recording_20250804_143022.avi/info
+   ```
+3. **Test direct stream**: `GET /videos/{file_id}/stream`
+   ```bash
+   curl -I http://localhost:8000/videos/camera1_recording_20250804_143022.avi/stream
+   ```
+4. **Validate video file**: `POST /videos/{file_id}/validate`
+   ```bash
+   curl -X POST http://localhost:8000/videos/camera1_recording_20250804_143022.avi/validate
+   ```

 ### Performance Issues
-1. Check cache status: `GET /admin/videos/cache/cleanup`
-2. Monitor system resources
-3. Adjust cache size in configuration
+1. **Check cache status**: Clean up cache if needed
+   ```bash
+   curl -X POST "http://localhost:8000/admin/videos/cache/cleanup?max_size_mb=100"
+   ```
+2. **Monitor system resources**: Check CPU, memory, and disk usage
+3. **Adjust cache size**: Modify configuration parameters
+4. **Invalidate specific cache**: For updated files
+   ```bash
+   curl -X POST http://localhost:8000/videos/{file_id}/cache/invalidate
+   ```

 ### Format Issues
- AVI files are automatically converted to MP4 for web compatibility
- Conversion requires FFmpeg (optional, graceful fallback)
+- **AVI files**: Automatically converted to MP4 for web compatibility
+- **Conversion requires FFmpeg**: Optional dependency with graceful fallback
+- **Supported formats**: AVI (with conversion), MP4 (native), WebM (native)
+
+### Common HTTP Status Codes
+- **200**: Success - Video streamed successfully
+- **206**: Partial Content - Range request successful
+- **404**: Not Found - Video file doesn't exist or isn't streamable
+- **416**: Range Not Satisfiable - Invalid range request
+- **500**: Internal Server Error - Failed to read video data or generate thumbnail
+
+### Browser Compatibility
+- **Chrome/Chromium**: Full support for MP4 and range requests
+- **Firefox**: Full support for MP4 and range requests
+- **Safari**: Full support for MP4 and range requests
+- **Edge**: Full support for MP4 and range requests
+- **Mobile browsers**: Generally good support for MP4 streaming
+
+### Error Scenarios and Solutions
+
+#### Video File Issues
+```bash
+# Problem: Video not found (404)
+curl http://localhost:8000/videos/nonexistent_video.mp4
+# Response: {"detail": "Video nonexistent_video.mp4 not found"}
+# Solution: Verify file_id exists using list endpoint
+
+# Problem: Video not streamable
+curl http://localhost:8000/videos/corrupted_video.avi/stream
+# Response: {"detail": "Video corrupted_video.avi not found or not streamable"}
+# Solution: Use validation endpoint to check file integrity
+```
+
+#### Range Request Issues
+```bash
+# Problem: Invalid range request (416)
+curl -H "Range: bytes=999999999-" http://localhost:8000/videos/small_video.mp4/stream
+# Response: {"detail": "Invalid range request: Range exceeds file size"}
+# Solution: Check file size first using /info endpoint
+
+# Problem: Malformed range header
+curl -H "Range: invalid-range" http://localhost:8000/videos/video.mp4/stream
+# Response: {"detail": "Invalid range request: Malformed range header"}
+# Solution: Use proper range format: "bytes=start-end"
+```
+
+#### Thumbnail Generation Issues
+```bash
+# Problem: Thumbnail generation failed (404)
+curl http://localhost:8000/videos/audio_only.mp4/thumbnail
+# Response: {"detail": "Could not generate thumbnail for audio_only.mp4"}
+# Solution: Verify video has visual content and is not audio-only
+
+# Problem: Invalid timestamp
+curl "http://localhost:8000/videos/short_video.mp4/thumbnail?timestamp=999"
+# Response: Returns thumbnail from last available frame
+# Solution: Check video duration first using metadata
+```
+
+#### System Resource Issues
+```bash
+# Problem: Cache full or system overloaded (500)
+curl http://localhost:8000/videos/large_video.mp4/stream
+# Response: {"detail": "Failed to read video data"}
+# Solution: Clean cache or wait for system resources
+curl -X POST "http://localhost:8000/admin/videos/cache/cleanup?max_size_mb=50"
+```
+
+### Debugging Workflow
+```bash
+# Step 1: Check system health
+curl http://localhost:8000/health
+
+# Step 2: Verify video exists and get info
+curl http://localhost:8000/videos/your_video_id
+
+# Step 3: Check streaming capabilities
+curl http://localhost:8000/videos/your_video_id/info
+
+# Step 4: Validate video file
+curl -X POST http://localhost:8000/videos/your_video_id/validate
+
+# Step 5: Test basic streaming
+curl -I http://localhost:8000/videos/your_video_id/stream
+
+# Step 6: Test range request
+curl -I -H "Range: bytes=0-1023" http://localhost:8000/videos/your_video_id/stream
+```
+
+### Performance Monitoring
+```bash
+# Monitor cache usage
+curl -X POST "http://localhost:8000/admin/videos/cache/cleanup?max_size_mb=100"
+
+# Check system resources
+curl http://localhost:8000/system/status
+
+# Monitor video module status
+curl http://localhost:8000/videos/ | jq '.total_count'
+```

 ## 🎯 Next Steps

--- a/reindex_videos.py
+++ b/reindex_videos.py
@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+"""
+Video Reindexing Script for USDA Vision Camera System
+
+This script reindexes existing video files that have "unknown" status,
+updating them to "completed" status so they can be streamed.
+
+Usage:
+    python reindex_videos.py [--dry-run] [--camera CAMERA_NAME]
+"""
+
+import os
+import sys
+import argparse
+import logging
+from pathlib import Path
+from datetime import datetime
+
+# Add the project root to Python path
+project_root = Path(__file__).parent
+sys.path.insert(0, str(project_root))
+
+from usda_vision_system.core.config import Config
+from usda_vision_system.core.state_manager import StateManager
+from usda_vision_system.storage.manager import StorageManager
+
+
+def setup_logging():
+    """Setup logging configuration"""
+    logging.basicConfig(
+        level=logging.INFO,
+        format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+    )
+    return logging.getLogger(__name__)
+
+
+def reindex_videos(storage_manager: StorageManager, camera_name: str = None, dry_run: bool = False):
+    """
+    Reindex video files with unknown status
+    
+    Args:
+        storage_manager: StorageManager instance
+        camera_name: Optional camera name to filter by
+        dry_run: If True, only show what would be done without making changes
+    """
+    logger = logging.getLogger(__name__)
+    
+    logger.info(f"Starting video reindexing (dry_run={dry_run})")
+    if camera_name:
+        logger.info(f"Filtering by camera: {camera_name}")
+    
+    # Get all video files
+    files = storage_manager.get_recording_files(camera_name=camera_name)
+    
+    unknown_files = [f for f in files if f.get("status") == "unknown"]
+    
+    if not unknown_files:
+        logger.info("No files with 'unknown' status found")
+        return
+    
+    logger.info(f"Found {len(unknown_files)} files with 'unknown' status")
+    
+    updated_count = 0
+    
+    for file_info in unknown_files:
+        file_id = file_info["file_id"]
+        filename = file_info["filename"]
+        
+        logger.info(f"Processing: {file_id}")
+        logger.info(f"  File: {filename}")
+        logger.info(f"  Current status: {file_info['status']}")
+        
+        if not dry_run:
+            # Update the file index directly
+            if file_id not in storage_manager.file_index["files"]:
+                # File is not in index, add it
+                file_path = Path(filename)
+                if file_path.exists():
+                    stat = file_path.stat()
+                    file_mtime = datetime.fromtimestamp(stat.st_mtime)
+                    
+                    new_file_info = {
+                        "camera_name": file_info["camera_name"],
+                        "filename": filename,
+                        "file_id": file_id,
+                        "start_time": file_mtime.isoformat(),
+                        "end_time": file_mtime.isoformat(),  # Use file mtime as end time
+                        "file_size_bytes": stat.st_size,
+                        "duration_seconds": None,  # Will be extracted later if needed
+                        "machine_trigger": None,
+                        "status": "completed",  # Set to completed
+                        "created_at": file_mtime.isoformat()
+                    }
+                    
+                    storage_manager.file_index["files"][file_id] = new_file_info
+                    logger.info(f"  Added to index with status: completed")
+                    updated_count += 1
+                else:
+                    logger.warning(f"  File does not exist: {filename}")
+            else:
+                # File is in index but has unknown status, update it
+                storage_manager.file_index["files"][file_id]["status"] = "completed"
+                logger.info(f"  Updated status to: completed")
+                updated_count += 1
+        else:
+            logger.info(f"  Would update status to: completed")
+            updated_count += 1
+    
+    if not dry_run and updated_count > 0:
+        # Save the updated index
+        storage_manager._save_file_index()
+        logger.info(f"Saved updated file index")
+    
+    logger.info(f"Reindexing complete: {updated_count} files {'would be ' if dry_run else ''}updated")
+
+
+def main():
+    """Main function"""
+    parser = argparse.ArgumentParser(description="Reindex video files with unknown status")
+    parser.add_argument("--dry-run", action="store_true", 
+                       help="Show what would be done without making changes")
+    parser.add_argument("--camera", type=str, 
+                       help="Only process files for specific camera")
+    parser.add_argument("--log-level", choices=["DEBUG", "INFO", "WARNING", "ERROR"], 
+                       default="INFO", help="Set logging level")
+    
+    args = parser.parse_args()
+    
+    # Setup logging
+    logging.basicConfig(
+        level=getattr(logging, args.log_level),
+        format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+    )
+    logger = logging.getLogger(__name__)
+    
+    try:
+        # Initialize system components
+        logger.info("Initializing USDA Vision Camera System components...")
+        
+        config = Config()
+        state_manager = StateManager()
+        storage_manager = StorageManager(config, state_manager)
+        
+        logger.info("Components initialized successfully")
+        
+        # Run reindexing
+        reindex_videos(
+            storage_manager=storage_manager,
+            camera_name=args.camera,
+            dry_run=args.dry_run
+        )
+        
+        if args.dry_run:
+            logger.info("Dry run completed. Use --no-dry-run to apply changes.")
+        else:
+            logger.info("Reindexing completed successfully!")
+            logger.info("Videos should now be streamable through the API.")
+        
+    except Exception as e:
+        logger.error(f"Error during reindexing: {e}")
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()