Add comprehensive video streaming module and AI agent integration guide
- Introduced a new video streaming module with endpoints for listing, streaming, and managing video files. - Added detailed API documentation for video streaming, including features like HTTP range requests, thumbnail generation, and caching. - Created a new AI Agent Video Integration Guide with step-by-step instructions for integrating with the video streaming API. - Implemented a video reindexing script to update video statuses from "unknown" to "completed". - Enhanced error handling and logging throughout the video streaming and reindexing processes.
This commit is contained in:
Alireza Vaezi
415
docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md
Normal file
415
docs/AI_AGENT_VIDEO_INTEGRATION_GUIDE.md
Normal file
@@ -0,0 +1,415 @@
|
||||
# 🤖 AI Agent Video Integration Guide
|
||||
|
||||
This guide provides comprehensive step-by-step instructions for AI agents and external systems to successfully integrate with the USDA Vision Camera System's video streaming functionality.
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
The USDA Vision Camera System provides a complete video streaming API that allows AI agents to:
|
||||
- Browse and select videos from multiple cameras
|
||||
- Stream videos with seeking capabilities
|
||||
- Generate thumbnails for preview
|
||||
- Access video metadata and technical information
|
||||
|
||||
## 🔗 API Base Configuration
|
||||
|
||||
### Connection Details
|
||||
```bash
|
||||
# Default API Base URL
|
||||
API_BASE_URL="http://localhost:8000"
|
||||
|
||||
# For remote access, replace with actual server IP/hostname
|
||||
API_BASE_URL="http://192.168.1.100:8000"
|
||||
```
|
||||
|
||||
### Authentication
|
||||
**⚠️ IMPORTANT: No authentication is currently required.**
|
||||
- All endpoints are publicly accessible
|
||||
- No API keys or tokens needed
|
||||
- CORS is enabled for web browser integration
|
||||
|
||||
## 📋 Step-by-Step Integration Workflow
|
||||
|
||||
### Step 1: Verify System Connectivity
|
||||
```bash
|
||||
# Test basic connectivity
|
||||
curl -f "${API_BASE_URL}/health" || echo "❌ System not accessible"
|
||||
|
||||
# Check system status
|
||||
curl "${API_BASE_URL}/system/status"
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"timestamp": "2025-08-05T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: List Available Videos
|
||||
```bash
|
||||
# Get all videos with metadata
|
||||
curl "${API_BASE_URL}/videos/?include_metadata=true&limit=50"
|
||||
|
||||
# Filter by specific camera
|
||||
curl "${API_BASE_URL}/videos/?camera_name=camera1&include_metadata=true"
|
||||
|
||||
# Filter by date range
|
||||
curl "${API_BASE_URL}/videos/?start_date=2025-08-04T00:00:00&end_date=2025-08-05T23:59:59"
|
||||
```
|
||||
|
||||
**Response Structure:**
|
||||
```json
|
||||
{
|
||||
"videos": [
|
||||
{
|
||||
"file_id": "camera1_auto_blower_separator_20250804_143022.mp4",
|
||||
"camera_name": "camera1",
|
||||
"filename": "camera1_auto_blower_separator_20250804_143022.mp4",
|
||||
"file_size_bytes": 31457280,
|
||||
"format": "mp4",
|
||||
"status": "completed",
|
||||
"created_at": "2025-08-04T14:30:22",
|
||||
"start_time": "2025-08-04T14:30:22",
|
||||
"end_time": "2025-08-04T14:32:22",
|
||||
"machine_trigger": "blower_separator",
|
||||
"is_streamable": true,
|
||||
"needs_conversion": false,
|
||||
"metadata": {
|
||||
"duration_seconds": 120.5,
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"fps": 30.0,
|
||||
"codec": "mp4v",
|
||||
"bitrate": 5000000,
|
||||
"aspect_ratio": 1.777
|
||||
}
|
||||
}
|
||||
],
|
||||
"total_count": 1
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Select and Validate Video
|
||||
```bash
|
||||
# Get detailed video information
|
||||
FILE_ID="camera1_auto_blower_separator_20250804_143022.mp4"
|
||||
curl "${API_BASE_URL}/videos/${FILE_ID}"
|
||||
|
||||
# Validate video is playable
|
||||
curl -X POST "${API_BASE_URL}/videos/${FILE_ID}/validate"
|
||||
|
||||
# Get streaming technical details
|
||||
curl "${API_BASE_URL}/videos/${FILE_ID}/info"
|
||||
```
|
||||
|
||||
### Step 4: Generate Video Thumbnail
|
||||
```bash
|
||||
# Generate thumbnail at 5 seconds, 320x240 resolution
|
||||
curl "${API_BASE_URL}/videos/${FILE_ID}/thumbnail?timestamp=5.0&width=320&height=240" \
|
||||
--output "thumbnail_${FILE_ID}.jpg"
|
||||
|
||||
# Generate multiple thumbnails for preview
|
||||
for timestamp in 1 30 60 90; do
|
||||
curl "${API_BASE_URL}/videos/${FILE_ID}/thumbnail?timestamp=${timestamp}&width=160&height=120" \
|
||||
--output "preview_${timestamp}s.jpg"
|
||||
done
|
||||
```
|
||||
|
||||
### Step 5: Stream Video Content
|
||||
```bash
|
||||
# Stream entire video
|
||||
curl "${API_BASE_URL}/videos/${FILE_ID}/stream" --output "video.mp4"
|
||||
|
||||
# Stream specific byte range (for seeking)
|
||||
curl -H "Range: bytes=0-1048575" \
|
||||
"${API_BASE_URL}/videos/${FILE_ID}/stream" \
|
||||
--output "video_chunk.mp4"
|
||||
|
||||
# Test range request support
|
||||
curl -I -H "Range: bytes=0-1023" \
|
||||
"${API_BASE_URL}/videos/${FILE_ID}/stream"
|
||||
```
|
||||
|
||||
## 🔧 Programming Language Examples
|
||||
|
||||
### Python Integration
|
||||
```python
|
||||
import requests
|
||||
import json
|
||||
from typing import List, Dict, Optional
|
||||
|
||||
class USDAVideoClient:
|
||||
def __init__(self, base_url: str = "http://localhost:8000"):
|
||||
self.base_url = base_url.rstrip('/')
|
||||
self.session = requests.Session()
|
||||
|
||||
def list_videos(self, camera_name: Optional[str] = None,
|
||||
include_metadata: bool = True, limit: int = 50) -> Dict:
|
||||
"""List available videos with optional filtering."""
|
||||
params = {
|
||||
'include_metadata': include_metadata,
|
||||
'limit': limit
|
||||
}
|
||||
if camera_name:
|
||||
params['camera_name'] = camera_name
|
||||
|
||||
response = self.session.get(f"{self.base_url}/videos/", params=params)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
def get_video_info(self, file_id: str) -> Dict:
|
||||
"""Get detailed video information."""
|
||||
response = self.session.get(f"{self.base_url}/videos/{file_id}")
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
def get_thumbnail(self, file_id: str, timestamp: float = 1.0,
|
||||
width: int = 320, height: int = 240) -> bytes:
|
||||
"""Generate and download video thumbnail."""
|
||||
params = {
|
||||
'timestamp': timestamp,
|
||||
'width': width,
|
||||
'height': height
|
||||
}
|
||||
response = self.session.get(
|
||||
f"{self.base_url}/videos/{file_id}/thumbnail",
|
||||
params=params
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.content
|
||||
|
||||
def stream_video_range(self, file_id: str, start_byte: int,
|
||||
end_byte: int) -> bytes:
|
||||
"""Stream specific byte range of video."""
|
||||
headers = {'Range': f'bytes={start_byte}-{end_byte}'}
|
||||
response = self.session.get(
|
||||
f"{self.base_url}/videos/{file_id}/stream",
|
||||
headers=headers
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.content
|
||||
|
||||
def validate_video(self, file_id: str) -> bool:
|
||||
"""Validate that video is accessible and playable."""
|
||||
response = self.session.post(f"{self.base_url}/videos/{file_id}/validate")
|
||||
response.raise_for_status()
|
||||
return response.json().get('is_valid', False)
|
||||
|
||||
# Usage example
|
||||
client = USDAVideoClient("http://192.168.1.100:8000")
|
||||
|
||||
# List videos from camera1
|
||||
videos = client.list_videos(camera_name="camera1")
|
||||
print(f"Found {videos['total_count']} videos")
|
||||
|
||||
# Select first video
|
||||
if videos['videos']:
|
||||
video = videos['videos'][0]
|
||||
file_id = video['file_id']
|
||||
|
||||
# Validate video
|
||||
if client.validate_video(file_id):
|
||||
print(f"✅ Video {file_id} is valid")
|
||||
|
||||
# Get thumbnail
|
||||
thumbnail = client.get_thumbnail(file_id, timestamp=5.0)
|
||||
with open(f"thumbnail_{file_id}.jpg", "wb") as f:
|
||||
f.write(thumbnail)
|
||||
|
||||
# Stream first 1MB
|
||||
chunk = client.stream_video_range(file_id, 0, 1048575)
|
||||
print(f"Downloaded {len(chunk)} bytes")
|
||||
```
|
||||
|
||||
### JavaScript/Node.js Integration
|
||||
```javascript
|
||||
class USDAVideoClient {
|
||||
constructor(baseUrl = 'http://localhost:8000') {
|
||||
this.baseUrl = baseUrl.replace(/\/$/, '');
|
||||
}
|
||||
|
||||
async listVideos(options = {}) {
|
||||
const params = new URLSearchParams({
|
||||
include_metadata: options.includeMetadata || true,
|
||||
limit: options.limit || 50
|
||||
});
|
||||
|
||||
if (options.cameraName) {
|
||||
params.append('camera_name', options.cameraName);
|
||||
}
|
||||
|
||||
const response = await fetch(`${this.baseUrl}/videos/?${params}`);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
return response.json();
|
||||
}
|
||||
|
||||
async getVideoInfo(fileId) {
|
||||
const response = await fetch(`${this.baseUrl}/videos/${fileId}`);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
return response.json();
|
||||
}
|
||||
|
||||
async getThumbnail(fileId, options = {}) {
|
||||
const params = new URLSearchParams({
|
||||
timestamp: options.timestamp || 1.0,
|
||||
width: options.width || 320,
|
||||
height: options.height || 240
|
||||
});
|
||||
|
||||
const response = await fetch(
|
||||
`${this.baseUrl}/videos/${fileId}/thumbnail?${params}`
|
||||
);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
return response.blob();
|
||||
}
|
||||
|
||||
async validateVideo(fileId) {
|
||||
const response = await fetch(
|
||||
`${this.baseUrl}/videos/${fileId}/validate`,
|
||||
{ method: 'POST' }
|
||||
);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
const result = await response.json();
|
||||
return result.is_valid;
|
||||
}
|
||||
|
||||
getStreamUrl(fileId) {
|
||||
return `${this.baseUrl}/videos/${fileId}/stream`;
|
||||
}
|
||||
}
|
||||
|
||||
// Usage example
|
||||
const client = new USDAVideoClient('http://192.168.1.100:8000');
|
||||
|
||||
async function integrateWithVideos() {
|
||||
try {
|
||||
// List videos
|
||||
const videos = await client.listVideos({ cameraName: 'camera1' });
|
||||
console.log(`Found ${videos.total_count} videos`);
|
||||
|
||||
if (videos.videos.length > 0) {
|
||||
const video = videos.videos[0];
|
||||
const fileId = video.file_id;
|
||||
|
||||
// Validate video
|
||||
const isValid = await client.validateVideo(fileId);
|
||||
if (isValid) {
|
||||
console.log(`✅ Video ${fileId} is valid`);
|
||||
|
||||
// Get thumbnail
|
||||
const thumbnail = await client.getThumbnail(fileId, {
|
||||
timestamp: 5.0,
|
||||
width: 320,
|
||||
height: 240
|
||||
});
|
||||
|
||||
// Create video element for playback
|
||||
const videoElement = document.createElement('video');
|
||||
videoElement.controls = true;
|
||||
videoElement.src = client.getStreamUrl(fileId);
|
||||
document.body.appendChild(videoElement);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Integration error:', error);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🚨 Error Handling
|
||||
|
||||
### Common HTTP Status Codes
|
||||
```bash
|
||||
# Success responses
|
||||
200 # OK - Request successful
|
||||
206 # Partial Content - Range request successful
|
||||
|
||||
# Client error responses
|
||||
400 # Bad Request - Invalid parameters
|
||||
404 # Not Found - Video file doesn't exist
|
||||
416 # Range Not Satisfiable - Invalid range request
|
||||
|
||||
# Server error responses
|
||||
500 # Internal Server Error - Failed to process video
|
||||
503 # Service Unavailable - Video module not available
|
||||
```
|
||||
|
||||
### Error Response Format
|
||||
```json
|
||||
{
|
||||
"detail": "Video camera1_recording_20250804_143022.avi not found"
|
||||
}
|
||||
```
|
||||
|
||||
### Robust Error Handling Example
|
||||
```python
|
||||
def safe_video_operation(client, file_id):
|
||||
try:
|
||||
# Validate video first
|
||||
if not client.validate_video(file_id):
|
||||
return {"error": "Video is not valid or accessible"}
|
||||
|
||||
# Get video info
|
||||
video_info = client.get_video_info(file_id)
|
||||
|
||||
# Check if streamable
|
||||
if not video_info.get('is_streamable', False):
|
||||
return {"error": "Video is not streamable"}
|
||||
|
||||
return {"success": True, "video_info": video_info}
|
||||
|
||||
except requests.exceptions.HTTPError as e:
|
||||
if e.response.status_code == 404:
|
||||
return {"error": "Video not found"}
|
||||
elif e.response.status_code == 416:
|
||||
return {"error": "Invalid range request"}
|
||||
else:
|
||||
return {"error": f"HTTP error: {e.response.status_code}"}
|
||||
except requests.exceptions.ConnectionError:
|
||||
return {"error": "Cannot connect to video server"}
|
||||
except Exception as e:
|
||||
return {"error": f"Unexpected error: {str(e)}"}
|
||||
```
|
||||
|
||||
## ✅ Integration Checklist
|
||||
|
||||
### Pre-Integration
|
||||
- [ ] Verify network connectivity to USDA Vision Camera System
|
||||
- [ ] Test basic API endpoints (`/health`, `/system/status`)
|
||||
- [ ] Understand video file naming conventions
|
||||
- [ ] Plan error handling strategy
|
||||
|
||||
### Video Selection
|
||||
- [ ] Implement video listing with appropriate filters
|
||||
- [ ] Add video validation before processing
|
||||
- [ ] Handle pagination for large video collections
|
||||
- [ ] Implement caching for video metadata
|
||||
|
||||
### Video Playback
|
||||
- [ ] Test video streaming with range requests
|
||||
- [ ] Implement thumbnail generation for previews
|
||||
- [ ] Add progress tracking for video playback
|
||||
- [ ] Handle different video formats (MP4, AVI)
|
||||
|
||||
### Error Handling
|
||||
- [ ] Handle network connectivity issues
|
||||
- [ ] Manage video not found scenarios
|
||||
- [ ] Deal with invalid range requests
|
||||
- [ ] Implement retry logic for transient failures
|
||||
|
||||
### Performance
|
||||
- [ ] Use range requests for efficient seeking
|
||||
- [ ] Implement client-side caching where appropriate
|
||||
- [ ] Monitor bandwidth usage for video streaming
|
||||
- [ ] Consider thumbnail caching for better UX
|
||||
|
||||
## 🎯 Next Steps
|
||||
|
||||
1. **Test Integration**: Use the provided examples to test basic connectivity
|
||||
2. **Implement Error Handling**: Add robust error handling for production use
|
||||
3. **Optimize Performance**: Implement caching and efficient streaming
|
||||
4. **Monitor Usage**: Track API usage and performance metrics
|
||||
5. **Security Review**: Consider authentication if exposing externally
|
||||
|
||||
This guide provides everything needed for successful integration with the USDA Vision Camera System's video streaming functionality. The system is designed to be simple and reliable for AI agents and external systems to consume video content efficiently.
|
||||
Reference in New Issue
Block a user