diff --git a/SESSION_SUMMARY.md b/SESSION_SUMMARY.md new file mode 100644 index 0000000..209f109 --- /dev/null +++ b/SESSION_SUMMARY.md @@ -0,0 +1,251 @@ +# Session Summary: MQTT Auto-Recording Debugging & Theme Implementation + +## Overview +This session focused on debugging MQTT-triggered auto-recording functionality and implementing dark/light theme toggle in the management dashboard. Multiple issues were identified and resolved related to camera recording, MQTT message handling, and UI theming. + +--- + +## Major Tasks Completed + +### 1. Scheduling Module Extraction (Previous Session) +- Extracted the Scheduling module into a separate microfrontend (`scheduling-remote`) +- Configured Module Federation for the new microfrontend +- Updated `docker-compose.yml` to include the new service + +### 2. Dark/Light Theme Toggle Implementation +- Created `useTheme` hook (`management-dashboard-web-app/src/hooks/useTheme.ts`) +- Added theme toggle button to `TopNavbar.tsx` +- Applied dark mode classes across multiple components: + - `DashboardLayout.tsx` - Main content area + - `Login.tsx` - Login page background + - `DataEntry.tsx` - Widgets, text, borders + - `UserManagement.tsx` - Stats cards, tables, forms + - `ExperimentPhases.tsx` - Phase cards, badges + - `index.css` - Body background + - `index.html` - Inline script to prevent FOUC (Flash of Unstyled Content) + +### 3. MQTT Auto-Recording Debugging + +#### Issues Identified: +1. **Manual recording buttons not working** - "Record" buttons for cameras couldn't start manual recording +2. **Automatic recording unavailable** - MQTT-triggered auto-recording wasn't functioning +3. **Camera2 initialization error** - Error code 32774 when initializing camera2 +4. **Button visibility issues** - "Restart Camera" and "Stop Streaming" buttons appeared transparent + +#### Fixes Applied: + +##### A. Transparent Buttons Fix +**File:** `vision-system-remote/src/components/CameraCard.tsx` +- Added inline `style` attributes to "Restart Camera" and "Stop Streaming" buttons +- Explicitly set `backgroundColor` and `color` to prevent transparency + +##### B. Camera Initialization Error (32774) +**File:** `camera-management-api/usda_vision_system/camera/monitor.py` +- Enhanced error handling for camera initialization error code 32774 +- Added logic to check if camera is already in use by existing recorder/streamer +- If error 32774 occurs, camera is marked as "available" with warning instead of failing completely +- Provides better diagnostics for cameras that might be in use by another process + +##### C. StandaloneAutoRecorder Integration +**File:** `camera-management-api/usda_vision_system/recording/standalone_auto_recorder.py` +- **Problem:** `StandaloneAutoRecorder` was creating its own MQTT client and camera recorders, causing conflicts +- **Solution:** + - Modified constructor to accept `camera_manager`, `state_manager`, and `event_system` instances + - Removed internal MQTT client setup when using shared instances + - Now subscribes to `EventType.MACHINE_STATE_CHANGED` events from the event system + - Uses `camera_manager.manual_start_recording()` and `manual_stop_recording()` instead of creating its own recorders + - Added comprehensive logging for event reception and recording operations + +**File:** `camera-management-api/usda_vision_system/main.py` +- Updated `USDAVisionSystem` constructor to pass `camera_manager`, `state_manager`, and `event_system` to `StandaloneAutoRecorder` + +##### D. Missing Constants Import +**File:** `camera-management-api/usda_vision_system/camera/recorder.py` +- **Error:** `NameError: name 'CAMERA_TEST_CAPTURE_TIMEOUT' is not defined` +- **Fix:** Added missing imports: + - `CAMERA_GET_BUFFER_TIMEOUT` + - `CAMERA_INIT_TIMEOUT` + - `CAMERA_TEST_CAPTURE_TIMEOUT` + - `DEFAULT_VIDEO_FPS` + - `BRIEF_PAUSE_SLEEP` + - All imported from `.constants` + +##### E. Recording Routes Enhancement +**File:** `camera-management-api/usda_vision_system/api/routes/recording_routes.py` +- Made `StartRecordingRequest` parameter optional for `start_recording` endpoint +- Added logic to create default `StartRecordingRequest` if `request` is `None` +- Added INFO-level logging for recording attempts and success/failure +- Improved error messages for `manual_start_recording` failures + +##### F. MQTT Logging Improvements +**Files:** +- `camera-management-api/usda_vision_system/mqtt/client.py` +- `camera-management-api/usda_vision_system/mqtt/handlers.py` + +**Changes:** +- Changed MQTT message logging from `DEBUG` to `INFO` level for visibility in production +- Added detailed logging for: + - Message reception: `๐ก MQTT MESSAGE RECEIVED - Topic: {topic}, Payload: '{payload}'` + - Message processing: `๐ก Processing MQTT message for machine '{machine_name}': '{payload}'` + - Payload normalization: `๐ก Normalized payload '{payload}' -> '{normalized_payload}'` + - Event publishing: `๐ก Publishing MACHINE_STATE_CHANGED event` + - Connection status: `๐ MQTT CONNECTED` with broker details + - Subscription confirmation: `๐ MQTT subscribed to {count} topics` + +##### G. MQTT Test Script +**File:** `camera-management-api/test_mqtt_simple.py` (NEW) +- Created standalone MQTT test script to verify connectivity and message reception +- Connects to MQTT broker and subscribes to configured topics +- Displays received messages in real-time with timestamps +- Validates payload format (on/off/true/false/1/0) +- Shows connection statistics + +### 4. Configuration Updates +**File:** `camera-management-api/config.json` +- Confirmed `log_level` is set to `INFO` +- `auto_recording_enabled` is set to `true` + +--- + +## Key Files Modified + +### Frontend (React/TypeScript) +1. `management-dashboard-web-app/src/hooks/useTheme.ts` - NEW +2. `management-dashboard-web-app/src/components/TopNavbar.tsx` +3. `management-dashboard-web-app/src/components/DashboardLayout.tsx` +4. `management-dashboard-web-app/src/components/Login.tsx` +5. `management-dashboard-web-app/src/components/DataEntry.tsx` +6. `management-dashboard-web-app/src/components/UserManagement.tsx` +7. `management-dashboard-web-app/src/components/ExperimentPhases.tsx` +8. `management-dashboard-web-app/src/index.css` +9. `management-dashboard-web-app/index.html` +10. `vision-system-remote/src/components/CameraCard.tsx` + +### Backend (Python/FastAPI) +1. `camera-management-api/usda_vision_system/recording/standalone_auto_recorder.py` +2. `camera-management-api/usda_vision_system/main.py` +3. `camera-management-api/usda_vision_system/camera/recorder.py` +4. `camera-management-api/usda_vision_system/camera/monitor.py` +5. `camera-management-api/usda_vision_system/api/routes/recording_routes.py` +6. `camera-management-api/usda_vision_system/mqtt/client.py` +7. `camera-management-api/usda_vision_system/mqtt/handlers.py` +8. `camera-management-api/test_mqtt_simple.py` - NEW + +--- + +## Current Status + +### โ Completed +- Dark/light theme toggle fully implemented +- Transparent buttons fixed +- Camera initialization error handling improved +- `StandaloneAutoRecorder` properly integrated with shared instances +- Missing constants imported +- Recording routes enhanced with better logging +- MQTT logging improved to INFO level +- MQTT test script created + +### ๐ Testing Needed +- **MQTT Auto-Recording**: Needs verification that MQTT messages trigger recording + - Test script available: `camera-management-api/test_mqtt_simple.py` + - Monitor logs for MQTT message flow: `docker compose logs api -f | grep -i "mqtt\|๐ก"` + - Check MQTT status: `curl http://localhost:8000/mqtt/status` + - Check recent events: `curl http://localhost:8000/mqtt/events?limit=10` + +### ๐ Expected MQTT Message Flow +When a machine turns on/off, the following should appear in logs: +1. `๐ก MQTT MESSAGE RECEIVED` - Message received from broker +2. `๐ก Processing MQTT message` - Message being processed +3. `๐ก Normalized payload` - Payload normalization +4. `โ Published MACHINE_STATE_CHANGED event` - Event published to event system +5. `๐ก AUTO-RECORDER: Received MACHINE_STATE_CHANGED event` - Auto-recorder received event +6. `โ Started auto-recording` - Recording started + +--- + +## MQTT Configuration + +From `config.json`: +```json +{ + "mqtt": { + "broker_host": "192.168.1.110", + "broker_port": 1883, + "username": null, + "password": null, + "topics": { + "vibratory_conveyor": "vision/vibratory_conveyor/state", + "blower_separator": "vision/blower_separator/state" + } + } +} +``` + +Machine-to-Camera Mapping (from `standalone_auto_recorder.py`): +- `vibratory_conveyor` โ `camera2` +- `blower_separator` โ `camera1` + +--- + +## Testing Commands + +### Test MQTT Connectivity +```bash +# Option 1: Standalone test script +docker compose exec api python test_mqtt_simple.py + +# Option 2: Check MQTT status via API +curl -s http://localhost:8000/mqtt/status | python -m json.tool + +# Option 3: Monitor API logs for MQTT messages +docker compose logs api -f | grep -i "mqtt\|๐ก" + +# Option 4: Check recent MQTT events +curl -s http://localhost:8000/mqtt/events?limit=10 | python -m json.tool +``` + +### Restart API Container +```bash +docker compose restart api +``` + +--- + +## Architecture Notes + +### Event Flow for Auto-Recording +1. **MQTT Message Received** โ `MQTTClient._on_message()` +2. **Message Processed** โ `MQTTMessageHandler.handle_message()` +3. **State Updated** โ `StateManager.update_machine_state()` +4. **Event Published** โ `EventSystem.publish(EventType.MACHINE_STATE_CHANGED)` +5. **Event Received** โ `StandaloneAutoRecorder._on_machine_state_changed()` +6. **Recording Started/Stopped** โ `CameraManager.manual_start_recording()` / `manual_stop_recording()` + +### Key Components +- **MQTTClient**: Connects to MQTT broker, subscribes to topics, receives messages +- **MQTTMessageHandler**: Processes MQTT messages, normalizes payloads, updates state +- **EventSystem**: Publishes/subscribes to internal events +- **StandaloneAutoRecorder**: Subscribes to `MACHINE_STATE_CHANGED` events, triggers recording +- **CameraManager**: Manages camera operations including recording + +--- + +## Notes for Future Sessions + +1. **MQTT Testing**: If auto-recording still doesn't work, verify: + - MQTT broker is reachable from API container + - Topics match exactly (case-sensitive) + - Messages are being published to correct topics + - Payload format is recognized (on/off/true/false/1/0) + +2. **Camera Initialization**: Error 32774 typically means camera is in use. The system now handles this gracefully but may need investigation if cameras are not accessible. + +3. **Theme Persistence**: Theme preference is stored in `localStorage` and persists across sessions. + +4. **Log Level**: Currently set to `INFO` in `config.json` for comprehensive event tracking. + +--- + +## Session Date +Session completed with focus on MQTT debugging and enhanced logging for troubleshooting auto-recording functionality. + diff --git a/camera-management-api/config.json b/camera-management-api/config.json index 2bd1516..a9158a0 100644 --- a/camera-management-api/config.json +++ b/camera-management-api/config.json @@ -17,7 +17,7 @@ }, "system": { "camera_check_interval_seconds": 2, - "log_level": "WARNING", + "log_level": "INFO", "log_file": "usda_vision_system.log", "api_host": "0.0.0.0", "api_port": 8000, diff --git a/camera-management-api/test_mqtt_simple.py b/camera-management-api/test_mqtt_simple.py new file mode 100755 index 0000000..54939e9 --- /dev/null +++ b/camera-management-api/test_mqtt_simple.py @@ -0,0 +1,162 @@ +#!/usr/bin/env python3 +""" +Simple MQTT Test Script + +This script tests MQTT connectivity and message reception. +It connects to the broker and listens for messages on the configured topics. + +Usage: + python test_mqtt_simple.py +""" + +import paho.mqtt.client as mqtt +import time +import signal +import sys +from datetime import datetime + +# MQTT Configuration (from config.json) +MQTT_BROKER_HOST = "192.168.1.110" +MQTT_BROKER_PORT = 1883 +MQTT_USERNAME = None +MQTT_PASSWORD = None + +# Topics to monitor (from config.json) +MQTT_TOPICS = { + "vibratory_conveyor": "vision/vibratory_conveyor/state", + "blower_separator": "vision/blower_separator/state" +} + +class SimpleMQTTTester: + def __init__(self): + self.client = None + self.message_count = 0 + self.running = True + + def on_connect(self, client, userdata, flags, rc): + """Callback when client connects""" + if rc == 0: + print(f"โ Connected to MQTT broker: {MQTT_BROKER_HOST}:{MQTT_BROKER_PORT}") + + # Subscribe to all topics + for machine_name, topic in MQTT_TOPICS.items(): + result, mid = client.subscribe(topic) + if result == mqtt.MQTT_ERR_SUCCESS: + print(f"๐ Subscribed to: {topic} (machine: {machine_name})") + else: + print(f"โ Failed to subscribe to {topic}: {result}") + else: + print(f"โ Connection failed with return code {rc}") + + def on_disconnect(self, client, userdata, rc): + """Callback when client disconnects""" + if rc != 0: + print(f"โ ๏ธ Unexpected disconnection (rc: {rc})") + else: + print("๐ Disconnected from broker") + + def on_message(self, client, userdata, msg): + """Callback when a message is received""" + try: + topic = msg.topic + payload = msg.payload.decode("utf-8").strip() + timestamp = datetime.now().strftime('%H:%M:%S.%f')[:-3] + + self.message_count += 1 + + # Find machine name + machine_name = "unknown" + for name, configured_topic in MQTT_TOPICS.items(): + if topic == configured_topic: + machine_name = name + break + + # Display message + print(f"\n๐ก [{timestamp}] Message #{self.message_count}") + print(f" ๐ญ Machine: {machine_name}") + print(f" ๐ Topic: {topic}") + print(f" ๐ Payload: '{payload}'") + print(f" ๐ Total messages received: {self.message_count}") + + # Check if payload is valid on/off + payload_lower = payload.lower() + if payload_lower in ["on", "off", "true", "false", "1", "0"]: + state = "ON" if payload_lower in ["on", "true", "1"] else "OFF" + print(f" โ Valid state: {state}") + else: + print(f" โ ๏ธ Unusual payload format: '{payload}'") + print("-" * 60) + + except Exception as e: + print(f"โ Error processing message: {e}") + + def start(self): + """Start the MQTT tester""" + print("๐งช Starting MQTT Test") + print("=" * 60) + print(f"Broker: {MQTT_BROKER_HOST}:{MQTT_BROKER_PORT}") + print(f"Topics to monitor: {len(MQTT_TOPICS)}") + for name, topic in MQTT_TOPICS.items(): + print(f" - {name}: {topic}") + print("=" * 60) + print("\nWaiting for messages... (Press Ctrl+C to stop)\n") + + # Create client + self.client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION1) + self.client.on_connect = self.on_connect + self.client.on_disconnect = self.on_disconnect + self.client.on_message = self.on_message + + # Set authentication if provided + if MQTT_USERNAME and MQTT_PASSWORD: + self.client.username_pw_set(MQTT_USERNAME, MQTT_PASSWORD) + + # Connect + try: + self.client.connect(MQTT_BROKER_HOST, MQTT_BROKER_PORT, 60) + except Exception as e: + print(f"โ Failed to connect: {e}") + return False + + # Start loop + self.client.loop_start() + + # Wait for messages + try: + while self.running: + time.sleep(1) + except KeyboardInterrupt: + print("\n\n๐ Stopping test...") + self.running = False + + # Cleanup + self.client.loop_stop() + self.client.disconnect() + + print(f"\n๐ Test Summary:") + print(f" Total messages received: {self.message_count}") + print("โ Test completed") + + return True + + +def main(): + """Main entry point""" + tester = SimpleMQTTTester() + + # Setup signal handler for graceful shutdown + def signal_handler(sig, frame): + print("\n\n๐ Received interrupt signal, shutting down...") + tester.running = False + + signal.signal(signal.SIGINT, signal_handler) + signal.signal(signal.SIGTERM, signal_handler) + + success = tester.start() + sys.exit(0 if success else 1) + + +if __name__ == "__main__": + main() + + diff --git a/camera-management-api/usda_vision_system/api/routes/recording_routes.py b/camera-management-api/usda_vision_system/api/routes/recording_routes.py index ec9c1ab..2219bbf 100644 --- a/camera-management-api/usda_vision_system/api/routes/recording_routes.py +++ b/camera-management-api/usda_vision_system/api/routes/recording_routes.py @@ -4,6 +4,7 @@ Recording-related API routes. import logging from fastapi import FastAPI, HTTPException +from typing import Optional from ...camera.manager import CameraManager from ..models import StartRecordingResponse, StopRecordingResponse, StartRecordingRequest from ...core.timezone_utils import format_filename_timestamp @@ -17,12 +18,19 @@ def register_recording_routes( """Register recording-related routes""" @app.post("/cameras/{camera_name}/start-recording", response_model=StartRecordingResponse) - async def start_recording(camera_name: str, request: StartRecordingRequest): + async def start_recording(camera_name: str, request: Optional[StartRecordingRequest] = None): """Manually start recording for a camera""" try: if not camera_manager: + logger.error("Camera manager not available") raise HTTPException(status_code=503, detail="Camera manager not available") + # Handle case where request body might be None or empty + if request is None: + request = StartRecordingRequest() + + logger.info(f"๐น Starting recording for {camera_name} - filename: {request.filename}, exposure_ms: {request.exposure_ms}, gain: {request.gain}, fps: {request.fps}") + success = camera_manager.manual_start_recording( camera_name=camera_name, filename=request.filename, @@ -37,19 +45,28 @@ def register_recording_routes( if request.filename: timestamp = format_filename_timestamp() actual_filename = f"{timestamp}_{request.filename}" + else: + timestamp = format_filename_timestamp() + camera_config = camera_manager.get_camera_config(camera_name) + video_format = camera_config.video_format if camera_config else "mp4" + actual_filename = f"{camera_name}_manual_{timestamp}.{video_format}" + logger.info(f"โ Recording started successfully for {camera_name}: {actual_filename}") return StartRecordingResponse( success=True, message=f"Recording started for {camera_name}", filename=actual_filename ) else: + logger.error(f"โ Failed to start recording for {camera_name} - manual_start_recording returned False") return StartRecordingResponse( success=False, - message=f"Failed to start recording for {camera_name}" + message=f"Failed to start recording for {camera_name}. Check camera status and logs." ) + except HTTPException: + raise except Exception as e: - logger.error(f"Error starting recording: {e}") + logger.error(f"โ Error starting recording for {camera_name}: {e}", exc_info=True) raise HTTPException(status_code=500, detail=str(e)) @app.post("/cameras/{camera_name}/stop-recording", response_model=StopRecordingResponse) diff --git a/camera-management-api/usda_vision_system/camera/monitor.py b/camera-management-api/usda_vision_system/camera/monitor.py index a55f9f6..56ab7cb 100644 --- a/camera-management-api/usda_vision_system/camera/monitor.py +++ b/camera-management-api/usda_vision_system/camera/monitor.py @@ -188,6 +188,34 @@ class CameraMonitor: self.logger.info(f"Attempting to initialize camera {camera_name} for availability test...") + # Check if camera is already in use by recorder or streamer before trying to initialize + recorder = self.camera_manager.camera_recorders.get(camera_name) if self.camera_manager else None + streamer = self.camera_manager.camera_streamers.get(camera_name) if self.camera_manager else None + + camera_in_use = False + if recorder and recorder.hCamera: + try: + # Check if recorder has camera open + if mvsdk.CameraIsOpened(recorder.hCamera): + camera_in_use = True + self.logger.info(f"Camera {camera_name} is already in use by recorder (handle: {recorder.hCamera})") + except: + pass + + if not camera_in_use and streamer and streamer.hCamera: + try: + # Check if streamer has camera open + if mvsdk.CameraIsOpened(streamer.hCamera): + camera_in_use = True + self.logger.info(f"Camera {camera_name} is already in use by streamer (handle: {streamer.hCamera})") + except: + pass + + # If camera is already in use, mark as available (since it's working, just occupied) + if camera_in_use: + self.logger.info(f"Camera {camera_name} is in use by system components - marking as available") + return "available", "Camera is in use by system", self._get_device_info_dict(device_info) + # Suppress output to avoid MVCAMAPI error messages during camera testing hCamera = None try: @@ -195,7 +223,26 @@ class CameraMonitor: hCamera = mvsdk.CameraInit(device_info, -1, -1) self.logger.info(f"Camera {camera_name} initialized successfully, starting test capture...") except mvsdk.CameraException as init_e: - self.logger.warning(f"CameraInit failed for {camera_name}: {init_e.message} (error_code: {init_e.error_code})") + error_msg = f"CameraInit failed for {camera_name}: {init_e.message} (error_code: {init_e.error_code})" + + # Special handling for error code 32774 (camera already in use) + if init_e.error_code == 32774: + error_msg += " - Camera may be in use by another process or resource conflict. " + error_msg += "This camera may still be functional if accessed through existing recorder/streamer." + self.logger.warning(error_msg) + # Mark as "available" but with warning, since it might be usable through existing connections + # The UI can show a warning but camera operations might still work + try: + device_info_dict = self._get_device_info_dict(device_info) + device_info_dict["init_error"] = "Camera appears in use (error 32774) but may be accessible" + device_info_dict["init_error_code"] = 32774 + except Exception as dev_info_e: + self.logger.warning(f"Failed to get device info dict after CameraInit failure: {dev_info_e}") + device_info_dict = None + return "available", "Camera may be in use (error 32774) - check if recorder/streamer is active", device_info_dict + else: + self.logger.warning(error_msg) + # Get device info dict before returning - wrap in try/except in case device_info is corrupted try: device_info_dict = self._get_device_info_dict(device_info) diff --git a/camera-management-api/usda_vision_system/camera/recorder.py b/camera-management-api/usda_vision_system/camera/recorder.py index 67331c5..7e7a233 100644 --- a/camera-management-api/usda_vision_system/camera/recorder.py +++ b/camera-management-api/usda_vision_system/camera/recorder.py @@ -26,6 +26,13 @@ from ..core.events import EventSystem, publish_recording_started, publish_record from ..core.timezone_utils import now_atlanta, format_filename_timestamp from .sdk_config import ensure_sdk_initialized from .utils import suppress_camera_errors +from .constants import ( + CAMERA_GET_BUFFER_TIMEOUT, + CAMERA_INIT_TIMEOUT, + CAMERA_TEST_CAPTURE_TIMEOUT, + DEFAULT_VIDEO_FPS, + BRIEF_PAUSE_SLEEP, +) class CameraRecorder: diff --git a/camera-management-api/usda_vision_system/main.py b/camera-management-api/usda_vision_system/main.py index 197c57f..ae4688d 100644 --- a/camera-management-api/usda_vision_system/main.py +++ b/camera-management-api/usda_vision_system/main.py @@ -46,7 +46,12 @@ class USDAVisionSystem: self.storage_manager = StorageManager(self.config, self.state_manager, self.event_system) self.mqtt_client = MQTTClient(self.config, self.state_manager, self.event_system) self.camera_manager = CameraManager(self.config, self.state_manager, self.event_system) - self.auto_recording_manager = StandaloneAutoRecorder(config=self.config) + self.auto_recording_manager = StandaloneAutoRecorder( + config=self.config, + camera_manager=self.camera_manager, + state_manager=self.state_manager, + event_system=self.event_system + ) self.api_server = APIServer(self.config, self.state_manager, self.event_system, self.camera_manager, self.mqtt_client, self.storage_manager, self.auto_recording_manager) # System state diff --git a/camera-management-api/usda_vision_system/mqtt/client.py b/camera-management-api/usda_vision_system/mqtt/client.py index 6fea69a..fc0078f 100644 --- a/camera-management-api/usda_vision_system/mqtt/client.py +++ b/camera-management-api/usda_vision_system/mqtt/client.py @@ -172,14 +172,15 @@ class MQTTClient: self.connected = True self.state_manager.set_mqtt_connected(True) self.event_system.publish(EventType.MQTT_CONNECTED, "mqtt_client") - self.logger.info("๐ MQTT CONNECTED to broker successfully") + self.logger.info(f"๐ MQTT CONNECTED to broker successfully at {self.mqtt_config.broker_host}:{self.mqtt_config.broker_port}") print(f"๐ MQTT CONNECTED: {self.mqtt_config.broker_host}:{self.mqtt_config.broker_port}") # Subscribe to topics immediately after connection self._subscribe_to_topics() + self.logger.info(f"๐ MQTT subscribed to {len(self.mqtt_config.topics)} topics") else: self.connected = False - self.logger.error(f"โ MQTT CONNECTION FAILED with return code {rc}") + self.logger.error(f"โ MQTT CONNECTION FAILED with return code {rc} to {self.mqtt_config.broker_host}:{self.mqtt_config.broker_port}") print(f"โ MQTT CONNECTION FAILED: {self.mqtt_config.broker_host}:{self.mqtt_config.broker_port} (code: {rc})") def _on_disconnect(self, client, userdata, rc) -> None: @@ -201,7 +202,8 @@ class MQTTClient: topic = msg.topic payload = msg.payload.decode("utf-8").strip() - self.logger.debug(f"MQTT message received - Topic: {topic}, Payload: {payload}") + # Log at INFO level so we can see messages in production + self.logger.info(f"๐ก MQTT MESSAGE RECEIVED - Topic: {topic}, Payload: '{payload}'") # Update MQTT activity and tracking self.state_manager.update_mqtt_activity() @@ -211,19 +213,20 @@ class MQTTClient: # Get machine name from topic machine_name = self.topic_to_machine.get(topic) if not machine_name: - self.logger.warning(f"โ MQTT UNKNOWN TOPIC: {topic}") - print(f"โ MQTT UNKNOWN TOPIC: {topic}") + self.logger.warning(f"โ MQTT UNKNOWN TOPIC: {topic} (payload: '{payload}')") + print(f"โ MQTT UNKNOWN TOPIC: {topic} (payload: '{payload}')") return - # Show MQTT message on console + # Show MQTT message on console with machine name print(f"๐ก MQTT MESSAGE: {machine_name} โ {payload}") + self.logger.info(f"๐ก Processing MQTT message for machine '{machine_name}': '{payload}'") # Handle the message self.message_handler.handle_message(machine_name, topic, payload) except Exception as e: self.error_count += 1 - self.logger.error(f"Error processing MQTT message: {e}") + self.logger.error(f"โ Error processing MQTT message: {e}", exc_info=True) def publish_message(self, topic: str, payload: str, qos: int = 0, retain: bool = False) -> bool: """Publish a message to MQTT broker""" diff --git a/camera-management-api/usda_vision_system/mqtt/handlers.py b/camera-management-api/usda_vision_system/mqtt/handlers.py index f890ecd..0767a43 100644 --- a/camera-management-api/usda_vision_system/mqtt/handlers.py +++ b/camera-management-api/usda_vision_system/mqtt/handlers.py @@ -31,10 +31,11 @@ class MQTTMessageHandler: self.message_count += 1 self.last_message_time = datetime.now() - self.logger.info(f"Processing MQTT message - Machine: {machine_name}, Topic: {topic}, Payload: {payload}") + self.logger.info(f"๐ก Processing MQTT message - Machine: {machine_name}, Topic: {topic}, Payload: '{payload}'") # Normalize payload normalized_payload = self._normalize_payload(payload) + self.logger.info(f"๐ก Normalized payload '{payload}' -> '{normalized_payload}' for machine {machine_name}") # Update machine state state_changed = self.state_manager.update_machine_state(name=machine_name, state=normalized_payload, message=payload, topic=topic) @@ -44,9 +45,12 @@ class MQTTMessageHandler: # Publish state change event if state actually changed if state_changed: + self.logger.info(f"๐ก MQTT: Machine {machine_name} state changed to: {normalized_payload}") + self.logger.info(f"๐ก Publishing MACHINE_STATE_CHANGED event for {machine_name} -> {normalized_payload}") publish_machine_state_changed(machine_name=machine_name, state=normalized_payload, source="mqtt_handler") - - self.logger.info(f"Machine {machine_name} state changed to: {normalized_payload}") + self.logger.info(f"โ Published MACHINE_STATE_CHANGED event for {machine_name} -> {normalized_payload}") + else: + self.logger.info(f"๐ก Machine {machine_name} state unchanged (still {normalized_payload}) - no event published") # Log the message for debugging self._log_message_details(machine_name, topic, payload, normalized_payload) diff --git a/camera-management-api/usda_vision_system/recording/standalone_auto_recorder.py b/camera-management-api/usda_vision_system/recording/standalone_auto_recorder.py index a3c48f7..dbd406a 100644 --- a/camera-management-api/usda_vision_system/recording/standalone_auto_recorder.py +++ b/camera-management-api/usda_vision_system/recording/standalone_auto_recorder.py @@ -30,13 +30,13 @@ sys.path.insert(0, str(Path(__file__).parent.parent.parent)) from usda_vision_system.core.config import Config from usda_vision_system.camera.recorder import CameraRecorder from usda_vision_system.core.state_manager import StateManager -from usda_vision_system.core.events import EventSystem +from usda_vision_system.core.events import EventSystem, EventType, Event class StandaloneAutoRecorder: """Standalone auto-recording system that monitors MQTT and controls cameras directly""" - def __init__(self, config_path: str = "config.json", config: Optional[Config] = None): + def __init__(self, config_path: str = "config.json", config: Optional[Config] = None, camera_manager=None, state_manager=None, event_system=None): # Load configuration if config: self.config = config @@ -45,9 +45,9 @@ class StandaloneAutoRecorder: # Setup logging (only if not already configured) if not logging.getLogger().handlers: - # Use WARNING level by default to reduce INFO log noise - log_level = getattr(self.config.system, 'log_level', 'WARNING') - log_level_num = getattr(logging, log_level.upper(), logging.WARNING) + # Use configured log level + log_level = getattr(self.config.system, 'log_level', 'INFO') + log_level_num = getattr(logging, log_level.upper(), logging.INFO) logging.basicConfig( level=log_level_num, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", @@ -59,16 +59,17 @@ class StandaloneAutoRecorder: self.logger = logging.getLogger(__name__) # Ensure this logger respects the configured log level if hasattr(self.config, 'system') and hasattr(self.config.system, 'log_level'): - self.logger.setLevel(getattr(logging, self.config.system.log_level.upper(), logging.WARNING)) + self.logger.setLevel(getattr(logging, self.config.system.log_level.upper(), logging.INFO)) - # Initialize components - self.state_manager = StateManager() - self.event_system = EventSystem() + # Use provided components or create new ones + self.state_manager = state_manager if state_manager else StateManager() + self.event_system = event_system if event_system else EventSystem() + self.camera_manager = camera_manager - # MQTT client + # MQTT client (only if not using event system) self.mqtt_client: Optional[mqtt.Client] = None - # Camera recorders + # Camera recorders (only if not using camera_manager) self.camera_recorders: Dict[str, CameraRecorder] = {} self.active_recordings: Dict[str, str] = {} # camera_name -> filename @@ -82,8 +83,17 @@ class StandaloneAutoRecorder: self.running = False self._stop_event = threading.Event() + # Subscribe to machine state change events if using event system + if self.event_system and self.camera_manager: + self.event_system.subscribe(EventType.MACHINE_STATE_CHANGED, self._on_machine_state_changed) + self.logger.info("Subscribed to MACHINE_STATE_CHANGED events") + self.logger.info("Standalone Auto-Recorder initialized") self.logger.info(f"Machine-Camera mapping: {self.machine_camera_map}") + if self.camera_manager: + self.logger.info("Using provided camera_manager for recording") + else: + self.logger.info("Will create own camera recorders (standalone mode)") def _build_machine_camera_map(self) -> Dict[str, str]: """Build mapping from machine topics to camera names""" @@ -162,80 +172,137 @@ class StandaloneAutoRecorder: except Exception as e: self.logger.error(f"Error processing MQTT message: {e}") + def _on_machine_state_changed(self, event: Event): + """Handle machine state change event from event system""" + try: + machine_name = event.data.get("machine_name") + state = event.data.get("state", "").lower() + source = event.source + + self.logger.info(f"๐ก AUTO-RECORDER: Received MACHINE_STATE_CHANGED event from {source}") + self.logger.info(f"๐ก AUTO-RECORDER: Event data - machine_name: {machine_name}, state: {state}") + + if not machine_name or not state: + self.logger.warning(f"โ AUTO-RECORDER: Invalid event data - machine_name: {machine_name}, state: {state}") + return + + self._handle_machine_state_change(machine_name, state) + + except Exception as e: + self.logger.error(f"โ AUTO-RECORDER: Error handling machine state change event: {e}", exc_info=True) + def _handle_machine_state_change(self, machine_name: str, state: str): """Handle machine state change""" try: # Check if we have a camera for this machine camera_name = self.machine_camera_map.get(machine_name) if not camera_name: + self.logger.debug(f"No camera mapped to machine: {machine_name}") return - self.logger.info(f"Handling state change: {machine_name} ({camera_name}) -> {state}") + self.logger.info(f"๐ก MQTT: Machine {machine_name} ({camera_name}) -> {state}") if state == "on": self._start_recording(camera_name, machine_name) elif state == "off": self._stop_recording(camera_name, machine_name) + else: + self.logger.debug(f"Ignoring state '{state}' for machine {machine_name}") except Exception as e: - self.logger.error(f"Error handling machine state change: {e}") + self.logger.error(f"Error handling machine state change: {e}", exc_info=True) def _start_recording(self, camera_name: str, machine_name: str): """Start recording for a camera""" try: # Check if already recording - if camera_name in self.active_recordings: - self.logger.warning(f"Camera {camera_name} is already recording") + camera_info = self.state_manager.get_camera_status(camera_name) if self.state_manager else None + if camera_info and camera_info.is_recording: + self.logger.info(f"Camera {camera_name} is already recording, skipping") return - # Get or create camera recorder - recorder = self._get_camera_recorder(camera_name) - if not recorder: - self.logger.error(f"Failed to get recorder for camera {camera_name}") - return + # Use camera_manager if available, otherwise use standalone recorder + if self.camera_manager: + # Generate filename with timestamp and machine info + from ..core.timezone_utils import format_filename_timestamp + timestamp = format_filename_timestamp() + camera_config = self.config.get_camera_by_name(camera_name) + video_format = camera_config.video_format if camera_config else "mp4" + filename = f"{camera_name}_auto_{machine_name}_{timestamp}.{video_format}" - # Generate filename with timestamp and machine info - timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") - camera_config = self.config.get_camera_by_name(camera_name) - video_format = camera_config.video_format if camera_config else "mp4" - filename = f"{camera_name}_auto_{machine_name}_{timestamp}.{video_format}" + # Use camera manager to start recording with camera's default settings + success = self.camera_manager.manual_start_recording( + camera_name=camera_name, + filename=filename, + exposure_ms=camera_config.exposure_ms if camera_config else None, + gain=camera_config.gain if camera_config else None, + fps=camera_config.target_fps if camera_config else None + ) - # Start recording - success = recorder.start_recording(filename) - if success: - self.active_recordings[camera_name] = filename - self.logger.info(f"โ Started recording: {camera_name} -> {filename}") + if success: + self.logger.info(f"โ Started auto-recording: {camera_name} -> {filename}") + self.active_recordings[camera_name] = filename + else: + self.logger.error(f"โ Failed to start auto-recording for camera {camera_name}") else: - self.logger.error(f"โ Failed to start recording for camera {camera_name}") + # Standalone mode - use own recorder + recorder = self._get_camera_recorder(camera_name) + if not recorder: + self.logger.error(f"Failed to get recorder for camera {camera_name}") + return + + # Generate filename with timestamp and machine info + timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") + camera_config = self.config.get_camera_by_name(camera_name) + video_format = camera_config.video_format if camera_config else "mp4" + filename = f"{camera_name}_auto_{machine_name}_{timestamp}.{video_format}" + + # Start recording + success = recorder.start_recording(filename) + if success: + self.active_recordings[camera_name] = filename + self.logger.info(f"โ Started recording: {camera_name} -> {filename}") + else: + self.logger.error(f"โ Failed to start recording for camera {camera_name}") except Exception as e: - self.logger.error(f"Error starting recording for {camera_name}: {e}") + self.logger.error(f"Error starting recording for {camera_name}: {e}", exc_info=True) def _stop_recording(self, camera_name: str, machine_name: str): """Stop recording for a camera""" try: - # Check if recording - if camera_name not in self.active_recordings: - self.logger.warning(f"Camera {camera_name} is not recording") - return - - # Get recorder - recorder = self._get_camera_recorder(camera_name) - if not recorder: - self.logger.error(f"Failed to get recorder for camera {camera_name}") - return - - # Stop recording - filename = self.active_recordings.pop(camera_name) - success = recorder.stop_recording() - - if success: - self.logger.info(f"โ Stopped recording: {camera_name} -> {filename}") + # Use camera_manager if available + if self.camera_manager: + success = self.camera_manager.manual_stop_recording(camera_name) + if success: + self.logger.info(f"โ Stopped auto-recording: {camera_name}") + if camera_name in self.active_recordings: + filename = self.active_recordings.pop(camera_name) + self.logger.debug(f"Recording filename was: {filename}") + else: + self.logger.warning(f"Camera {camera_name} may not have been recording") else: - self.logger.error(f"โ Failed to stop recording for camera {camera_name}") + # Standalone mode - use own recorder + if camera_name not in self.active_recordings: + self.logger.warning(f"Camera {camera_name} is not recording") + return + + recorder = self._get_camera_recorder(camera_name) + if not recorder: + self.logger.error(f"Failed to get recorder for camera {camera_name}") + return + + # Stop recording + filename = self.active_recordings.pop(camera_name) + success = recorder.stop_recording() + + if success: + self.logger.info(f"โ Stopped recording: {camera_name} -> {filename}") + else: + self.logger.error(f"โ Failed to stop recording for camera {camera_name}") except Exception as e: - self.logger.error(f"Error stopping recording for {camera_name}: {e}") + self.logger.error(f"Error stopping recording for {camera_name}: {e}", exc_info=True) def _get_camera_recorder(self, camera_name: str) -> Optional[CameraRecorder]: """Get or create camera recorder""" @@ -356,19 +423,27 @@ class StandaloneAutoRecorder: try: self.logger.info("Starting Standalone Auto-Recorder...") - # Setup MQTT + # If using event system and camera_manager, we don't need our own MQTT client + if self.event_system and self.camera_manager: + self.logger.info("Using event system - no need for separate MQTT client") + self.running = True + self.logger.info("โ Standalone Auto-Recorder started successfully (event-based mode)") + return True + + # Otherwise, setup MQTT client for standalone mode if not self._setup_mqtt(): + self.logger.error("Failed to setup MQTT client") return False # Wait for MQTT connection time.sleep(2) self.running = True - self.logger.info("โ Standalone Auto-Recorder started successfully") + self.logger.info("โ Standalone Auto-Recorder started successfully (standalone MQTT mode)") return True except Exception as e: - self.logger.error(f"Failed to start auto-recorder: {e}") + self.logger.error(f"Failed to start auto-recorder: {e}", exc_info=True) return False def stop(self) -> bool: diff --git a/scheduling-remote/src/components/Scheduling.tsx b/scheduling-remote/src/components/Scheduling.tsx index 25d4345..8d028e1 100644 --- a/scheduling-remote/src/components/Scheduling.tsx +++ b/scheduling-remote/src/components/Scheduling.tsx @@ -995,8 +995,8 @@ function ScheduleExperiment({ user, onBack }: { user: User; onBack: () => void } return ( -
Fetching conductors, phases, and experiments.