Files
usda-vision/management-dashboard-web-app/SEED_DATA_CORRECTIONS.md
salirezav e675423258 Remove deprecated CSV files and update experiment seeding scripts
- Deleted unused CSV files: 'meyer experiments.csv' and 'phase_2_experimental_run_sheet.csv'.
- Updated SQL seed scripts to reflect changes in experiment data structure and ensure consistency with the latest experiment parameters.
- Enhanced user role assignments in the seed data to include 'conductor' alongside 'data recorder'.
- Adjusted experiment seeding logic to align with the corrected data from the CSV files.
2025-09-28 21:10:50 -04:00

4.7 KiB

Seed Data Corrections

Problem Identified

The "Schedule Experiment" component was showing "undefined" for soaking duration in the top right panel. This was caused by a mismatch between the database schema and frontend expectations:

  • Database: Used soaking_duration_hours column
  • Frontend: Expected soaking_duration_minutes field
  • Seed Files: Were trying to insert into non-existent soaking_duration_minutes column

Root Cause

  1. The migration 20250101000004_data_entry_tables.sql created the soaking table with soaking_duration_hours column
  2. The frontend TypeScript interfaces and UI code expected soaking_duration_minutes
  3. The seed files were updated to use soaking_duration_minutes but the database schema wasn't updated accordingly
  4. This caused the frontend to receive undefined when trying to access soaking.soaking_duration_minutes

Solution Implemented

1. Database Schema Fix

File: supabase/migrations/20250103000001_fix_soaking_duration_column.sql

  • Added soaking_duration_minutes INTEGER column
  • Backfilled data from soaking_duration_hours * 60
  • Updated trigger function to use minutes
  • Dropped the old soaking_duration_hours column

2. Views Update

File: supabase/migrations/20250103000002_update_views_for_soaking_minutes.sql

  • Updated experiments_with_phases view to use soaking_duration_minutes
  • Updated repetitions_with_phases view to use soaking_duration_minutes

3. Corrected Seed Files

Phase 2 JC Experiments

File: supabase/seed_04_phase2_jc_experiments_corrected.sql

  • Experiment Numbers: 0-19 (matching CSV data, not 1-20)
  • Soaking Duration: Converted from hours to minutes (e.g., 34 hours = 2040 minutes)
  • Air Drying Duration: Used exact values from CSV (19-60 minutes)
  • JC Cracker Parameters: All parameters match CSV data exactly
  • Repetitions: Each experiment has 3 repetitions as specified

Meyer Experiments

File: supabase/seed_05_meyer_experiments_corrected.sql

  • Experiment Numbers: 1-40 (matching CSV data)
  • Soaking Duration: Converted from hours to minutes (e.g., 27 hours = 1620 minutes)
  • Air Drying Duration: Used exact values from CSV (11-56 minutes)
  • Meyer Cracker Parameters: All parameters match CSV data exactly
  • Repetitions: Each experiment has 1 repetition as specified

Data Mapping from CSV

Phase 2 JC Experiments (phase_2_JC_experimental_run_sheet.csv)

  • Unique Experiments: 20 (numbers 0-19)
  • Repetitions per Experiment: 3
  • Soaking Duration Range: 10-38 hours (600-2280 minutes)
  • Air Drying Duration Range: 10-60 minutes
  • Machine Type: JC Cracker

Meyer Experiments (post_workshop_meyer_experiments.csv)

  • Unique Experiments: 40 (numbers 1-40)
  • Repetitions per Experiment: 1
  • Soaking Duration Range: 10-54 hours (600-3240 minutes)
  • Air Drying Duration Range: 11-56 minutes
  • Machine Type: Meyer Cracker

Files Created/Modified

New Migration Files

  1. supabase/migrations/20250103000001_fix_soaking_duration_column.sql
  2. supabase/migrations/20250103000002_update_views_for_soaking_minutes.sql

New Seed Files

  1. supabase/seed_04_phase2_jc_experiments_corrected.sql
  2. supabase/seed_05_meyer_experiments_corrected.sql

Deployment Instructions

  1. Run the migrations in order:

    -- First fix the column
    \i supabase/migrations/20250103000001_fix_soaking_duration_column.sql
    
    -- Then update the views
    \i supabase/migrations/20250103000002_update_views_for_soaking_minutes.sql
    
  2. Clear existing data and run corrected seeds:

    -- Clear existing data (optional, use with caution)
    DELETE FROM public.experiment_repetitions;
    DELETE FROM public.jc_cracker_parameters;
    DELETE FROM public.meyer_cracker_parameters;
    DELETE FROM public.cracking;
    DELETE FROM public.airdrying;
    DELETE FROM public.soaking;
    DELETE FROM public.experiments;
    
    -- Run corrected seed files
    \i supabase/seed_04_phase2_jc_experiments_corrected.sql
    \i supabase/seed_05_meyer_experiments_corrected.sql
    

Expected Result

After applying these changes:

  • The "Schedule Experiment" component will show correct soaking durations in minutes
  • All experiment data will match the CSV source files exactly
  • The database schema will be consistent with frontend expectations
  • Phase timing calculations will work correctly

Verification

To verify the fix:

  1. Check that soaking.soaking_duration_minutes is populated in the database
  2. Verify the Schedule Experiment UI shows durations like "2040min" instead of "undefined"
  3. Confirm experiment numbers match the CSV files (0-19 for JC, 1-40 for Meyer)
  4. Validate that all machine parameters match the CSV data