Add comprehensive validation system and migrate to .env configuration

This commit includes:

1. Database Configuration Migration:
   - Migrated from DB.txt (Java JDBC) to .env (python-dotenv)
   - Added .env.example template with clear variable names
   - Updated database.py to use environment variables
   - Added python-dotenv>=1.0.0 to dependencies
   - Updated .gitignore to exclude sensitive files

2. Validation System (1,294 lines):
   - comparator.py: Statistical comparison with RMSE, correlation, tolerances
   - db_extractor.py: Database queries for all sensor types
   - validator.py: High-level validation orchestration
   - cli.py: Command-line interface for validation
   - README.md: Comprehensive validation documentation

3. Validation Features:
   - Compare Python vs MATLAB outputs from database
   - Support for all sensor types (RSN, Tilt, ATD)
   - Statistical metrics: max abs/rel diff, RMSE, correlation
   - Configurable tolerances (abs, rel, max)
   - Detailed validation reports
   - CLI and programmatic APIs

4. Examples and Documentation:
   - validate_example.sh: Bash script example
   - validate_example.py: Python programmatic example
   - Updated main README with validation section
   - Added validation workflow and troubleshooting guide

Benefits:
-  No Java driver needed (native Python connectors)
-  Secure .env configuration (excluded from git)
-  Comprehensive validation against MATLAB
-  Statistical confidence in migration accuracy
-  Automated validation reports

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2025-10-13 15:34:13 +02:00
parent 876ef073fc
commit 23c53cf747
25 changed files with 7476 additions and 83 deletions

View File

@@ -4,4 +4,11 @@ version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.9"
dependencies = []
dependencies = [
"mysql-connector-python>=9.4.0",
"numpy>=2.0.2",
"openpyxl>=3.1.5",
"pandas>=2.3.3",
"python-dotenv>=1.0.0",
"scipy>=1.13.1",
]