-
3a53564bb5
con web ui
main
alex
2026-01-11 21:56:50 +01:00
-
6306006f82
Fix: Update RAWDATACOR partition end_year and mapping logic
alex
2026-01-11 15:55:41 +01:00
-
53cde5f667
Fix: Correct RAWDATACOR partition mapping logic
alex
2026-01-11 15:33:08 +01:00
-
d1dbf7f0de
aggiunta log
alex
2026-01-05 15:14:39 +01:00
-
931fec0959
fix logger x partition
alex
2026-01-05 14:00:29 +01:00
-
a7d2d501fb
fix: Use timezone-aware timestamps for migration state tracking
alex
2026-01-04 16:04:15 +01:00
-
23e9fc9d82
feat: Add error logging and fix incremental migration state tracking
alex
2026-01-01 19:49:44 +01:00
-
03e39eb925
fix docs
alex
2025-12-30 15:33:32 +01:00
-
bcedae40fc
fix .env.example var and docs
alex
2025-12-30 15:29:26 +01:00
-
5f6e3215a5
clean docs
alex
2025-12-30 15:24:19 +01:00
-
5c9df3d06f
fix incremental
alex
2025-12-30 15:16:54 +01:00
-
79cd4f4559
fix: Fix duplicate group insertion in consolidation generator
alex
2025-12-27 10:26:39 +01:00
-
418da67857
fix: Properly handle incomplete consolidation groups at batch boundaries
alex
2025-12-27 09:54:42 +01:00
-
3576a8f354
fix: When --partition is specified, ignore migration_state partition tracking
alex
2025-12-27 09:45:15 +01:00
-
f3768d8174
fix: Allow dry-run even when migration is in progress
alex
2025-12-27 09:43:45 +01:00
-
58624c0866
feat: Add --partition flag to migrate only specific partition
alex
2025-12-27 09:05:11 +01:00
-
287a7ffb51
feat: Add consolidation support to incremental migration
alex
2025-12-27 08:49:40 +01:00
-
f2b6049608
fix: CRITICAL - Don't prematurely yield incomplete groups at batch boundaries
alex
2025-12-26 20:44:40 +01:00
-
0e52f72dbe
fix: Track last_migrated_id during migration, not just at final update
alex
2025-12-26 20:27:00 +01:00
-
8c48e5eecb
fix: Pass last_completed_partition to ALL migration_state updates, not just final
alex
2025-12-26 20:23:11 +01:00
-
49b9772dba
fix: CRITICAL - Do not re-sort rows by NodeNum after MySQL ORDER BY consolidation key
alex
2025-12-26 20:15:00 +01:00
-
ca2f7c5756
fix: Ensure last_completed_partition is saved on final migration state update
alex
2025-12-26 18:24:28 +01:00
-
1430ef206f
fix: Ensure complete node consolidation by ordering MySQL query by consolidation key
alex
2025-12-26 18:22:23 +01:00
-
681812d0a4
cleanup: Remove unnecessary debug logging from consolidation logic
alex
2025-12-26 14:55:22 +01:00
-
55bbfab8b8
fix: Simplify consolidation grouping to use NodeNum decrease as boundary
alex
2025-12-26 14:49:32 +01:00
-
49dbd98bff
fix: Add last_completed_partition column to migration_state table schema
alex
2025-12-26 11:39:30 +01:00
-
ff0187b74a
debug: Add logging for key changes within batch grouping
alex
2025-12-26 09:46:52 +01:00
-
c9088d9144
fix: Merge consolidation groups with same key across batch boundaries
alex
2025-12-26 09:16:21 +01:00
-
6ca97f0ba4
fix: Only update last_completed_partition when partition is fully processed
alex
2025-12-26 00:49:14 +01:00
-
32f90fdd47
refactor: Remove legacy consolidation methods from MySQLConnector
alex
2025-12-26 00:20:46 +01:00
-
d6564b7f9e
refactor: Improve logging for consolidation group tracking
alex
2025-12-26 00:10:16 +01:00
-
4277dd8d2c
fix: Yield all groups in final batch, not just last group
alex
2025-12-26 00:03:45 +01:00
-
3687f77911
debug: Add detailed logging to consolidation group buffering
alex
2025-12-26 00:03:17 +01:00
-
9ef65995d4
feat: Add granular resume within partitions using last inserted ID
alex
2025-12-25 23:41:57 +01:00
-
e5c87d145f
feat: Track last completed partition for accurate resume capability
alex
2025-12-25 23:30:37 +01:00
-
3532631f3f
fix: Reduce INSERT buffer size and update state after every flush
alex
2025-12-25 23:01:33 +01:00
-
dfc54cf867
perf: Batch INSERT statements to reduce database round-trips
alex
2025-12-25 22:53:20 +01:00
-
d513920788
fix: Buffer incomplete groups at batch boundaries for complete consolidation
alex
2025-12-25 22:36:15 +01:00
-
c30d77e24b
Fix N+1 query problem - use single ordered query with Python grouping
alex
2025-12-25 22:32:41 +01:00
-
fe2d173b0f
Optimize consolidation fetching with GROUP BY and reduced limit
alex
2025-12-25 22:22:30 +01:00
-
b6886293f6
Add detailed partition progress logging
alex
2025-12-25 22:10:43 +01:00
-
255fb1c520
Simplify resume logic for partition-based consolidation
alex
2025-12-25 21:54:40 +01:00
-
bb27f749a0
Implement partition-based consolidation for ELABDATADISP
alex
2025-12-25 21:49:30 +01:00
-
a394de99ef
Fix ELABDATADISP consolidation by consolidating across batches
alex
2025-12-25 20:23:31 +01:00
-
9cc12abe11
fix: Order rows by consolidation key to keep related nodes together in batches
alex
2025-12-25 19:32:52 +01:00
-
648bd98a09
chore: Add debug logging to ELABDATADISP consolidation
alex
2025-12-25 19:27:34 +01:00
-
72035bb1b5
fix: Convert MySQL Decimal values to float for JSON serialization in ELABDATADISP
alex
2025-12-25 19:06:50 +01:00
-
3c0a6f72b4
fix: Use correct ID column for ELABDATADISP in fetch_rows_ordered_for_consolidation()
alex
2025-12-25 19:04:51 +01:00
-
e75cf4c545
fix: Support ELABDATADISP in fetch_rows_ordered_for_consolidation()
alex
2025-12-25 19:02:00 +01:00
-
5045c8bd86
fix: Add updated_at column back to ELABDATADISP table
alex
2025-12-25 18:46:03 +01:00
-
42c0d9cdaf
chore: Update column order for ELABDATADISP to exclude node/state/calc_err
alex
2025-12-25 18:42:19 +01:00
-
693228c0da
feat: Implement node consolidation for ELABDATADISP table
alex
2025-12-25 18:41:54 +01:00
-
4d72d2a42e
chore: Add validation queries for default timestamp records
alex
2025-12-23 20:47:01 +01:00
-
d3ada1ded2
fix: Mark migration as completed when migration finishes
alex
2025-12-23 20:41:46 +01:00
-
8d9e63081a
chore: Add detailed logging for migration state update
alex
2025-12-23 20:32:33 +01:00
-
26b3ccb06e
fix: Ensure migration_state updates are committed to database
alex
2025-12-23 20:26:10 +01:00
-
1708969616
fix: Update migration state with final count when migration completes
alex
2025-12-23 20:16:28 +01:00
-
0461bb3b44
fix: Handle invalid MySQL dates (0000-00-00) gracefully
alex
2025-12-23 19:06:38 +01:00
-
4f4ba6af51
fix: Import date type explicitly to fix isinstance checks
alex
2025-12-23 18:56:12 +01:00
-
eb315c90ff
fix: Handle date conversion for string dates in data transformer
alex
2025-12-23 18:52:42 +01:00
-
262edd0ed2
chore: Revert throughput reporting feature from progress tracker
alex
2025-12-23 16:47:10 +01:00
-
678cd22c89
fix: Use print_status() for throughput reporting in progress tracker
alex
2025-12-23 16:34:47 +01:00
-
38b359a72d
feat: Add periodic throughput reporting to progress tracker
alex
2025-12-23 16:31:12 +01:00
-
7cb4783385
fix: Reduce expensive COUNT(*) queries to every 10 batches
alex
2025-12-23 16:10:40 +01:00
-
0cb4a0f71e
fix: Update progress tracking to use MySQL row count instead of PostgreSQL count
alex
2025-12-23 15:40:50 +01:00
-
0f217379ea
fix: Use actual PostgreSQL row count for total_rows_migrated tracking
alex
2025-12-23 15:33:27 +01:00
-
b09cfcf9df
fix: Add timeout settings and retry logic to MySQL connector
alex
2025-12-21 09:53:34 +01:00
-
821cda850e
fix: Change from COPY to parameterized INSERT for batch inserts
alex
2025-12-10 20:48:20 +01:00
-
e2377d4191
fix: Add explicit commit/rollback in PostgreSQL context manager exit
alex
2025-12-10 20:39:04 +01:00
-
e381618255
fix: Support both uppercase and lowercase table names in TABLE_CONFIGS
alex
2025-12-10 20:28:19 +01:00
-
de6bde17c9
feat: Add sequences for auto-incrementing IDs
alex
2025-12-10 20:20:52 +01:00
-
2834f8b578
fix: Remove unsupported constraints from partitioned tables
alex
2025-12-10 20:18:20 +01:00
-
410b253808
fix: Update Pydantic v2 configuration for .env loading
alex
2025-12-10 20:11:12 +01:00
-
9b18db029b
docs: Add quick navigation guide (START_HERE.md)
alex
2025-12-10 20:00:50 +01:00
-
8e705e33da
docs: Add detailed example workflow
alex
2025-12-10 19:59:22 +01:00
-
38c6b4c6d8
docs: Add implementation summary
alex
2025-12-10 19:58:49 +01:00
-
fccc83eb74
docs: Add comprehensive documentation and helper scripts
alex
2025-12-10 19:58:20 +01:00
-
62577d3200
feat: Add MySQL to PostgreSQL migration tool with JSONB transformation
alex
2025-12-10 19:57:11 +01:00