Problem: Resume was re-processing all partitions from the beginning because
migration_state didn't track which partition was the last one completed.
This caused duplicate data insertion and wasted time.
Solution:
1. Added 'last_completed_partition' column to migration_state table
2. Created _get_last_completed_partition() method to retrieve saved state
3. Updated _update_migration_state() to accept and save last_partition parameter
4. Modified migration loop to:
- Retrieve last_completed_partition on resume
- Skip partitions that were already completed (partition <= last_completed_partition)
- Update last_completed_partition after each partition finishes
- Log which partitions are being skipped during resume
Now when resuming:
- Only processes partitions after the last completed one
- Avoids re-migrating already completed partitions
- Provides clear logging showing which partitions are skipped
For example, if migration was at partition d5 when interrupted, resume will:
- Skip d0 through d5 (logging each skip)
- Continue with d6 onwards
🤖 Generated with Claude Code
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
alex
e5c87d145f