Optimize consolidation fetching with GROUP BY and reduced limit

Changed consolidation_group_limit from 100k to 10k for faster queries.

Reverted to GROUP BY approach for getting consolidation keys:
- Uses MySQL index efficiently: (UnitName, ToolNameID, NodeNum, EventDate, EventTime)
- GROUP BY with NodeNum ensures we don't lose any combinations
- Faster GROUP BY queries than large ORDER BY queries
- Smaller LIMIT = faster pagination

This matches the original optimization suggestion and should be faster.

🤖 Generated with Claude Code

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
2025-12-25 22:22:30 +01:00
parent b6886293f6
commit fe2d173b0f
2 changed files with 22 additions and 18 deletions

View File

@@ -49,7 +49,7 @@ class MigrationSettings(BaseSettings):
)
batch_size: int = 10000
consolidation_group_limit: int = 100000
consolidation_group_limit: int = 10000
log_level: str = "INFO"
dry_run: bool = False