fix: Add timeout settings and retry logic to MySQL connector
Configuration improvements: - Set read_timeout=300 (5 minutes) to handle long queries - Set write_timeout=300 (5 minutes) for writes - Set max_allowed_packet=64MB to handle larger data transfers Retry logic: - Added retry mechanism with max 3 retries on fetch failure - Auto-reconnect on connection loss before retry - Better error messages showing retry attempts This fixes the 'connection is lost' error that occurs during long-running migrations by: 1. Giving MySQL queries more time to complete 2. Allowing larger packet sizes for bulk data 3. Automatically recovering from connection drops Fixes: 'Connection is lost' error during full migration
This commit is contained in:
@@ -156,13 +156,13 @@ _rawdatacor_config = {
|
||||
"mysql_table": "RAWDATACOR",
|
||||
"postgres_table": "rawdatacor",
|
||||
"primary_key": "id",
|
||||
"partition_key": "event_date",
|
||||
"partition_key": "event_timestamp",
|
||||
}
|
||||
_elabdatadisp_config = {
|
||||
"mysql_table": "ELABDATADISP",
|
||||
"postgres_table": "elabdatadisp",
|
||||
"primary_key": "idElabData",
|
||||
"partition_key": "event_date",
|
||||
"partition_key": "event_timestamp",
|
||||
}
|
||||
|
||||
TABLE_CONFIGS = {
|
||||
|
||||
Reference in New Issue
Block a user