The issue causing delayed River runs and extended execution times has been fully resolved. This incident started at 06:05 UTC on November 3rd and was completely addressed by 13:00 UTC on November 4th.
Due to these delays: * Rivers with shorter timeout settings may have reached their limits and been canceled. * Rivers with longer timeouts were completed successfully, with no data loss expected.
The next scheduled River run will retrieve data for any canceled runs, ensuring continued data integrity.
We are investigating the root cause and will provide a detailed RCA once our review is complete.
We appreciate your patience and understanding. Best regards, Rivery Support Team
Posted Nov 04, 2024 - 14:06 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Nov 04, 2024 - 13:26 UTC
Update
We are continuing to work on a fix for this issue.
Posted Nov 04, 2024 - 12:09 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted Nov 04, 2024 - 12:09 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Nov 04, 2024 - 12:07 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted Nov 04, 2024 - 10:19 UTC
Investigating
We are currently experiencing longer-than-usual execution times for Rivers across both our EU and US environments. Our team is actively investigating the issue to identify the root cause and restore optimal performance as quickly as possible.
While Rivers are still running, users may experience delays in execution times. We understand the importance of timely data workflows and apologize for any inconvenience this may cause.
We will provide updates here as soon as more information becomes available, including an estimated time to resolution.
Thank you for your patience and understanding as we work to resolve this issue. Rivery support tream
Posted Nov 04, 2024 - 08:27 UTC
This incident affected: Rivery Pipelines (Rivers (US), Rivers (EU)).