[Tokyo Region] Workflow performance degradation

Incident Report for Treasure Data

Resolved

The Workflow performance degradation caused by the underlying database capacity. We will revisit the database capacity and find the way to mitigate the impact.

We have been monitoring for the last 1 hour, and we've confirmed that there is no more performance degradation.

This incident has been resolved.

Posted Aug 17, 2022 - 19:25 PDT

Monitoring

Based on our investigation, the scheduled task delay started from 2022-08-17 14:01:05 UTC and we won't be able to see the delays since 2022-08-18 01:19:00 UTC, for the last 30 minutes.

Now we are monitoring continuously.

Posted Aug 17, 2022 - 18:55 PDT

Investigating

We are experiencing the performance degradation from Workflow, so the scheduled workflow and/or tasks can be delayed. We are currently investigating the issue.

Posted Aug 17, 2022 - 18:20 PDT

This incident affected: Tokyo (Workflow).