[US Region] Job Queuing Issue
Incident Report for Treasure Data
Resolved
The issue of our partition metadata system has been fixed.
Posted 20 days ago. Jul 31, 2019 - 16:25 PDT
Update
Stream import API is working normally.
Posted 20 days ago. Jul 31, 2019 - 16:15 PDT
Update
We are continuing to monitor for any further issues.
Posted 20 days ago. Jul 31, 2019 - 16:03 PDT
Update
We are continuing to monitor for any further issues.
Posted 20 days ago. Jul 31, 2019 - 15:58 PDT
Monitoring
A fix has been made. We are checking affected components and adding more capacity for processing queued jobs.
Posted 20 days ago. Jul 31, 2019 - 15:49 PDT
Update
The system is now gradually recovering.
Posted 20 days ago. Jul 31, 2019 - 15:44 PDT
Update
We are continuing to work on a fix for this issue.
Posted 20 days ago. Jul 31, 2019 - 15:37 PDT
Update
We are applying a fix to the metadata system.
Posted 20 days ago. Jul 31, 2019 - 15:36 PDT
Identified
We've found performance degradation of our partition metadata system. It is causing delay of data import and query processing. We are now working on fixing the issue.
Posted 20 days ago. Jul 31, 2019 - 15:31 PDT
Update
We have an increased number of queued jobs and delay of data import. We are now investigating the cause.
Posted 20 days ago. Jul 31, 2019 - 15:25 PDT
Investigating
We are currently investigating this issue.
Posted 20 days ago. Jul 31, 2019 - 15:21 PDT
This incident affected: US (Streaming Import REST API, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow).