[US Region] Presto Cluster Stability Issue
Incident Report for Treasure Data
Resolved
Clusters are back to normal state.
Posted 6 months ago. May 16, 2018 - 01:40 PDT
Update
Presto clusters are gradually recovering to a normal state. We will keep monitoring.
Posted 6 months ago. May 16, 2018 - 01:27 PDT
Monitoring
We have increased the capacity of Presto clusters. Keep monitoring
Posted 6 months ago. May 16, 2018 - 01:20 PDT
Identified
We have distributed the query workload to mitigate the impact of memory consuming queries.
Posted 6 months ago. May 16, 2018 - 01:06 PDT
Update
We detected one of our Presto clusters is having frequent major GCs. We are now working on the fix.
Posted 6 months ago. May 16, 2018 - 00:51 PDT
Investigating
We have detected an increase of internal failure rate of Presto queries. Investigating.
Posted 6 months ago. May 16, 2018 - 00:25 PDT
This incident affected: US (Presto Query Engine).