Hive job on one Hadoop cluster slow down

Incident Report for Treasure Data

Resolved

From 4:25pm PT (8:25am JST) to 5:15pm PT (9:15am JST), Hive jobs took longer than usual. We confirmed our system is operating normally. This incident was resolved.
Posted Oct 29, 2019 - 18:40 PDT

Monitoring

We were experiencing slowdown in scheduling in one of our clusters due to a GC issues. We have since restarted the scheduling service and things seems to be back to normal. To be sure there are no issues with the underlying hardware we will replace the cluster, over the next couple of hours
Posted Oct 29, 2019 - 17:42 PDT

Investigating

We were experiencing slowdown in scheduling in one of our clusters due to a GC issues. We have since restarted the scheduling service and things seems to be back to normal. To be sure there are no issues with the underlying hardware we will replace the cluster, over the next couple of hours
Posted Oct 29, 2019 - 17:31 PDT
This incident affected: US (Hadoop / Hive Query Engine).