[US Region] Hadoop performance degradation

Incident Report for Treasure Data

Resolved

All Hadoop jobs at the new cluster are running smoothly. This incident was resolved.

Posted Apr 04, 2018 - 16:42 PDT

Update

The jobs that were running at the impaired Hadoop cluster already restarted at the new cluster. We will keep monitoring the new cluster in detail for more 15 minutes.

Posted Apr 04, 2018 - 16:19 PDT

Monitoring

New jobs started to run at the new Hadoop cluster. We are working for rerouting old jobs to restart at the new cluster.

Posted Apr 04, 2018 - 15:38 PDT

Identified

We identified the infrastructure trouble of Hadoop master node in the impaired Hadoop cluster. We are recreating the cluster.

Posted Apr 04, 2018 - 15:16 PDT

Investigating

We are experiencing performance degradation at one of Hadoop clusters from 14:15 PDT.

Posted Apr 04, 2018 - 15:01 PDT

This incident affected: US (Hadoop / Hive Query Engine).