This incident has been resolved. The job system is performing normally, and all jobs affected by the performance impairment have successfully completed.
Posted Mar 31, 2019 - 04:01 PDT
Monitoring
We are now monitoring the job system. Scale-out is completed and new jobs are being queued and executed normally. Some jobs that were in progress on impaired instances will gracefully progress to completion.
Posted Mar 31, 2019 - 03:41 PDT
Identified
We have identified a performance issue in the job queueing system. A small number of jobs are being processed slower than expected causing delays consuming new jobs. The system is scaling to add capacity and should be back to normal shortly.
Posted Mar 31, 2019 - 02:46 PDT
Investigating
We are currently investigating hadoop/hive execution delays in the eu01 site.
Posted Mar 31, 2019 - 02:16 PDT
This incident affected: EU (Hadoop / Hive Query Engine).