Elevated API error rate again

Incident Report for Treasure Data

Resolved

All systems are working normally since 08:07 am PDT. This incident is resolved.

Posted Apr 21, 2017 - 10:52 PDT

Monitoring

After the previous incident, we are again observing the same problems from 06:10 to 06:16 PDT and 07:50 to 08:05 PDT. The cause is the same, storage IO problem at one of our backend DB server.
We have implemented interim mitigation fix to API servers.

Posted Apr 21, 2017 - 08:20 PDT

This incident affected: US (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API).