Treasure Data
All Systems Operational

About This Site

This is Treasure Data's status page. Our belief: "Trust starts with full transparency".

Web Interface Operational
REST API Operational
REST API (Import) Operational
Operational
Degraded Performance
Partial Outage
Major Outage
System Metrics Month Week Day
REST API Response Time ?
Fetching
REST API (Import) Response Time ?
Fetching
REST API Error Rates ?
Fetching
Web Interface Response Time ?
Fetching
Number of Queued Imports ?
Fetching
Past Incidents
Nov 23, 2014

No incidents reported today.

Nov 22, 2014

No incidents reported.

Nov 21, 2014

No incidents reported.

Nov 20, 2014

No incidents reported.

Nov 19, 2014

No incidents reported.

Nov 18, 2014

No incidents reported.

Nov 17, 2014

No incidents reported.

Nov 16, 2014

No incidents reported.

Nov 15, 2014

No incidents reported.

Nov 14, 2014

No incidents reported.

Nov 13, 2014

No incidents reported.

Nov 12, 2014
Postmortem - Read details
Nov 16, 23:51 PST
Resolved - This incident has been resolved.
Nov 12, 03:55 PST
Update - Most of the pending jobs have been completed and the cluster workload is back to normal now. We've reverted the configuration to normal status.
Nov 11, 22:35 PST
Monitoring - We confirmed that the issue is identified and fixed.

We increased the number of available Hive cores for the service and doubled the number of maximum mappers and reducers for every account. This is done to provide more computation resources to the affected queries running slow (because of the lack of time filtering) and speed them up to complete sooner.

We continue to monitor the situation until the number of job queued and execution time will be stable again.
Nov 11, 19:36 PST
Identified - We have identified and fixed an issue concerning the time filtering of all Hive queries.

Hive queries that were created between ~2:30 PM and ~5:00 PM PST are affected and therefore their execution, especially if scanning large tables which were meant to be sliced by time, is significantly slower.

A fix for this problem has been deployed at ~5:00 PM PST.
* New queries are once again able to leverage time filtering.
* Existing queries in 'running' state will however still experience the problem and may cause new query to wait for an execution slot. Customers that are able to do so, are invited to terminate the execution of their old queries and rerun them to leverage the fix.

At the same time, more query execution resources are being added to help consume the job backlog for all customers.
Nov 11, 17:41 PST
Nov 10, 2014

No incidents reported.

Nov 9, 2014

No incidents reported.