All Systems Operational

About This Site

This is Arm Treasure Data's status page.
We believe that trust starts with full transparency.

US ? Operational
90 days ago
99.99 % uptime
Today
Web Interface Operational
90 days ago
100.0 % uptime
Today
REST API Operational
90 days ago
100.0 % uptime
Today
Streaming Import REST API Operational
90 days ago
100.0 % uptime
Today
Mobile/Javascript REST API Operational
90 days ago
100.0 % uptime
Today
Data Connector Integrations Operational
90 days ago
100.0 % uptime
Today
Hadoop / Hive Query Engine Operational
90 days ago
99.98 % uptime
Today
Presto Query Engine Operational
90 days ago
99.98 % uptime
Today
Presto JDBC/ODBC Gateway Operational
90 days ago
100.0 % uptime
Today
Workflow ? Operational
90 days ago
100.0 % uptime
Today
CDP API ? Operational
90 days ago
100.0 % uptime
Today
CDP Personalization - Lookup API ? Operational
90 days ago
100.0 % uptime
Today
CDP Personalization - Ingest API ? Operational
90 days ago
100.0 % uptime
Today
Data Access API (beta) ? Operational
90 days ago
100.0 % uptime
Today
Tokyo ? Operational
90 days ago
99.99 % uptime
Today
Web Interface Operational
90 days ago
100.0 % uptime
Today
REST API Operational
90 days ago
100.0 % uptime
Today
Streaming Import REST API Operational
90 days ago
100.0 % uptime
Today
Mobile/Javascript REST API Operational
90 days ago
100.0 % uptime
Today
Data Connector Integrations Operational
90 days ago
100.0 % uptime
Today
Hadoop / Hive Query Engine Operational
90 days ago
99.98 % uptime
Today
Presto Query Engine Operational
90 days ago
99.98 % uptime
Today
Presto JDBC/ODBC Gateway Operational
90 days ago
100.0 % uptime
Today
Workflow ? Operational
90 days ago
100.0 % uptime
Today
CDP API ? Operational
90 days ago
100.0 % uptime
Today
CDP Personalization - Lookup API ? Operational
90 days ago
99.97 % uptime
Today
CDP Personalization - Ingest API ? Operational
90 days ago
99.97 % uptime
Today
Data Access API (beta) ? Operational
90 days ago
100.0 % uptime
Today
EU ? Operational
90 days ago
99.99 % uptime
Today
Web Interface Operational
90 days ago
100.0 % uptime
Today
REST API Operational
90 days ago
100.0 % uptime
Today
Streaming Import REST API Operational
90 days ago
100.0 % uptime
Today
Mobile/Javascript REST API Operational
90 days ago
100.0 % uptime
Today
Data Connector Integrations Operational
90 days ago
100.0 % uptime
Today
Hadoop / Hive Query Engine Operational
90 days ago
99.98 % uptime
Today
Presto Query Engine Operational
90 days ago
99.98 % uptime
Today
Presto JDBC/ODBC Gateway Operational
90 days ago
100.0 % uptime
Today
Workflow ? Operational
90 days ago
100.0 % uptime
Today
CDP API ? Operational
90 days ago
100.0 % uptime
Today
CDP Personalization - Lookup API ? Operational
90 days ago
100.0 % uptime
Today
CDP Personalization - Ingest API ? Operational
90 days ago
100.0 % uptime
Today
Data Access API (beta) ? Operational
90 days ago
100.0 % uptime
Today
Global ? Operational
90 days ago
99.98 % uptime
Today
Reporting ? Operational
90 days ago
99.98 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
had a major outage
had a partial outage
Scheduled Maintenance
[EU Region] Database maintenance Mar 2, 16:15-17:15 PST
On Mar 3rd, 2020 from 01:15 to 02:15 CET, the databases supporting Treasure Data system will undergo a short maintenance.

During the maintenance window, the databases will experience short failover within 120 seconds, after which the respective systems are expected to recover in approximately 10 minutes. During the failover period, customers will observe the following symptoms:

* The Web Console will not be functional.

* Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.

* Javascript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Real-time CDP event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Requests to the following APIs will receive HTTP 5XX errors.
- REST API
- CDP API
- Workflow API

* The Presto JDBC / ODBC Gateway will report authentication failures.

* td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors.

* Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.

* Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.

Beyond this notice, we will provide updates approximately 1 hour before the beginning of the maintenance window, at the start and completion of the operation, and once the verification is completed. At that time, all systems will have returned to full functionality and the Scheduled Maintenance will be closed.

If you have any question or concern about this maintenance, please feel free to reach out to our Support team at support@treasure-data.com.
Posted on Feb 7, 17:38 PST
On Mar 3rd, 2020 from 10:15 to 11:15 JST, the databases supporting Treasure Data system will undergo a short maintenance.

During the maintenance window, the databases will experience short failover within 120 seconds, after which the respective systems are expected to recover in approximately 10 minutes. During the failover period, customers will observe the following symptoms:

* The Web Console will not be functional.

* Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.

* Javascript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Real-time CDP event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Requests to the following APIs will receive HTTP 5XX errors.
- REST API
- CDP API
- Workflow API

* The Presto JDBC / ODBC Gateway will report authentication failures.

* td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors.

* Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.

* Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.

Beyond this notice, we will provide updates approximately 1 hour before the beginning of the maintenance window, at the start and completion of the operation, and once the verification is completed. At that time, all systems will have returned to full functionality and the Scheduled Maintenance will be closed.

If you have any question or concern about this maintenance, please feel free to reach out to our Support team at support@treasure-data.com.
Posted on Feb 7, 17:40 PST
[US Region] Database maintenance Mar 2, 18:15-19:15 PST
On Mar 2nd, 2020 from 18:15 to 19:15 PST, the databases supporting Treasure Data system will undergo a short maintenance.

During the maintenance window, the databases will experience short failover within 120 seconds, after which the respective systems are expected to recover in approximately 10 minutes. During the failover period, customers will observe the following symptoms:

* The Web Console will not be functional.

* Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.

* Javascript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Real-time CDP event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Requests to the following APIs will receive HTTP 5XX errors.
- REST API
- CDP API
- Workflow API

* The Presto JDBC / ODBC Gateway will report authentication failures.

* td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors.

* Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.

* Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.

Beyond this notice, we will provide updates approximately 1 hour before the beginning of the maintenance window, at the start and completion of the operation, and once the verification is completed. At that time, all systems will have returned to full functionality and the Scheduled Maintenance will be closed.

If you have any question or concern about this maintenance, please feel free to reach out to our Support team at support@treasure-data.com.
Posted on Feb 7, 17:42 PST
US - REST API - Response Time ?
Fetching
US - Streaming Import REST API - Response Time ?
Fetching
US - Queued Streaming Import requests ?
Fetching
US - Mobile/Javascript REST API - Response Time ?
Fetching
US - Web Interface - Response Time ?
Fetching
Tokyo - REST API - Response Time ?
Fetching
Tokyo - Streaming Import REST API - Response Time ?
Fetching
Tokyo - Queued Streaming Import requests ?
Fetching
Tokyo - Web Interface - Response Time ?
Fetching
EU - REST API - Response Time ?
Fetching
EU - Streaming Import REST API - Response Time ?
Fetching
EU - Queued Streaming Import requests ?
Fetching
EU - Web Interface - Response Time ?
Fetching
US - CDP API - Response Time
Fetching
Tokyo - CDP API - Response Time
Fetching
EU - CDP API - Response Time
Fetching
US - CDP Personalization Lookup API - Response Time
Fetching
Tokyo - CDP Personalization Lookup API - Response Time
Fetching
EU - CDP Personalization Lookup API - Response Time
Fetching
US - CDP Personalization Ingest API - Response Time
Fetching
Tokyo - CDP Personalization Ingest API - Response Time
Fetching
EU - CDP Personalization Ingest API - Response Time
Fetching
US - REST API - Error Rates
Fetching
Tokyo - REST API - Error Rates
Fetching
EU - REST API - Error Rates
Fetching
Past Incidents
Feb 18, 2020

No incidents reported today.

Feb 17, 2020

No incidents reported.

Feb 16, 2020

No incidents reported.

Feb 15, 2020

No incidents reported.

Feb 14, 2020

No incidents reported.

Feb 13, 2020

No incidents reported.

Feb 12, 2020

No incidents reported.

Feb 11, 2020

No incidents reported.

Feb 10, 2020

No incidents reported.

Feb 9, 2020

No incidents reported.

Feb 8, 2020

No incidents reported.

Feb 7, 2020
Resolved - This incident has been resolved.
The error occurred during 2020-02-07 13:22 PDT ~ 2020-02-07 16:14 PDT.
The failure workflow will be recovered by retrying the workflow.
If you have any questions about it, please contact support@treasure-data.com
Feb 7, 17:30 PST
Monitoring - A fix has been implemented and we are monitoring the results.
Feb 7, 16:36 PST
Investigating - We have observed some Treasure Workflow failures due to "Failed to process variables" error.
This error started to happen 4 hours ago.
We'll provide an update shortly.
Feb 7, 16:26 PST
Feb 6, 2020

No incidents reported.

Feb 5, 2020

No incidents reported.

Feb 4, 2020
Resolved - During the planned Presto migration that happened between 14:40 JST and 14:50 JST,
we observed some job failures in several customer accounts because of mis-configuration in our system.
The failed jobs showed the following error message:

NoMethodError: undefined method `sub' for nil:NilClass

If you see this error in your Presto queries, please rerun the job.
We sincerely apologize for this issue
Feb 4, 22:42 PST