[US Region] Database maintenance
Scheduled Maintenance Report for Treasure Data
Completed
*Maintenance* and system *recovery* have been fully completed.
Treasure Data system maintenance period was from 18:15:57 - 18:17:42 PST.

The scheduled maintenance is now complete.
Posted Mar 02, 2020 - 18:24 PST
Verifying
The scheduled maintenance is complete.

We are monitoring the system closely to ensure all systems successfully complete their *recovery*.
Posted Mar 02, 2020 - 18:19 PST
In progress
The scheduled maintenance is starting now.
Posted Mar 02, 2020 - 18:15 PST
Update
In about one hour, from 18:15 PST, we will start the scheduled maintenance on the databases supporting Treasure Data system.

During the maintenance window, the databases will experience short failover within 120 seconds, after which the respective systems are expected to recover in approximately 10 minutes. During the failover period, customers will observe the following symptoms:

* The Web Console will not be functional.

* Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.

* Javascript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Real-time CDP event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Requests to the following APIs will receive HTTP 5XX errors.
- REST API
- CDP API
- Workflow API

* The Presto JDBC / ODBC Gateway will report authentication failures.

* td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors.

* Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.

* Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.

Beyond this notice, we will provide updates at the start and completion of the operation, and once the verification is completed. At that time, all systems will have returned to full functionality and the Scheduled Maintenance will be closed.
Posted Mar 02, 2020 - 17:15 PST
Scheduled
On Mar 2nd, 2020 from 18:15 to 19:15 PST, the databases supporting Treasure Data system will undergo a short maintenance.

During the maintenance window, the databases will experience short failover within 120 seconds, after which the respective systems are expected to recover in approximately 10 minutes. During the failover period, customers will observe the following symptoms:

* The Web Console will not be functional.

* Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.

* Javascript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Real-time CDP event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.

* Requests to the following APIs will receive HTTP 5XX errors.
- REST API
- CDP API
- Workflow API

* The Presto JDBC / ODBC Gateway will report authentication failures.

* td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors.

* Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.

* Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.

Beyond this notice, we will provide updates approximately 1 hour before the beginning of the maintenance window, at the start and completion of the operation, and once the verification is completed. At that time, all systems will have returned to full functionality and the Scheduled Maintenance will be closed.

If you have any question or concern about this maintenance, please feel free to reach out to our Support team at support@treasure-data.com.
Posted Feb 07, 2020 - 17:42 PST
This scheduled maintenance affected: US (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API, Data Connector Integrations, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow, CDP API, Data Access API (beta)).