[AP03] Backend Database maintenance
Scheduled Maintenance Report for Treasure Data
Completed
The scheduled maintenance has been completed.
Posted Mar 22, 2022 - 20:00 PDT
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Mar 22, 2022 - 18:00 PDT
Update
In about an hour, from 01:00 to 03:00 UTC, the backend databases supporting the Treasure Data system will undergo a short maintenance.

#Impact

During the two-hour maintenance window, the RDS backend databases supporting the Treasure Data Platform will be updated to the latest Operating System with critical security fixes as recommended by AWS. During this time frame, despite the 2 hours window, the databases will only become unreachable for a portion of the operation. Customers will experience momentary failures, instabilities, or service unreachable for all services. As we approach the end of the maintenance, services will gradually return to their full throughput and operations.

During the database failover period, customers may observe the following symptoms:

- The Web Console will not be functional.

- Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.

- Javascript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.

- Real-time Segment event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.

- Requests to REST, CDP, Workflow APIs will receive HTTP 5XX errors.

- Presto JDBC / ODBC Gateway return errors during downtime

- Audit log will be delayed

- td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors.
Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.

- Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.

- CDP services in the audience studio would be unavailable.

- Request to ADL REST services will experience error and data that are ingested may experience longer than normal delay before they are queryable.
Posted Mar 22, 2022 - 17:01 PDT
Update
We will be undergoing scheduled maintenance during this time.
Posted Mar 03, 2022 - 03:43 PST
Scheduled
On March 23, 2022 from 01:00 to 03:00 UTC, the backend databases supporting the Treasure Data system will undergo a short maintenance.

During the two-hour maintenance window, the RDS backend databases supporting the Treasure Data Platform will be updated to the latest Operating System with critical security fixes as recommended by AWS. During this time frame, despite the 2 hours window, the databases will only become unreachable for a portion of the operation. Customers will experience momentary failures, instabilities, or service unreachable for all services. As we approach the end of the maintenance, services will gradually return to their full throughput and operations.

During the database failover period, customers may observe the following symptoms:

- The Web Console will not be functional.

- Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.

- Javascript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.

- Real-time Segment event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.

- Requests to REST, CDP, Workflow APIs will receive HTTP 5XX errors.

- Presto JDBC / ODBC Gateway return errors during downtime

- Audit log will be delayed

- td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors.
Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.

- Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.

- CDP services in the audience studio would be unavailable.

- Request to ADL REST services will experience error and data that are ingested may experience longer than normal delay before they are queryable.
Posted Feb 22, 2022 - 14:00 PST
This scheduled maintenance affected: AP03 (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API, Data Connector Integrations, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow, CDP API, CDP Personalization - Lookup API, CDP Personalization - Ingest API, ADL).