[US Region] Backend Database Maintenance
Scheduled Maintenance Report for Treasure Data
Completed
The scheduled maintenance has been completed.
Posted Aug 23, 2023 - 19:28 PDT
Verifying
Verification is currently underway for the maintenance items.
Posted Aug 23, 2023 - 19:18 PDT
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Aug 23, 2023 - 19:00 PDT
Scheduled
On August 23, 2023 19:00 - 20:00 PDT (August 24, 2023 02:00 - 03:00 UTC), the backend databases supporting the Treasure Data system will undergo hardware maintenance again as there was an issue with the previous maintenance on the side of the infrastructure provider.

We sincerely apologize for any inconvenience this may have caused. We are communicating closely with the infrastructure provider to make sure that no further issues would arise from this said maintenance.

# Impact
During the maintenance window, the RDS backend databases supporting the Treasure Data Platform will undergo maintenance on the underlying hardware. During this time frame, the databases will only become unreachable for the time it takes for the failover to complete on the affected DB instances, usually about 60 seconds. Customers will experience momentary failures, instabilities, or service unreachable for all services. As we approach the end of the maintenance, services will gradually return to their full throughput and operations.

During the database failover period, customers may observe the following symptoms:
- The Web Console will not be functional.
- Streaming import requests will fail: Where fluentd / td-agent is being used (as recommended), event collection will continue locally on each device/server and will recover automatically once the maintenance is complete due to the built-in buffering and retry mechanisms.
- JavaScript SDK and Mobile SDK event collection will continue undisturbed but events will not be available for querying until the maintenance is completed.
- Real-time Segment event ingestion will continue undisturbed but events will not be available for querying until the maintenance is completed.
- Requests to REST, CDP, Workflow, Treasure Insights APIs will receive HTTP 5XX errors.
- Presto JDBC / ODBC Gateway return errors during downtime
- Premium Audit log will be delayed
- td command­-line (CLI) commands will encounter failures and execution will be delayed until the maintenance is complete. In some cases, the commands will be returning errors. Presto, Hive, Bulk Import and Data Connector jobs will be delayed. Already running jobs will be completed successfully or retried internally until they are.
- Workflows using Treasure Data operators (e.g. td>) will retry and regain full functionality again after the failover. In case of workflow sessions failures, the customer could elect to resume them manually.
- CDP services in the audience studio would be unavailable.
Posted Aug 18, 2023 - 01:45 PDT
This scheduled maintenance affected: US (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API, Data Connector Integrations, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow, CDP API, CDP Personalization - Lookup API, CDP Personalization - Ingest API, Data Access API (beta), Insights (beta), ADL, Data Tank, Consent Configuration API).