[AP03 Region] Backend Database Maintenance
Scheduled for Mar 12, 18:00 - 19:00 PDT
On Wednesday, March 13 from 01:00 to 02:00 UTC (March 13 from 10:00 to 11:00 JST), we will be performing routine maintenance on the backend databases that power the Treasure Data Platform.

During this maintenance, we will apply patches and updates to databases used by the Treasure Data Platform. These updates are necessary to ensure we have the latest security and performance improvements available. We expect downtime to be brief, lasting only a few minutes. The remainder of this time will be used for our team to validate that the maintenance was successful. Customers should expect some attempts to reach Treasure Data may time out or produce errors while we are deploying database improvements. See below for detailed information about the impact of this change on each Treasure Data service.

We will provide ongoing communications via our status page (https://status.treasuredata.com) to keep customers updated. If you notice any problems outside this maintenance window, we encourage you to contact our support team for assistance (https://docs.treasuredata.com/display/public/PD/Contacting+Technical+Support).

# Impact

During the database downtime and recovery period, customers may experience the following:

- Streaming, Mobile, and JavaScript/Browser imports delay

Streaming import (through td-agent or fluentd) requests will continue to be accepted as usual, but the requests will remain queued until after the database maintenance is complete. We expect stream import processing to be further delayed during recovery. The same impact will apply to import requests from Browsers (Javascript SDK), Mobiles (Android, iOS, and Unity SDKs), Data Ingestion APIs, and Postback APIs.

- Jobs execution delay

All jobs (Presto, Hive, Data Connector Integrations, Bulk Import, Bulk Export, Result Export, and Partial Delete jobs submitted from Console, API, Workflow or triggered by our system according to the configured schedule) will be delayed and automatically retried until they succeed after the downtime period. We expect job processing to ramp up over 30 minutes after the maintenance is complete before returning to full throughput.

- Table creation errors while maintenance is underway

During the maintenance period, any REST API requests to create tables will result in an error. SQL or Workflow requests to create tables will be delayed and automatically retried after the maintenance period.

- Console execution delay

Data Workbench and Audience Studio will show that jobs are running longer due to job execution delays. These delays will show because the Parent Segments, Segments, and Workflows jobs will be unavailable. After maintenance is complete, we expect processing of Data Workbench and Audience Studio jobs to follow the same recovery pattern as all other jobs (see above).

- Treasure Workflow delay and failure

We will attempt to retry any workflows, but some long-running workflows may be terminated if they run more than a few minutes. We encourage you to check any failed workflows and re-run if necessary.

- Workflow REST API will be unavailable

It won’t be possible to create or update Workflow Projects, Settings related to Audience Studio (including Parent Segment, Segment, Customer Journey Orchestration, etc.), and Predictive Scoring during the maintenance window. If you face errors using the Workflow REST API during maintenance, please wait for an update on our status page and re-execute your tasks when we’ve indicated maintenance is complete.

- Workflow operations in Console and CLI will be unavailable

Workflow-related operations in the Console and CLI may fail during the maintenance. Once we’ve indicated service has returned to normal on our status page, you can resume regular Workflow-related operations.

- Presto JDBC/ODBC queries and CDP segmentation queries failure

JDBC/ODBC queries which access “api-presto.treasuredata.com” will result in an error. Additionally, queries running from the CDP Segmentation UI in Console will be unavailable. We suggest retrying queries after the database maintenance is complete. Scheduled Presto queries in Workflow or Console won't be affected and will automatically run once the maintenance has been completed.

- Console Table preview update delay

The Table preview (sample data and records counter) in Data Workbench may show old information while maintenance is underway. This will return to normal once maintenance is complete.

- The REST API to submit and cancel jobs will be unavailable

The REST API endpoints for job submission and cancellations will be unavailable during the maintenance and will respond with a 500 error code.

If customers experience errors running Data Connector Integration Export jobs, please contact our Support team at support@treasuredata.com to recover the jobs. To avoid data duplication in your systems from Treasure Data exports, Integration Export jobs won’t be automatically retried.

# Communication

Beyond this notice, we will provide updates on our status page (https://status.treasuredata.com) approximately 1 hour before the beginning of the maintenance window, at the start and completion of the operation, and once the verification is complete. At that time, all systems will have returned to normal operations, and the Scheduled Maintenance will be closed.

If you have any questions or concerns about this upgrade, please feel free to contact our Support team at support@treasuredata.com.
Posted Feb 12, 2024 - 19:44 PST
This scheduled maintenance affects: AP03 (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API, Data Connector Integrations, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow, CDP API).