[US Region] Scheduled Metadata Database maintenance
Scheduled Maintenance Report for Treasure Data
Completed
*Maintenance* and system *recovery* processes have been fully executed.
Streaming import and job execution delay was around 9 minutes from 6:19 pm PDT.

The scheduled Metadata Database maintenance is now complete.
Posted May 08, 2018 - 18:45 PDT
Verifying
The scheduled Metadata Database maintenance is complete.

We are monitoring the system closely to ensure all systems successfully complete their *recovery* and return to full functionality and throughput as quickly as possible.
Posted May 08, 2018 - 18:33 PDT
In progress
The scheduled Metadata Database maintenance is starting now.

The database will become unreachable for the duration of the *maintenance* procedure, which should last no longer than 15 minutes. We expect this to be followed by a *recovery* period of around 15 minutes during which the system will gradually reach back to full throughput. During the maintenance and recovery, customers may experience delays with Streaming (td-agent, fluentd) imports, Mobile (Android, iOS, Unity SDK) imports, Browser (JavaScript SDK) imports, and job execution as well as Presto JDBC / ODBC request failures.
Posted May 08, 2018 - 18:15 PDT
Update
In about an hour, at 6:15 PM PDT (10:15 AM JST), we will start the scheduled maintenance on our PlazmaDB Metadata database.

The maintenance is necessary to upgrade the PostgreSQL database to keep up with feature advancement and take advantage of the latest performance improvements.

The database will become unreachable for the duration of the *maintenance* procedure, which should last no longer than 15 minutes. We expect this to be followed by a *recovery* period of around 15 minutes during which the system will gradually reach back to full throughput. During the maintenance and recovery, customers may experience the following:

* Streaming, Mobile, and JavaScript/Browser imports delay
Streaming import (through td-agent or fluentd) requests will continue to be accepted as usual but the requests will remain queued until after the database maintenance is complete. We expect stream import processing to be further delayed during recovery. Thus we expect the stream import delay to extend to up to 30 minutes.
The same will apply to import requests from Browsers (Javascript SDK) and Mobiles (Android, iOS, and Unity SDKs).

* Jobs execution delay
All jobs (Presto, Hive, Result Export, Data Connector, Bulk Import, Export, and Partial Delete jobs submitted from Console, API, Workflow or triggered by our system according to the configured schedule) will remain queued during maintenance. During recovery, we expect jobs to begin processing slowly: within 30 minutes job processing should reach back to full throughput.

* Presto JDBC / ODBC Gateway errors
The Presto JDBC / ODBC Gateway will report errors during maintenance due to the unreachability of the Metadata database: errors will be propagated to the clients. We expect processing of Presto JDBC / ODBC jobs to follow the same recovery pattern as all other jobs (see above).

Beyond this notice, we will provide updates at the start and completion of the operation and once the verification has completed: at that time, all systems will have returned to full functionality and this Scheduled Maintenance will be closed.
Posted May 08, 2018 - 17:07 PDT
Scheduled
On Tuesday, May 8th from 6 to 7 PM PDT (May 9th from 10 to 11 AM JST) we will be performing maintenance on the PlazmaDB Metadata database.

The maintenance is necessary to upgrade the PostgreSQL database to keep up with feature advancement and take advantage of the latest performance improvements.

# Impact

NOTE: This maintenance notice only interests *AWS customers using the US region* and not AWS customers using the Tokyo region.

The database will become unreachable for the duration of the *maintenance* procedure, which should last no longer than 15 minutes. We expect this to be followed by a *recovery* period of around 15 minutes during which the system will gradually reach back to full throughput.

During the maintenance and recovery, customers may experience the following:

* Streaming, Mobile, and JavaScript/Browser imports delay
Streaming import (through td-agent or fluentd) requests will continue to be accepted as usual but the requests will remain queued until after the database maintenance is complete. We expect stream import processing to be further delayed during recovery. Thus we expect the stream import delay to extend to up to 30 minutes.
The same will apply to import requests from Browsers (Javascript SDK) and Mobiles (Android, iOS, and Unity SDKs).

* Jobs execution delay
All jobs (Presto, Hive, Result Export, Data Connector, Bulk Import, Export, and Partial Delete jobs submitted from Console, API, Workflow or triggered by our system according to the configured schedule) will remain queued during maintenance. During recovery, we expect jobs to begin processing slowly: within 30 minutes job processing should reach back to full throughput.

* Presto JDBC / ODBC Gateway errors
The Presto JDBC / ODBC Gateway will report errors during maintenance due to the unreachability of the Metadata database: errors will be propagated to the clients. We expect processing of Presto JDBC / ODBC jobs to follow the same recovery pattern as all other jobs (see above).

# Communication

Beyond this notice, we will provide updates approximately 1 hour before the beginning of the maintenance window, at the start and completion of the operation, and once the verification is completed. At that time, all systems will have returned to full functionality and the Scheduled Maintenance will be closed.

If you have any question or concern about this upgrade, please feel free to reach out to our Support team at support@treasuredata.com.
Posted Apr 06, 2018 - 13:42 PDT
This scheduled maintenance affected: US (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API, Data Connector Integrations, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow).