*Maintenance* and system *recovery* have been fully completed. At US region, Streaming import and job execution delay was less than 70 seconds at 6:37 pm PDT. At Tokyo region, Streaming import and job execution delay was less than 40 seconds at 6:38 pm PDT.
The scheduled Metadata Database maintenance is now complete.
Posted Oct 30, 2018 - 18:47 PDT
Verifying
The scheduled Metadata Database maintenance is complete.
We are monitoring the system closely to ensure all systems successfully complete their *recovery* and return as quickly as possible to full functionality and throughput.
Posted Oct 30, 2018 - 18:42 PDT
In progress
The scheduled Metadata Database maintenance is starting now.
We expect the operation to cause *no downtime*: the database will be online after the upgrade within less than 1 minute. This will be followed by a *recovery* period of around 5 minutes during which the system will gradually reach back to full throughput. During the maintenance and recovery, customers may experience delays with Streaming (td-agent, fluentd) imports, Mobile (Android, iOS, Unity SDK) imports, Browser (JavaScript SDK) imports, Postbacks, and job execution.
Posted Oct 30, 2018 - 18:17 PDT
Update
In about 30 minutes, from 6:15 PM PDT, we will start the scheduled maintenance on our PlazmaDB Metadata database.
The maintenance is necessary to upgrade the PostgreSQL database engine with the latest patches (including security related ones).
We expect the operation to cause *no downtime*: the database will be online after the upgrade within less than 1 minute. This will be followed by a *recovery* period of around 5 minutes during which the system will gradually reach back to full throughput. During the maintenance and recovery, customers may experience the following:
* Streaming, Mobile, and JavaScript/Browser imports delay Streaming import (through td-agent or fluentd) requests will continue to be accepted as usual but the requests will remain queued until after the database maintenance is complete. We expect stream import processing to be further delayed during recovery. Thus we expect the stream import delay to extend to up to 5 minutes. The same will apply to import requests from Browsers (Javascript SDK),Mobile devices (Android, iOS, and Unity SDKs), and Postback requests.
* Jobs execution delay All jobs (Presto, Hive, Result Export, Data Connector, Bulk Import, Export, and Partial Delete jobs submitted from Console, API, Workflow or triggered by our system according to the configured schedule) will remain queued during maintenance. During recovery, we expect jobs to begin processing slowly: within 5 minutes job processing should reach regular capacity.
Beyond this notice, we will provide updates at the start and completion of the operation and once the verification has completed: at that time, all systems will have returned to full functionality and this Scheduled Maintenance will be closed.
Posted Oct 30, 2018 - 17:39 PDT
Scheduled
On Tuesday, October 30th from 6 to 7 PM PDT (October 31st from 10 to 11 AM JST) we will be performing maintenance on the PlazmaDB Metadata database.
The maintenance is necessary to upgrade the PostgreSQL database engine with the latest patches (including security related ones).
We expect the operation to cause *no downtime*: the database will be online after the upgrade within less than 1 minute.
# Impact
The database will become unreachable for the duration of the *maintenance* procedure, which should last no longer than 1 minute. We expect this to be followed by a *recovery* period of around 5 minutes during which the system will gradually reach back to full throughput.
All components reading/writing the Metadata Database have built-in fault tolerance that will allow them to retry the request in case of failures: when the connection incurs into an error, the request is retried several times and for long enough to ensure maintenance and recovery periods are successfully completed.
During the maintenance and recovery, customers may experience the following:
* Streaming, Mobile, and JavaScript/Browser imports delay Streaming import (through td-agent or fluentd) requests will continue to be accepted as usual but the requests will remain queued until after the database maintenance is complete. We expect stream import processing to be further delayed during recovery. Thus we expect the stream import delay to extend to up to 5 minutes. The same will apply to import requests from Browsers (Javascript SDK),Mobile devices (Android, iOS, and Unity SDKs), and Postback requests
* Jobs execution delay All jobs (Presto, Hive, Result Export, Data Connector, Bulk Import, Export, and Partial Delete jobs submitted from Console, API, Workflow or triggered by our system according to the configured schedule) will remain queued during maintenance. During recovery, we expect jobs to begin processing slowly: within 5 minutes job processing should reach regular capacity.
# Communication
Beyond this notice, we will provide updates approximately 1 hour before the beginning of the maintenance window, at the start and completion of the operation, and once the verification is completed. At that time, all systems will have returned to full functionality and the Scheduled Maintenance will be closed.
If you have any question or concern about this upgrade, please feel free to reach out to our Support team at support@treasuredata.com.
Posted Oct 30, 2018 - 09:40 PDT
This scheduled maintenance affected: US (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API, Data Connector Integrations, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow) and Tokyo (Web Interface, REST API, Streaming Import REST API, Mobile/Javascript REST API, Data Connector Integrations, Hadoop / Hive Query Engine, Presto Query Engine, Presto JDBC/ODBC Gateway, Workflow).