The API service with new instances are running properly.
Jun 13, 14:24 HKT
Between 2017-06-12 21:18 UTC and 2017-06-12 22:16 UTC, the API service was down. Meanwhile Secure Web Portal and Apps were affected as they were depending on the tracking API service. The issue was resolved and we keep monitoring the service.
Our API service is running with automatic scaling mechanism. During the incident, our system failed to allocate new instances automatically. Hence no service is available.
We have resolved the issue by changing the instance type manually and restart our API service. While instances are successfully allocated, the API services are resumed back to normal.
Our API service is running on Amazon Web Service (AWS) EC2 platform. It failed to allocate any EC2 instances automatically (instance type c3.xlarge, c4.xlarge, m3.xlarge, m4.xlarge) in AWS region "us-east-1" during the incident. We manually changed the instance type (c3.2xlarge and m4.2xlarge) and EC2 instances are successfully allocated.
Jun 13, 10:45 HKT