Remote Access service disruption - Singapore Traffic Manager

Mitigation steps:

1. We will now issue manual restart of the traffic manager immediately when such an alert is received.

2. We are working on adding automation steps on the traffic managers that will auto-fail over in this scenario.

On October 11th at 12:15am UTC we received alerts that our Singapore traffic manager stopped communication with our main data centers - Causing users routed to it to not be able to use the Remote Access service. We use traffic managers to manage users' communications with our main data centers via SSH tunnel.

This should result in an automatic failover where the connector is routed to a different traffic manager, however in this incident this was not the case due to the connector kept on communicating with the traffic manager - But traffic didn't reach the main data center.

At 4:15am UTC we manually restarted the Singapore traffic manager which resolved the issue.

Mitigation steps:

1. We will now issue manual restart of the traffic manager immediately when such an alert is received.
2. We are working on adding automation steps on the traffic managers that will auto-fail over in this scenario.