Some analysis of DNS denial of service attack

You can read about the recent denial of service attack here.

New DNS provider

Our production DNS infrastructure has been migrated to a new provider. We are awaiting complete propagation, but services should be returning to normal for the majority of users

DDOS of DNS

A distributed denial of service attack (DDOS) occurred (and is still happening) with our DNS provider (Zerigo) -  this caused many users to be unable to look up DNS names (web addresses) - this affected quite a few people. We switched to a DNS server that was not under attack, and then to a new provider. However, changes with DNS take time to propagate (it often depends on your ISP). Services should be returned to normal for most users by now. More posts to follow with details on this and mitigation.

Application Deployment Outage - Resolved

We detected a high number of errors related to application deployments.  During the investigation, the API endpoint was taken offline and a fix was applied to deal with the error.  Any applications that failed to deploy were restarted.

The API endpoint is now back online and all systems are operating normally.

Applications restored

Maintenance work completed. Our apologies for any apps affected. Systems are normal now. Should you have specific problems with your apps from this point, please do issue a restart.

RUN@cloud rolling restarts

The restarts/maintenance work on RUN@cloud is taking longer than expected, some applications may notice ongoing outages until this is completed, sorry for the interruption to service.