Please see http://status.cloudbees.com for status indicators and high level system status information.
For support, please visit support.cloudbees.com or email support@cloudbees.com

Tuesday 3 November 2015

DEV@cloud global restart - Java 7 update

We will be performing a Java upgrade and global restart of all Jenkins instances in DEV@cloud.

Purpose:

  • patch Java 7 to latest update
  • deploy Java 8 so it can be used on beta customers (in preparation for global rollout)
  • allow individual customers to be switched to Java 8

Window

  • 4th November 7am UTC - 9am UTC

Impact

The outage will be momentary for customers as their Jenkins restarts.

Due to how this patch to the environment is applied it is not possible for us to hold off this restart for individual customers.

Our monitoring systems will tell us if your Jenkins has not come back up cleanly, however in the event that you do experience issues, please raise a support request via the normal means.

Post Outage Review

There were a small number of Jenkins servers in our production environment running an older base operating system.  These older instances did not upgrade to our satisfaction - and so we made the decision to terminate these instances and reprovision customer Jenkins on newer and faster hardware.

While this was not ideal timing, the work was completed largely within the outage window - but not as quickly as we would like.

Improvements

We are reviewing the way we communicate outages with customers - in this case we did not have sufficient time (for operational scheduling reasons) to communicate this particular upgrade.

We are also reviewing the Jenkins behaviour of displaying a stack-trace to the user rather than something more useful.

There are also changes being made to our hosted Jenkins platform to improve the resilience and stability.