-tom
[UPDATE:]
[UPDATE:] The system is still having problems and needs to be taken down again. More updates shortly...
[UPDATE:] We're up and running for the moment but will have to take the system down again in the late morning to replace the hardware.
[UPDATE 11:48am:] We will be doing a quick reboot at noon to put us in a better position to replace our hardware and gather additional information. We should see appx. 12 minutes of downtime during this procedure.
[UPDATE 2pm:] The previous reboot ended up taking appx. 4 minutes. We're in much better shape but still have some testing to do which will require some downtime for reboots. We plan on initiating these late tonight so as to minimize the general impact on things. We should all wake up to a more permanent solution to the problems we've seen today.
Once again, my apologies for the outage and extra work these outages must cause you as a result.
-tom