On the afternoon of May 8^th after a brief ceremony attended by all the BSouth e-business, and our staff, the east entrance was officially opened to the public.  Congratulations are extended to all that contributed to make such a success for our clients and us.  Special thanks to J R in his excellent quest for perfection, and for making us the best data center in BSouth.  As a Vendor branch we should feel very proud of our accomplishments, but also more aware of our new added responsibilities.

Clean, Clean, Clean

   This is the new motto of our branch, but for some of us this sounds like a bell going off somewhere.  You have noticed the change in our data center, and so the customers.  Just look at our loading dock!  One of our operators was so impressed that he thought he was in another data center.  Some of us have degrees and others technical licenses and none of us like to clean.  We must be honest and fair since some of our operators are doing most of the cleaning.  We all should do our part, and enjoy when a customer makes a comment on the excellent condition of our data center.

   Commendations to: Chris, Fred, and Ray . "The lights in the  control room turned to darkness, alarms and horns of every kind went off, equipment stopped… there was silence for a moment…. Anxiously waiting for the lights to come back.  It seemed endless. 

   This is what happened on the afternoon of April 25, 2001 at about 5 PM when a storm-related power failure hit the facility, an experience we haven't encountered for a long time.  But the spirits of our operators on shift were up.  There was fear to journey to the unknown and unexpected…but there was courage…. For an operator was not alone by himself.  He has others to help him out… and together as a team they succeeded. 

Their struggles - to bring equipment up, to stabilize the plant, to troubleshoot and repair an electrical problem - came out to be their shining moment, …an absolute test of their skills, their responses, their decisions, their persistence and their enduring commitment altogether to maintain a high level of performance.  These operators evolved to be our heroes this week. They have performed very well in front of our client, under the scrutiny of our customers. They received commendations, written and verbal for a job well done." (As quoted by Nestor , Site Operations Manager)

What can we learn from this incident?

Rule No. 1.  Do not panic.  Nestor quoted an important truth, "We are not alone".  There will always be someone we can turn to for help in case of need.  You have procedures that can guide you step by step in any problem.

Rule No.2.  The operator assigned as monitor has the control of the monitor station, and he is solely responsible for the actions of his crew. (Check the monitor's responsibilities) It should be clear that no one that is not a certified operator should be at the monitor station during an emergency.  The monitor has the right to request help from other persons present during the emergency.  He will dispatch his crew to the different areas to be checked.

Rule No. 3.  Establish secondary flow at the chiller plant immediately to avoid a loss of cooling on the floor.

Rule No.4.  Do not pay too much attention to the System 600-alarm bell.  Do not ignore the bell, just at the beginning of the emergency, concentrate on your main systems, The UPS, Generators, Chiller plant.  You will have time to acknowledge all alarms.  There is an average of 99 alarms going off at the same time.  If you try to clear all alarms and at the same time you try to access a graphic on the System 600, the System 600 network will slow down and you will not be able to access your graphics fast enough.  In turn this will confuse you more and will delay more the restoration of the chiller plant. 

Remember: at the chiller plant, verify secondary pump operation, check the towers for any alarm flags.  In case of a repeated FPL bump, put your chillers in operator mode, to guard against having more than one chiller going to autocycling mode. Clear you alarms from the running chiller and re-start the chiller, or select another on. In this case, since your chillers are in operator mode, select "Operator On" to start the desired chiller.  After the plant is stable, you may return all your chillers "START BUTTONS ONLY" back to none.  Remember that no other equipment should be in OPERATOR MODE.  Only the chiller START buttons.

During the last emergency, after Fred had the chiller plant stable, he noticed that the chiller was surging (this sounds like a huffing and puffing noise) this is caused by high condenser pressure, which can be caused by lack of condenser water cooling.  Fred discovered that the cooling tower #2 frequency drive had stopped, due to a blown fuse.  They replaced the fuse, and the system started lowering the head pressure on the chiller.

Commendations also go to Doug.  During one of his many trips to the chiller plant, (with the log clipboard under his arm) he found one of the air dryers not working.  After consulting with Nestor he replaced the dryer, and solved an impending problem that could have possibly escalated to a more serious one.

Three new members have been added to our group.  Mark; power plant operator with FPL.  After a small incident in which he was slightly bloodied, Mark has bounced back to be and excellent operator in our group. Bien, a reliable and able operator with an affinity with dark under the floor spaces.  John came to us from the windy city, looking for some warmth.  We welcome all in their respective duties and wish you the best of times with us.

We have now 846 days without an incident.  Let's keep up the good work.

Tip of the week: Chiller 4 take longer to reset after a shutdown.  Keep this in mind when re-starting the system.