HarleyGuy said:
You know, when I started this thread I really just wanted to know what happened. At least more specifically than the computers 'went down.' I was curious because it seemed unfathomable to me that all the flights would be cancelled. I certainly didn't mean to open the door for guys bickering over weather forecasts, regs, and professionalism vs being a sucker. I've managed to gleam some info, but I still don't understand why people didn't simply adhere to the schedules they had. Could the company not dispatch? Were they unable to adjust to any delays and it all snowballed?
ps- let's take it easy on each other!
Your question was a reasonable one and you didn't get an answer. I don't know all the answers but I'll attempt a more direct explanation.
With respect to the storm -
Management was very aware that the storm was approaching. There is a physical limit to the quantity of deicing fluid that can be stored. That limit is the capacity of the tanks, which are the property of the airport, not the airline. They were full to maximum capacity before the storm arrived.
A senior CMR manager attempted to hold some flights at out stations so that they would no come to CVG and be trapped by the storm. He was overruled by his bosses.
The storm was more intense than forecast; much more. They type of precip required much more extensive deicing than anyone could have anticipated. As the supply began to deplete, more was ordered and was enroute by truck, which is the only way to get it there. Due to the heavy ice, the entire supply was exhausted within 24 hours. The delivery truck convoy was stopped by the State police, which shut down the hiway system.
That is why the supply was exhausted. It was not due to poor planning.
The Computer System Failure -
The tracking and crew scheduling systems are two different software programs from the same vendor that interface with each other. They are both older technology that needs replacement. There is no "mirror" backup system for these. The back up system is a manual system that has not been used in years. Most schedulers are trained in it, but most have never had to use it except for a few "old timers".
Due to the very large number of flight changes, reroutes, aircraft groundings (unable to deice) etc., the systems were overloaded and both failed. That causes the loss of all data.
To operate a manual system, you must know where you are before you can begin. When flying an aircraft, you can't navigate to any point if you do not already know where you are. To start a manual system, you must also know where you are. When all data was lost, there was no starting point. Therefore, the only thing that could be done was to cancel all flights, manually locate all aircraft and crews, and start from scratch. That's what we're doing while the IT's are restoring the computer systems.
There should have been a "mirror" backup system which stores data on separate servers and updates itself every 5 minutes. However, that costs a fortune. It may surprise you (and others) but most airlines, including the very largest, do not have "mirror" backup systems.
In fact I believe that UAL is the only large carrier that does. American now has a partial mirror, why ... because they had a failure resulting from an improper entry that canceled many flights at two major airlines (AA and U). In August, NWA canceled 120 flights because of a comparatively minor failure in their computer system. In May, Delta canceled 40 flights because of a minor failure in their dispatch computer. To my knowledge, no one has had to deal with a total failure of their tracking and scheduling systems, in the middle of a major blizzard. That's what Comair is dealing with.
After the system failure, the phone calls from and to crews were so high in volume that they overloaded the telephone system and the schedulers, preventing calls from being made or received for long periods due to busy signals.
I'm not making any excuses, just trying to tell you what happened and why.
Those who complain about not being called at their hotels (in this thread) couldn't be called because no one knew who was in what hotel. That's what happens when you lose all data. You don't know where anything or anyone is. All of our scheduling has been exclusively computerized for years. There are no "paper" schedules other than a print out of your line at the beginning of the month. When trips are disrupted and rerouted due to massive weather systems, those original printouts become useless. The trip you were originally assigned doesn't even exist any more.
Crews are not the only thing that has to be tracked. We also have to track the airplanes themselves and there limitations, maint., etc., and the flights and their routings and re-routings. Both the tracking and the crew scheduling systems must operate together. The main failure here is not the crew scheduling system, its the tracking system.
The majority of those ranting in this thread about the justification of "just going home" without company contact, are not Comair pilots. I'm not saying that some CMR pilots didn't leave and go home. What I am saying is that most of the ravers in this thread about why that's OK are not Comair pilots. We do have some people with that type of attitude, but most of our people are not that way and will help the Company in time of crisis. Like all other airlines, we also have an a$$hole factor. Don't judge the book by its cover, especially in an anonymous forum where we don't know who is who.
On Sunday we were able to operate about 175 flights. Today the target was 60% (I don't know if we made it). It's a big problem and most of our people are doing everything humanly possible to recover as soon as possible. We are doing our best to help our customers and each other in the crisis.
The major flaw in all of this is the absence of a "mirrored" backup system. Why don't we have one? Because it is extraordinarily expensive and no one anticipated such a massive failure happening all at once. Perhaps we should have, but we didn't. The only consolation is that most other airlines, much larger than us, don't have such sophisticated backup systems either. After this debacle they will probably get them. I bet we will too.
Maybe George Bush should have ordered all airlines grounded before 4 of them were highjacked and flown into those buildings on 9/11. But, he didn't. Maybe the earthquake in Indonesia should have been forseen and Tsunami warning systems installed in the Indian Ocean to avoid the waves that haven't occurred in the last 100 years. They weren't. Maybe the US should have such a system in the Atlantic Ocean, just in case there's a big wave there. We don't.
Comair and its people regret this failure more than anyone. We also regret that we did not anticipate it and therefore avoid it. All we can do now is our best to fix it after the fact. We are doing that.
Meanwhile, we sincerely hope that all the other airlines with their perfect systems do not experience a similar failure, before they can install their own mirror backups and fail-safe programs. Who knows, maybe after Mineta conducts his "investigation" the government can create a new Agency that will prevent future computer failures. I'm sure it will be just as good as the TSA.
I hope that gives you a better idea of what happened. I hope it also gives the naysayers a chance to achieve their own perfect systems. We will learn from our mistake, and I hope others do as well.
One final word --- it's all the pilots fault. If we would just agree to fly for free, the Company would be able to afford a mirror backup system.