NY Times: German power plant's Y2K problems

greenspun.com : LUSENET : Electric Utilities and Y2K : One Thread

From Friday's NY Times:

**It was March 1998, and most people here were greeting warnings about Year 2000 failures with a lack of interest. It seemed like a trivial problem, a matter of making sure that computer clocks would not misread the year 2000 as 1900.

**Then came the test at the Hanover city power company, organized by a confident data processing manager in the spirit of public education, with local media invited to watch. At first, everything seemed fine. But within a few minutes after "midnight," the computer began spewing out thousands of error messages. Then it froze up entirely, and the monitors went blank.

**Hanover did not go dark. But for a few minutes, it was impossible to monitor the electric grid or to trace equipment breakdowns. It took seven months to eradicate all the problems.

**"I really thought it would be fine," said Juergen Rehmer, the blue-jeans-clad manager who arranged the event. "We had made a lot of changes already, and I was quite certain that a full-system test wouldn't present any great difficulty."

-- Anonymous, July 26, 1999

Answers

Drew,

What did the rest of the article say? Your quoting a section of the article that talks about a test that happened 16 months ago. What section of the paper was this article in?

-- Anonymous, July 27, 1999


The key phrase here seems to be, "I really thought it would be fine." It appears that this belief is the driving motivation behind any Y2K-related lack of action. Unfortunately, for agencies, corporations, and utility companies, this attitude leads to missed bad code. It also leads to the "allay, allay" speeches from the government and others. I used to teach assessment and diagnosis in a large medical school. One of the things we taught our students was that if a child presents with a handicapping condition (e.g. Down Syndrome), then the physician must assume that every system in the child's body has a defect UNLESS AND UNTIL PROVEN OTHERWISE. So, the physician's job is to make multiple referrals to the cardiologist, pulmonary specialist, endocrinologist, etc. Very frequently, additional medical problems which are completely unsuspected are thus discovered which would otherwise be missed until they proved possibly life-threatening. I remember when I was still a student, discovering one little girl's deafness while on a home visit. Within weeks, her well-respected pediatrician was on the phone to me, livid and screaming that this child "does not have a hearing problem, and you're just a stupid student, etc. etc.) (Despite a previous bout of meningitis) Eight months later, the parents called me to tell me that their daughter had just been fitted for hearing aids to help her with her profound deafness. Posters on this site: Keep making the noises you're making, but please make them to the people who can change the rules. Write to your Senator (repeatedly) about the NRC changes, and stay on the senators' doorsteps until they do something. It doesn't help to continue to kvetch to each other in a web-site vacuum. I'm very grateful for the people on this website. This place proves to me that there are some very thoughtful, caring, and tenacious people around.

-- Anonymous, July 27, 1999

?,

The rest of the story does not deal specifically with power; it deals with the general state of readiness of Germany and other European nations. My excerpt (which includes the statement that the problems were fixed, although it took seven months) is the only part of the story which focused on power per se.

Ann is right that the key sentence here is "I really thought it would be fine." And yet, despite this confidence, it took seven more months to get rid of a problem officials believed didn't exist at all at the time of the test. One can only imagine that prior to this test these plant officials were "increasingly confident" that "there will be no problem" with a "we're the professionals here" attitude, etc etc. Note that the excerpt says that the test was organized by a "confident data processing manager."

Ann is also correct that in something like Y2K, which is a risk management situation, you have to assume the worst in your Y2K reviews, and go looking for the problems. From what I have read, it seems as if a lot of power companies in the US have done exactly this- ie, testing things they thought would have no problem, but they wanted to be sure nonetheless. But of course, the ones who did this would have also been the ones who started on time. What about the rest?

-- Anonymous, July 27, 1999


I think that one of the biggest problems we have with Y2K -- everywhere, not just in utilities -- is the corporate, bureaucratic attitude starting with "This Y2K Problem just cannot possibly be real".

That attitude, entrenched and widespread, has caused organizations of all kinds to short-shrift research and testing. Instead of doing serious, deliberate, probing research to determine if there is a problem and how big it is, far too many companies have substituted half-hearted, casual, shallow research designed to "allay" worries and to allow clue-free management to say aloud with a clear conscience that there really is no problem -- not a big problem, anyway -- and (whispering to one another) we really, really, oh, we really, really do not have to spend lots of money on this....

-- Anonymous, July 27, 1999


Drew,

BTW you do not include an quote saying that the systems have since been fixed. My only point in asking the question is that while Herr Rehmer had egg on his face from this test (rightfully so) this test occurred last year and they have had a significant amount of time since then to correct the problems. This is not (necessarilly) an accurate depiction of their current status. And in the end you have to give them some credit (even though it might be small) for conducting the testing effort, just imagine if they would have been arrogent enough to say they do not need to test.

-- Anonymous, July 27, 1999



Lane,

If, as you say, companies are not spending lots of money on this problem, how do you explain the projected $600 Bilion price tag for Y2k projects.

-- Anonymous, July 27, 1999


?,

>>BTW you do not include an quote saying that the systems have since been fixed.

Yes, the original excerpt says: "It took seven months to eradicate all the problems." I assume that means "fixed." And, oh, sure, I give them credit for admitting their mistake. Of course, they had no choice, given the public nature of this particular instance.

With respect to your question to Lane, the fact that an estimated $600 billion is being spent is not in & of itself sufficient to show that Y2K has been taken seriously enough everywhere. Perhaps the figure should be $1.6 trillion. Also, the fact that a lot of money is being spent does not mean it is being spent wisely (checked the federal budget lately? Or worse, what it was like pre-Reagan- not that it's not still bad now).

Lane is quite correct that Y2K has not been taken seriously in a number of organizations- although that number has probably dropped in the last several months to a year. But there have been various cases of brain-dead nonsense going on in important facilities. But, this gets a bit off-topic...

-- Anonymous, July 27, 1999


Drew,

First of all thanks for the excerpts. I would like to have seen the entire feature but if this was used as a Y2K war story to make a point for the entire article, it was a good one. Just as a successful demonstration would have been a good one. As I understand it, there have been successful ones. Interestingly, according to the excerpts, Hanover did not go dark.

Well then, what did happen and what were the equipment breakdowns? How long was the recovery? Surely not seven months! What was the primary failure, on what system and was it confirmed as Y2K related? Was it related to remediated code? What problems were eradicated? Is he now prepared to give it another try? Sure leaves a lot of questions. I have to assume that Mr. Rehmers confidence in his public demonstration was for a good reason because I cant imagine inviting the public and the media in witness a presumed failure. I am sure Mr. Rehmer had exhausted every effort to ensure that it was a successful demonstration. If this is not the case well.. go figure!

However, the important issue that I see in this case should be What did they learn from the failure? Was the lessons learned applicable to other systems?

Perhaps this example in Hanover can provide some answer to the intriguing Y2K vs Ready disagreement mentioned in this forum. Obviously Hanover was not Y2K ready. Just as clearly, only hardware and software systems can be made Y2K compliant. Our own mission- critical and some non-mission critical systems have been tested as compliant and we are confident, as was Mr. Rehmer, that electric service availability during the rollover will not differ from today or any other day. Your excerpts suggest that being compliant is just not good enough. Therefore, I submit to you that it is at least equally, if not more important to be Y2K Ready. We are Y2K Ready. ;)

-- Anonymous, July 27, 1999


?,

The primary concern I have with the excerpts that Drew posted is the fact that this company was testing so early. With my limited knowledge that would seem to put them by FAR in the minority.

I still cannot for the life of me figure out how my power company is now compliant or ready or whatever. Last May, they stated they started in 1995 and were halfway through fixing the problem. Simple math would indicate a problem with this. But at the same time they stated that they would be "finished" in Dec/98 but that they hadn't yet fully evaluated embedded systems. Testing isn't mentioned at all and yet I have heard nothing but the fact that testing is the part of the process that takes the most time. Now, in June they are compliant or ready or whatever. Geez, talk about confusing.

This kind of thing is what has me concerned the most. Concern not just for my own comfort, but more importantly for our country to operate at capacity without the proverbial "domino effect" hitting many areas of our economy.

Norm

-- Anonymous, July 27, 1999


If anyone wants to read the entire story, it's at:

http://www.nytimes.com/library/tech/99/07/biztech/articles/23year.html

You do need to register (free) for the Times on the Web.

-- Anonymous, July 28, 1999



Thanks, Drew. That about covers what I would have said.

-- Anonymous, July 28, 1999

Moderation questions? read the FAQ