On the Necessity of Contingency Planning

greenspun.com : LUSENET : Electric Utilities and Y2K : One Thread

I have found myself avoiding the topic of contingency planning as it relates to the electric industry and Y2K in general. It seems too fatalistic, not glamorous enough, certainly not as riveting as trying to tease out possible failures and grand remediation strategies. This is disturbing on many levels. First, it was this lack of attention to what might strike one as mundane that initially contributed to the design flaw of Y2K. Remediation of Y2K, is often described as tedious or boring work, I suspect the original date dependent programs where themselves often not as enticing as designing the latest graphical user interface or java applet.

Two previous posts also, have me reassessing the need for emphasizing contingency planning in the electric industry. Gordon explained how the loss of manual versus, computerized hydraulic, back-up controls in airplanes leads to a dependence on complex systems. In another recent post, Fact Finder, asks for model numbers and part numbers documenting bugs that would cause a generator to trip off line and states he knows of next to none. I disagree that the electric industry is not substantially exposed to the Y2K design flaw, however, many experts have expressed an opinion that the degree of exposure to Y2K problems in the electric industry is not as great as many other industries. As De Jager said "They lucked out". This may be nonsense or right on, how am I to tell. But it certainly raises the possibility of successfully reverting to "manual" control. The key though to successful contingency planning, is not to rely on a redundant system of great complexity, but to use simple, human controlled back-up systems, or to use Gordon's airliner analogy don't add a second and third hydraulic system but keep intact the mechanical, human operated, flight controls. To emphasize the need for simple (KISS) type contingency plans I want to go way out on an interesting limb for a moment, a journey into the field of CHAOS Theory (oooh scary). I'm not a mathematician but this example illustrates why I remain uneasy about complex systems which will encounter a new perturbation to their equilibrium in Y2K.

Consider the very simple equation f(x) = rx(1-x) or put in words take a number between 0 and 1 call it x, multiply x by (1 minus x), take this result and multiply it by a constant r. The result equals f(x). For instance, lets set r equal to 2 and x equal to 0.2 the equation then writes out as f(x) = 2*0.2 *(1 - 0.2) or 2 * 0.2 * 0.8 = 0.32 To keep playing with this equation one then sets x equal to the solution, 0.32, and the equation is performed or iterated again. This could be repeated numerous times in a computer loop. Eventually, x should settle down to an equilibrium value, in this case 0.5. No matter what initial value is used for x it will over a number of iterations settle out to an equilibrium value of 0.5 when r is set equal to 2. Not very interesting huh, could be a Junior High school math problem. Could an equation this simple ever misbehave. Well, yes. Here's the neat part, if the constant r is raised to equal 3.2, the value of x settles out to two separate values that it oscillates back and forth between. At a slightly higher r value x bounces between 4 "answers". At r values slightly higher, x never settles down at all, it behaves chaotically. (There are some underlying patterns even within this chaotic behaviour and I refer the reader to the book referenced below for a fuller discussion. It also helps to use a computer to graph the equation, Fractal Programming in C might be of help there, it graphs out what is called a bifurcation diagram). What in the heck does this have to do with real world systems? The following quotes James Gleik's (A polly BTW last I heard) Chaos: The Making of a New Science.

"In the physical systems from which these equations were borrowed, that parameter (the r value) corresponded to the amount of heating, or the amount of friction, or the amount of some other messy quantity. In short, the amount of nonlinearity."

Possibly even on an intellectual level we are still understanding Y2K "only through a glass darkly". I suspect, that to date nonlinearity has been avoided in complex systems through a combination of common sense and trial and error. However, in even seemingly simple systems like that modeled above, nonlinearity or chaotc behavior is lurking. When the operating parameters in the electric industry are altered in unexpected ways will even compliant systems fail. For instance, if a Ford plant and a GM plant suddenly shut down will a compliant grid suddenly be thrown into an unstable, chaotic state leading to damage and/or shutdown of the grid in that area. I can't reference the best example of this type of worry that I have however the story as I read it concerned a disaster analysis specialist who told of a scenario where at a large internet hub, a single fuse blew. Everyone in the room waited breathlessly to see if the small waft of smoke from that fuse would trigger the buildings sprinkler system, destroying the hub and disabling a large portion of the internet. The sprinkler system didn't go off, but if it had the fuse and sprinkler system still would have been performing exactly as designed, but they would have created a system of chaotic behavior. The upcoming problem may in fact reveal that the electric grid is very robust. I imagine, though, that many programmers have used logic more complicated then the above equation in their embedded chip logic and other programs. Who knows, perhaps unused attempts at some of these programs had behaved erratically (chaotically?) for unclear reasons. In short, especially at this late date, I think we need to stop focusing on attempting to perfect and perfectly remediate complex systems whose behavior we may not understand as well as we think, and start focusing on the practical common sense part of contingency planning. Contingency manuals, making sure that experienced operators are available and in the right places at the right time, low tech drilling. Let's start focusing on the human element of the equation, the individuals who have a world of experience in dealing with chaotic situations.

-- Anonymous, September 05, 1999

Answers

christ, paul, i'm still reeling... you talk about keeping it simple and then went off into an amazingly complex diatribe.

there are a myriad of posts under nuclear that outline the dangers of reverting to manual. one of the most obvious is that few remember how to do it and if they do... they are not employed at that plant any longer.

i suggest that you research the impact of deregulation on the safety and staff at the electrical facilities and then...

read what caused the melt down at tmi and see if you still come back and suggest the benefits of reverting to a manual mode of operation.

-- Anonymous, September 05, 1999


RIGHT ON SISSSSSSSSSTAH!! Damn, if that is a 'simple' equation, I hope he never gets into the complicated stuff! Sometimes manual is the way to go.... and sometimes its not all that its cracked up to be. Ever cranked the landing gear down on a twin engine plane as you circled around and around in front of the tower and the fuel gauage was bouncing off empty?? I have...I get diarrhea and break out in a cold sweat just thinking about it.

Taz

-- Anonymous, September 05, 1999


I believe one of the reasons why Rick asked us to begin using a password system on this ListServ was to protect all of us from the extremes of emotionalism. Granted what we are all concerned about has to do with our fundamental wellbeing. However, by judging someone's way of presenting useful information does not bring us together in ways that can lead to solutions.

True, P.D.'s presentation is based on math that many of us are not familiar with. But, it would be useful to understand that 1) he did not use the math to talk down to anyone nor to muddy the issue he presented; and 2) there are many of us who do understand the math, if not completely, certainly enough to see the logic it alludes to.

Please understand, I am not trying to pick sides here. Just pointing out that it is more productive for us to engage in a dialog if we don't understand rather than to take issue with someone's editorial style.

It comes down to this: We as a society have not been smart enough to avoid the problems we are currently facing. Will we, or, are we smart enough to know how to work together so that we may address meaningful solutions?

-- Anonymous, September 05, 1999


Let me make it simple for everyone:

Not enough/no fuel = no electricity.

-- Anonymous, September 06, 1999


Simplicity is often in the eye of the beholder. Pauls fine post may speak simply to some readers yet confuse other equally intelligent readers whose frame of reference in everyday life is not mathematically based. Chaos Theory has been brought up twice on this forum recently, and since I believe that a frame of reference can be found to explain a theory to anybody so it can be understood, Im going to tackle Chaos. I think it can be tamed and now that schools are going to be in session again, what better time?

So who wants to do Chaos Theory 101 with me? Wow! Look at that show of hands! (Hear those groans, more likely....*wink*. ) Seriously, gentle readers, follow along and I think youll discover that you already intuitively know a whole lot about the basic principles of Chaos Theory. No math here, nobodys going to fail this course and better still theres not going to be a test at the end!

Chaos Theory is also called Complex System Theory. For centuries scientists and mathematicians tried, and often succeeded, in coming up with various formulas for predicting (or modelling) the outcome of certain physical behaviors. If youve seen the recent movie, "October Sky", youll remember the scene where the two boys use mathematical formulas to help them discover where one of their missing test rockets should have landed. When they get to the area they still cant locate the rocket. One of the boys asks, "Was the wind blowing that day? If it was, it probably was coming from the west. It could have pushed the rocket so it would have landed over there instead...". Sure enough, they find the test rocket.

Now, a man experienced in rocket science would have known about factoring in wind velocity and direction when trying to determine a rockets path, or trajectory. Some things are quite predictable because of various Laws of Physics already formulated over the centuries. What this same rocket scientist could not predict any better than anyone else is how any woman he might meet would react to the question, "How ya doin, Babe?" Nor could he always accurately predict the weather. People and the weather are examples of complex systems. They constantly surprise us. In fact, one of the most essential components of a complex system is unpredictability, simply because there are so many, many tiny factors which are part of the system.

About forty years ago the common sense notion expressed by the following ditty hit the world of mathematics and Chaos Theory was officially born:

"For want of a nail, the shoe was lost; For want of a shoe, the horse was lost; For want of a horse, the rider was lost; For want of a rider, the battle was lost; For want of a battle, the kingdom was lost!"

This is now a big scientific deal, you ask? Yep. Everyone may now repeat after me, "Duh! Those scientists don't get out much do they?" *laughing* In all fairness, however, the mathematics involved can be as complex as the systems, they are a big deal, and certainly even attempting to discover new predictive patterns among such complexity is an ambitious undertaking. But the basic concept is a no brainer for anybody who has ever tried to figure out why a man or a woman did or said something completely different from what you had expected.

THIS is Chaos Theory? See? I said youd already know about it, didnt I? Another way of talking about Chaos phenomena is called the Butterfly Effect. In common parlance you might hear a phrase like, "A butterfly flapping its wings in Hong Kong might cause a thunderstorm in New York." In more scientific terms, to be able to precisely predict the weather youd need an exact replica of the earth, because all the tiny things like butterfly wings flapping would have to be factored in to any model.

Ill bet youve all watched the Olympics or various sports competitions at one time or another. When there are two athletes competing who are nearly equal in their abilities, maybe youve heard an announcer say something similar to, "Boy, these guys are both so good this match could come down to who ate the most breakfast this morning!"

Youve got the idea, dont you? Chaos Theory is a science of surprises; of trying to understand and pattern things that are almost impossible to predict like weather, emotional interactions, stock market vacillations, and on and on. One of the defining features of Chaos Theory is that chaotic systems are sensitive to initial conditions and even a very slight change in the starting point can lead to large or importantly different outcomes.

I can envision a lot of "Clicks" going on in readers brains while youve been reading this. Some of you are bound to have thought to yourself, "Heck, since I first started looking into Y2K I figured our global system of computers was just too complex and interconnected for anybody to able to predict for sure whats going to happen when little date problems are introduced into it."

BINGO!! You are a Y2K Chaotician! You get to call yourself Genius for the day. You also get bonus points if you saw the movie Jurassic Park and remember the character of Dr. Malcolm, played by Jeff Goldblum. He was a Chaotician. Remember the part where he drops water on the leading ladys hand to explain all the tiny little aspects which could affect which way the water ran off her hand? Remember his later phrase, "Life will find a way,"? Are you now mumbling to yourself that, "Y2K will find a way."?

If you are, theres only one thing to keep in mind. Unpredictable effects do not necessarily mean disastrous effects, nor do they mean benign effects. Unpredictable is just that. What can be predicted is that changes in the starting condition of a very complex system _will_ cause significant variations in outcomes from the original behavior. And when the original behavior was predicated upon reliability and accuracy, it's logical to extrapolate that changes in the outcome could include unreliability, inaccuracy and other unpleasant conditions.

The Year 2000 date problem will enter changes in the interconnected complexity of our global computer systems, in combination with human interaction with those systems. While small or individual system changes can be predicted and/or mitigated to a large extent by testing, these individual systems and the people who designed and work with them are only a small part of the vast complex system of computer processing. Certain prediction of the results of the millions of tiny 2000 complex system changes is impossible, but we know there _will be_ outcome changes.

Therefore, that makes contingency plans to deal with potentially bad outcomes a smart conclusion, both from a mathematical and common sense position. Since I think most of the readers of this forum arrived at a decision many months ago which supported both Y2K personal risk management strategies and corporate/governmental contingency plans , you were already on top of the implications of Chaos Theory, now werent you? Give yourself an A+, and what say we go watch Jurassic Park again, just for fun? Better yet, rent October Sky, it's a great film!

Oh - one last thing. Best case is to always study up and think things through for yourself. Don't take any "teacher's" word for anything unless it makes sense in the context of your own life and endeavors. I'm sure a Yahoo search (or any other) will bring up info on Chaos Theory.

-- Anonymous, September 06, 1999



Bonnie,

That was a fabulous posting. I loved the simple way you brought that whole matter into focus. Could you do one like this on the Theory of Relativity. I'm still struggling with that one, and it does have a lot to do with "time" I think. Perhaps Y2k is relative, for that matter. Anyway, you are truly a star asset on this forum. I hope you are getting a decent cut of the profits. Rick, she is getting a partner's cut of the profits, right? Sure. ;-)

-- Anonymous, September 06, 1999


Thanks, Gordon, I'm glad if the post was helpful. I know you were joking about the profits, but since it may start others wondering, I'll post here what I once had to put on the Yourdon forum. First, I believe very few people are making any appreciable amount of money due to Y2K involvement, in contrast to what the popular conception is. I think there are many more individuals, both well known and lesser known local activists, who have expended a lot of money in Y2K awareness efforts for which they will never be reimbursed, and don't expect to be.

Second, it doesn't bother me a whit if anyone does see a profit for their efforts. That said, I went into Y2K research and awareness promotion as a volunteer effort and long ago decided I would keep it that way. That's why I have turned down paid offers to write, and not accepted various gifts of free what-evers. This works for me and does not lock me into any kind of schedules which might interfere with other aspects of my life, especially since there are times when I must (and want) to be available for family needs. ( Yes, I DO have a life away from this computer! ...grin )

Now on to the Theory of Relativity. You just had to bring that one up, didn't you? *laughing* Fortunately for me, I don't think it's on topic for this forum so I get to pass you on to a paper by Kenny Felder titled, "The Day The Universe Went All Funny". I personally think this is a very fine introduction to special relativity, and it doesn't assume any math ability except basic addition and subtraction. Give it a try if you haven't already come across it:

http://www2.ncsu.edu/unity/lockers/users/f/felder/public/kenny/papers/relativity.html

-- Anonymous, September 07, 1999


Thanks Bonnie,

Yes, I was just joking about the profits thing. I really didn't think Rick was making anything on this forum. Heck, I don't see any advertising around here. I shouldn't have even suggested such a thing, but at least it gave you an opportunity to set the record straight, for anyone who might have been wondering.

I have gone over to the relativity site and bookmarked it for further study. It looks very interesting, but will have to wait for when I have some more time, soon I hope. I appreciate the URL. You are surely a wealth of interesting information. Say, you don't have any links to pictures of those EDGs, do you? All this talk and all I have is my own imagery which is probably making them too small or too large. A picture is worth a thousand words, right?=8^]

-- Anonymous, September 07, 1999


Bonnie, when I started reading your post I immediately thought that you should read the book, "Jurassic Park," because Dr. Malcolm explained the chaos theory beautifully. The book was superior to the movie in that regard. For me, I believe the chaos theory can be explained by the word, "unknown." When I say "not enough/no fuel = no electricity," the chaos theory is implied in "not enough/no fuel." It is the interconnectedness of systems and the people working in these systems that spells out the chaos theory. The equation that fits, it seems to me, is "systems interdependency + personnel = outcome unknown."

-- Anonymous, September 08, 1999

Bonnie,

Since posting my request that you supply a picture of an EDG, I note that FactFinder has done exactly that for us. If you drop down a few threads to the one the starts Emergency Declared at Nuke.... you will find his link to a very nice picture and some data about EDGs. So, thanks again FactFinder!

-- Anonymous, September 08, 1999



This got more press than I would have thought and a special thank you Bonnie for making a difficult subject more palatable, I warned that I never majored in math. If we skip the math, what I am getting at is that complex even "intelligent" systems have no "common sense". The vast amounts of data we can order and process is astounding, is not equaled by computers and often involves tasks we perform without even considering them. No computer can recognize a face, yet we can with a glance, recognize a face last seen ten years prior, all with hardly a thought. To try another analogy here, if we are paddling in a canoe in smooth waters right now, we can easily let a program say one stroke to the left one stroke to the right. But there are rapids ahead, at that point you want an experienced canoist guiding things with subtle shifts of body weight and well timed paddle strokes. The EUY2K equivalent to this is seeing to it that an operator with 20 years experience is making the decisions about the stability of the electric grid, not a flawed piece of programing. To allow for this you need to have manual back-ups available, you need to have that hand crank that Taz mentioned. You of course don't try to complicate things by doing everything manually when there is no pressing indication for it. You do leave the decision making, to the extent that automated process can be controlled, in the hands of a thinking person. Then as Noah asks, "are we smart enought to work together?" Maybe all this is, As GN seems to take so much delight in proclaiming, an insurrmountable task. But whether it is or not, let's at least try to have some dignity and trust our fate to God and one another and not if we can help it to the flawed, and all too controlling, works of our hands.

-- Anonymous, September 09, 1999

Moderation questions? read the FAQ