More infornation from Webpal on embedded chips

greenspun.com : LUSENET : Electric Utilities and Y2K : One Thread

Here is another piece from Webpal on the subject of embedded chips.

The material was posted by "faith" on the Yourdon site.

http://www.Webpal.org/Gas.htm

-- Anonymous, April 09, 1999

Answers

I hope someone who is experienced in the "embedded systems" field will respond to this. I found it very interesting, but don't know how accurate his conclusions are.

Thanks for bringing it to our attention.

-- Anonymous, April 10, 1999


HOUSTON - WE HAVE A PROBLEM!, Boy, do we ever. So many of the engineers have stated (with complete conviction and honesty) that there is almost NO problem with the embedded systems, that it was a bunch of ballyhoo to begin with. Guess what? The situation is just SLIGHTLY different in reality.

The minute I read this article (quite, quite enlightening), I printed it off and brought it to the attention of TWO different college professors, both chair their respective departments in computer science and engineering. After reading the paper, they BOTH said the same thing, "This guy is dead on the money". Unfortunately, the people who are constantly working (and testing) around these "devices", are more often than not, not aware of the idiosyncrosies (sp?) of the microprocessors.

If all of this is true, and I must admit that even after verifying it I'm having a problem accepting the facts, then we have a much more serious problem than anyone ever contemplated. Consider the ramifications: All of the processors in the chemical plants, electric utilities, nuclear facilities,etc., etc., start to act quirky come the new year. These would be the same ones that had been certified as Y2K complient. The same ones nobody will be paying much attention to, considering they need to watch over the systems not made "complient".

Anyone else have a different take on this? Engineers? Utility Techs? Rick?

I hope this guy is wrong or we are in an awful lot of trouble! Jeff

-- Anonymous, April 11, 1999


If I can understand the maze of circular logic and lengthy nonsense this link is spewing forth, all this guy is trying to say is that some devices not only have a "clock" that operates at 50-500 MHz, just like on your computer at home, but also a Real Time Clock (RTC).

This is nothing new, and the way it is being presented is just more Y2K hype.

bob

-- Anonymous, April 11, 1999


What I think he is trying to say, is the primary clock is in fact the oscillator running the micro-controller, and the secondary clock is a possible built-in RTC, either in the micro-controller, or the ASIC.

I agree with what he is saying, for the most part, but I think he glossed over the epoch timing issue for the RTC's. If they are set at the factory to some arbitrary value, then they will rollover at some arbitrary time, and not necessarily at midnight on Dec 31.

Now, if a third-party vendor programmed the micro-controller, and knew it had a battery-backed up RTC, and set the clock properly before the embedded system was delivered to the next tier in the supply chain, then it is quite possible that those systems will do what he describes come midnight Dec. 31.

BTW, Dallas Semiconductor makes several different RTCs with built-in (in the chip) 10-year lithium batteries...

Jon

-- Anonymous, April 11, 1999


I would also like for Rick to give us an opinion on this piece. I read the whole thing last night and I will be the first to admit I understand little of embedded systems but from what Mr. Beach says, there a lot of others that do not understand either.

-- Anonymous, April 11, 1999


What will happen when the secondary clock stops ticking ?

You have to reset the clock everytime the power goes off.

This is not a y2k-problem, but it can happen anytime.

-- Anonymous, April 12, 1999


There's also a post on the Yourdon forum that addresses the Webpal thing. Scroll down to the second Paul Davis post at the following thread: http://www.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=000hbh

-- Anonymous, April 12, 1999

Menno wrote:

> What will happen when the secondary clock stops ticking ? > You have to reset the clock everytime the power goes off. > This is not a y2k-problem, but it can happen anytime.

In many cases, that is true. However, as I stated before, some real-time clocks come with built-in lithium batteries (ten year lifespan). I can provide manufacturer URLs if anyone is interested.

Jon

-- Anonymous, April 12, 1999


I mailed a copy of this forum discussion to Bruce Beach himself and he was kind enough to reply, here it is: Subject: Microprocessors on the Y2K forum Date: Sun, 11 Apr 1999 23:07:22 -0500 From: Bruce Beach To: "'cats7@prodigy.net'" CC: "'mbec2@home.com'" Hello Jeff and Mike, I received inquiries from both of you regarding the postings to the Y2K forum. Perhaps Jeff could take the lead on this because it seemed more of the thrust of his email and post the following reply since I am not a member of that forum. Thank you both very much for bringing this to my attention. My reply------------------> Let me assure the writer that you have identified the correct Bruce Beach. My degree is in Economics, a subject which I taught in addition to computer science. But, my being a Great Grandfather, I sort of got into the computer field under the grandfather clause, because degrees in Computer Science were not that popular when I got into the field. Actually, I had over sixty courses regarding computers, from companies like IBM, Honeywell, Control Data and others. So you will just have to forgive me for being OLD. However, I wouldn't expect you to forgive me for being wrong. I can actually add little to what I said before, but I CAN add a lot more AUTHORITATIVE sources. In the last 48 hours (actually now over 50 hours) since I published the page on the web http://www.webpal.org/Gas.htm there have come to many, many other sources of sustaining information. I have also had the privilege of dialog with several Ph.D,s (Whom I somewhat assume to be in the field of Computer Science, although I cannot know for sure). The end consensus of those who are not greatly concerned seems to be as follows (and these are actual quotes)----------> "Nothing that you say is wrong per se." "the possibility to which you refer does exist but is extemely unlikely" On the other hand, there were numbers more that were not nearly so sanguine, and they provided me with a great number of what I consider to be very authoritative (and usually conservative) resources that quite increased my pessimism over when I wrote the article. Still, I do not know the answers, and I willing share with everyone such information as I have on which to base such conclusions as I have. First, before I clarify some terms, I do NOT wish to apologize for not using "professional" terms like "oscillator" and "RTC" because it is the very use of such well defined concepts that leads to confusion when trying to explain a new and different concept, particularly in terms that can be understood by the layman. Still, while every knowledgeable professional could understand that by the PRIMARY clock, the heart beat, the drum beat, that I was referring to what they call the oscillator-- there was still confusion about what I was referring to as an External Clock By the External Clock, I meant what most of the correspondents refer to as the RTC. (Real Time Clock). Most of the responses referred to only these two types of clocks but I tried to distinguish a third type of clock, which some of the correspondents also discussed, but which they also identified as a RTC. It was to avoid this confusion that I described a SECONDARY clock and how it could be conceptually distinguished. Jerry B (skeptic76@erols.com) described very well the confusion: "Bruce uses the terms primary clock, secondary clock, and external clock. He clearly describes what he means by the term primary clock; it is simply an oscillator. His term secondary clock seems to refer to what most would call a real time clock (RTC). Unlike what he calls the primary clock, the RTC would keep track of dates, and could have Y2K problems. Whether or not the RTC might have Y2K problems, software that uses it could have Y2K problems if the software uses two digit years. Bruce also refers to an external clock, but is not clear what he means by that term." To clarify further the situation: The RTC is the clock that one sets on their PC to give the time of day. The RTC is the clock that the FAA sets ahead when they test the Air Traffic Control system and airplanes. The RTC is the one that the engineers in testing time reliant systems set forward in the 14 tests listed by Michael Harden. (see in the reference at the end). But there is another clock concept, and that is the clock embodied in the firmware. (Heck, another new term. It is not software - that has to be reloaded each time the computer is started. It is not hardware - that is built into the computer and cannot be modified. But it is pretty FIRM and is not easily modified.) This firmware programming is stored as I explained at GREAT length in the ROMs). I should add that when we are talking about PC,s we are only talking about two clocks, the oscillator and the RTC. It is in OTHER devices like PLCs (Programmable Logic Controllers) that we find this third clock concept in the ROMs and ASICs. There is VERY often NO WAY to EXTERNALLY set this clock. That is why I do not call it an EXTERNAL clock and differentiate it from those which can be set externally (ala FAA, NERC, etc). It is true, that this SECONDARY clock (sometimes, many times, maybe even oftentimes) interacts with a RTC that can be set externally, and that it interacts with the RTC and in those cases becomes virtually indistinguishable from the RTC. But there are other times when a SECONDARY clock will not make itself known by either providing, or requiring, external time values, and it is in that function that I have tried to distinguish it by a separate nomenclature. Other respondents to the forum also clearly identified how these "program clocks" (gee, to use another expression, I am just full of them) are developed and came to be embedded in the software (firmware), with no current documentation to describe their performance or parameters. While momentarily, I shall give you the sources for notable authorities views on this point let me also point out that one writer to the forum quoted the following relevant IEEE document (which after describing the tests for PC,s stated -) 1.9.2 Industrial Controllers At this time no test programs have been identified which apply to industrial controls. Several may be in development for scanning PLC programs for potential year 2000 problems. Without going into more defense of what I said in the original web page, because the facts remain what they are, and other authoritative sources have been found to substantiate them, I will respond to two points in Paul Davis presentation. First and FOREMOST I clearly stated (and I quote DIRECTLY from my original posting): Now comes the interesting part,
as it was explained to me.
These PLC,s are NEVER tested on-line.
EACH and EVERYONE of them is removed from the system,
and taken off-line for testing.

Yet, Paul says quoting me: -------------------------- FIRST - they replaced a component with another component WHILE THE COMPONENT WAS UNDER POWER. THAT IS A NO NO. First day of the first class in any electronics based technical field you are told - DON'T EVER DO THAT. I have to say they were blowing stuff out by bad testing methods. Did not catch that in Beechs paper? Here is the snip - says exactly what I said it did. !QUOTE> But, there is one that I can remember. It is where they could they would interface the system with an advanced clock. They would watch the system go through the clock change from 99 to 00. Even if the system successfully performed in that function they would then turn the system off and see if they could restart it. Much to their initial surprise, (and I had previously heard this from other engineers) some of the systems would not then restart. !ENDQUOTE> ---------------------------- Even in that quote I said they turned the power off. -Bruce --------------------------- Then Paul says:------------> I almost fell out of my chair when I read that! And they are in there soldering away, causing all kinds of thermal spikes and Lord knows what other problems. If it ain't broke, don't fix it. -------------------------- He is really building a straw man here. No where in the article did I suggest in any way or manner that anyone was soldering an live boards. - Bruce ------------------------- Still later, Paul lectures (and the caps are his): YOU NEVER, EVER CONNECT CIRCUITS WITH POWER ON -------------------------- I feel like I am in a discussion with someone who when they can't discuss the point at hand brings up another and then just SHOUTS and SHOUTS about it although I never said anything about soldering to hot boards, and in fact said quite the opposite (see above). -------------------------- Paul states--------------> 1. You cannot interface the system with an advanced clock properly, using the SAME TYPE OF CLOCK AND ASSOCIATED CIRCUITS, without the clock running while you do this, since he specifies a clock that can not be set from the existing circuitry. 2. If you connect it under power you invalidate the test because YOU NEVER, EVER CONNECT CIRCUITS WITH POWER ON. 3. IF YOU USE ANY OTHER TYPE OF CLOCK OR CONTROL CIRCUITS TO ALLOW YOU TO SET THE CLOCK ON STARTING THE MACHINE BACK UP, YOU HAVE INVALIDATED THE TEST FROM THE GET GO. (His emphasis again) 4. There is no valid way to test the program - any way you can test it will introduce new modes of failure and invalidate the test! ---------------------------------- Bruce responds: These four points are exactly what it was the purpose of the article to set out and prove. This points are again reaffirmed by the IEEE quote above. What more can I say, except that through the kindness of others there have now come to me many other documented authorities saying the same thing. Which sources I shall leave you with momentarily. ----------------------------- Now Paul says:--------------> ND WHY WOULD YOU WANT TO TEST IT? That one really blows my mind. The machine has been running since power up X many years ago. You KNOW the date in the RTC does not match the real date in any way. --------------------------------- The latter part of this statement answers the first. We know that at some point, at Y2K or relatively soon thereafter, the Secondary Clock can advance to 00. Paul of course is skeptical of this. However, other members of the forum have already explained how that can be, and I did as well, in the article, so there is no need to repeat myself here. Moreover, I will now refer your to a number of related articles that have since been brought to my attention. This is all the evidence that I have. This is all the understanding that I have. These are representative of all the sources that I have. I have already provided you with such poor credentials as I have. I have provided you with as good (circular, foggy or otherwise) reasoning as I have. Based on these you can equally well make up your own mind, as I have. Knowing Paul's "discussion" method, I doubt that I will really be able to discuss with him further, but nevertheless I feel that he really made my points for me, and he well knows in what little esteem he would hold my capacity for responding, anyhow. Peace and love, Bruce Beach survival@webpal.org =========================================================== The references that I have received in the last two days--------------> I thought you might be interested in knowing of my white paper........ Paula D. Gordon, Ph.D. Visiting Research Professor and Director of Special Projects, Research Program in Social and Organizational Learning, School of Business and Public Management George Washington University Please direct all communications to pgordon@erols.com Http://www.gwu.edu/~y2k/keypeople/gordon ----------------------------------------------------------- Executive Summary: A Working White Paper on Y2K "A Call to Action: National and Global Implications of the Year 2000 and Embedded Systems Crisis" February 25, 1999 by Paula D. Gordon, Ph.D. Director of Special Projects George Washington University Research Program in Social and Organization Learning (The White Paper and accompanying Selected List of Y2K References and Resources are available at http://www.gwu.edu/~y2k/keypeople/gordon.) ----------------------------------------- (I have taken the following complete statement from the above - Bruce) According to Dr. Gordon, Y2K is not a solvable problem. She believes that all that we can do now is to work as smartly and rapidly as we can to minimize the damaging impacts on all fronts. The primary emphasis of the President's Council has been on information gathering, monitoring, assessing progress, and coordinating communication and activity ~ approaches which are neither crisis-oriented nor adequately designed to minimize to the extent possible the harm that can be expected, such as: ~ the failure of weapons systems ~ the failure of nuclear power plants ~ the failure of chemical plants (80% of the American public live within close proximity to a chemical plant), ~ the failure of pipelines and refineries, ~ the failure of hazardous material sites, etc., etc. (note particularly the mention of refineries - Bruce) ---------------------------------------------------------------------- -------------- The following is an interview similar to the one that we did, but at the U.S. Postoffice-----------> During the tour of the Old Post Office Building on Thursday, March 4, I heard several people say that only embedded systems that the manufacturer said were date sensitive were being tested and that other systems (presumably including those which were only "time" sensitive) were not being tested. Perhaps, the statements that I heard were something of an oversimplification of what is actually being done. But in speaking with the technical people in the computer room who were describing the testing process, I got the impression that they were not aware that time sensitive embedded systems needed to be assessed and tested. For instance, when time sensitive embedded systems are integrated with systems that use real time clocks, it is necessary that all the components of the integrated system be tested. References on the need for assessment and testing can be found in the September 1998 Datamation article (The World's Biggest Easter Egg Hunt" at http://www.datamation.com/PlugIn/issues/1998/september/09y2k.html -------------------------------------------------- Michael Harden's list of fourteen things to look out for when assessing embedded systems. This list appears in Harden's book Millennia Minefields Century Technology Services and in the above Datamation article. 1. Does the system display or print a date or time? This would indicate some type of date function is integral to the operation of the device. 2. Does the system produce regular reports? If reports are generated by the device, and dates are part of the report, there may be a problem. 3. Does the system store historical records? If dates are stored, they may also be manipulated and sorted. 4. Does the system time-stamp data? If a system date-stamps records, logos, or products, it will likely be dependent on utilizing a date that may not be able to handle the year 2000. 5. Does the system implement a timed sequence? If the system starts or stops a function based on date or time, it may have a problem. 6. Does the system perform an operation on a time or date basis? Systems that perform a function based on date or time, such as locking doors on weekends, depend on the correct date. ***7. Does the system perform a calculation based on the differences between time or date? Systems that determine intervals, averages, or total times could be at risk for year 2000 problems. 8. Does the system request the date/time on start-up? When power is turned on, a system dependent on date may request it as input. 9. Does the system send date or time information to other systems?. If a system receives date information from other systems, it may have a date problem. Systems that must synchronize themselves with other systems will typically be dependent on knowing the exact date and time. 10. Does the system receive date information from other systems? If it doesn't have a date problem, it may be dependent on another system that does. 11. Does the system have a command that allows the date to be set? If the device or system allows a date to be input, there is likely a need for a correct date. 12. Does the system know which day of the week is based on a particular date? For example, if the system can tell that June 1, 1998 is a Monday, then some kind of calendaring function exists, and consequently a year 2000 problem is likely. ***13. Does the system generate an alert based on some type of interval? If a system creates some kind of notification based on an elapsed period, an elapsed time counter may be involved, which has no date problem, but a real time clock may also be involved, which does. It is difficult to know which is being used, so these systems are suspect. 14. Does the system display or print data based on a time sequence? Logs or listings of events by date or time indicate a dependency upon knowing the correct date. (This list is typical of the type of list that concerns me. The question is always one of obvious time dependency. Only items 7 and 13 can be construed to involve the type of problem that I think is most serious because it is the least obvious. - Bruce) ---------------------------------------------------------------------- ---------------- The following is a response from a very respected authority in the field, when he was sent a query on the subject by one of my readers--------> Mark Frautschi : 3/10/99 I believe that the logic "If it does not use dates, it is Y2k immune" is rooted as much in common sense (which, unfortunately, in this case is wrong) as in wishful thinking, an unfortunate combination. It may be useful to emphasize that the failure rates are low for embedded systems overall, and that for systems that do not "need" dates, the failure rates are even lower. This is a minority of a minority. Larger haystack, fewer needles. Thus the outcome of this failed logic happens to be right, in the overwhelming majority of cases. But not all cases....... ----------------------------------------------- The following was sent to me this morning: California State Water Resources Board Summary of YEAR 2000 Embedded Systems Issues http://www.swrcb.ca.gov/html/y2ksumry.html Based on experience, it takes 21 months to inventory, analyze, fix and test the embedded systems in a small to medium sized plant. Testing of embedded chips must be done very carefully; there are instances where tests have damaged chips beyond repair. Testing of embedded microchips can be very complex, requiring trained specialists with specialized equipment. Approximately 10% of embedded chips have date functions. Approximately 4% of those may not be year 2000 compliant. That equates to an estimated 160 million non-compliant chips in the world. Just because a chip does not perform a date function does not guarantee it will work properly after 1/1/2000 (date sensitive chips have been used in devices that do not perform date functions). Just because an embedded chip works properly on 1/1/2000 does not guarantee that it will continue to operate properly after that. There are over 30 tests that must be performed on each embedded system to ensure that it is year 2000 compliant. Vendors of electronic devices cannot be certain that they are year 2000 compliant (therefore, all devices must be tested). Just because one device is Y2k compliant, there is no guarantee that all other devices that have the same manufacturer, model, version, lot number, etc. will also be compliant (therefore, all devices must be tested). A system containing numerous Y2k compliant embedded microchips (from different manufacturers) may not be compliant, because there is no date standard followed by all manufacturers. It will not be possible to test all embedded systems and repair the non-compliant ones (therefore, efforts should be focused on the most critical systems). It is possible that major utilities (electric, telecommunications, water, etc.) will be unable to operate for some period of time after 1/1/2000 (therefore, it is essential to have a contingency plan). Businesses will be held legally responsible for damages caused by non-compliant embedded systems. (This California Water Board statement shows that some organizations are aware of what I am saying. However, one must still wonder how many are not. For those who are not, the 21 month lead time is far past. And this is not something that one can hurry up and do in 9 months. It may still be possible to have a baby before the new year, but it is NOT possible to remedy all the Y2K systems in a large facility if one has not started long ago. - Bruce) ---------------------------------------------------------------- (And here is a VERY conservative and responsible web page that REALLY reinforces what we discussed and found. - Bruce) http://www.jsonline.com/bym/tech/0214chips.asp (Look particularly at the Occidental petroleum experience. -Bruce) ------------------------------------------------------------- (And compare our experience with this attributed to Chevron. I do not have an original source. - Bruce) Because of the scope of Chevron's operations, the company believes it is impractical to seek to eliminate all potential Year 2000 problems before they arise. As a result, Chevron expects that its Year 2000 assessment and corrections will include ongoing remedial efforts into the year 2000." "While the company believes that the impact of any individual Year 2000 failure will most likely be localized and limited to specific facilities or operations, the company is not yet able to assess the likelihood of significant business interruptions occurring in one or more of its operations around the world. Such interruptions could prevent the company from being able to manufacture and deliver refined products and chemicals products to customers. The company could also face interruptions in its ability to produce crude oil and natural gas." ------------------------------------- (And while this one is a bit off topic, and much less a problem, it still is quite informative. - Bruce) http://www.iee.org.uk/2000risk/w-87.htm ----------------------------------------------- (And some people might find this a real eye opener from a very responsible source. - Bruce) http://www.chemsafety.gov/y2k ------------------------------------------- (And here is still ANOTHER that is much the same. - Bruce) http://www.news.com/News/Item/0,4,33752,00.html ------------------------------------------------------- End of the references. Once again: Peace and love, Bruce Beach Survival@webpal.org --------------------------------- end of email

-- Anonymous, April 12, 1999


Jon said: "I agree with what he is saying, for the most part, but I think he glossed over the epoch timing issue for the RTC's. If they are set at the factory to some arbitrary value, then they will rollover at some arbitrary time, and not necessarily at midnight on Dec 31."

This seems to be the aspect of embedded chips that EVERYBODY is overlooking. If they given a date at the factory of 1980 (pretty common date for that sort of thing),and that date doesn't begin advancing until they are powered up, when will they fail? The ones installed in devices in 1995, for example fill fail 2015. This is especially true of the SECONDARY (sic) clocks Bruce talks about.

-- Anonymous, April 12, 1999



Hilarious! Facts mixed with fiction, the old hidden epoc date myth...sigh...Paul Davis has a brain at least! Regards,

-- Anonymous, April 12, 1999

fellow inhabitants of the earth: this Y2K thing is but the beginning of the end. all the prophesies of the ages coalesce upon our time (1997-2012). the earth is about to reclaim itself. humanity will be reset again, as it has in previous ages, until we get it right. see you on the other side my friends...

-- Anonymous, April 13, 1999

Because trying to read it made me dizzy <G>, I reformatted Jeff Sullivan's post of Bruce Beach's reply and put it on the server I use (to save space at MIT). It's probably still got a couple of bugs in it, but if you want to read it there, just click here

And by the way... For anyone posting copies of emails (or anything, far as that goes), the trick to preventing things from coming out in one big text block is to:

A) Take the < left and > right arrow symbols out of notes (you'll find them bracketing <email@addresses>); and

B) Be sure to put two carriage returns in between each paragraph or distinct line... As in -

Date: April 13th, 1999

From: Me

To: You

It's an idiosyncracy of the forum software: Right and left arrow symbols are HTML "tag" brackets which makes the software think your post has HTML in it which makes it treat its contents differently than normal; and for some reason it simply doesn't recognize single carriage returns. (It views them as single spaces... That's why you'll see things like email headers and numbered lists get jammed into a single paragraph.)

And by the way. Someone (Jeff?) ought to let Mr. Beach know that he doesn't need to be a "member" to participate in this forum: He wrote: "Perhaps Jeff could take the lead on this because it seemed more of the thrust of his email and post the following reply since I am not a member of that forum." (And please be sure to suggest he read the above for when it's time to copy and past those notes from other people in! <G>)

Okay... Now I'm going to go finish reading what he had to say...

-- Anonymous, April 13, 1999


Bill -

Thank You!

Jeff

-- Anonymous, April 13, 1999


For Paul Davis' FINAL reply to Beach and his clocks

-- Anonymous, April 14, 1999


It shouldn't of course have any bearing on the credibility on the research content, but it might interest the forum to know there is an article on Mr. Beach online in the Ottawa Citizen. Isn't it strange: the photos of Mr. Beach sitting in his 10,000 foot bunker are exactly as I imagined him. Even his claim to owning "North Americas largest and most advanced survival community", chimes perfectly with the mindset I had attributed to him.

This link comes courtesy of Mutha Nacha on yet another noisy thread on the Yourdon forum (including a bit more information on Mr. Beach's patents in microprocessing), entitled (honest, I didn't make this up) Gary North: most significant document that I have posted

-- Anonymous, April 14, 1999


Moderation questions? read
the FAQ