So, it seems that someone doesn't believe that people are doing a thorough enough job testing embedded systems.

Surprise surprise.

http://library.northernlight.com/FC19991201560000218.html?cb=0&dx=1006&sc=0#

Jon

-- Anonymous, December 02, 1999

Answers

Better late than never? Hmmm...

-- Anonymous, December 02, 1999

Rick,

What are the ramifications for EU's and Nuclear? Any guesses? How about you, FactFinder?

Jeff

-- Anonymous, December 02, 1999

Can we believe this? Is it credible? It looks like it is but at this stage who knows what to believe about embedded systems. If Rick will respond to this I will believe him. If it is true we are down the tube.

-- Anonymous, December 02, 1999

Don't think I made myself clear. I know it is a legitimate posting. My question is are their conclusions correct?

-- Anonymous, December 02, 1999

marcella...

isn't this similar to what jim lord's 'friend' has puportedly said?

hasn't mark frautschi warned of the embedded systems problem and the lack of understanding regarding same?

there has been an inordinate amount of schizophrenic reports regarding the embedded systems.

didn't the earlier reports from the oil companies remark on the difficulty/impossibility of reaching and remediating same?

-- Anonymous, December 02, 1999

Marcella, the Final Answer is: yes.

I know what to believe about embedded systems. I have been working with them for twenty years.

Here are some choice quotes from The President's Commission's view on Embedded Systems thread:

"Participants in the meeting included technicians that had done work in the bio-medical, defense, electric power, gas, manufacturing, oil, shipping, and telecommunications industries."

"Embedded systems are at risk of problems during Y2K rollover if they conduct a calculation that depends on a representation of the date. The date could be in relative or absolute form. The participants presented a number of specific cases where they had found Y2K problems in embedded systems. Several of these involve calculations of time increments inside an embedded system without the date being displayed or apparently used."

Here's one for FactFinder: "All except one of the examples were large, complex processes where embedded systems inter-relate with each other and, in some cases, with external computer systems."

"There was considerable discussion of potential failure rates of embedded systems. Estimates ranged from a 1 - 2% potential failure rate of processes containing embedded systems in some sectors to 4 - 6% in others, but no conclusion was reached."

"Where possible, all mission critical systems should be tested end-to-end, whether or not the systems appear to have date sensitive functions. Failure to do so means a small level of risk has been assumed that, at minimum, should be addressed with a contingency plan."

"The majority of failures of embedded systems are expected to occur on or about December 31st through January 1st. However, simply turning a system off during that time frame is generally not a solution."

"Anyone taking a fix-on-failure approach for Y2K, particularly with embedded systems, runs a significant risk of collateral damage and a difficult recovery. There was little discussion leading to this statement. Remedying the kinds of Y2K problems participants had found in embedded systems was difficult and time-consuming."

and, on and on... Almost every statement in this report is a loaded gun. And yet there is nothing in the report that has not been said in this and many other forums over the past couple of years. I am sure I have covered everything in this report in my postings alone.

Everybody, repeat after me: over 4 billion MCUs and MPUs ship each and every year; over one hundred million PCs now ship each and every year; over one hundred million sophisticated embedded systems requiring the power of a PC class system ship each and every year; not all products are yet Y2K compliant, and still shipping; only a fraction of the installed base has been -- or ever will be -- remediated; there are going to be problems; nobody knows how small or large those problems are going to be; nobody knows how predictable or random, how systemic or chaotic the problems will be; so, the final answer, the Y2K mantra, is this: prepare for the worst and hope for the best. Be like our grandparents, be like a Boy Scout or Girl Scout, be a conscientious citizen of the world, be prepared. Be prepared to help others.

-- Anonymous, December 02, 1999

Where oh where is the old factfinder when ya needem. Hey there old buddy, dispute this will ya?

Sounds like you big brained types really blew it on testing and IV&V.

-- Anonymous, December 02, 1999

F. Snyder Gokey's post below was very interesting and credible information, see my reply there. The Century Corp (a y2k remediation firm) PR release was not too impressive on the other hand. AJ, I see you're taking a few facts, throwing in your own theories, and extrapolating into hyperspace again, sigh....

Regards,

-- Anonymous, December 02, 1999

Help! Rick, somebody with some credentials please post a thumbs up or down on this. I read it as credible, and extremely disturbing at this very late date.

I would like to send this up my ladder to my collegues, can anyone vouch for Cherry and company or NIST?

As far as their points on embeddeds who agrees or disagrees?

-- Anonymous, December 02, 1999

Jim - I can't vouch for the Cherry outfit, simply because I don't know enough about them.

I can vouch for the NIST alert. I can most certainly vouch for Snyder himself, his contacts with John the K, and the legitimacy of the previous posting from Snyder. What I don't want to do is jump immediately into the breach - I've had a very long day, and as I posted in response to Snyder's article, I want to consider not only the Koskinen email, but the NIST alert and Dale Way's (IEEE) recent contributions.

I'll post an analysis here in the forum or as a column on energyland.net sometime tomorrow. Right now (9:30PM EST), it's time to go rest my aching brain

-- Anonymous, December 02, 1999

Thanks Rick. I really appreciate it. I also appreciate everything you've done in this forum and out. And thanks to all the members as we head to the starting line.

I haven't posted in a while but have been lurking. I've been watching the oil industry slowly, slowly wake up to this potential supply interruption. But they still don't get it. Not many in the industry think there will be probs beyond Jan 1. How little they know. I'm now personally on a 7-8 scenario myself as I think to little attention has been paid to real work and to much to spin control.

God bless all of you for helping me for the past year. I am truly frightened at the events I see unfolding. It's like a slow motion train wreck. Sorry to be so gloomy, but I'm seeing some really scary things here lately.

-- Anonymous, December 02, 1999

Thanks, Marianne and A.J., for slapping me back into reality. Due to the controversy about embedded systems my brain had set that aside as a "won't know until it happens but maybe it is not as big a deal as first thought." Maybe it isn't as big a deal as first thought, but one embedded system that fails in a refinery can bring down the refinery. Persons who work in refineries know that a refinery is a "controlled explosion." That explosion is necessary to break oil down into its various parts so that fuel oil, gasoline, kerosene, gas, plastics, etc. can be made. If you could see one of those plants and the maze of pipes you would understand the possibility of failure.

Yesterday evening a chemical plant in the Houston Ship Channel area exploded. I don't yet know the details. Any of you take the Houston newspaper? If you do perhaps you could post the reason for this explosion.

-- Anonymous, December 03, 1999

Have more info. on the Houston area chemical plant (at LaPorte) that exploded yesterday evening. Two workers were slightly hurt. There was also a fire at this plant this past Monday, 5 people hurt. Plant officials say the two aren't related. They do not know why the plant exploded. Residents close to plant were told to stay inside. Fire lasted about 3 hrs. The chemical product is polymer chemicals (needed to make some plastics.) Plant name is Akzo Nobel.

-- Anonymous, December 03, 1999

FactFinder wrote: "AJ, I see you're taking a few facts, throwing in your own theories, and extrapolating into hyperspace again, sigh...."

Dear FactFinder, please explain to me what you think my theories are, please show me where I am extrapolating into hyperspace. I don't think I have done that. While you're at it why don't you rebut some of the specific things you disagree with using some facts. There are still open threads that you said you were going to research and comment on, which you have not.

All I have ever said is this: 1. embedded systems are a problem 2. x86 based embedded systems are a worse problem 3. the installed base of large, complex embedded systems is huge 4. the installed base has been growing rapidly over the past few years 5. the majority of the installed base has not and will not be made Y2K compliant 6. some small percentage of embedded systems will fail 7. the general populace should be prepared for some problems over the coming months

P.S. How's the weather in Wisconsin lately?

-- Anonymous, December 03, 1999

Maybe it's just a Canadian or North-Eastern thing, to want to be prepared. I was born and raised in Montreal.

Ontario Hydro's Y2K warning

The power utility suggests customers 'be prepared' and stock up on supplies in case the lights go out on New Year's Eve. Jake Rupert reports .

-- Anonymous, December 03, 1999

A.J., when Factfinder (who is that masked man?) has no answer, he adopts a flip attitude and used to end with "LOL" (maybe he still does?) Your position is clear and valid. Ff is a nameless individual that may work somewhere doing something, or maybe not. There is no way to judge his credibility.

-- Anonymous, December 04, 1999

When I have time, I would like to address the statements in the Koskinen email that Snyder posted in some detail. For now, I will provide a quick response to one of the items AJ mentions from the email:

Koskinen (Presidents Commission on y2k): "Embedded systems are at risk of problems during Y2K rollover if they conduct a calculation that depends on a representation of the date. The date could be in relative or absolute form. The participants presented a number of specific cases where they had found Y2K problems in embedded systems. Several of these involve calculations of time increments inside an embedded system without the date being displayed or apparently used."

AJ's emphasis: "Here's one for FactFinder: "All except one of the examples were large, complex processes where embedded systems inter-relate with each other and, in some cases, with external computer systems."

AJ, I find this very credible, as would others who have worked on y2k in embedded systems have for the past two years or more, nothing new here. So whats your point, just because this is news to you, doesn't mean it's news to everyone! There are a number of devices that I know of that have date functions that are not apparent on the surface, with no visable date display. These devices communicate data to higher level systems that DO have date display and date entry. One example is RTUs, these devices process analog process signals and send digital data (including date stamps for some models of RTUs) to a computerized system that DOES have date display and the ability to enter the date. Though there are no dates to be seen on the device, its quite easy to determine if a device has date functionality by reviewing the manuals/technical literature for the device. Typically, dates are set on such devices by the computerized system wich downloads the current date to the device. Another example is the Bentley-Nevada vibration monitoring systems - there are low level embedded devices that send digital data to the PC based monitoring/trending part of the system. You set the date at the PC.

The document that most of the utility industry used for remediation even addressed devices that may not have apparent date functions, the devices that might communicate date data for example.

Thats why I find most of the Koskinen email credible, the information provided is what we already knew. Nothing new here for those who are knowledgable of y2k in embedded systems.

Regards,

-- Anonymous, December 04, 1999

italics off.

-- Anonymous, December 04, 1999

AJ, regarding this statement: "Several of these involve calculations of time increments inside an embedded system without the date being displayed or apparently used." This prompted me to remember one system that Malcom had mentioned in the forum that used the time off of the absolute date value to perform a rate calculation. Rick also cited a UK problem that appeared to use the date this way. This is quite an unusual programming method for rate functions (or PID control), and I had intended to ask him about it at the time but never got to it. I sent him an email to get further info, so perhaps he can describe it further here. Since it was identified in their testing, I assumed that you could indeed enter the date through some means, but I want to verify this.

Regards

-- Anonymous, December 04, 1999

FactFinder, you really are a piece of work. If I didn't know better I'd think you were an NSO plant who's job it is to provide baseless information for Poly's to grab onto so they can maintain their position and sanity. ;-)

It's now apparent to me that you spend far more of your precious brain power rationalizing and justifying your position than you do seriously trying to understand anybody elses. When you make a statement like this, "So whats your point, just because this is news to you, doesn't mean it's news to everyone!". It really shows your ignorance on a couple of levels. First, if you had truly understood some of the things I have posted in this forum you would know that this is not news to me, that in fact it is one of the things that I am most concerned about. Secondly, you completely misinterpretted the point of my having selected that particular quote specifically for you. In this thread you are on record as saying:

"A little more information regarding "end to end" testing and the frequently cited "interconnectedness of it all", the majority of embedded system devices have VERY LITTLE INTERCONNECTEDNESS at ALL. Only higher level control or monitoring systems are likely to have multiple data exchanges. On the software level, many of the software applications on desktop PCs are run stand-alone, and aren't exchanging data with other programs. Those programs that interfaced with other software AND HARDWARE, we tested. We also tested programs that interfaced with mainframe databases.

Like Y2K, the "interconnectedness of it all" problem is way overblown.

The hype ends in two months - enjoy the last leg of the ride ;)"

Which is it? Complex systems that inter-relate are a problem, or complex systems that inter-relate have been tested and are good to go?

Let me ask you a specific question, with a simple yes or no answer. If there exists a simple control loop that consists of two sensors, a vibration sensor and an over temperature sensor, a PLC, an RTU and a PC based MMI and control computer, is it possible that this control loop will fail given the following conditions? The conditions are these: the sensors do not relay any time/date information, the PLC does relay time/date information with each sample but only for information purposes, it does not process the time/date in any way, the RTU simply takes RS-232/RS-422/RS-485/Current Loop input and packetizes it for TCP/IP output to the PC control box, this control system has not been remediated, when the PLC's clock ticks over to 00:00:00 1/1/2000 it starts sending packets with time stamps that look like this "00:00:00 01/01/@@".

Given this scenario will the control cell malfunction in the hours following midnight 01/01/2000, yes or no?

Yes the hype ends in twenty-six days, and I'll be happy for that.

-- Anonymous, December 05, 1999

The two broken links are: FactFinder refers to himself as an "outspoken Polyanna"; FactFinder talks about "VERY LITTLE INTERCONNECTEDNESS at ALL".

-- Anonymous, December 05, 1999

Sorry, it appears the forum software can't handle the specific length of these URLS as hot links. Bugs! Funny that.

You'll have to cut'n'paste them.

The two broken links are: http://www.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=000tsF http://www.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=001iGj

-- Anonymous, December 05, 1999

Test message: polyanna

-- Anonymous, December 05, 1999