Here's what I've learned from Delta-T Probe technicians in the field testing embedded systems at NASA, the DOD, Defense contractors and elsewhere: 1) All too often, vendor statements about compliance are dead wrong, and 2) with distressing frequency, vendors have no idea how many firmware versions of their chips they have issued. Since vendor statements are currently a major, if not the major, compliance verification method, I'd say we have a problem.

-- Keith R. Wolgamuth (krwolg@aol.com), October 14, 1999

Answers

Houston, I think we have a problem here.

-- relocating now (relocating@relocatinggg.com), October 14, 1999.

Actually, given the resisitance to doing anything except relying on vendor statements where roll forward is not possible, somewhere near the moon might not be a bad spot. I also understand from one of those infamous insiders that oil company pipeline sensors, valves and other controls are microprocessor controlled (read: embedded systems) that generally can't be rolled forward. Care to guess what they are relying on? Or could it be fix on failure? I understand that one view holds that businesses will do their best in order to remain competitive, but what about when they all know they are in the same boat? Any reason to work extra hard?

-- Keith R. Wolgamuth (krwolg@aol.com), October 14, 1999.

What is a Delta-T Probe technician ... and why should I care what one thinks.

-- Dr. Chip Dasher (funtime94@hotmail.com), October 14, 1999.

Dr. Dasher: A Delta-T Probe is a device that clips directly onto RAM, ROM, RTC and other embedded chips to determine whether the system is time and/or date sensitive. If it is, the Probe copies segments of code for analysis of compliance risk. Essentially, it determines directly whether or not an embedded system will fail. Therefore, a Delta-T Probe technician is one who uses the tool to perform the testing. Got it?

-- Keith R. Wolgamuth (krwolg@aol.com), October 14, 1999.

Keith --

It has been estimated that there may be somewhere in the neighborhood of 50 BILLION embedded chips. Even though only a percentage of these are date sensitive, no one knows for sure which are which, ergo the Delta Probe tester. How many techs working for how long will it take to insure something less than 4% failure of systems that incorporate embedded chips in their architecture? Also what percentage of added software failure will be needed to equal TSHHTF? In YHO.

-- Michael (mikeymac@uswest.net), October 15, 1999.

Michael -- To help calculate the answer, 1 technician can test about 5 systems in 8 hours, and there are about 60 Probes in the U.S. at present. It takes about 2 days to train a tech. Finally, I confess my ignorance as I am new to this posting game -- please spell out your abbreviations. Thanks.

-- Keith R. Wolgamuth (krwolg@aol.com), October 15, 1999.

Keith R. Wolgamuth, thank you for posting here!

-- Ashton & Leska in Cascadia (allaha@earthlink.net), October 15, 1999.

If systems can be verified in such short order, why isn't everybody doing it?

-- Dasher (funtime94@hotmail.com), October 15, 1999.

The DELTA-T probe is as you described. It is used for remediating Large Scale Embedded Systems (LSES), which may have a Mainboard the size of a playing card. Most of these systems are date sensitive (data logging and control) but not all will fail come January. In LSES situations, defect rates have been found up to 37% and each one MUST be tested. The good news is that there are nowhere near 50 Billion, the bad news is that they are EVERYWHERE...firetrucks (ladder companies) Municipal water, sewer, traffic, Chemical plants, pipeline pumping stations, CNC 5 axis machining centers, etc.

For a wonderful descussion in clear, readily understandable language, go to the website and download the free DELTA-T Probe Operator's Manual. You will find a complete Cominity College level course in Embeddeds there. It is in Adobe PDF format. I have my copy on Zip disk and refer to it from time to time. The website is www.embedded-science.com (sorry, I am Hyperlink challenged).

-- K. Stevens (kstevens@ It's ALL going away in January.com), October 15, 1999.

Oops, that is Community College level course. Sorry.

-- K. Stevens (kstevens@ It's ALL going away in January.com), October 15, 1999.

"1) All too often, vendor statements about compliance are dead wrong"

Something that struck me very funny. A month or 2 (?) ago, I think it was Dodd on C-span, said that (see above) was a concern FOR MEDICAL DEVICES. Now, I wonder, why just medical devices? This is a small percentage of embedded systems. What's so special about medical devices??????????

"2) with distressing frequency, vendors have no idea how many firmware versions of their chips they have issued"

Yup, I've been there. We updated a ROM on an almost monthly basis. Added features, fixed bugs, just like a SP for Win, but you had to install a new "hardware chip". The last version of "the chip" for our machine was in 1986. The company dropped support in 1988.

Are there any of these "boxes" still working in the real world? Yes, I know for a fact. They were built like a tank, still doing the job.

Do they have a Y2K problem? Not that I know of. But my "code" is maybe 1/16th of the ROM, and I didn't use any time/date services. But they were available. Did any of the other programmers, that wrote the other 15/16th of the code, use them? I don't know, but the service was available.

Tick... Tock... <:00=

-- Sysman (y2kboard@yahoo.com), October 15, 1999.

Thanks Keith,

Also, Sysman and others, for your postings on this matter. Now, I wonder what kind of debunking we'll get from our debunker experts?

Keith, YHO = your humble opinion. TSHHTF = the sh*t has hit the fan, or some variation on that thought.

-- Gordon (gpconnolly@aol.com), October 15, 1999.

I'm not a 'debunker expert', but I know a thing or two.

One: people will not sit around with their thumbs up their rears, bemoaning the sorry state of affairs, whining, "what ever will we do?" should anything happen Jan. 1. People will get to work and fix whatever's wrong, in short order.

Two: you folks seem to have no faith in the ability of people to fix problems. That, to me, is a psychological shortcoming on your parts; not a shortcoming on the part of any other humans that has any basis in fact.

Three: Y2k has been a known fact for years. Hurricane Floyd, and the flood which followed, caught us completely by surprise here. Yet the utilities engineers right here where I live managed to keep the power on continuously, after an initial 24-hour outage, despite the fact that the main transformer that supplies this entire county with power was 6 feet underwater. That's 100,000 people. But they came through. The power stayed on. By virtue of a 'workaround'.

You people love to whine and moan with all your pessimistic theories. The actual proof has been seen right here where I live in the past few weeks. People with know-how can solve problems, even when said problems hit unexpectedly. And people who are hit with unexpected problems pitch in to help one another. That has happened here, out the wazoo. The 'marauding bands of bloodthirty mobs' Doomer theory is, quite simply, sheer insanity.

The events surrounding Hurricane Floyd and the flood that followed, in eastern North Carolina, have completely BURIED the Doomer philosophy; for anyone who's paying attention.

-- Chicken Little (panic@forthebirds.net), October 15, 1999.

Chicken Little,

For once, you make some very valid points! Please ignore my flames in other posts tonight...

One) I agree. People (tech people) will not give up. They will take their best shot. It's human nature. The question is how much can they do?

Two) See one.

Three) Yes, it has been known for years. I knew about it in 1968 when I IPLed my first mainframe. My boss at the time said "Don't worry about it, it'll be fixed long before then." But it wasn't. The boss boss didn't want to spend the money. "We can get bye for another year, can't we?" It's business as usual.

You can't compare Joe Sixpack stringing wires from a cherry-picker, and John Public running a post-hole digger, after a hurricane, with Y2K. It has nothing to do with computers, and what they do. Y2K isn't a physical problem, it's a logical problem. I hear ya, we have a few towns here in NJ (Bound Brook, Manville) that are "dead" because of Floyd. But it just ain't the same!

Listen CL, I've changed my opinion of you. What you say makes sense. If you keep it up, and stop with the kid stuff, I'll no longer think of you as a moron troll. All I ask is that you look at it from my point of view, the doomer point of view. Can't you see what could happen? Do you have that much faith in "the system"??? Are you willing to bet your life on happy-face PR reports?

Our old truce is still valid, as far as I'm concerned. As long as we can keep this on an adult level... Can we talk?

Tick... Tock... <:00=

-- Sysman (y2kboard@yahoo.com), October 15, 1999.

How do you fix in SHORT ORDER something so pervassive, that hasn't been fixed in 4 years? How do you fix in SHORT ORDER a machine that has taken 40 years to evolve? Somebody got a majgic wand?

-- FLAME AWAY (BLehman202@aol.com), October 15, 1999.

I agree with CL that things are "working" in NC. On the other hand, all of these people that are out there trying to salvage their homes and possessions have been removed from the "system" or playing field. Along about Jan 1, what happens when they decide they need to get a bank loan to try and bring their home back to what it was before Floyd? Or in Dec the bank says they have to start making their house payments again, even tho they are not living in the house due to damage and then in Jan, the bank decides to foreclose as the computer shows they are 4 months behind in payments. Right now an awful lot of the people are living out of the loop. And if they are NOT living out of the loop, then its not as bad as CL has claimed it to be. With regards to power co and military, I believe CL and I believe my power company will do all it can when the time comes. I also can foresee events taking place where we would be happy to have the military come in. And all my blathering is about to be tested. The first bands of rain are coming via Irene. The projected path is right over my house. We live in the forest and while we make it a point of not having any trees close enuff to fall on the house or out building, it doesn't take much wind to bring down these tall pines and large oaks out here. When the trees come down, so do the power lines. A few years ago, a strong wind took the power out for over a week in this area. nthere are the floods. We live on a knoll and it should drain off into the forest, but sometimes the rain comes down a lot faster than it can run down hill. So we may be getting tested here in another 12 to 24 hours. Then I guess we are sending it up to CL to deal with. After all, he has experience! LOL Really, I am very concerned about CL's area getting pounded again. I hope the power men have had time to rest and the military is still there and restocked with provisions. I will try and keep you posted re "Irene".

Taz

-- Taz (Taz@aol.com), October 15, 1999.

CL:

Your points are very valid. The engineers and techs and linesmen have done yeoman's work in your area.

However, did not some to significant support come from outside your region?

With the Y2K situation, it remains doubtful whether any area will remain unaffected enough to be able to send help initially. Perhaps, perhaps not. This affects most current contingency and disaster recovery planning.

As to civil disturbance, I am glad it went down as it did. I most fervently hope that, if there ARE serious Y2K problems, that our people, instead of expecting outside help, pitch in and solve their own problems.

My fears, though, are that if outside help is delayed for days or weeks or, heaven help us, months, if Medicare, Medicaid, ADC, AFDC, Food Stamps, unemployment, and/or other government benefits are disrupted, that our people or some percentage of them will NOT react as well as they did during and after Floyd.

Where any large cities affected by the flooding? These urban areas are the main concern of many of us.

Never have so few wished so fervently to be wrong........

-- mushroom (mushroom_bs_too_long@yahoo.com), October 15, 1999.

I'm curious about the Delta-T probe in light of Sysman's observation. I can see how the probe can determine that a time service (RTC) is available. Not necessarily whether it is used, much less whether it is used incorrectly, and far from determining what problems the incorrect usage may cause.

RTC logic is common in embedded systems for several reasons -- they are cheap, they are often built in (you get them "free"), there are routine libraries already built for accessing them, they are handy for longer time intervals (minutes rather than microseconds). But almost every IC we use contains features we don't use in any given system.

From my reading, the Delta-T probe can determine whether a system contains RTC logic, and therefore the *potential* to use that logic incorrectly somehow. It cannot tell you whether the device, as implemented, is noncompliant. And it certainly can't tell you whether the device won't perform its essential function properly, should the code misuse the year.

As a rule of thumb (not always accurate), if there is no way to SET the date, then the date is either not used by the code logic, or the failure could happen at any time randomly over a 100-year period.

-- Flint (flintc@mindspring.com), October 15, 1999.

Hey chicken,

You call that living what the people in eastern NC are doing??? I know its not that bad but is worst than being reported. In fact outside NC where is it being reported? Not here.. I calledmy brother in Charlotte and he's very glad not to be in eastern NC..

-- y2k dave (xsdaa111@hotmail.com), October 15, 1999.

Hey Chicken, Floyd just grazed NY and our power was out for 5 full days and nights. Good thing it wasn't the middle of the winter! I'm not as confident in the quick efforts of people as you, especially our utilities and emergency services when problems are widespread ... my fingers are crossed!

-- you must be kidding (ktwenterprises@hotmail.com), October 15, 1999.

There is a huge difference between Y2K and a natural disaster.

The natural disasters are accompanied by shock, awe of nature, the "Act of God" sense of fatality.

Humanimals will react VERY differently to Y2K, the Blunder of Man.
Rage, fury, betrayal, rampage, opportunistic acting out.

Revolution.

It won't be Mother Nature who has been fooled.

-- Ashton & Leska in Cascadia (allaha@earthlink.net), October 15, 1999.

Chicken Little: I (want to?) thoroughly agree with you. I do think that given the resources and ability, most people most of the time will dig in and find a way to keep each other going, and I admire the people who work tirelessly to ke3ep essential services running for those that have suffered disasters like hurricanes, tornadoes, etc. My greater concern is that the resources will not be available to provide those services. For instance, if there are spotty power outages due to embedded chip failures or otherwise, I think that the power grid can generally be rerouted to provide power while those utilities get it fixed. What if, however, there is only spotty power available? Further, it is entirely possible that there will not be replacement chips available. In my own service area, I know that every one of the manual substation operators was laid off over a year ago. The cascade effects can be enormous: The national power grid system is operated via phone line. Without power, that system goes down. When I am being fearful, I fear that we are so used to assuming the existence of the tools that support our everyday tasks that their existence is being assumed in contingency planning. Embedded systems may turn out to be the piece of this puzzle that removes those tools. Think upside down pyramid, embedded system at the bottom. In any event I join you in the hope that the character of which you speak will prevail. Flint: I am not a Delta-T technician, although I am going to ask one to join us on the technical side, or at least give me an answer to relay. As I understand the process, the Probe also reviews the source code and copies those areas of code that are performing time and/or date comparisons for reading by an expert, who then makes a call based on the code. I am told that it performs very reliably, I suspect particularly in excluding systems from further concern. I do recall a technician telling me that the doors in a jail were found to be likely to fail -- the best guess was that they would fail open, but no one was certain. (He also suggested that this latter issue might actually have political overtones, but that's a topic for another day). Mushroom: Add me to your "Never have so few wished..." list.

-- Keith R. Wolgamuth (KRWolg@aol.com), October 15, 1999.

It has been estimated that there may be somewhere in the neighborhood of 50 BILLION embedded chips.

WRONG!

Even though only a percentage of these are date sensitive, no one knows for sure which are which, WRONG

-- Michael (mikeymac@uswest.net), October 15, 1999.

Keith -- **** If systems can be verified in such short order, why isn't everybody doing it?

-- Dasher (funtime94@hotmail.com), October 15, 1999.

most technitions have done it. Thry don't need the probe because they can read the schmetacs and know how an embedded system works, and contrary to popular belief, there is documentation on the "embedded" chips as well as regular, non embedded ones.

*****

The DELTA-T probe is as you described. It is used for remediating Large Scale Embedded Systems (LSES), which may have a Mainboard the size of a playing card. Most WRONG!! of these systems are date sensitive (data logging and control) but not all will fail come January. In LSES situations, defect rates have been found up to 37% and each one MUST be tested. The good news is that there are nowhere near 50 Billion, the bad news is that they are EVERYWHERE...firetrucks (ladder companies)

This has been de-bunked, over a YEAR ago, but is still up on websites Municipal water, sewer, even Dave Hall, the so called expert on embeddeds who got the 50 billion embedded hype started in the first place now admits water and sewer systems have a very small possibility of "embedded" problems. traffic, Been checked- no problems expected Chemical plants, chemical plants do not leave safety to "chips" which could fail in the best of times pipeline pumping stations, another exageration which has been left up on old websites CNC 5 axis machining centers, What are those? etc. not as etc as is assumed.

-- K. Stevens (kstevens@ It's ALL going away in January.com), October 15, 1999.

*************

Do they have a Y2K problem? Not that I know of. But my "code" is maybe 1/16th of the ROM, and I didn't use any time/date services. But they were available. Did any of the other programmers, that wrote the other 15/16th of the code, use them? I don't know, but the service was available.

Tick... Tock... <:00=

-- Sysman (y2kboard@yahoo.com), October 15, 1999.

Thank you

***

Also, Sysman and others, for your postings on this matter. Now, I wonder what kind of debunking we'll get from our debunker experts?

-- Gordon (gpconnolly@aol.com), October 15, 1999.

Gordon, by posting that the postings will be debunked, do you believe they cannot be debunked?

****

How do you fix in SHORT ORDER something so pervassive, that hasn't been fixed in 4 years? How do you fix in SHORT ORDER a machine that has taken 40 years to evolve? Somebody got a majgic wand?

I guess it is a good thing that what you describe does not exist

-- FLAME AWAY (BLehman202@aol.com), October 15, 1999.

********

From my reading, the Delta-T probe can determine whether a system contains RTC logic,

SNIP As a rule of thumb (not always accurate), if there is no way to SET the date, then the date is either not used by the code logic, or the failure could happen at any time randomly over a 100-year period.

-- Flint (flintc@mindspring.com), October 15, 1999.

Flint,

The myth about a clock being preset by the manufacturer is just that, a myth. There is also no way any tech would not know a chip contained a RTC. Crystal oscilators are difficult to hide and the purpose of the RTC would be defeated if it did not contain a battery backup - also difficult to hide from a tech.

****

Humanimals will react VERY differently to Y2K, the Blunder of Man. Rage, fury, betrayal, rampage, opportunistic acting out.

-- Ashton & Leska in Cascadia (allaha@earthlink.net), October 15, 1999.

Or trust in those they have learned that can, will.

****

if there are spotty power outages due to embedded chip failures or otherwise, I think that the power grid can generally be rerouted to provide power while those utilities get it fixed.

Power lines are not internet connections, they cannot be rerouted as you seem to think. Further, it is entirely possible that there will not be replacement chips available.WRONG

In my own service area, I know that every one of the manual substation operators was laid off over a year ago.

I'm sorry I find that impossible to believe, every ONE of them? Name the town, city please... Those substations are not running themseves... a statement like this causes one to question the rest of what you said.

The national power grid system is operated via phone line. That is NOT true.

I fear that we are so used to assuming the existence of the tools that support our everyday tasks that their existence is being assumed in contingency planning. Those who run the power companies do not assume, they KNOW what does what and what does not do whatever.

As I understand the process, the Probe also reviews the source code and copies those areas of code that are performing time and/or date comparisons for reading by an expert, who then makes a call based on the code. Nope. The probe stores digital (binary) bits and compares the results with a database - done by computer - not a human, and gives the match to the techs. The information on the database then can tell you if there is the possibility of a date problem. Just like the manufacturer does.

I do recall a technician telling me that the doors in a jail were found to be likely to fail -- the best guess was that they would fail open, but no one was certain.

Oh really??? It appears you are repeating what you have read as apposed to what a "Technition" has told you. The doors in jails and prisons fail in the locked position, they do get the ocational storm related power outages and no one has heard of a mass jail break due to that have they? (Your credibility is going downhill FAST)

-- Keith R. Wolgamuth (KRWolg@aol.com), October 15, 1999.

Keith, I believe you have just read a lot on the web and have not been doing it long enough to know which "myths" were debunked. There are just too many points you tried to make that are known to be incorrect that you attribute to a "technition".

-- Cherri (sams@brigadoon.com), October 15, 1999.

Keith,

I apologise. The reason I point out what I have is that this is basically a forum of people who believe, in varying degrees, that Y2K will cause the end of the world as we know it up to those who expect little more than a bump in the road with 10 year depressions, New world order, Clinton taking over as dictator, with contrails full of biochemicals, concentration camps, mass sewage overflow, white vans (they are selling a bunch down the road from me), black helecoptors and a few odd ideas thrown in.

Unfortunatly there are a lot of people who post anything they can find, without understanding it in the least, that has been researched and explained.

You are new? How did you find this place? Basically this is considered a "doomer" site, with pollies who come over from their (debunking sites) to debunk and all manner of persons in between. If you would like the information about some of the subjects you spoke of, they are available. (This is an excellent place to learn about people by the way) You will have to develope a thick skin and be aware that a "technition" telling yopu something will convince those who wish to be convinced it is gonna be bad, and earn you a correction by those who are convinced or (as in my case) know better.

I have used logic analyzers for 3 decades (literally started playing with O'scopes when I was 3 years old!) and knew what the Delta-T Probe was all about before it was ever sold to anyone. We discussed it way back in the early dark ages of Y2K. (almost 2 years ago!) It cannot do much more than a real technition with a decade or two of experience cannot do, but it only takes a day or two to train someone to become a "tech" on one so basically they do not have the knowledge or experience to give you correct information on all of the other subjects you asked about.

My e-mail is real and if you want to know about embedded stuff feel free to mail me and I will attempt to send you to where you can find the information for yourself.

-- Cherri (sams@brigadoon.com), October 16, 1999.