I read conflicting opinions on embedded systems. I want to know the truth - if it is out there- Are embedded chips date sensitive or not? I have recently read a post by someone who claims to be an engineer who "knows" that embedded chips are not date sensitive because in his experience implementing hundreds of thousands of them none were date sensitive. I want to respond to this with something concrete. Can anyone respond please - with examples?

-- April (Alwzapril@home.com), August 25, 1999

Answers

Embedded systems fault casebook:

http://www.iee.org.uk/2000risk/Casebook/eg_index.htm#TopOfPage

Allen-Bradley:

http://domino.automation.rockwell.com/webstuff/y2k.nsf

Texas Instruments:

http://www.ti.com/corp/docs/year2000/dspsds.htm

Motorola:

http://mot-sps.com/y2k/realtimeclocks.html

A good, basic article about embedded systems:

http://www.jsonline.com/bym/tech/0214chips.asp

"Problems lurk in more than just computers"

-- Linkmeister (link@librarian.edu), August 25, 1999.

[Fair Use/For Educational/Research Purposes Only]

http://www.jsonline.com/bym/tech/0214chips.asp

Problems lurk in more than just computers

By Douglas Armstrong of the Journal Sentinel staff

February 14, 1999

Embedded chips are the wild cards of Y2K.

Only a tiny percentage of them are expected to fail when the calendar rolls over into the next century after 11:59:59 Dec. 31. But there are literally tens of billions of these dedicated processors out there in everything from microwave ovens to airliner cockpit controls (a Boeing 777 has 1,000).

Some, obviously, perform critical duties. And, according to many experts, there isn't time to check them all and tell which are bad and which are not by the time the new millennium ticks ominously in.

One reason is that the programming in embedded chips is not always readily accessible for inspection. And there are hundreds of different varieties. It's like looking for burned out light bulbs in Las Vegas -- with the power switch turned off.

"Most of the failures will be nuisance issues," says Bill Thompson, senior analyst with Automation Research Corp., a consulting firm in Dedham, Mass.

Not everyone is so sanguine.

"The embedded systems problem is still a black hole," says Harlan Smith, a Y2K analyst who moderates an online forum on the issue at y2knews.com.

"Identifying the devices that are not compliant and assessing the effect of them on the environment in which they operate is complicated."

Corporations spent a lot of time and money bug-checking the front office software code on their mainframe computers for Y2K compliance before realizing an even bigger problem existed on the plant floor in automation controls and other systems running on embedded chips.

A massive catch-up effort is under way, at least in the United States. How big is the job? Experts can only estimate.

Tava Technologies, a Colorado software and consulting firm that specializes in assessment and repair of plant Y2K problems, says that in its experience at more than 400 sites, it has "yet to find a single site that did not require some degree of remediation (repairs)."

At a pharmaceutical firm with operations in 39 countries, for example, Tava found 4,457 embedded processors in the laboratory equipment and manufacturing facilities of one location.

Based on an inventory it conducted, 18% of the items were not Y2K compliant and 17% could cause a plant shutdown or affect production.

"The chance of these systems failing was 70% for the lab and 80% for manufacturing and facilities," says Bill Heerman of Tava's Denver office.

Tava estimated that it would take 39 weeks to inventory and analyze the firm's 125 plants at a cost of $11.5 million. The fix would take another 31 weeks and cost $54.8 million.

Is there time to fix it all?

"There is little reasonable prospect of timely correction of all Y2K exposures that exist," says a report from Manufacturers Alliance/MAPI Inc. "The effort to achieve compliance is one of damage mitigation."

The effects of a maverick embedded processor are unpredictable. It depends on where it exists in the chain and what is connected to it. Typically, these chips gather a lot of information to make limited decisions.

If a single temperature sensor tied to an embedded chip in a complex chain of measuring instruments used in manufacturing were to go haywire because of a Y2K problem, for example, the manufacturer could end up with a product with different ingredients -- if the product came out at all.

The stakes involved in locating and repairing these chips are huge, given the dependence of our systems on them. The size of the chore is every bit as large, given the proliferation of embedded chips in number and design.

"They are everywhere," says Steve Barnicki, an associate professor of electrical engineering and computer science at Milwaukee School of Engineering.

Why?

"They are cheaper and more trouble-free than mechanical systems," says Barnicki. As a result, they have played a pivotal role in powering productivity improvements everywhere since first introduced in the 1970s.

Fortunately, many (like the one in your portable CD player) couldn't care less about dates.

"There are embedded systems that don't have the faintest idea what year it is," Barnicki says.

So why not hunt down those that compute dates and fool them by turning back the year to play it safe, you ask?

The answer lies in the sheer number of chips and the independent way many have been programmed. These processors also work in tandem with chips and systems that would experience their own set of problems if a false date turned up.

The issue is made more difficult by ubiquitous quirks, such as chips that have the ability to disguise that they have date capabilities and escape detection until they fail. Or those that can have a delayed reaction.

"We encountered a controller on a process line recently that rolled over to Jan. 1, 2000, just fine," says Kurt Schmidt of Tava Technologies' Denver office.

"And it kept working just fine until it went to Jan. 32, then Jan. 33, Jan. 34 and so on all the way up to Jan. 54. Some of these systems won't show the date problems immediately."

Embedded chips come in a number of varieties from a host of manufacturers.

On the low end are ROM (read only memory) chips that contain basic instructions that cannot be changed. If these have a Y2K problem, they cannot be saved. The machine they are attached to may have to go as well, if a compatible substitute chip cannot be found.

Next are PROM (programmable read only memory) chips, which typically can be reprogrammed only once, according to Barnicki.

EPROM chips (erasable programmable) can be reprogrammed thousands of times after they are exposed to ultraviolet light. Finally, EEPROM (electrically erasable programmable) chips and similar Flash ROM chips have the potential to be reprogrammed tens of thousands of times.

Rockwell Automation, based in Milwaukee, is a leading maker of programmable logic controllers (which use embedded chips) to run factory automation configurations. The brand name is Allen-Bradley.

The company lists 17 different known year 2000 issues with its controllers on its Y2K Web site.

In addition, it outlines a procedure to test its controllers for other potential problem dates, such as Feb. 29, 2000 (leap year), Jan. 10, 2000 (1/10/2000 -- first seven character date) and Sept. 9, 1999 (the "9999" date field matches an end-of-data "9999" input signal in some computer programming codes).

Rockwell/Allen-Bradley's programmable logic controller issues are a microcosm of the complexity of the problem. They have:

Processors that won't roll over on their own and must manually be set to 2000.

Processors that roll over to a new century only if the power is on at the time of century change. (Jan. 1, 2000, falls on a Saturday in a holiday weekend when many plants would ordinarily be dark.)

Processors that won't roll over without new software or bug fixes.

Processors that are dependent on the compliance of the system they are connected to.

Processors that are totally dependent on systems that are not prepared for 2000 at all, such as 286 and 386 computers.

Many programmable logic controllers don't have clocks.

"You don't put a date in there unless you need it because it wastes power," Barnicki says. "Embedded processors are stripped down to fit the application."

Although a vast database of embedded chip compliance has been assembled by Tava Technologies and others, manufacturer assessments of the chips can only help so much.

"They can test all they want," says Automation Research Corp.'s Thompson, "but it's really up to the end user with the local application to test out the system. (The processor) might work in a vacuum.

"Once it's installed with custom add-ons and special report functions that have been locally written, there is no way for suppliers to help the users predict what will happen."

Says Tava's Schmidt, "There are going to be hiccups."

And some hiccups may occur in places that cause more than just a nuisance or harm to a negligent company.

The American Chemical Society has warned that chips automating control pumps and valves to prevent spills and other hazards may have problems that have not been addressed by small to medium-size firms.

"Even chemical companies that have actively addressed the Y2K problem may have underestimated its depth," says an article in the society's Chemical and Engineering News.

"Consultants hired by Occidental Chemical found 10 times more systems with potential Y2K problems than the company's own engineers found."

The new assessment of Y2K progress by larger American companies from Manufacturers Alliance/MAPI Inc., on the other hand, found cause for "cautious optimism" among big companies, given the level of awareness and the amount of effort.

Larger companies surveyed said they were on track to be compliant by 2000, while smaller firms were having trouble finding technical help that was affordable and competent.

"In the final analysis, the Y2K issue is an annoying, resource- intensive exercise in triage and damage mitigation," the report concludes. "Time is short and the stakes are high.

"The century rollover could be a nuisance or a calamity depending on the diligence with which Y2K correction is pursued."

----------------------------------------------------------------------

-- Linkmeister (link@librarian.edu), August 25, 1999.

Thanks - I don't know whether to be glad I now am armed with this info or even more scared! The following was the only response I had to the embedded chip discussion:

There are hundreds of thousands of small to medium sized companies that won't make it next year - not because of embedded chips but because they have non-compliant software they can't afford to replace, or they have chosen to ignore the whole thing hoping it would go away. The down-the-line results is many more small businesses folding because of the reduction in spending because of so many out of work. The delicate balance is at risk.

Until February I had a quaint art and gift shop in a historic district artist's colony. One of the reasons I closed was because there just wasn't enough cash flow for me to continue. Everyone loved the shop - they ohd and ahd and loved to come in. They looked at everything and then thanked me as they left. I wanted to put a sign on the door "This is not a museum!" And this is in a booming economy where many people have descretionary income.

The point is - and I know I sound like a broken record - it is our fragile, credit-based economy that is going to take a karate chop to the gut. Even the experts agree that there are computers out there running businesses that won't work at the roll over. I know several that I have been in recently -they haven't a clue their old computers are about to crash. This isn't speculation. How will they fix themselves-when they can't produce -if they can't afford to do it now.

So, why is it everyone ignores what is so obviously going to happen to us when many of us are out of work? It does matter, you know, because, out-of-work people tend to do things they wouldn't normally do in a society where "doing" and "having" is worshipped.

Non-start embedded chips - non-compliant software, utilites up and running, all of this pales to the reality of rampant unemployment. Ask the citizens of Flint Michigan what it was like when the GM plant closed. Then mulitply this by hundreds.

IMHO, we need to be preparing for a worldwide economic crisis. The key word being worldwide. If it is everywhere - predictably worse in some places - what new system do they have plans to put in place to "fix it"? There's a scary thought.

How arrogant of us to think the US economy couldn't be dramatically effected by what happens to our myriad of small businesses not to mention the rest of the world.

Get out of debt- Get some supplies- Get out of the way.

-- April (Alwzapril@home.com), August 25, 1999.

Maxiumise your debt, to finance your supplies, get out of the way

-- Les (yoyo@tolate.com), August 25, 1999.

EXAMPLE NO EG-07

Linkmeister thanks!!! Excellent, I had no idea this site existed...

All it took was two clicks through the IEE site to find this under petrochemical... GO TO THAT SITE!

==========================

Equipment Type DCS Industry Sector Oil & Gas

PC or Computer based No

System Age 6 years

Application

DCS control system control for petrochemical plant Description of the Problem

Online rollover to Year 2000

How was it Identified

During testing. Offsite testing on a testbed was performed with satisfactory results. Upon testing of stations on site, control was no longer possible after the system had rolled over to Year 2000. It was not until this problem was evident on three of the four operating stations was testing aborted. What was the Solution

No known workaround. Plant had to be operated from one station until problem could be rectified

Consequences for the SYSTEM System Stops

Consequences of failure to the BUSINESS Near catastrophic. Limited reliability and operability of plant. Reduced production

===================================================================

Worst fears realized with a simple mouse click...I don't see how people can play "down" the embedded system problem. Clicking through these sites makes it very evident that things will occur.

*sigh*

Mike

===========================================================

-- Michael Taylor (mtdesign3@aol.com), August 25, 1999.

Barnicki...Barnicki...That guy owes me MONEY!

-- nothere nothere (notherethere@hotmail.com), August 25, 1999.

The complete link to M. Taylor's info is at http://www.iee.org.uk/ 2000risk/Casebook/eg-07.htm (sorry, I don't know how to hotlink).

But Gosh! This can't be right! After all, Steve Hewitt gave his word that, " There are no chips in the bottom of oil wells that is going to cause Y2K problems (see API public report, links are everywhere, even on Hyatt's site)." On this very forum-- http:// www.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=001Ef6

But maybe he was being specific. No chips in bottom of oil wells that will cause y2k problems. Plenty of chips elsewhere that will. Although I never did find those 'links' that were supposed to be 'everywhere' either. At the API website (American Petroleum Institute) there is no flat out assurance that embedded chips will not fail. In fact from their faqs page for consumers they basically say that it's in the pipeline company operators' and drilling rig operators' interest (and ours) to keep pipelines/drilling platforms running smoothly.

(http://www.api.org/ecit/y2k/faqs/consumer.html#pipelines)

I hope all those embedded chips realize it's in our best interest for them not to fail.

-- murphy (none@right.now), August 25, 1999.

This seems as good a place as any to post this. Yesterday we received "reassurances" from our water company that they've come upon the most wondrous solution to y2k: they'll have a 16-hour window for fixing any problems that might occur by waiting for news from New Zealand (we're in Florida) on what happened to the Kiwi water companies. They plan to link up to NZ via the Internet. How wonderful! Oh, joy!

These people live in a vacuum. That's like saying I'll avoid the hurricane by going next door. Unbelievable. Glad the well's going in next month.

Jeannie

-- jhollander (hollander@ij.net), August 25, 1999.

short and simple,

some embedded chips are date sensitive, some are not. Some will work and some will not.

-- Moore Dinty moore (not@thistime.com), August 26, 1999.

MDM,

One of the greatest tools to writing a very scary movie or book is to "keep it simple".

What you write is true and simple and it scares the hell out of me.

Mike

===================================================================

-- Michael Taylor (mtdesign3@aol.com), August 26, 1999.