This is a repost but some people may have not read this so I am posting it again Folks

This is a truly impressive document and recommended reading. Once again we have a pdf file but this is an  edited overview of the file. Very scary material in regards to those that may live around a plant of one sort or another. If people are still in denial after reading the complete document then they have no hope. Recommended.

This document is referenced extensively and has several appendices

Technology Problems and Industrial Chemical Safety

http://www.csb.gov/y2k/y2k01.pdf

On December 18, 1998, at the request of the U.S. Senate Special
Committee on the Year 2000 Technology Problem, Dr. Gerald
Poje, one of Chemical Safety and Hazard Investigation Boards
Board Members convened a Year 2000 (Y2K) workshop
(Appendix I). The workshop brought together professionals from
the public and private sectors for the purpose of drawing on the
participants expertise in order to assess the impact of the Y2K
problem with regard to catastrophic events in the chemical
process plants. Appendix II contains a copy of the workshop
agenda and a list of those individuals who attended.

Snip

The Technical Workshop as well as the research conducted for
this report concluded that the Y2K problem is one of major
proportions and has the potential for causing disruption of normal
operations and maintenance at the nations chemical and
petroleum facilities. It is important to point out that Y2K
compliance 1 activities reported to the Chemical Safety Board to
date have not found a single failure (embedded microchips or
software) which by itself could cause a catastrophic chemical
accident. However, it is unclear what the outcome might be from
multiple failures, e.g., multiple control system failures, multiple
utility failures, or a combination of multiple utility and control
system failures. Surveillance of the industrial sector that handles
high hazard chemicals is insufficient to draw detailed
conclusions.

Snip
 

Chemical workers, emergency responders and local governmental
agencies that focus on environmental health and emergency
response should be provided with training and tools (e.g.,
guidelines, checklists, and software) to address Y2K issues.
Power outages and other utility failures could constitute as much
of a threat, or even more so, than internal process plant Y2K-related
failures. Thus, utilities and oversight agencies should
expend every effort to preserve the integrity of the national power
grid system, local power supplies and other appropriate utilities.
In addition, contingency plans should incorporate specific
elements for communicating with utilities regarding each others
status.

Snip

Small and Mid-sized Enterprises (SMEs) Control and Instrumentation
Vendors

One of the major concerns regarding Y2K-related catastrophic
events may be associated with SMEs. (For the purposes of this
report, SMEs are defined as facilities that have less than 50
employees, facilities that have between 51-200 employees or are
not part of a multinational national corporation, or public sector
facilities, e.g., municipal water and wastewater facilities.) SMEs
managing high hazard chemicals can pose large risks to works
and the surrounding community. While some exceptional SMEs
are highly resourced, more generally, SMEs lack awareness
regarding the Y2K impact, resources, and the technical know-how
for fixing the problems. Given the time constraints, there is very
little chance of changing that reality. The best we can do now is
try to increase awareness, provide easy-to-use tools and
accessible resources, and provide attractive incentives for Y2K
compliance efforts. The only hope is that the SMEs, as well as
other organizations who are coming into the game late, can take
advantage of the work done by others. In this context, it should
be mentioned that the new federal law designed to encourage
disclosure of information has not yet yielded the desired results.
It is quite clear that additional work needs to be done to provide
increased incentives for information sharing. This is an area
where intervention and action by the federal government can
yield positive and fruitful results.

The major control and instrumentation vendors canvassed in this
study are involved in an extensive program to provide Y2K
compliance for their products. There is, however, reason to
believe that some independent control systems integrators may
have developed and implemented control systems for which there
is little or no paper trail. In addition, some vendors are no longer
in business, and some are not as cooperative as the major control
and instrumentation vendors. One area of concern is the small
vendors and system integrators who may have done very little to
test and certify their products. These system integrators will have
difficulty in responding to the calls from their clients. In addition,
many of these vendors and system integrators are not aware of the
Year 2000 Information and Readiness Disclosure Act designed to
promote information exchanges.

Snip

Finally, regulated facilities must also
compile a risk management plan (RMPlan) which consists of a
description of the facilitys RMProgram (i.e., the hazard
assessment, the prevention program, and the emergency response
program). The regulation also requires that the RMPlan be
submitted to the EPA no later than June 21, 1999. EPA plans to
make the RMPlan available to the public.

Snip

Another important consideration is that the RMPlans will be
available in June 1999 when the general awareness about Y2K
will be significantly elevated. It is quite likely that these two
issues will be linked for facilities regulated through the
RMProgram. Facilities should therefore expect public queries
regarding their Y2K readiness.

The federal strategy is to provide the public with candid
information and assessments of Y2K compliance status 4 .
Overreaction and panic occur when people have insufficient,
inaccurate and irrelevant information and thereby assure that
rumors hold sway. The federal government through Presidents
Council on the Year 2000 Conversion and activities of different
agencies is trying to encourage the following:

United States Chemical Safety and Hazard Investigation Board Page 17

* Get pertinent and candid information out to the public,

* Demonstrate that organizations are managing against the
problem,

* Establish that normal emergency response mechanisms have
been reviewed and updated, and

* Share technical information via the Information Disclosure
Act.

However, there are some practical problems regarding disclosures
of technical information. For example, if a company discloses
adverse information about Y2K compliance, does its market value
go down? There are also issues regarding liability from lawsuits
notwithstanding the federal Good Samaritan law.

Snip

unclear what action EPA or any other regulatory agency could
manage receipt of certification documents from tens of thousands
of companies, as only a very small number of trained staff exist to
review and validate the submissions. Given the time frames
required, it appears that the unmovable deadlines imposed by
Y2K prevent regulatory approaches from being viable options to
recommend to the U.S. Congress.

Snip

 OSHAs process safety management program has
been in place since 1992. It may be inferred that the systems and
procedures in place have gone through significant continuous
improvement, and thus may reduce the possibility of catastrophic
accidents caused by Y2K-triggerred failures. However, similar to
the EPA RMProgram, the application of the process safety
management program is also quite limited in scope, and does not
include all the Y2K-vulnerable facilities.

Snip

The main problem is that in many cases these local and state
governments are oblivious of the threat. In the limited research
conducted for this study, the following information was
identified 11 :

* According to a survey by the National Association of
Counties announced on 12/8/98, half of the county
governments lack a plan to deal with Y2K preparedness,
contingency planning and emergency response.12 This will
impact the potential availability of emergency response
services, 911 communications, and sewer and water treatment
systems.

* According to a survey by the Emergency Response Research
Institute, released on 12/4/98, less than a third of the
emergency response organizations surveyed have begun Y2K
contingency planning activities, and less than a quarter have
looked at the external effects of other organizations Y2K
compliance on their ability to provide emergency response
services.13

Snip
 

The chemical process industry relies on software and microchips
for the operation, maintenance, and control activities that are vital
to the safe operation of the plants as well as the profitable
manufacture and distribution of chemical products. Software or
microchips that store dates as two digits could render incorrect
results. For example, a control device may have been programmed
to provide a reading or report every six months using the two-digit
arithmetic. Such a device could interpret the year 2000 as 00
and calculate a negative number when measuring time intervals.
The outcome of such an event could pose a problem. The question
is: would the computer ignore the incorrect answer, or could it
cause the hardware to malfunction, or cause a major process upset?
Other such date-programming or date-embedded problems can be
categorized as follows:

* Dates stored as two-digits may assume the year 1900 instead of
the year 2000;

* 00 may not be allowed as a valid date;

* Dates may be required to begin with 19;

* Dates may have assumed a range that ends in 1999;

* Reports may assume and print a 19 as the first two digits of the
year;

United States Chemical Safety and Hazard Investigation Board Page 24

* Dates such as 9/9/99 may cause hardware and software
problems;

* Leap year may be incorrectly calculated for the Year 2000,
resulting in problems around February 29, 2000 and December
31, 2000 on the 366 th day of the year.

Potential for Catastrophic Events Stemming from Year 2000 Non-Compliance

The potential for catastrophic events stemming from Year 2000
Non-Compliance can be divided into three categories. First,
failures in software or embedded microchips within the process
plants may cause process excursions or control problems resulting
in accidents. Second, external Y2K-related problems, such as
power outages may cause various problems, such as accelerated
shutdown of processing, monitoring, and safety systems.
Accelerated shutdowns may cause other problems such as the
triggering of fire suppression systems, causing loss of water
pressure for actual fires, and disabling such systems. Third,
multiple Y2K-related incidents may exceed the capacity of
emergency response organizations to respond.

Other factors that must be considered are applications that are
purchased from a supplier and customer applications that are
developed by the users. In addition, the current utilization of
integrated operations using multiple applications all of which pass
on information/data, or use information/data makes it mandatory
that users consider this in their readiness and operational
contingency plans.

Failures in Software or Embedded Microchips

The chemical process industries, irrespective of size and type of
operations, use a variety of software and embedded microchips to
operate, maintain, and control their processes. Y2K-related
failures, can at the minimum, cause off-specification products or
shutdown of the process and at the extreme cause process
malfunctions leading to accidents. For example, the agitator on a
batch reactor may fail to operate causing the initiation of a
runaway reaction. The emergency shutdown system (ESD) is
expected to stop the runaway reaction but the ESD itself may have
an embedded chip that may be susceptible to Y2K-related failure.
Many other examples exist for both batch processes as well as
continuous processes used by the chemical process industries.15
Chemical processes are usually built with multiple layers of
safeguards that require the congruent failure of various systems to
precipitate an accident. However, many accidents in the U.S. and
overseas have occurred when multiple simultaneous failures
resulted in catastrophic accidents. In addition, some automated
safeguard systems are on-demand or in reserve, making
recognition of the potential for failure very difficult. Thus, it is
prudent to explore the catastrophic potential of single Y2K-related
failures as well as combinations of various failures.

Power Outages

No effort was made in this study to assess the potential of power
outages from Y2K-related failures. However, potential Y2K-related
power outages represent another set of problems for
chemical and petroleum facilities. While many chemical and
petroleum manufacturing facilities have backup power generators,
Y2K failures may include concurrent loss of power, cooling water
and other system malfunctions. First, plants without auxiliary
power backup systems face a threat to parts of their processes that
may not shutdown in a fail-safe mode. Batch chemical processes
are especially susceptible because the safety of the process is quite
often dependent on time-dependent factors such as precisely timed
mixing, heating or cooling requirements. Second, a potential
scenario is that widespread power outages may cause shutdowns of
many plants, which in turn will require simultaneous startups.
Although startups of chemical plants are infrequent and their
durations are short compared with the life cycle of a plant, process
safety incidents occur five times as often during startup as they do
during normal operations 16 . Thus, a large number of simultaneous
startups may increase the potential of incidents in one or more
process plants. In addition, the simultaneous restarts of large
power-consuming facilities will impose large demands on the
electrical grid.

Snip

Plant Operations

Chemical process operations are heavily dependent on control
systems, predominately automated process control systems. These
systems consist of field instrumentation, which often contain
microprocessors, in addition to the programmable logic controllers
(PLC) or distributed control system (DCS) that are used as the
brains of the automated system. When a PLC or DCS is not able
to maintain control of the process unit, safety interlocks are
utilized to bring the plant to a safe state, regardless of anything
else going on in the plant. These safety interlocks may utilize
either hard-wired (relay) or PLC based systems to define the
actions taken when the specific process variable reaches its defined
"out of control" set point. Y2K compliance programs for chemical
process control systems consist of the following steps:

Snip

During the investigation step, it is important not to spend all the
time working on the "means to the end". The best course of action
is to identify, assess, and remediate problems as quickly as
possible. While not a particularly demanding issue, there are some
important subtleties about Y2K. For example, the clock cycle
issue, e.g., the issue of register overflow. An untrained technician
may not perceive that a device uses a date and may observe that it
does not print a date. However, does not guarantee that
somewhere in the device a date is not being used, is not critical and
may not cause a Y2K failure. Another critical factor is that some
Y2K failures may not occur in the year 2000 23 ; thus it is important
to integrate Y2K thinking into everyday business.

Snip

* In some instances, as much as 3% of vendor information was
incorrect.

* Some corporations are planning to shut down operations
through the millennium transition. It is planned that their
plants will be idle but staffed during the transition. (It should
be pointed out that most of the processes for these corporations
are batch processes and usually do not produce chemicals on
New Years Eve.) Because of higher financial costs and other
safety considerations, continuously operating plants are less
likely to shut down. However, many are evaluating
contingency plans taking into consideration safety, utility
continuity, supply reliability, and customer needs.

Snip

Contingency planning for Y2K-related emergencies is categorized
into three broad groupings. Contingency planning for continued
safe operations, safe shutdown, and then finally emergency
response:

Contingency Level 1: Continued Safe Operations

The first level of contingency planning addresses those operations
that are necessary to keep the facility running in a safe and
environmentally sound manner. This includes pre-planning of
actions that can be taken to allow the facility to continue to run in a
safe and environmentally sound manner, even if the Y2K
compliance efforts fail to prevent a Y2K-related failure. With all
these additional activities resource training and refresher training
must be addressed, not just a one time effort but something
sustained that insures operations people are fully oriented and
qualified to implement these alternative strategies and operational
activities. The important issue is whether operators will be able to
recognize operational problems, and be able to respond quickly
and correctly to what they recognize from the indications from
their control room CRTs. Examples of activities that can be
categorized under Contingency Level 1 are given below:

* Minimize finished product inventories and waste effluent
levels to allow as much reaction time as possible to address
unusual situations

United States Chemical Safety and Hazard Investigation Board Page 37

* Maximize raw material inventories (within safe limits) in case
a supplier fails (Note: In addition to limitations imposed by the
transportation system, this action may create problems with
facility siting issues, and should be addressed through process
hazard analyses)

* If facility operations depend upon a small steam supply source,
consider renting a backup mobile steam generator in case the
supplier fails

* Consider using bottled gas and/or portable compressors for air
and nitrogen backup

* Consider using low-tech/cheap radios to backup sophisticated
communication systems

* Increase Operations and Craftsman personnel staffing during
critical periods in order to respond quickly to unusual
situations

* Shutdown non-essential units; restart them after critical times
have passed and essential units are running well

* Make pre-arrangements with alternate transportation sources
to handle material if primary transportation modes are not
available

* Develop a plan to manually control output from normally
automatic controllers (switch to fixed speed and control
volume output via dampers, valves, etc.)

* Identify and test manual overrides for security and safety
systems

Contingency Level 2: Safe Shutdown

Contingency level 2 is activated if the activities described in
contingency level 1 do not work. Since continued safe operations
are not possible, the facility must consider safe shutdown. This
contingency level planning includes ensuring the availability of all
personnel, equipment, utilities, services, and other resources
needed to ensure safe shutdown. These issues or items could arise
from something overlooked at the site or it may also be caused by
an external influence. Shutdown systems and other devices that
ensure safe shutdown are tested as part of the contingency planning
process. Examples of activities that are covered under
Contingency Level 2 follow:

United States Chemical Safety and Hazard Investigation Board Page 38

* Rent portable electric generators or lights for emergency use

* Increase operations and craftsman staffing during critical
periods to monitor and react quickly for shutdown purposes

* Shutdown non essential equipment before critical periods to
allow more attention time for shutdown of critical systems

* Ensure all emergency shutdown equipment and safety systems
are fully functional before critical periods (test them)

* Test Uninterruptible Power Supply (UPS) and other backup
systems to ensure power is supplied to control systems for safe
shutdown

* Consider having a backup low/tech radio system for use if the
main system fails

* Pre-test emergency vent scrubbing systems to eliminate or
minimize emissions during shutdown

* Conduct Shutdown Drills -- consider more than one system
failure and limited access to external resources

* Alert the emergency response community

* Alert utilities whenever shut down will result in a significant
change in the demand.

Contingency Level 3: Emergency Response

Contingency Level 3 is activated when contingencies in level 1 fail
to ensure continued safe operation followed by failure of
contingencies in level 2 to ensure safe shutdown. This may
indicate the initiation of a process safety incident. Thus planning
for contingency level 3 requires that things necessary for an
adequate and proper emergency response to Y2K-precipitated
incidents are available. Examples of activities that are included in
Contingency Level 3 are given below:

* Consider having the Plant Emergency Response Team on
stand-by at the facility

* Work with "outside" responders and pre-plan a backup
communication mechanism and practice a response plan

* Develop a system to alert neighbors in case the local
emergency warning system fails

* Conduct drills considering multiple system failures

* Within the facility

* With "outside" response agencies

Snip
 

Of major concern are small and mid-sized enterprises (SMEs).
Individual SMEs were not present at the Y2K Technical
Workshop, but vendors, consultants, and association leaders with
expert knowledge of SMEs were present.25 Based on their input,
it is reasonable to conclude that in these companies the level of
Y2K compliance efforts are not proportionately comparable to
those in larger companies. In the little time left, there is very little
chance of changing that reality.

Snip

Process Controls

Process Controls are used in a wide variety of applications in the
hazardous chemicals industries.27 While the overwhelming
majority of control systems continue to function with the date
change, occasional problems are encountered. Mr. Dan Daley,
Maintenance Director of Occidental Chemical, said, we have
found situations, and there are situations with some of the older
operator consoles for DCSs that effectively will go to black
screen. Mr. Jordan Corn, Rohm and Haas, stated, to date, we
have found only one catastrophic control system failure, and let me
qualify that a bit. Catastrophic meaning that the control system
itself went to an unpredictable state from which you could not
recover. The process could still have been shutdown safely, but

United States Chemical Safety and Hazard Investigation Board Page 41

the control system itself was rendered completely inoperative.
Prepared facilities anticipating such situations should manage the
problems well. However, the certainty of safety at unprepared
facilities is unassured.

Snip

Another vendor source indicated that pharmaceutical companies
are all engaged and well ahead of other industries, with power
companies second, and chemical, pulp and paper, oil and gas
following about equal to each other. The same vendor also reports
excellent feedback from its customers, who say the vendor, has
done the right thing in a proactive and honest environment
regarding the Y2K issue. This vendor also uses its web site to
provide information on its products for customers.34

Snip

Internal Y2K Equipment Audits may miss some devices.

Equipment with embedded microchips can encompass a diversity
United States Chemical Safety and Hazard Investigation Board Page 43
of devices; some of, which may not be apparent, even to facility
personnel with extensive experience. Mr. Daley stated, Then
we brought in a couple of consultants and they were named as
specialists in Y2K, and they did inventories. And we found in both
plants that we did the pilots in, we found a 10:1 ratio. We found
that they identified 10 times as many devices as we had identified.
Demonstrating Y2K Compliance by the vendor does not assure
compliance in the chemical-manufacturing environment. An
examination of the vendors test procedure and retesting data is
necessary and prudent. In some cases, vendors are unable to
assure compliance for equipment that does not operate according
to original design configuration, and after having been subject to
customer modifications.

Snip
 

SMEs have lesser access to associations such as API and CMA,
which have helped corporate entities become educated on safety
issues. An exception to this may be the propane distributors who
have a well-developed organization that is engaged in dialogue
with the government. Also, the Chlorine Institute is making Y2K
information available through their website. The information that
is being provided is quite generic and fragmented, and has not been
assessed for its utility to SMEs. The experiences with some SMEs
on other issues seems to indicate that in order to be useful, the
information provided has to be very detailed and specific to the
SMEs.

Snip

Utility Issues

A major concern of the participants at the Y2K Technical
Workshop was that the main threat to facilities could be from
external failures, such as electrical, natural gas, water and waste
water utilities. The issue is much larger than any company,

United States Chemical Safety and Hazard Investigation Board Page 46

municipality, or state. Only the federal government can adequately
address the issue.

Many members of the chemical process industry are concerned
about the reliability of power supply and are seeking ways to
assess the vulnerability of their specific utility. Individual
companies and local associations are encouraged to engage in
dialogue with their individual power suppliers to find out what
they are doing regarding Y2K. Accurate and pertinent
information about utility status is essential for contingency
planning purposes. However, for the purpose of this study, no
effort was made to assess the potential of power outages from
Y2K-related failures.

For some managers of facilities that draw high power loads
prudent safety practice may determine that the plant be shut down
during critical time periods and restarted at a later date. However,
such decisions should not be made without communicating these
planned actions with their utilities in order to prevent problems on
the power grid. As a further complication, cumulatively, small
power consumers can impact on power distribution through the
nearly simultaneous shut down of many facilities without
coordinating with their utility. Utilities can bring up or shutdown
generators as demands vary, but they have trouble responding to
unexpected changes in load or demand.

Insufficient electrical demand coupled with increased numbers of
generators supplying the electric grid could overload the power
distribution system, threaten the integrity of equipment, and/or trip
breakers. If that happened, then there could be power outages for
all the customers on the affected distribution line. The January 11,
1999 report, Preparing the Electric Power Systems of North
America for Transition to the Year 2000-A Status Report and
Work Plan-Fourth Quarter 1998, issued a specific
recommendation that would affect any advice given for facilities
considering shutting down during rollover to Year 2000:35
Unusual Loading Patterns and Minimum Generation
Conditions. Another priority concern that is emerging from the
contingency planning process stems from the need to have
additional generating units on line as a precaution against Y2K

United States Chemical Safety and Hazard Investigation Board Page 47

events. With additional generators on line and the possibility
of customer demand being low through the extended holiday
period, utilities must consider what is called a *minimum
generation* condition. When there is too much generation on
line in relation to demand, system voltages and frequency can
rise. Planning for the rollover into the Year 2000 must trade
off the need to have additional reserves to respond to possible
generator contingencies with the potential for excessive
voltages. Customers should be encouraged during the period
not to take unusual steps such as shutting down facilities that
would normally operate through the holiday weekend.
Extremely low demand or unusual pattern demand can present
additional challenges for operation of the electric system.
The response to the utility problem has to be two-pronged,
governmental leadership and corporate accountability. The federal
government should ensure the integrity of the nations electrical
grid. In addition, state and local governments should make every
effort to ensure the integrity of other utilities within their purview.
The chemical process facilities should on the other hand design
their Y2K compliance activities, particularly the contingency
planning activities with the assumption that most utilities will fail,
or at the best be under maximum strain.

Snip

Stakeholder communication has various aspects.
While logistic and timing problems may prevent a regulatory
approach for assuring and communicating Y2K compliance to the
public, the government should provide incentives to facilities to
encourage them to voluntarily communicate to the public as clearly
as possible the status of Y2K compliance. Given the extent of
work being done for Y2K compliance, this communication will

United States Chemical Safety and Hazard Investigation Board Page 48

avoid creating chaos and panic, allay public fears and promote
rational behavior. Contingency planning, risk management, and
decisions concerning shutdown must also involve communication
amongst stakeholders.

-- Brian (imager@home.com), July 05, 1999