Why the future doesn't need us.

Our most powerful 21st-century technologies - robotics, genetic engineering, and nanotech - are threatening to make humans an endangered species.

By Bill Joy

http://www.wired.com/wired/archive/8.04/joy_pr.html

-- Anonymous, April 18, 2000

Answers

Subject: The best article yet on Bill Joy's epiphany
Date: Mon, 17 Apr 2000 11:17:21 -0700
From: Tom Atlee < cii@igc.org >

Dear friends,

Sorry for the recent overload, but my brother sent me this new article about Bill Joy. And I haven't sent you much in the previous three weeks, so I'm catching up...

To me, this article feels like the proverbial Berlin Wall coming down. If you had asked me even a couple of years ago if a leading technological hero would be lucidly proclaiming the evolutionary threat of unbridled technology and it would be being covered in a very long, sympathetic and brilliantly-written article featured in the Washington Post (no less) just as we step through the gate to the 21st Century, I would have said "No way! Not possible!" But it's happening. And here, Bill Joy begins to give some inkling of how society _could_ monitor its technological developments. If only he would add citizen consensus panels.... (I wrote him, but I'm sure my email was lost in the deluge since his April WIRED article....)

Please pass this around.

Coheartedly,

Tom

PS: If any of you know folks who would like to be on my list, send me their email addresses. I'll be happy to add them on, for free.

_ _ _ _ _

RAVING ROBOTS, MAD MICROBES: TEKKIE SAYS BEWARE
By Michael Powell
Washington Post Staff Writer
Sunday, April 16, 2000; Page F01

http://washingtonpost.com/wp-dyn/articles/A14079- 2000Apr14.html

[Fair Use: For Educational/Research Purposes Only]

"I may be working to create tools that will enable the construction of the technology to replace our species," Bill Joy says. (Steve Castillo - for The Washington Post)

SAN FRANCISCO A WHITEWASHED SPANISH CHURCH, and bay window rowhouses rising higgledy-piggledy up a hill toward an azure sky. A hundred Chinese ladies raising silken knees in balletic tai chi. Morning's cool shadow slanting across the park.

On a green bench a high priest of silicon sits and talks heresy.

Heresy about a golden Silicon Valley and the scientific Darwinism that runs amok there. About the technocratic culture he helped birth, and how it just might be the death of us.

"We are dealing now with technologies that are so transformatively powerful that they threaten our species," he says. "Where do we stop, by becoming robots or going extinct?"

His name is Bill Joy, a tall, 45-year-old man with the dreamy intensity of a scientist. He is a co-founder of Sun Microsystems, one of the world's most profitable companies. A widely respected computer expert who one night sat in an Aspen diner and on a place mat designed a software language that allows your refrigerator to talk to your car.

Now he watches as three fingers of doom arise from the world he helped shape. He looks at modern genetics, robotics and the ability to build molecule-size machines and sees a technology exploding beyond our ability to control it. He sees manmade plagues and malevolent robots overshadowing even the nuclear era in the capacity for utter destruction.

He warns that our scientists, his friends, must relinquish their Promethean fire.

"I'm sorry but there are certain technologies so terrible that you must say no." He leans forward on the bench. "We have to stop some research. It's one strike and you're out."

The broader crusade he and a handful of influential scientists are embarked on is no less imposing: to challenge a scientific culture soaring too close to the sun on wings of wax. It's a culture where pursuit of "truth" is all, where researching the next hot thing is what matters.

"For Aristotle, an argument based on a poem was as valid as one based on science. We've lost that. I don't sense in this community that an ethical, spiritual-based argument carries nearly as much weight as a capitalist imperative or the notion that progress is the ultimate. That whatever happens happens.

"It's scientific fatalism and it could be fatal for us."

Apocalypse now? The mind manufactures caveats and yes buts and well, whatevers. The stock bull runs the wired decade, dot.com-ism multiplies, the Cold War has passed . . . and Joy's talking death by technology?

The temptation is to say: Cut the dark talk and pass the IPO.

But that would reckon without our cultural schizophrenia. Popular culture celebrates its bigger, faster, 999,000-megabyte future even as it devours stories of cyber-terror and robo-cops, dystopian dreams and fears that chill.

American scientific culture pays less heed to its shadows. In the stock-optioned ecosystem of Silicon Valley, where scientific brilliance and venture capital leapfrog toward the horizon, the talk is of satisfying our deepest hungers. Scientists would stamp out disease, use computers to make us hear and see and feel better, maybe even beat back death itself.

And they recoil when Joy, who was co-chairman of a presidential commission on information technology, talks of barring some research.

"The evolution of technology just continues the . . . explosion of biological evolution," says Ray Kurzweil, a cutting-edge scientist and author of "The Age of Spiritual Machines." "In the 21st century we will make 20,000 years of progress."

Perhaps. But maybe the four-century-long process that freed the modern scientific mind from the bonds of religion, a philosophical string that runs from Copernicus to Descartes, has had a curious effect. Maybe our culture has made science so powerful that it's replaced God at the center of our universe.

"The technological impulse to break free from the constraints of nature and emancipate the human being is noble and a central archetype of mankind," says Richard Tarnas, the philosopher and author of "The Passion of the Western Mind." "The great danger of our time is that the quest hasn't been matched by a moral and psychological awareness of our limits."

Joy spoke at Stanford University a few weeks back, at a symposium asking: "Will Spiritual Machines Replace Humanity by 2100?" Midway through, Joy peered quizzically at scientist Hans Moravec, a husky man who has established the world's largest robotics research program at Carnegie Mellon. Moravec is convinced he can create a conscious robot in 30 years, and is quite enthused by the prospect.

"Isn't there a point at which you worry?" asks Joy.

"Well," Moravec allows, "I do worry about intelligent robots getting strange ideas . . ."

GRAY GOO

Joy traces the kernel of his discontent deep into the technological cocoon, of which he is a creature. He recalled talking with Kurzweil about computers in the autumn of 1998. It is theory and faith among denizens of the digital world that computer chip speed doubles every 18 months.

Joy always believed this rate would slow down as science rubs against the boundary of the physically possible. And he drew comfort from knowing there's a limit to the power of computers.

Now Kurzweil told him that molecular-level advances would allow for computers a million times faster and smarter by 2030. So a world of possibilities revealed itself, and cast Joy into darkness. He's married, has two children and a beautiful home in Aspen; he saw the clouds moving in.

He wrote of his epiphany in a much-noticed essay for Wired magazine:

"It was only [then] that I became anxiously aware of how great are the dangers facing us in the 21st century. . . . We have yet to come to terms with the fact that the most compelling 21st century technologies--robotics, genetic engineering and nanotechnology"--pose a more dangerous threat than any past technologies.

These computers and genes and micro machines, Joy writes, "share a dangerous amplifying factor: They can self replicate: A bomb is blown up only once, but one bot can become many, and quickly get out of control."

Genetic research is the most advanced. Already scientists splice the fish cell with the strawberry, the salmonella bacterium with the sperm cell. (What is not known is how these new cells and bacteria will behave once released outside the lab; the only certainty is that cells and bacteria replicate and mutate.)

A bit further into the future, scientists hope to build sub-microscopic nanotech machines so small they can fly by the billions into the atmosphere or cruise the bloodstream like galleons. Scientists envision computer robots so smart that, decades or centuries into the future, a metallic eye might blink and a silicon mind awaken.

The research fueling this micro industrial age is hurtling forward. It is not taking place in the crucibles of the nuclear age, the government-funded labs where scientists held information confidential on pain of imprisonment. That's the dinosaur age.

The market rules now. The venture boys erect a thousand elegantly outfitted labs, and the drive for riches and pure research attracts scientists of the highest caliber. Information is swapped on the Web and the ethos is libertarian and triumphal: Leave us alone and we will transform your world.

To its devotees, this is the technological great leap forward. Joy divines a darker shadow.

"Silicon Valley is like the Burgess Shale of the Darwinian experiment," he says. "Every scientist has become a new phylum and everyone is running around thinking they'll be the biggest mammal. They all want to be billionaires and no one wants to think about implications of their research."

Joy acknowledges that new technologies might cure diseases and increase crop yields, and bring great surcease from suffering. But at the crossroads of knowledge and the individual genius, the threat to humanity metastasizes.

Ruin might come from a laboratory experiment gone awry. Or a college student reading a Web recipe and mixing test tube viruses. Or the genius psychopath, shooing a boutique virus or self-replicating robots out the door.

It is what Eric Drexler, the father of nanotechnology, calls the gray goo problem: that someone might, inadvertently or otherwise, manufacture a ravenous and invisible man-made machine or bacterium that would outstrip natural competitors and, propelled by wind and rain, turn life to dust in a matter of days.

"There is a tremendous, market-driven haste to get into invisible technologies that are unstable and dangerous to life," says Tom Valovic, a research manager for International Data Corp. and author of "Digital Mythologies." "It requires this religious leap of faith that science knows best."

Joy has renounced work on nanotechnology, seeing it as too perilous. But he acknowledges that software he invented now enables one computer to talk with another, and so might help another scientist in another lab create a menace.

"I may be working to create tools that will enable the construction of the technology to replace our species. How do I feel about this? Very uncomfortable."

I AM BECOME DEATH

Men sometimes speak as though the progress of science must necessarily be a boon to mankind, but that, I fear, is one of the comfortable nineteenth-century delusions which our more disillusioned age must discard.

--Bertrand Russell in 1924 <> Many scientists respect Joy, even as they dismiss his fears as primitivist. From Faust to Mary Shelly to Einstein, they say, man has recoiled from the shadow of the new. And as often--refrigeration, telephones and penicillin come to mind--our technological and chemical handiwork makes life immeasurably healthier and easier.

It was once said labor-saving machines would cause mass unemployment, that recombinant DNA would be the end of us. Haven't we heard a thousand siren calls of doom?

"The fact is that he is right; this could happen," says John Perry Barlow, founder of the Electronic Frontier Foundation. "By the same token an asteroid could hit the planet Earth and have an even worse effect."

To which Freeman Dyson, the Princeton University physicist, adds: "There is a hidden cost of saying no. And that's the advances in health and food production that we will forgo."

Against such optimism lies the weight of homo sapiens history. So many technologies, from chariots to flying machines, become killing tools in our hands. And the bacteriological world spawns as many surprises: the Spanish flu, Ebola, AIDS. The grandfather of these 20th century disasters was the medieval plague.

Arriving in so many guises that 14th Century doctors termed it the "strange chameleon," the plague ravaged Europe, killing one-quarter of the population and causing bailiffs in London and Venice to nail shut houses of the dead and dying. Today, scientists can identify but a few plague strains. Many more strains have disappeared, or lie dormant.

We tend, also, to underrate our scientific ability, and our capacity for mayhem. Scientists in the 1930s laughed at the suggestion that they might soon explode "dangerous quantities of available subatomic energies." The nuclear bomb, a renowned physicist argued in 1930, was "a hobgoblin."

Fifteen years later, Edward Teller oversaw the construction of such a bomb. Warned that it might ignite the Earth's atmosphere, he endorsed an explosion anyway. And his colleague, Robert Oppenheimer, stared at the nuclear dawn and quoted the Hindu God Vishnu: "I am become Death."

The point is not that technology is evil. It is amoral. Human nature is the wild card.

Joy has been talking for hours this morning. The white around his blue eyes is burnt red from lack of sleep. We leave a restaurant and walk across the street in San Francisco. A brilliant western sun beats down, a dog barks, an old man sips an espresso under a palm tree.

Joy talks of the golden valley of silicon an hour's drive to the south. He discerns a thirst for scientific knowledge--and the money to underwrite it--that frankly scares him.

"In the endless pursuit of truth, we're about to bring back the notion of fate. By empowering everyone to hold the powers of destruction, everyone controls our fate as a species."

His eyes catch yours. "That's not a pleasant thought, is it?"

UPLOAD YOUR BRAIN

"We need to distinguish between artificial intelligence and downloading your brain into a computer, which is controversial," says Ray Kurzweil, on the coming age of spiritual robots.

We're in a lecture hall at Stanford University, jammed with scientists and aficionados of technology, listening to a panel of geneticists and experts on robotics and nanotechnology discuss how and when man might become silicon. Bill Joy is on the panel, too, in his new role as house skeptic.

To listen to these scientists, to swim in their virtual realities, is to understand how profoundly Joy's message disquiets them. This is a world where evolution equals technology equals progress. Where scientists talk about what technology "wants" and "needs" and "demands."

The passing of our carbon-based bodies is seen as inevitable, like a butterfly shedding its chrysalis. It's no insult to call this science fiction; science and fiction are tectonic plates that long ago converged in Silicon Valley.

"We're in for a 100-year identity crisis as human beings, and we're launching right now," says Kevin Kelly, an influential writer and member of the high-tech digerati. "We should figure out what technology wants and train it to be a good citizen."

A few of the scientists see intelligent robotic life as centuries away. Others discern it around the bend of the next decade. Some believe that microscopic nanotech centurions will guard our bloodstreams against man-made viruses. Others argue that we may have to manufacture pathogens to combat other manufactured pathogens.

Uploading human brains into computers, blending the biological with the silicon . . . the point of agreement is that swift, irrevocable change is inevitable. Technology will be our savior.

What's striking is the bedrock faith in the church of science as the ultimate truth.

"Progress and technology are inevitable," Kurzweil says. "We are talking about the next evolutionary step. In virtual reality, we will have bodies and riches. And we will take a walk on a virtual beach in a virtual Cancun."

Hans Moravec, the computer scientist with the blue T-shirt and the blue work shirt with the collar pulled up clerical style, is the least sentimental of the panel. In his view, sensory pleasure, ethics, morality, all of these are evolutionary outcroppings that could shear away as we evolve.

"One way to evade the biological threat is just to become non-biological." Moravec plays with a smile that suggests he's aware that the homo sapiens in the audience might find his proposal discomfiting.

"The evolution of our descendants will push them into entirely different realms." He's still smiling. "They will become something else entirely. I don't know why you are disturbed by that."

On and on the panelists talk, a wizard's brew of possibilities. Joy pulls at his upper lip and listens with the detachment of an apostle strayed. Their assumptions about the world and technology are no longer his.

"To use the word 'want' about technology, to talk about technology as our 'children'--" Joy shakes his head, disbelieving. "We are using an incredibly misleading vocabulary to try to get around vast ethical questions."

And he's not comforted by the notion of uploading the human brain into the computer maw. Hit the delete button on that one.

"Robot existence will never be a human existence," Joy says. "It is not three-dimensional. We're not a native form in cyberspace. If we allow evolution to go on in there, what results will not resemble us."

Roam the world of Web thinkers and scientists, in California and elsewhere, and it's not hard to find people who share Joy's fears. But the prevailing ethos, that scientific research cannot be constrained by man, is so strong that few even talk of reining it in.

Christine Peterson of the Foresight Institute, a California new-technology think tank, counsels fear followed by acceptance.

"Bill's correct to be nervous and seriously scared," Peterson says. "But research is not going to stop, that's not an option. If you try to stop people, they'll go underground."

Peterson would ask research labs to agree to outside surveillance. Other Foresight board members are working on legal and ethical protocols for nanotech research.

As for an age of intelligent computers?

Peterson suggests we might want to be careful. It will be hard to know what annoys them. (When you thumb your nose at a conscious computer, will it know it's being dissed?) We'll want to keep the computers' competitive impulses in check.

"We want a system where computers have no need to wipe us out. We want to make sure that physical property rights [translation: Our right to breathe and live] is protected."

She likens the human race to the Amish. Our legal and ethical system ensures that the Amish can live happily insulated from a technological world.

So one day, by implication . . . ?

"Right. When computers rule, we're going to be the Amish."

CRASH THE SYSTEM

"A Joyless Future" . . . "No Joy Here" . . . "Silicon Valley Killjoy"

So the headlines read in the days after Joy's essay appeared, the heretic nailing his manifesto to the door of the high-tech temple. National and international media interviews, scientists reacting, columns mixing praise with dismissive shrugs.

He e-mailed copies to lawmakers, to the Supreme Court, the president and his Cabinet secretaries. He's yet to hear back from any politician of note. The Republicans and Democrats are clamoring to claim credit for the technological dawn. Who wants to talk about the night?

Nor do many of his friends in Silicon Valley want to hear these arguments. In a way, it goes to why he decamped several years ago, left the Valley and its hungry young men and women sleeping on lab floors. He left for the vertical beauty of Aspen. Different time, different culture.

"The Valley culture has kind of gotten away from me," Joy says. "They're in this Darwinian mind-set. They can't imagine not doing it."

So he hopes to take the conversation outside the church door, to like-minded scientists and activists. Several high-tech thinkers, from Valovic at Intel to physicist Amory Lovins to John Seely Brown, director of the Xerox Palo Alto Research Center, now argue that our high-tech culture suffers from technological tunnel vision, without concern for protocols and the social good.

And the Clinton administration has taken halting steps, such as declaring emerging diseases a national security threat.

Joy would go much further. Prohibit research on dangerous technologies and put stringent international controls on sensitive equipment, much as nations do with plutonium and biological war agents. Create an international whistleblowers' fund and pay vast sums to anyone who exposes life-threatening research work.

And unleash the lawyers. Demand that private labs take out insurance against terrible accidents, and let the raptor tort types have at them. Death by lawsuit, a strategy immediately recognizable to any anti-tobacco activist.

"We need to reset the social contract," he says. "Just as we did in response to the technologies of the first industrial age."

In the end, perhaps, it comes back to faith. A former colleague or two has suggested that Joy has gone round the bend, become a primitive spiritualist in a rationalist world. But talk to the digerati long enough and you hear a sometimes unspoken belief in a "hidden hand" that shields us from the most terrifying use of our inventions.

We've had the bomb for half a century, the argument goes, and we haven't destroyed ourselves. AIDS rose up along with the technology emerged to keep it at bay. We have survived so far, therefore . . . .

"There is a co-evolution between society and technology," argues Barlow of the Electronic Frontier Society. What keeps us in balance? "At a certain stage you just have to go theological," he says. ". . . I do believe that there are larger forces than ourselves at work."

Joy comes to quite the opposite conclusion. He is no atheist, he left that certainty behind in college. But the world, he believes, is in our hands. It is the price of our maturity as a species.

He walks through the park toward his car. He stops and watches the Chinese ladies do a slow tai chi dip and rise. There's not a cloud in the sky.

"If you don't believe extinction can happen," he says, "check the fossil record."

c 2000 The Washington Post Company

_ _ _ _ _ _ _ _ _ _

Tom Atlee * The Co-Intelligence Institute * Eugene, OR
http://www.co- intelligence.org
http:// www.co-intelligence.org/CIPol_Index.html

-- Anonymous, April 18, 2000

"If you don't believe extinction can happen," he says, "check the fossil record." Absolutely. Thank you Diane.

By the way...the earth has been strangely quiet and still in Cali lately. I feel a shaker comin' on.

Mike

==================================================

-- Anonymous, April 19, 2000

Extinction for humans may be best for all concerned.

Vernor Vinge has some interesting works out dealing with the surpassing of human intelligence by some of our techical products and the dire outcomes that can result, I recall an article of his in Co- Evolution (Brand, et.al) from the mid-1980's so its not breaking news but well worth looking into. A major point he makes is that we will have no way to understand or deal with the motivations of superior machine or non-human intellegence.

-- Anonymous, April 19, 2000

From a particularly good edition of NetFuture by Steve L. Talbott Netfuture

Genome Hackers

Last month a 17-year-old girl won first place in the Intel Science Talent Search for an impressive bit of cryptographic work using DNA sequences. Many would be surprised to learn how common it has become for secondary school students to work with DNA. In what passes for high school biology these days, students are often given sophisticated, highly automated kits enabling them to carry out various recipe-like manipulations of isolated DNA.

Of course, this training as lab technicians has little to do with understanding the world of plants and animals -- and a good deal to do with the cultivation of false and one-sided notions about living organisms. But there's no denying the glamor in those kits. And while such abstractions as the students employ may reveal almost nothing of the world's biological richness, there is nevertheless power in them -- a power that is all the more fearsome for the fact that it is mostly blind.

If there's one thing hackers understand, it is the appeal of blind power, which might be described as throwing a wrench into the works and seeing what happens. This brings to mind a recent comment by Donella Meadows, who teaches environmental studies at Dartmouth: "It is only a matter of time before [biological] hackers appear who think it might be fun, as computer hackers do, to create and release their own viruses".

If and when this happens, we'll get a fresh perspective on the shallow characterization of computer viruses (and their hosts) as living things. The real danger is not in the fanciful prospect of raising our machines to life, but rather in the already entrenched practice of treating living things as if they were machines. In this game, it is not only Intel prize winners, but also hackers, who will feel quite at home.

(Donella Meadows' brief, excellent article referred to above is available here)

Hallyx

"We are a culture obsessed by new technical capabilities for their own sake, rather than the worthwhile activities and institutions that all technical capabilities presumably exist to support." ---James Howard Kunstler (The Geography of Nowhere)

-- Anonymous, April 19, 2000

Have to say, after reading the original Bill Joy article, and the above, theres a part of me that wants to hide away in a natural world, on a mountaintop or deep in the woods and enjoy life until we cant anymore. But then the other part internally yells... DO something!

Probably means well go down with the global ship, or eleventh hour, find the planetary liferaft. Donno. 50-50 chance. Or not.

Diane

BTW, from the original Bill Joy Wired article...

[snip]
A second dream of robotics is that we will gradually replace ourselves with our robotic technology, achieving near immortality by downloading our consciousnesses; it is this process that Danny Hillis thinks we will gradually get used to and that Ray Kurzweil elegantly details in The Age of Spiritual Machines.
[snip]

This sounds suspiciously like the science fiction story... Vacuum Flowers by Michael Swanwick. (Thanks David).

Will read that article next Hallyx.

-- Anonymous, April 19, 2000

Good one Hallyx. Reposting here.

Diane

Donella Meadows' The Global Citizen, March 16, 2000

Moments of Shocked Silence About Biotech
http:// iisd1.iisd.ca/pcdf/meadows/trinity.html

[Fair Use: For Educational/Research Purposes Only]

Biotech stocks plummeted this week as President Clinton and British Prime Minister Tony Blair requested that companies make their data on the human genome public.

Private firms are racing madly to read and patent the genetic code that makes you you and me me. They are trying to beat publicly funded labs, which are required as a condition of their grants to publish the gene sequences they unravel. One company, Celera Genomics, is funded by drug companies with the understanding that the funders will see the code before anyone else does.

If it strikes you as alarming that private investors can patent and keep secret and sell something that sits within every cell of your body, you ought to pay much closer attention to the new, jaw-dropping biotech industry. I have just spent several weeks with my students listening to biotech enthusiasts, critics, and a lot of folks in between. There were three particular moments I'd like to tell you about, all of them moments of stunned silence.

The first came when we heard from an ecologist who sits on a USDA panel that approves the release of genetically engineered crop plants. Of the 71 applications currently pending, one is for the implantation of the gene by which scorpions make their toxin. Splice that gene into a plant, and anything that nibbles on a leaf, from woodchucks to bugs, falls down dead. Of course people who eat the plant fall down dead too, so there must also be a package of genes to turn the scorpion gene on and off. Turn it on in the roots and leaves and stems, turn it off in the flower and fruit.

But what happens to the poison, the students asked, when roots or leaves decompose in the soil? What happens if the turn-off gene doesn't work infallibly? Would we have to check every fruit or grain for traces of scorpion poison?

Don't know, said the ecologist.

Silence.

The second moment came when a geneticist described a new rice with a pasted-in gene that allows the plant to make and store beta-carotene, the yellow pigment from which our bodies make vitamin A. Thousands of poor children in Asia, who eat little but rice, go blind or die for lack of vitamin A. The "golden rice" could solve that problem.

A hand went up, and one of the students asked, "Why not just splice the beta-carotene gene into the child?"

Silence. Finally another visiting expert said, "Within five years that could be possible. Fasten your seat belts."

More silence. I guess everyone's mind was racing as mine was. I was picturing golden children. Then I thought, why not splice in the gene for chlorophyll while we're at it, and just send the kids out in the sun to photosynthesize their lunch? Gold-green children.

Moment number three came when I showed the students a documentary called "The Day After Trinity." It's the story of J. Robert Oppenheimer, the developer of the atomic bomb, told through interviews with some of the great physicists who worked with him at Los Alamos during the Second World War.

The cause was compelling: to stop Hitler. The science was thrilling. The effort was tremendous. The bomb was nearing completion when Hitler surrendered in May, 1945.

That surrender did not cause any slowdown in the work at Los Alamos. There was too much excitement. It was nearly time for the first test, called Trinity, which took place at Alamagordo, New Mexico, on July 16. The scientists said that on that day, as they watched the first atom bomb explosion in history, their reaction was joyous. "It worked!"

Less than a month later, when a similar bomb incinerated 100,000 people at Hiroshima, one scientist said his first thought was, "Thank goodness it wasn't a dud." His second thought was, "Oh my God, what have we done?"

The film ends with Oppenheimer testifying in Washington two decades later. When asked by a senator how to contain the nuclear arms race, Oppenheimer answered, "It's 20 years too late. We should have done it the day after Trinity."

I turned on the lights. The students just sat there. Didn't move. Didn't say a word. I couldn't either.

Geneticists are already cloning sheep and cows and mice and pigs. They can pick out a trait from almost any creature and paste it into any other, and they are on the verge of being able to turn a gene on or off at will. We already plant gene-spliced crops on tens of millions of acres. We can order genes from catalogs. Within a few years we will be able to read the code for our very selves and reach in and tinker with it. It is only a matter of time before hackers appear who think it might be fun, as computer hackers do, to create and release their own viruses.

The stock market is speculating on this stuff. National leaders ask companies, politely, to make their knowledge available to all. We need to do much more that, more than just fasten our seatbelts and go along for the ride. We need to slow down and think together about where this technology is going and who should own it and who should decide.

For genomics it is still the day after Trinity. We don't want or need to have to ask, helplessly, "Oh my God, what have we done?"

-- Anonymous, April 19, 2000