Chairman, Institute for Molecular Manufacturing
President and Chief Operating Officer, Teknowledge Corporation
The future development of molecular nanotechnology (MNT) will require an active risk management program to achieve huge technical and economic benefits with minimum negative security and environmental impacts. This paper covers defense security and ecological principles relevant to MNT designers, classifies the nature of the risks, and recommends candidate risk mitigation procedures. Specific recommendations include:
The need to manage risks increases as technologies become more powerful in the size, distribution, and time scale of their effects (Jacobstein, 1995). All technologies have risks, from chipped flint to automobiles. Hammers can be used to build houses for the homeless or to murder innocent children. We do not outlaw hammers to prevent the abuse of the technology. We outlaw its use for hurting people. Other technologies, such as the halogenated organic compounds used in pesticides have important agricultural uses, but they are now handled with increasing care due to unintended toxic effects resulting from their transport and biomagnification in the ecosystem (Odum, 1971). Some technologies, such as nuclear, chemical, and biological weapons of mass destruction (WMD) are designed specifically to injure or kill large numbers of people simultaneously, and thus, their development and possession are subject to strict controls (Falkenrath et. al., 1998).
It is difficult for educated and well intentioned technologists to consider the consequences of putting technologies that can be used as weapons of mass destruction (WMD) into the hands of people who are: ignorant, careless, greedy, vicious, or executing dutifully an ideological crusade against unbelievers. Terrorism or deliberate misuse (via designing out safety measures) could turn a vital and life supporting technology like MNT into a very difficult problem.
Abuse is a risk with almost any technology. The car used to drive children to school could also be used to mow over a crowd of people waiting in line for a movie. However, there are important differences in scale. A weapon of mass destruction might kill thousands or millions of people, not just a local crowd (Lappe, 1971). What has changed is both the scale and power of the technology we are putting into the hands of individuals. Most individuals are responsible, but what can society do to prevent the few from destroying the many? Kenneth Watts (1974) wrote: “The magnitude of disasters decreases to the extent that people believe that they are possible, and plan to prevent them, or minimize their effects.”
David Forrest (1989) was a graduate student at MIT and a member of the MIT Nanotechnology Study Group when he outlined a case for guiding nanotechnology R&D to avoid dangers, rather than blindly opposing development. If nanotechnology development was slowed in the U.S. or other democracies, it would increase the chances that the technology would be developed in a country without accountability or a free press. Forrest identified four specific phases which might require different types of regulatory control.
Phase 1 would put in place a minimalist regulatory framework and operative agency before a working assembler had been developed. Phase 2 would establish protection mechanisms to prevent working assemblers against theft by groups that might abuse the technology. Regulatory review distinctions would be made between nanoscale products that contain no assemblers or replicators, products that contain non-replicating assemblers, products that contain replicating assemblers, and products that contain replicating assemblers that are designed to mutate and evolve. Research in Phase 2 would pursue development of sealed assembler labs (Drexler,1986), so that experimentation and product review could be done effectively with less security. In Phase 3 sealed assembler labs would exist, and be made available to product developers on an open basis. Phase 4 marks the introduction of an active shield to protect the environment as described by Drexler (1986). If this technology proves viable, much of the prior regulatory framework could be gradually phased out.
Postrel (1998) suggests that those who would attempt to control or regulate new technologies are stasists, as opposed to dynamists who embrace the future and are willing to let technology flow freely. While it may seem attractive to embrace dynamism and avoid a label associated with being anti-future and uptight, the problem is that our technologies have become powerful enough so that a small group of terrorists can potentially ruin the game for everyone. The labels stasist and dynamists are useful for rhetorical purposes, but the real world calls for some sane synthesis between these polar opposites. A dynamic orientation that is tempered with the knowledge that some controls may be necessary seems more appropriate to the non-ideal technology deployment circumstances we are currently facing.
Burch (1998) reviews tort liability laws that may be invoked as a control mechanism for MNT. No doubt this strategy will eventually provide some viable regulation and consequence management. Liability cases, or the threat of them, may provide corporations with some insight about how much to invest in safety and environmental controls. A problem with this approach is that many hazards are capable of creating damages that are orders of magnitude larger than the total assets of those who create them. Bankruptcy places an effective limit on corporate liability, but the risk exposure of the public and the environment may be effectively unlimited. The problem of limited redress is compounded when individuals can weld molecular nanotechnologies of unprecedented power.
James Watson (1999) was quoted in Time magazine saying: “In retrospect, recombinant-DNA may rank as the safest revolutionary technology ever developed. To my knowledge, not one fatality, much less illness, has been caused by a genetically manipulated organism.” This technology has been in practice since 1973, and during much of that time, bioengineers operated under a fairly strict set of self imposed biotech safety R&D guidelines (NIH, 1976). Was drafting and implementing biotech safety guidelines a waste of time? Clearly, some of the guidelines were overly restrictive, but they established that the research community was capable of self-regulation, which may have been preferable to having less informed guidelines pressed upon them.
In addition, although most rDNA research is for medical and agricultural purposes, the guidelines probably informed much of the bioweapon research that was conducted for years in the U.S. and the former Soviet Union. To evaluate the effects of the guidelines, we would also have to know what didn’t happen as a result, and this is difficult to know. What is clear is that genetically manipulated organisms provide a means to augment natural selection in the design and deployment of bioweapons of mass destruction or WMD (Falkenrath et. al., 1998). The issue here is not whether rDNA is inherently hazardous, but rather, the use of the techniques for creating weapons.
Watson goes on to say: “Never postpone experiments that have clearly defined future benefits for fear of dangers that can’t be quantified.” This is indeed a high standard to meet. Given the considerable uncertainty that surrounds new technologies, quantifying risk remains a black art. Humans have a poor track record using foresight in anticipating the negative consequences of new technologies. Examples include: Thalidomide and DES given to mothers in the U.S., mercury in Minimata Japan, hydrogen cyanide gas release from a Union Carbide pesticide plant in Bophal India, PCB’s now ubiquitous in human body fat, broad-spectrum halogenated pesticides acting as genetically active agents and teratogens in the environment, and poorly constructed or operated nuclear power plants (eg., Three Mile Island and Chernobyl). In each of these cases, a technology that theoretically could have been handled safely was handled improperly, with disastrous, and sometimes lethal consequences. In light of this history, some reasonable and selective safeguards in dealing with MNT might prove wise.
The agencies responsible for controlling WMDs have evolved a set of unofficial working principles that guide their thinking about terrorism and defense security risks. These unofficial principles are summarized below.
These principles suggest that if the MNT design community wants to be able to capture the unique future benefits of the technology (eg., space development, life extension, inexpensive control over matter), it needs to pay attention to how the technology gets deployed, and how it will integrate with the systems with which it interacts. This is the world of maniacs, terrorists, ignorance, greed, rogue states (Reich, 1998), and the natural environment — which plays by its own rules, developed over millions of years of co-evolution and natural selection.
Concern about environmental consequences is unpopular in the high technology community. It tends to be seen as a nuisance that interferes with making money and making progress on technological goals. Some environmentalists are indeed irrational, unreasonable, unscientific, economically ignorant, biologically idealistic, and generally irresponsible about their “facts”. However, rational people also have concerns about the environment, and some of these people have a deep understanding of biological systems. This latter group can be divided into two subgroups: 1) those who think that nature is sacrosanct and technology is an enemy to the environment, and 2) those who think nature is a form of biotechnology, and that human technology is natural. Those in the second group also understand that although human technology may be “natural”, unless it has been specifically engineered to integrate well with natural biotechnology, it may cause serious damage to humans or other species. Given the inherent biochemical feedback lags in biological systems in general, and in multi-tiered ecosystems in particular, quantification of risk remains an elusive goal. What are the ecological principles, or facts of life, that should be considered in ameliorating the risks of new technologies and designing them to integrate well with the current biological systems context?
A reasonable set of principles was summarized by Commoner (1970) in the Four Laws of Ecology:
Each of these laws is stated in a somewhat simplistic and cute way that may substitute for a deeper understanding of the underlying biological principles. The more detailed descriptions below provide additional context on the meaning and relevance of these “laws” to MNT design and development.
1. An ecosystem is a network of interconnected systems including populations, species, individual organisms, and their physical and chemical surroundings. These systems have evolved over millions of years into complex cycles of dynamic interdependence and competition, controlled by cybernetic feedback mechanisms. Human caused alterations in one part of the cybernetic system are capable of causing delayed feedback changes, instability, and even collapse in other parts of the system. This is not speculation or dogma, it is well-documented fact — witness a classical predator-prey cycle in a complex food chain. Eliminate the predator, and the prey population will often boom and bust. The significance for MNT is that its production systems and products will inevitably interact with the biological environment, often in subtle and non-obvious ways. It is incumbent on the designers to understand in advance, as best they can, how these interactions may occur.
2. The first law of thermodynamic states that “Energy and matter cannot be created or destroyed”. In nature, what is excreted as waste by one organism is taken up by another as food”. Animals release carbon dioxide as waste into the atmosphere, and plants take up the CO2 as a feedstock for glucose synthesis, releasing oxygen as a byproduct. The excreted O2 is inhaled by animals for respiration. Over the past eighty years, humans have developed and released thousands of synthetic compounds which interact with organisms and ecological cycles in unplanned, and often harmful ways. Tracing the likely ecological pathways of MNT products will be critical to proactive, environmentally sound design.
3. There are effectively four plus billion years of evolutionary R&D behind living organisms. Living things have co-evolved complex internal components and external interactions. These interactions may be subtle and poorly understood, but they do exist. When humans design new components that interact with an organism’s biochemistry or ecology of relations with other organisms, they may improve on the original interactions, but this is highly improbable. This is particularly true if the designers were unaware of these interactions in the first place. Due to the complexity of the underlying “code”, they are more likely to intervene in negative, possibly destructive ways. Gregory Bateson (1968) explained that human design and manufacturing operations are driven not by a knowledge of these complex webs of nonlinear interaction, but rather by relatively simple, linear purposes — typically to make a new product efficiently and profitably. Humans tend to make linear connections from point A to point Z in a complex web of nodes, often cutting through or short circuiting a delicate network of highly evolved ecological interactions. Since the consequences of this type of linear, “ecologically blind” interaction are typically negative, MNT designers need to pay attention to the systemic pathologies that may result from release of new components. These components should be considered “actors” in a highly specialized and co-evolved environment.
The point here is not that nature is the ultimate technology designer, or that natural systems produce optimal designs. Many of nature’s component designs are ad hoc, inefficient, and from an engineering perspective, easily improved upon. However, it is hard to make “improvements” that work well with the rest of the finely tuned natural systems that form the bulk of the machinery we and other species are still using for life support. While it is possible to redesign whole life support systems, nature ultimately selects what species, systems, and components survive over millennia. It is the final judge of the fitness of any component or system, and point optimized components may make for poor systemic interactions.
4. The notion that every gain in one part of a system must be won at some cost in another compartment of the system is central to economics and ecology. This is not about the potential of technology to move the world beyond some types of zero-sum interactions. It is a recognition that systems are a connected whole, in which energy and matter can be transformed, but are not lost. These interactions have a distribution of consequences, and at any given point in time, some part of the system “pays” for changes in other parts of the system. The relevance to MNT designers is that MNT will provide unprecedented economic, environmental, and technological gains in some parts of the world, and the same changes may enable economic displacement, environmental destruction, and terrorist activity in other parts of the world. The relevance for MNT designers is to consider the range of likely consequences, recognizing that there is no “free lunch” available. MNT designers and entrepreneurs should attempt to exercise responsibility by using available foresight in the introduction and development of the technology.
Given the risks that are inherent in the introduction of MNT, why should we proceed? The answer is straightforward. We cannot forget what we have learned about how to control manufacturing processes. Given that many of our environmental and economic problems are directly related to our control or lack of control over production processes, it would be irresponsible not to push ahead with MNT. For example, most pollution is the result of poor manufacturing or transportation techniques that result in toxic chemical byproducts being released into the environment. MNT could provide clean manufacturing and transportation technology. Poverty has less to do with laziness or absolute resource availability, and more to do with insufficient amounts of energy, material goods, and education being distributed to four fifths of the world’s people. MNT could provide the material infrastructure to resolve some of those problems as well. So, it is likely that the environmental, economic, and manufacturing benefits of MNT are much greater than the costs of managing the risks intelligently.
What are the risks, and how can they be managed, albeit imperfectly? Table 1 shows some of the environmental risks that have been identified along with probable methods for managing them. Note that unintended environmental risks are probably easier to control than the deliberate abuse that creates security risks.
|Environmental Risks||Risk Management|
|Runaway replicators||Design rules for preventing unchecked replication, and damage to biological systems. Require synthetic fuels not abundant in nature (Merkle, 1992)|
|Runaway nanomachines||Distinguish nonreproducing nanomachines from replicators; special fuels & redundant stop code checks|
|Experimental replication system accidents||1 micron sealed assembler labs with self destructs, and 1 cm thick containment systems|
|Nanomachine induced environmental damage||Environmental improvements via zero emission manufacturing, transportation, and clean up are likely. Modify the TSCA, implement Design for Environment (DFE) guidelines; and enforce liability laws.|
|Nanomachine replicators that can mutate and evolve systematically||Design with multiple redundant checks, standard parts with encrypted genomes and embedded stop codes. Develop active shield “immune system” technology.|
Table 2 summarizes the major security risks that have been identified along with some recommended risk management or amelioration techniques.
|Security Risks||Risk Management|
|Arms race in nanoweapons||Add nanoweapons to international conventions on BCN weapons; International cooperation for inspections; penalties for states trafficking in WMDs of any kind|
|Abuse via terrorism||Monitor, track, and intercept terrorists. Severe legal penalty for carrying nanoweapons designed as WMDs. Reduce economic desperation levels via nanotechnology; increase education levels with WWW-based tutoring systems; introduce licensed components with tracking standards|
|Proliferating nano surveillance and security devices||Promote openness. Reduce incentives for surveillance, and/or use counter controls. Reactive product evolution cycles are almost inevitable.|
|Surprise attack||Develop active shield “immune system” technology.|
|Economic monopoly||International consortium; open source technology infrastructure; product competition; actively distribute products and education|
Given the huge positive potential of MNT, and the risk of an arms race, it would be useful if most of the intellectual activity surrounding nanotechnology were focused on productive economic competition and cooperation. A ban on the use of MNT for weapons would be a step in this direction. Why ban nanoweapons now, before the technology has matured?
The following list of recommendations includes assumptions, principles, and suggestions that could provide a basis for guidelines for the responsible development of MNT.
Arms Control and Disarmament Agency. 1998. Conventions on Weapons of Mass Destruction: nuclear: http://www.acda.gov/treaties/npt1.htm; chemical: http://www.acda.gov/cw.htm; biological: http://www.acda.gov/bw.htm.
Burch, Greg. 1998. “Tiny Torts: A Liability Primer for Nanotechnologists”. http://users.aol.com/gburch3/nanotort.html.
Drexler, Eric K. 1986. Engines of Creation: Challenges and Choices of the Last Technological Revolution. Anchor Press/Doubleday, New York pp. 184-188. See also: http://www.foresight.org/EOC/EOC_Chapter_11.html#section04of05
Drexler, Eric K. 1993. Nanosystems: Molecular Machinery, Manufacturing, and Computation. Wiley, New York.
Drexler, Eric K. 1994. Molecular Manufacturing and Prospects for a Sustainable World. Joint Economic Subcommittee US. Congress.
Drexler, Eric K., Peterson, Chris. With Pergamit, Gayle. 1991. Unbounding the Future: The Nanotechnology Revolution. William Morrow, New York.
Fiksel, Joseph. 1996. Design for Environment: Creating Eco-Efficient Products and Processes. McGraw Hill, New York.
Fischoff, Baruch, et. al. 1981. Acceptable Risk. Cambridge University Press, New York.
Forrest, David. 1989. “Regulating Nanotechnology Development”. Revision 1.1 for MIT course TPP32 on Law, Technology, and Public Policy. MIT, Cambridge. See also: http://www.foresight.org/NanoRev/Forrest1989.html
Jacobstein, Neil. 1995. Nanotechnology Research and Development Sponsorship. In Krumenmenacker, M. and Lewis, J. Prospects in Nanotechnology: Proceedings of the First General Conference on Nanotechnology: Development, Applications, and Opportunities. John Wiley, New York. pp. 207-220.
Jacobstein, Neil. 1997. “Molecular Manufacturing: Current Status and Regulatory Risks”. Address to the Fifth Foresight Conference on Nanotechnology and Molecular Manufacturing. Palo Alto. See also: http://www.foresight.org/Conferences/MNT05/Abstracts/Jacoabst.html
Kantrowitz, Arthur. 1992. “The Weapon of Openness” in BC Crandall and J. Lewis (eds.) Nanotechnology: Research and Perspectives, The MIT Press, Cambridge Mass. pp. 303-311. See also: http://www.foresight.org/Updates/Background4.html
National Institutes of Health, 1976. “Guidelines for Research Involving Recombinant DNA Molecules. Federal Register, 41: 27902-27943. June 23, 1976. See also: http://www.gene.com/ae/AB/IE/NIH_Revised_Guidelines.html.
Senge, Peter, 1997. “Through the Eye of the Needle”, in Rethinking the Future. Rowan Gibson Ed. Nicholas Brealey Publishing, London pp 122-146.
Paper prepared for a Molecular Nanotechnology Policy Workshop sponsored by the Foresight Institute and the Institute for Molecular Manufacturing. ©Neil Jacobstein.