"Do they not? Mr. Harriman, you have on your desk a computer input which, I understand, is connected with the organization's Multivac. A computer is a kind of sessile robot; a robot brain not attached to a body-"
"True, but that also is limited. The computers used by mankind have been steadily specialized in order to avoid too humanlike an intelligence. A century ago we were well on the way to artificial intelligence of the most unlimited type through the use of great computers we called Machines. Those Machines limited their action of their own accord. Once they had solved the ecological problems that had threatened human society, they phased themselves out. Their own continued existence would, they reasoned, have placed them in the role of a crutch to mankind and, since they felt this would harm human beings, they condemned themselves by the First Law."
"And were they not correct to do so?"
"In my opinion, no. By their action, they reinforced mankind's Frankenstein complex; its gut fears that any artificial man they created would turn upon its creator. Men fear that robots may replace human beings."
"Do you not fear that yourself?"
"I know better. As long as the Three Laws of Robotics exist, they cannot. They can serve as partners of mankind; they can share in the great struggle to understand and wisely direct the laws of nature so that together they can do more than mankind can possibly do alone; but always in such a way that robots serve human beings."
"But if the Three Laws have shown themselves, over the course of two centuries, to keep robots within bounds, what is the source of the distrust of human beings for robots?"
"Well"-and Harriman's graying hair tufted as he scratched his head vigorously-"mostly superstition, of course. Unfortunately, there are also some complexities involved that anti-robot agitators seize upon."
"Involving the Three Laws?"
"Yes. The Second Law in particular. There's no problem in the Third Law, you see. It is universal. Robots must always sacrifice themselves for human beings, any human beings."
"Of course," said George Ten.
"The First Law is perhaps less satisfactory, since it is always possible to imagine a condition in which a robot must perform either Action A or Action B, the two being mutually exclusive, and where either action results in harm to human beings. The robot must therefore quickly select which action results in the least harm. To work out the positronic paths of the robot brain in such a way as to make that selection possible is not easy. If Action A results in harm to a talented young artist and B results in equivalent harm to five elderly people of no particular worth, which action should be chosen."
"Action A, " said George Ten. "Harm to one is less than harm to five."
"Yes, so robots have always been designed to decide. To expect robots to make judgments of fine points such as talent, intelligence, the general usefulness to society, has always seemed impractical. That would delay decision to the point where the robot is effectively immobilized. So we go by numbers. Fortunately, we might expect crises in which robots must make such decisions to be few...But then that brings us to the Second Law."
"The Law of Obedience."
"Yes. The necessity of obedience is constant. A robot may exist for twenty years without every having to act quickly to prevent harm to a human being, or find itself faced with the necessity of risking its own destruction. In all that time, however, it will be constantly obeying orders...Whose orders?"
"Those of a human being."
"Any human being? How do you judge a human being so as to know whether to obey or not? What is man, that thou art mindful of him, George?"
George hesitated at that.
Harriman said hurriedly, "A Biblical quotation. That doesn't matter. I mean, must a robot follow the orders of a child; or of an idiot; or of a criminal; or of a perfectly decent intelligent man who happens to be inexpert and therefore ignorant of the undesirable consequences of his order? And if two human beings give a robot conflicting orders, which does the robot follow?"
"In two hundred years," said George Ten, "have not these problems arisen and been solved?"
"No," said Harriman, shaking his head violently. "We have been hampered by the very fact that our robots have been used only in specialized environments out in space, where the men who dealt with them were experts in their field. There were no children, no idiots, no criminals, no well-meaning ignoramuses present. Even so, there were occasions when damage was done by foolish or merely unthinking orders. Such damage in specialized and limited environments could be contained. On Earth, however, robots must have judgment. So those against robots maintain, and, damn it, they are right."
"Then you must insert the capacity for judgment into the positronic brain."
"Exactly. We have begun to reproduce JG models in which the robot can weigh every human being with regard to sex, age, social and professional position, intelligence, maturity, social responsibility and so on."
"How would that affect the Three Laws?"
"The Third Law not at all. Even the most valuable robot must destroy himself for the sake of the most useless human being. That cannot be tampered with. The First Law is affected only where alternative actions will all do harm. The quality of the human beings involved as well as the quantity must be considered, provided there is time for such judgment and the basis for it, which will not be often. The Second Law will be most deeply modified, since every potential obedience must involve judgment. The robot will be slower to obey, except where the First Law is also involved, but it will obey more rationally."