Clever Hans Effect

Clever Hans (; ) was a horse that appeared to perform arithmetic and other intellectual tasks during exhibitions in Germany in the early 20th century.

In 1907, psychologist Oskar Pfungst demonstrated that the horse was not actually performing these mental tasks, but was watching the reactions of his trainer. The horse was responding directly to involuntary cues in the body language of the human trainer, who was entirely unaware that he was providing such cues. In honour of Pfungst's study, this type of artifact in research methodology has since been referred to as the ''Clever Hans effect'' and has continued to be important to the observer-expectancy effect and later studies in animal cognition.
Pfungst was an assistant to German philosopher and psychologist Carl Stumpf, who incorporated the experience with Hans into his further work on animal psychology and his ideas on phenomenology.

Spectacle

During the early twentieth century, the public was especially interested in animal intelligence owing in large part to Charles Darwin's recent publications. The case of Clever Hans was taken to show an advanced level of number sense in an animal.

Hans was a horse owned by Wilhelm von Osten, who was a gymnasium mathematics teacher, an amateur horse trainer and phrenologist and was considered to be a mystic. Hans was said to have been taught to add, subtract, multiply, divide, work with fractions, tell the time, keep track of the calendar, differentiate between musical tones, and read, spell, and understand German. Von Osten would ask Hans, "If the eighth day of the month comes on a Tuesday, what is the date of the following Friday?" Hans would answer by tapping his hoof eleven times. Questions could be asked both orally and in written form. Von Osten exhibited Hans throughout Germany and never charged admission. Hans' abilities were reported in ''The New York Times'' in 1904.

After von Osten died in 1909, Hans was acquired by several owners. He was then drafted into World War I as a military horse and "killed in action in 1916 or was consumed by hungry soldiers".

Investigation

The great public interest in Clever Hans led the German board of education to appoint a commission to investigate von Osten's scientific claims. Philosopher and psychologist Carl Stumpf formed a panel of 13 people, known as the ''Hans Commission''. This commission consisted of a veterinarian, a circus manager, a cavalry officer, a number of schoolteachers, and the director of the Berlin zoological gardens. This commission concluded in September 1904 that no tricks were involved in Hans's performance.

The commission passed off the evaluation to Oskar Pfungst, who tested the basis for these claimed abilities by:
# Isolating horse and questioner from spectators, so no cues could come from them
# Using questioners other than the horse's master
# By means of blinders, varying whether the horse could see the questioner
# Varying whether the questioner knew the answer to the question in advance.

Using a substantial number of trials, Pfungst found that the horse could get the correct answer even if von Osten himself did not ask the questions, ruling out the possibility of fraud. However, the horse gave the right answer only when the questioner knew what the answer was and the horse could see the questioner. He observed that when Hans could see the questioner, the horse got 89 percent (50 out of 56) of the answers correct, but when Hans was not able to see the questioner, the horse answered only six percent (2 out of 35) of the questions correctly.

Pfungst was aware of the ability of circus trainers to train horses to respond to small gestures, and was aware of a number of cases of dogs, like that of English astrophysicist Sir William Huggins, who were able to point to an object their master was looking at or who were able to "bark" the answer to questions like square roots while staring at their master's face, and so after refuting his initial suspicion of a fraud involving whispering or the like, began to consider accidental communication with Hans.Oskar Pfungst, ''Clever Hans: (The Horse Of Mr. Von Osten)'', Pfungst 191
pp.177
185 Pfungst then examined the behaviour of the questioner in detail, and showed that as the horse's taps approached the right answer, the questioner's posture and facial expression changed in ways that were consistent with an increase in tension, which was released when the horse made the final, correct tap. This provided a cue that the horse could use to stop tapping. The social communication systems of horses may depend on the detection of small postural changes, and this would explain why Hans so easily picked up on the cues given by von Osten, even if these cues were subconscious.

Pfungst carried out laboratory tests with human subjects, in which he played the part of the horse. Pfungst asked subjects to stand on his right and think "with a high degree of concentration" about a particular number, or a simple mathematical problem. Pfungst would then tap out the answer with his right hand. He frequently observed "a sudden slight upward jerk of the head" when reaching the final tap, and noted that this corresponded to the subject resuming the position they had adopted before thinking of the question.

Some critics of Pfungst suggest that a proper way of validating the body language hypothesis should involve Pfungst acting out the same body language cues himself in order to reliably get Hans to give wrong answers. This would have validated the hypothesis that Hans is simply tapping in correspondence with certain body language cues, independently of the true answer; however, Pfungst never was able to successfully validate his hypothesis in this manner.

After the investigations by Pfungst were done, von Osten, who was never persuaded by Pfungst's findings, continued to show Hans around Germany, attracting large and enthusiastic crowds.

The Clever Hans effect

After Pfungst had become adept at giving Hans performances himself, and believed he was fully aware of the subtle cues which made them possible, he discovered that he would produce these cues involuntarily regardless of whether he wished to exhibit or suppress them. Recognition of this phenomenon has had a large effect on experimental design and methodology for all experiments whatsoever involving sentient subjects, including humans.

The risk of Clever Hans effects is one reason why comparative psychologists normally test animals in isolated apparatus, without interaction with them. However this creates problems of its own, because many of the most interesting phenomena in animal cognition are only likely to be demonstrated in a social context, and in order to train and demonstrate them, it is necessary to build up a social relationship between trainer and animal. This point of view has been strongly argued by Irene Pepperberg in relation to her studies of parrots (Alex), and by Allen and Beatrix Gardner in their study of the chimpanzee Washoe. If the results of such studies are to gain universal acceptance, it is necessary to find some way of testing the animals' achievements which eliminates the risk of Clever Hans effects. However, simply removing the trainer from the scene may not be an appropriate strategy, because where the social relationship between trainer and subject is strong, the removal of the trainer may produce emotional responses preventing the subject from performing. It is therefore necessary to devise procedures where none of those present knows what the animal's likely response may be.

The Clever Hans effect has also been observed in drug-sniffing dogs. A study at University of California, Davis revealed that cues can be telegraphed by the handler to the dogs, resulting in false positives.

A 2004 study of Rico, a border collie reported by his owners as having a vocabulary of over 200 words, avoided the Clever Hans effect by having the owner ask the dog to fetch items from an adjacent room, so that the owner could not provide real time feedback while the dog was selecting an object.

A study conducted in 2012 examined the socio-communicative ability of dogs with humans, looking at how much of an influence an owner, if present, would have on their dog during an object-choice pointing task, including looking at whether a Clever Hans effect was present. The study concluded that when the experimenter provided a pointing gesture regardless of the owner’s knowledge, and that when no pointing cue was given to the dog, it only performed at chance level. This study proved that, for this pointing task, there was no Clever Hans effect affecting the dogs' performance, as long as the owners did not actively influence them.Schmidjell, T., Range, F., Huber, L., & Virányi, Z. (2012). "Do owners have a Clever Hans effect on dogs? Results of a pointing study". ''Frontiers in Psychology'', 3.

Similarly, a study done in 2013 also examined whether a Clever Hans effect was present in a two-way object choice test and included an experimental group in which the owners actively tried to influence their dog's decision. The results showed that the experimenter had the strongest effect on the dog's choice in this task, regardless of the owner's knowledge or actions. This provides evidence that the Clever Hans effect is not always present when humans and dogs interact.Pongrácz, P., Miklósi, Á., Hegedüs, D., & Bálint, A. (2013). "Owners fail to influence the choices of dogs in A Two-choice, Visual pointing task". ''Behaviour'', 150(3–4), 427–443.

Pfungst's final experiment showed that Clever Hans effects can occur in experiments with humans as well as with animals. For this reason, care is often taken in fields such as perception, cognitive psychology, and social psychology to make experiments double-blind, meaning that neither the experimenter nor the subject knows what condition the subject is in, and thus what their responses are predicted to be. Another way in which Clever Hans effects are avoided is by replacing the experimenter with a computer, which can deliver standardized instructions and record responses without giving clues.

Clever Hans effect in artificial intelligence

In the field of artificial intelligence, the Clever Hans effect describes a phenomenon where an algorithm seems to make correct predictions without having the relevant data and/or by using incorrect reasoning. This effect can appear, when a deep neural networks's predictions are dominated by unknown features, which were not intended for the task, e.g. when an image classifier "reads" text on a photo instead of analysing the image motif.

The Clever Hans effect can also be seen as a "secret" overfitting of deep neural networks towards an unknown feature. This overfitting might not affect the algorithm at all as long as the unknown feature is present, while it could lead to unpredictable fails in the absence of the unknown feature. The existence of this effect and its practical imperceceptibility restrict the robustness and validity of most AI models. Explainable AI research tries to find methods to fully comprehend the decision making of AI models in order to detect such Clever Hans correlations.
The German conceptual artist Max Haarich demonstrated the Clever Hans effect in artificial intelligence with his art installation “Smart Hans”. Based on the information from Oskar Pfungst's book ''Clever Hans'' (1911), the installation shows an animated horse and uses facial detection and pose estimation to predict which number a person is currently thinking about.

See also

* List of historical horses
* Harass II, a dog used in criminal investigations
* Ideomotor phenomenon
* Lady Wonder, a horse with purported telepathic abilities.
* Beautiful Jim Key
* Muhamed (horse)
* Equine intelligence

References

Sources

* Hothersall, David. History of Psychology. McGraw-Hill. 2004.
* Pfungst, O. (1911)
''Clever Hans (The horse of Mr. von Osten): A contribution to experimental animal and human psychology''
(Trans. C.L. Rahn). New York: Henry Holt. (Originally published in German, 1907).
*

External links

Clever Hans the Math Horse
* {{librivox book , title=Clever Hans , author=Pfungst

Trick horses
Animal intelligence
1895 animal births
1916 animal deaths
Individual animals in Germany
Military animals of World War I