But why do people deliberately waste time when there are much more efficient ways of working? This is a very good question. In fact, it is such a good question that I decided to ask a good friend of mine, Lisa Feldman Barrett, who is professor of psychology at one of the top universities on the east coast of the USA. What she told me was no doubt a simplification so that a layman like myself could understand it. Despite such simplifications, her explanation both gave me a much better understanding of the phenomenon and some ideas about how to compensate for it.
Apparently, this phenomenon was investigated by the well-known psychologist Carol Dweck (who used somewhat different terminology).
My friend told me that (with respect to this phenomenon) people can be roughly divided into two categories that she called perfection-oriented and performance-oriented.
The people in the category perfection-oriented have a natural intellectual curiosity. They are constantly searching for better ways of doing things, new methods, new tools. They search for perfection, but they take pleasure in the search itself, knowing perfectly well that perfection can not be accomplished. To the people in this category, failure is a normal part of the strive for perfection. In fact, failure gives a deeper understanding of why a particular path was unsuccessful, making it possible to avoid similar paths in the future.
The people in the category performance-oriented on the contrary, do not at all strive for perfection. Instead they have a need to achieve performance immediately. Such performance leaves no time for intellectual curiosity. Instead, techniques already known to them must be applied to solve problems. To these people, failure is a disaster whose sole feature is to harm instant performance. Similarly, learning represents the possibility of failure and must thus be avoided if possible. To the people in this category, knowledge in other people also represents a threat. As long as everybody around them use tools, techniques, and methods that they themselves know, they can count on outperforming these other people. But when the people around them start learning different, perhaps better, ways, they must defend themselves. Other people having other knowledge might require learning to keep up with performance, and learning, as we pointed out, increases the risk of failure. One possibility for these people is to discredit other people's knowledge. If done well, it would eliminate the need for the extra effort to learn, which would fit very well with their objectives.
This model of learning also explains other surprising behavior that I frequently observe. I have seen novices in software development with knowledge of a single programming language explain to experienced expert developers why their choice of programming language was a particularly bad one. In one case, I talked to a student of computer science who told me why a particular programming language was bad. In fact he told me it was so bad that he had moved to a different university in order to avoid courses that used that particular language. When asked, he admitted he had never written a single program in that language. He simply did not know what he was talking about. And he was willing to fight for it. With respect to programming languages, negative opinions about a language that a person does not know, are usually based on very superficial aspects of it. To people obsessed with performance lack of such in a programming language is a favorite reason to advocate its eradication (even though performance is not a quality of a language, but of a particular implementation).
As the reader has probably already guessed, my surprising observations concern mostly performance-oriented people. The above discussion is obviously a simplification. In particular, a person can be in one category with respect to a particular domain, and in the other with respect to another domain. Thus, I have seen professors in mathematics who were obviously perfection-oriented with respect to mathematics, be firmly in the performance-oriented category with respect to the efficient use of (say) word processors. It is almost a surrealistic experience to see a person in one situation full of intellectual curiosity and wanting to know everything about everything, and in another situation argue why you should not use a particular method that he himself does not know anything about, for reasons that are obviously totally artificial.
Thus, what I have observed is not only what one might expect, i.e., some reluctance to learning new tools and methods, but a kind of reaction orders of magnitude stronger than I had expected. I have observed that people ignorant in a particular domain, or not knowing a particular tool or technique, would go to great trouble to explain why knowing this domain, tool, or technique, would be a complete waste of time. Usually these explanations were based on erroneous ideas of what it represented. To make things worse, they were perfectly willing to present their erroneous arguments to the very experts in the field in question.
Similarly, I have heard people argue against a tool that they ignore based on the fact that it can do too much. Too much functionality in a tools is a problem only if unneeded or unwanted functionality somehow makes it harder to use the needed and wanted parts. I have heard people argue about the amount of memory a particular tool requires, whereas the additional memory required might represent a cost equivalent to a few hours of work at most. A favorite idea is to label a particular tool with a name suggesting what it ought to be doing, and then arguing that it is doing more than that. For instance, a text editor that is capable of automatic indentation would be accused of being a ``kitchen-sink'' tool because after all it does much more than allowing the user to just edit text.
Needless to say, these people make complete fools of themselves. But that does not seem to bother them in any way whatsoever.
It is hard to overestimate the strength of this phenomenon. I myself recently discovered a marvelous feature in a programming language that I had purposely avoided for the past 10 years, simply because 10 years ago, a colleague (who did not know the feature) explained to me that it was no good. We were both victims of our own minds. My colleague because he obviously needed to defend that he had made a different choice, and myself because I subconsciously found it very appealing to be able to brush off the feature as useless and thus not having to learn it. It is hard to overestimate the wasted time I have put in during the past 10 years due to considerably lower productivity than I could have had, had I realized at the time what I now know about human psychology.
It is my hope that this explanation of a common phenomenon makes it possible for students to reflect upon their own motivations, and that it ultimately makes all students perfection-oriented.