There is always a market for speculation about how technology will change life and work. But information technologies shape how we interact socially. As McLuhan (1964) put it: “it is the medium that shapes and controls the scale and form of human association and action.” Whether it is the printing press or electronic media, the medium matters because it shapes how people share and acquire information. It even imposed limitations on where and at what scale. For example, the “google effect” is the phenomena whereby the internet has changed the way we retain information. It has lowered the rate at which we can recall information directly, such that people shift their efforts to remembering where to find it (Sparrow et al 2011). Just like google outsourced some memorization, Generative AI will shape how people acquire information.
Tools like ChatGPT change the nature of knowledge work by speeding up initial research, providing general information, and creating outlines or frameworks for creation. The extent to which these tools are helpful will be limited by the ability of those applying them to provide specific local knowledge to the model in applications. Preferably it is applied where it costs very little to make an error or where it is easy to identify errors. People are the principal agents providing local knowledge either through training or prompting. This tool speeds up the collection process of well-established knowledge by lowering the cost of initial research, but if poorly prompted or too specific the knowledge becomes sophomoric.
In my own experience using ChatGPT, it is very helpful at curating a base of general knowledge. It is a probabilistic large language model. This is what I expect. For example, I was considering writing about education the other day. I asked ChatGPT for a summary of the intellectual history of the philosophy of education, and I followed up asking for clarification on specific authors. After a few minutes I had a lengthy reading list. I have also used it extensively when coding. I can ask to create different object classes for a game, going through after the fact to add specific methods to these objects. But the general framework for the code is there. When writing I can ask for feedback on my essay, prompting it to look for specific types of errors or weaknesses in organization. In all of these applications I do not have to agree with the resulting advice from the black box. I see it as having a discussion with a soulless equation. The primary pitfall in my own experience using Generative AI to help with researching, writing, or coding, has been when I try to use it as a substitute for critical thinking.
The late Nobel Laureate Daniel Kahneman’s framework in his 2013 book, Thinking Fast and Slow, is helpful for understanding how to interact with this tool. In brief, he describes two modes of thinking: System 1 and System 2. System 1 thinking is a passive experience, intuitive and often unconscious. It is fast and efficient. It is based on available knowledge, past experience, and long-established mental models. Tasks that become second nature fall into this category. The other system, System 2 thinking or slow thinking, is analytical, deliberate, and conscious reasoning. It feels inefficient and tiresome. This is the part of the mind that relies on qualitative and quantitative methods to reach conclusions.
To explain this further, consider how anyone learns to drive or to read. How do we make decisions while driving? How do we comprehend a word that we have read 100,000 times? In the vast majority of cases, we do these things unconsciously. System 1 perceives the road and the marks on the page and prompts the conscious mind with the post-processed data. However, at one point both driving and reading were conscious experiences. When I was first learning to read I had to sound out every word on the page. When I was first learning to drive, I was fixated on keeping the car between the lines and at the right speed.
I am not making any claims as to how the brain works chemically, rather I am seeking to provide a framework for understanding how we might effectively interact with Generative AI. While I might want to outsource critical thinking to a probabilistic model, that would be a mistake. Any ongoing project I have undertaken breaks down when I start asking ChatGPT to do my thinking for me.
I have heard it said before that, “to write is to think.” Prompting is writing. ChatGPT has helped me clarify so many questions and issues succinctly, but indirectly. There have been numerous occasions when I started writing a prompt and through the development and clarification process, I understood the problem well enough myself to be inspired to take it on from a new tack. Generative AI is a useful tool for augmenting System 2 thinking when its limitations are kept in mind.
References
McLuhan, Marshall. 1964. “Chapter 1: The Medium Is the Message.” In The Extensions of Man.. https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf.
Kahneman, Daniel. 2013. Thinking, Fast and Slow. First Edition. New York: Farrar, Straus and Giroux.