The Literature Machine and Computer Programming
The following is a loose collection of thoughts and questions that deal broadly with how the advent of LLMs and their continuous improvement are poised to influence and shape our industry, or are already doing so. I mostly limit myself to those questions that deal with aspects of the ongoing discourse which I feel are currently under-discussed. This is obviously not to say that the concerns voiced by others regarding negative externalities are in any way less valid.
A few of these questions have at least partial answers, while for most I have not yet arrived at a satisfying conclusion for myself. As such I expect to update this document continuously over the following months.
-
Will computer programming spark less joy ?
Until now, computer programming entailed, besides the obviously necessary knowledge about computer science, detailed knowledge of lots of adjacent technologies and tools. These seem set to lose significance over the coming years or will change dramatically at least.
Ironically, these very tools, first and foremost the command line and the small UNIX utilities that give it such power, are what make LLMs as capable and versatile for developing software as they are. Learning to wield them productively takes years of practice. Plain text is often not the most suitable interface for humans, though it may be the most powerful. For machines though, things are different. Remembering endless command line flags and reading through dozens of man pages might prove a challenge for us humans, but for today's capable models this is a breeze. I for my part enjoy learning these tools and discovering their cultural significance in the process. But for better or worse, this knowledge may soon be a relic (Claude is already way better at grepping things than I am, so it just makes more sense to prompt it to look for certain things or perform operations over files and directories instead of doing this myself).
Naur argued already in the 1980s that the essence of programming lies in theory building, and that the exact methodology applied matters relatively little and is largely secondary. The accelerating development of these artificially intelligent assistants and how they are reshaping our profession seem to prove him correct.
Naur, Peter Programming as Theory Building We are faced anew with the question of what programming actually is, and have yet to find an answer, both as individuals and as an industry. -
Will the lack of manual labor mean the loss of hard-won tacit knowledge?
Polymath Michael Polanyi coined the term
tacit knowledge
in his work Personal Knowledge (1958) and developed it further in his later piece The Tacit Dimension (1966).Polanyi, Michael, The Tacit Dimension He popularized the concept behind it through a rather bold assertion stating "We know more than we can tell". A recurring term coming up in discussions around using LLMs in computer programming is taste, which hints at something similar that is equally difficult to express explicitly. In addition, Polanyi's epistemology ascribes to the body a special function in how we acquire and retain knowledge. When writing (by hand) the material we're about to learn, we are forced to proceed slowly, but in exchange we allow ourselves to attend from subsidiary sensations as an additional means that further facilitates knowledge acquisition.Before her retirement, my mother worked as a teacher and firmly believed in the value of handwriting as a learning tool. She felt that the mechanical act of writing helped students truly understand and retain material, and it was one of her convictions she would not back down from. I suspect the same to hold true when it comes to typing out programs ourselves as opposed to simply instructing an agent using natural language, which, due to its fuzzy nature, is farther detached from the actual artifacts we're about to produce.
For a more elaborate exploration see this rather famous rant by Dijkstra. EWD 667 I fear that giving up most of the manual and laborious parts of our work might mean the eventual loss of this kind of embodied knowledge. -
How can we advance as an industry while ensuring the continuity of our workforce?
While the question could be interpreted as referring to the obvious economic implications of the further automation that this new technology seems predetermined to bring upon us, I want to focus on something different here. These days, development speed seems to be the prime factor occupying the minds of even the most senior members of our trade. While we might churn out code faster than ever, I’m not sure our brains can necessarily keep up with that pace.
The fact that this concern does not seem to be more widely shared among experienced members of our industry is troublesome, to say the least.
See the following paper for one of the rare instances where leading industry figures seem to come to grasps with the reality they are about to create. Russinovich, Mark and Hanselman, Scott, Redefining the Software Engineering Profession for AI On the other hand, moving fast and breaking things has been the ethos of Silicon Valley all along, and LLMs merely accelerate this dynamic, which above all rewards the quick rather than the deliberate. -
What are the costs of frictionlessness?
Adjacent to the previous point, LLMs have tremendous potential to facilitate learning like nothing before. As models are evolving almost faster than we can keep track of, it is becoming evident that, especially when working on problems of limited depth, their rate of failure is surprisingly low. Consequently, private tutors for all - historically a luxury reserved for the children of society’s most prosperous - are already a reality. When used responsibly, they undoubtedly have the ability to level the educational playing field even further than the internet already did.
However, as others have observed: Learning entails struggle. I question whether real learning is possible if all that's left for us as humans is delegation, while the actual implementation is automated away. Sporadically reviewing the product of that probabilistic slot machine, or going even further, removing oneself from the loop entirely, erodes the attention necessary to reflect on problems and reach conclusions on our own.
See for instance the following study conducted by Anthropic whose results indicate similar concerns. Shen, Judy Hanwen and Tamkin, Alex, How AI assistance impacts the formation of coding skills Refreshing social media feeds has a similar effect: We no longer engage with content intellectually, but consume it compulsively and without satisfaction. Their addictive nature is, of course, nothing new, but to see this same dynamic now at play in the context of intellectual work is. Issues posted during AI service outages reveal how this behavior already has taken hold today. -
Will this contribute to the existing social alienation by between us?
Superficially broadening the knowledge of individuals might obviate the necessity of collaboration between them. Paraphrasing from some private correspondence: We may be trading the messy, generative friction of human collaboration for the frictionless isolation of individual sufficiency.
Conclusion
If I had to summarize these questions and concerns, I would be inclined to say the following: LLMs in their current form are incredibly powerful tools and provide undeniable value to us as practitioners. But it is not clear yet whether we are able to benefit from their usage in the long term as well. Yet these tools do not exist in a vacuum. They are shaped by the incentives of the environment that produced them, and the interfaces through which we are meant to engage with them are an extension of those same incentives. Which is to say: the manner in which this technology is presented to us can quietly erode the very benefits it promises.
If one thing seems certain, it is that our work as software engineers will change drastically over the coming months and years. Whether that change will diminish or enrich us as humans is still to be determined.