Against Chat

The most prevalent interface of recent fame is undeniably the chat window. Today it feels ubiquitous, to the extent that almost all interactions gravitate toward a single indistinguishable blob. This seems due in large part to an assumption that using LLMs necessities replacing existing interfaces. I want to argue the opposite: if we instead view them as an opportunity to augment existing interfaces, they can lend software a dimension of fluidity that was previously unattainable. Since LLMs rose to prominence, I have wondered how they might allow for ways of displaying information that traditional, rule-based algorithms never managed. As part of my work on Mochi, I had the chance to implement and experience for myself what this kind of fluidity feels like in practice.

The term Gen-AI is by now so entangled with that empty input field that the two are hard to pull apart. When hearing "fluidity" and "computing" in a single sentence, one might be tempted to think of the fever dreams some designers imagine as the infinitely malleable UI of the future. While this might seem counterintuitive at first glance, using LLMs to generate, rather than merely process, works best when we confine them to a narrow domain and interface with strict rules to adhere to. This limits the uncertainty on both ends: the model doesn't have to bend over backwards to interpret our instructions, and what it produces naturally stays within the bounds of what we envisioned.

I'm sure there are countless possible use cases I have not even considered, so I will limit myself for now to the one I do have personal experience with: dictionaries. Like many others, I use spaced repetition to study foreign languages. In practice this is a rather fancy name for (electronic) flashcards: a word on one side and its meaning on the other. Now, language is a delicate thing, and spanning semantic boundaries across languages means bridging differences in how cultures perceive the world. There is rarely a clean bijective mapping between concepts; instead, we end up with superficially equivalent words that are, in fact, essentially different. For any given word, a dictionary offers a fixed definition in our language of choice. But, just as concepts differ between languages, so does each learner's understanding of them, which rests on prior experience and knowledge. No pre-compiled resource can account for the countless questions individual learners might bring with them. Just as we might ask a teacher about the contexts in which a term applies, or how it relates to words we already know, we can now pose these questions to the computer and get an answer that accommodates these needs.

Let's make all this abstract talk more tangible: After two months of doing spaced repetition to study Korean, I ended up with a bit less than 1,000 entries in my database of flashcards. Each of these has a field prosaically named "question", which allows me to pose a question to an LLM, with the contents of the current card as context. The field may be unused, in which case it is simply invisible. I use this special field type on around 40% of all my cards (exactly 432 as of today). These mostly fall into three categories:

1. Disambiguation How is this different from 번호?[1]
2. Contextualization In what context is this commonly used?[2]
3. Collocation With what verbs is this commonly paired?

From a technical perspective, what these have in common is the following: The brevity and closed nature of a single question and answer pair, and the targeted context that the LLM is exposed to. True to the maxim less is more, we gain quite a lot. First, practical concerns like context pollution are non-existent as there, again, is (almost) no context that could get polluted to begin with. Second, such a limited application permits smaller, seemingly less capable models, a win on both economic and ecological grounds. Finally, there is no task being completed on our behalf. The output is not something to skim and verify but something we actively sought and attentively read. The artifact is the textual answer itself. The question about the human in the loop does not arise, because there is no loop in the first place.

Intentionally confining our interactions with LLMs like this, we lose the open-ended nature of longer conversations that the chat window provides, but in return the artifact we're left with is naturally focused precisely on what we need, while avoiding the pitfalls commonly associated with them.[3]