As we continue developing our software, we accumulate a growing amount of technical debt just to keep the system running. But I believe we are on the brink of an even larger issue. Cognitive debt.
Hope you enjoy this reading, all feedback is welcome.
enjoy this reading
It’s not a reading: we have to read it ourselves.
I’ll be honest, I had to look on the dictionary to understand this subtle meaning.
Thanks, I learned a new thing today! :D
This is the cognitive debt of the past. We already had it. We just didn’t have the name for it.
We always called it “domain knowledge” or “tribal knowledge”.
But in the days of old, the replacement of human, or generation of domain/tribal knowledge were in the range of human capability to learn the new parts of a system.
With the LLM of today this scale is far flipped on the wrong side, and no product fully LLM will be manageable in a few years.
“oral history”.
“The ropes”
It’s got many names. Why did this guy not know any of them? Why was “I don’t know the name” suddenly “there is no name”?
I enjoyed reading this. I don’t have anything more constructive to say (I’m only leaving this comment because I realised that you are the person who actually wrote the linked post, and I’ve found that leaving an appreciative comment is like a super-upvote).
I like this piece because it’s another voice saying effectively the same thing as many others I respect have argued: that pressure to ship leads to poor quality, difficult to maintain code (even before AI), and that AI is just making the problem worse. It’s cool to see the conversation evolve. For example, after I first saw the term “cognitive debt” used, I saw it crop up in more and more places, as other writers seemed to conclude “this problem is similar to technical debt, but it’s distinct enough that using a different but related term is a good idea”. And even if there are multiple people arguing effectively the same thing, each person tends to tackle the problem slightly differently
Hey there! Thanks for the comment, and reading it.
I know that sometimes hearing the same voice speaking on the same topic can be problematic. But on the other side of the same coin, have fewer voices pointing to the problem is also a problem. For me, writing the blog is a way to organize my own mind and be able to create a reasoning on some topic.
And sometimes, repeating the echoed words that the industry is repeating.
Either way, thanks again for reading, and even more for the comment.
I’m going to break this up into two. Feedback on content and feedback on style and presentation.
Content
LLMs didn’t create cognitive debt, but they do make it worse. Cognitive debt is paid every time a team member leaves or is promoted to a non-contributing role.
The interesting thing to me is that, in order for an LLM project to be successful, all of that cognition has to be done up front, and written down. If done well, you aren’t relying on the LLM to figure anything out, and it’s just rote execution by a simpler model. The build process then exposes holes in the initial design which must be plugged within the documentation until all of the cognition has been performed and the LLM can execute.
The beauty of that model is that you can be 2/3 of the way through a project, realize you missed something foundational and rearchitect and rebuild the whole project in hours instead of weeks. LLM saves no time or money if you have perfect planning and execution skills, but it does enable you to make significant pivots far faster than a human team.
One of my teams just finished a project that took two massive pivots just as design was finished and execution was underway. The result was all the time explaining v1 was wasted and left confusion for the developers, v2 was continuing to be developed even after v3 was provided, and months of time were wasted understanding and building the wrong thing. This was a six month effort when it could’ve been two.
Now, with a real developer, they eventually build their own cognition and their own mental model and are then capable of cognitive work — LLMs can assist documenting the cognitive work, but they are piss poor at performing it. LLMs can’t cognate and they can’t replace developers — but they can change the shape of development work to be more about requirements analysts and activities, and if used well can improve efficiency of teams.
I could write a whole article on that — I could write a book — but I don’t have time and AI would completely fuck it up, so it’s just in my head, still taking shape.
Presentation
I might have more specific feedback on your points, but I’d forgotten them by the time I got to the end.
I’d suggest moving your analysis of the examples to their own post which you can then link to support your points. The concept you are trying to convey isn’t difficult, but this article is about 10x the length it needs to be. You basically write well enough, and you attempt to engage the reader well, but this article is like having a chat with someone and they just launch into an hour long impromptu lecture and giving you no chance to absorb or respond.
By breaking it up into smaller chunks, you can give a reader an opportunity to engage with an idea fully, and then they can decide if they want to follow the links to read your takes on this project or the, or simply accept a one sentence reference in the primary article at face value.
There is writing out there so enchanting that you don’t mind investing the time because the reading is as much entertainment as it is learning. This article aspires to be that, but isn’t there.
Wooow, this message is as long as my post, love it!
The interesting thing to me is that, in order for an LLM project to be successful, […]
This is where the open spec tries to help/solve. But this is a bandaid. Without a lot of hand-holding the models will make shity decisions/code. And this is cumulative, the more you try to steer the wheel, the worse it gets. I can specify details, but a 3 years old project is in this shape, and no amount of LLM will solve/help.
The beauty of that model is that you can be 2/3 of the way through a project, […]
I understand this point, but it’s flaky at best. To be honest, with a large enough project a decision will take longer, if not impossible in a LLM driven codebase. It will be years to debugging, breaking down and rebuilding to even get close. Or spend the paycheck of 5 engineers annually for a single migration.
And this considers the the current LLM are heavily subsided.
Now, with a real developer, they eventually build their own cognition and their own mental […]
Wrong, from the experiments on the academia, we get the fact that engineers with AI are often slower, because the problem was never in the code, code is the tool, it was the acquiring the right data, model and get the knowledge of the product to make the changes needed.
Presentatio […]
I try to keep around 1.5k to 2k words a week. It’s not always the case, some weeks I’m somewhat more inspired, sometimes I need some refinement, but this is a personal view, I want the reader to have a “conversation” with the author. This is the type of metaphor I try to use.
Either way, thanks for reading.
Ps.: Do you always use LLMs to write your messages? I would love to read your own words. Don’t be the meat proxy for the LLM.