Context, Communication, the Keys to LLM Productivity

If you haven't read this article published by Antirez yet, it's a great perspective:

LLMs and Programming in the first days of 2024

I wouldn't say I'm at or even near the level of developer as the author, but the big picture they outline here aligns with a lot of the thoughts I've had on working with AI as a developer, and lead me to some new insights as well.

If you're a developer, or, frankly work in any sort of knowledge work, you have probably spent a decent amount of time thinking about how AI will affect your career. There's clearly more alarmist or cool headed ways to react to this paradigm shift, especially as it unfolds before our eyes, but I do think there are some really bright spots here.

Especially if you are already a good communicator.

Backing up a tiny bit . . . the author of the article lays out some specific use case examples from a developer perspective to illustrate a handful of LLMs' strengths and weaknesses. A general conclusion is that it's an amazing tool, but it is only as good as the user's ability to employ it correctly.

"I have also learned that LLMs are a bit like Wikipedia and all the video courses scattered on YouTube: they help those with the will, ability, and discipline, but they are of marginal benefit to those who have fallen behind. I fear that at least initially, they will only benefit those who already have an advantage."

If an LLM is essentially an 'erudite fool', then it leaves a whole lot of creative capacity still in the court of the human counterpart. As developers it's crucial to formulate our queries clearly, to try to provide an accurate picture and concrete examples that pertain to our applied use case, and to push back on the answers an LLM gives back, which we can reasonably assume are always going to be _best guesses_, _stock knowledge_ or straight up _bullshit_.

This is great for cutting down the time it takes to dig up basic operations or compose nuts and bolts functions or essentially peruse virtually any documentation directly towards your current problem space.

I've spent many hours spinning up AutoGPT and similar suites that promise to take inputs in and spit out complete programs, and so far I haven't been able to coax anything useful out of it. Some scaffolding, maybe, but nothing anywhere close to usable before heavy time commitment trying to comprehend code summoned out of the ether.

But as a coding companion and gateway to information, it is incredibly empowering. In the past year I have tackled more projects than perhaps any year prior, because all of the knowledge and resources that I usually have to painstakingly scrounge up from the internet hellscape are brought considerably within reach. Projects I back-burnered or abandoned or just didn't even bother with and now excitingly back on the menu.

Or as the author states:

"Would I have been able to do it without ChatGPT? Certainly yes, but the most interesting thing is not the fact that it would have taken me longer: the truth is that I wouldn't even have tried, because it wouldn't have been worth it. This fact is crucial. The ratio between the effort and the benefit of writing such a program, secondary to my project, would have been inconvenient."

Even with the sorts of small little utility or 'nice-to-have' ideas that bubble up in the context of any other larger project . . .

"Writing this type of program would be, in my opinion, an imprudent use of my time. The fact that an LLM can do it for me certainly speeds up my ability to produce code in a rather direct way: I can focus on the things that matter."

Obviously this is a broad and ever evolving topic, and this isn't much of a focused post TBH, just an attempt to capture some of the thoughts I have. Perhaps I'll get into things with more specificity in the future, but for now I'll leave it be and wrap up with a few main thoughts:

AI and LLMs won't end software development as a career, anytime soon. For now, I think it's a net enabler to build bigger, better things. But it demands higher level thinking and the ability to communicate needs and end goals to build the next generation of software.

Soon, anyone with an idea will have all the tools they need to summon all the knowledge they need to build virtually anything. Experienced developers are starting with an advantage, because they already have the language to communicate their context and goals to an AI and get better results. That curve will eventually flatten, and we will all be required to be more like designers and product thinkers than engineers (but it'll still help to have engineering chops)