In theory, there is no difference between theory and practice. But as anyone who’s ever tried to apply lessons from books/courses/posts about Best Practices to their work can tell you, the difference is real and it is vast.

I’ve featured Jen and Kyle’s Complexity Lounge on here before, but last month I finally got a chance to appear on the show myself. We talk about some of the things that make reality so…well, complex, compared to the neat simplicity of theory books.

As with many of my rants, the conversation eventually comes to the topic of systems thinking: being able to understand not only the thing you are working on, but the forces around that thing that frustrate your efforts to change it. Last week, I wrote about one way this can manifest: higher output velocity never brings you closer to valuable outcomes without your org being able to deliberately form intent and hold on to it.

The good and bad news is that solutions to systemic problems don’t live inside our tools. It’s good news because AI is never going to be able to take this part of the work away from us. It’s bad news because it means we have to leave our comfort zones (exit Figma, close vim, log out of Jira) and go talk to people about complex things that don’t fit into a Slack message.

This method works really well for solving seemingly intractable problems, for example engineers saying “that’s not possible” (which often simply means “I don’t know”):

However, since this is the “not so fast!” issue of the Picnic, we must immediately admit that “just talk to people” is not a magic cure-all. Any product manager will tell you that they became disenchanted with UX when “we talked to users, but the product was still bad.” Similarly, the way in which professionals (including designers!) talk to their colleagues often falls prey to the curse of knowledge, where we cannot imagine what it is like to not know all the stuff we know. Conversations with cross-functional partners are not a delivery system for our brilliant insights.

Erika Hall has some good advice for how we can avoid that curse. One line especially stuck with me: “seek feedback, not approval.” While Erika uses it in the specific context of self-improvement, it has many facets and all of them are true. After all, feedback is the core of feedback loops, which are the things that the entire product development lifecycle runs on.

Unfortunately…

Broadly speaking, any feedback loop has two stages: theory formation and experimentation. We come up with a thing to try, and then we try it. Design and development, if you want to be overly prescriptive. Each stage repeats until its output is fit for purpose; we continue to form theories until one is worth trying, and then we keep testing it until we learn what we wanted to learn.

Both of those loops get absolutely wrecked as collaborators retreat further into LLM-driven tools.

Let’s start with the first part: theory. The way we typically decide a theory is worth testing is through critique (which I talk about in an older issue). This was already something orgs struggled with: most people’s idea of a good critique was a brutally harsh critique, maximizing nitpicking and pushback for its own sake. Figuring out which feedback mattered has always been a challenge that collaborators struggled to meet.

Now let’s put our heads together and see if we can’t think of a technology that is really good at generating “feedback” up to an arbitrary word count, and then emailing it to the poor designer who now has to implement it.

An org that sees critique as something you can win by picking the most nits will never be able to adequately finalize a theory. It will keep getting sidetracked by low-priority minutia that doesn’t actually bother anyone, but costs far less to identify than to fix. And that’s assuming that the feedback is even correct, which it’s not likely to be. The “best practices” baked into LLM training data are not actually true: the 7±2 rule is bunk, the “less clicks is better” rule is bunk, the “laws” of hierarchy depth are bunk. And yet you can count on any or all of these appearing in feedback from a robot (or a human who trained themselves to think like a robot).

Some orgs have opted to downplay the necessity of theory formation entirely, appealing instead to velocity as the solution to all their problems: we don’t need to prioritize ideas if we can get them all out the door and see how they fare.

Just like bad critique, this is not an AI phenomenon (which is a theme that repeats over and over: most problems with AI are preexisting problems that have been intensified by thoughtlessly scaling the systems where they appear). This was a problem with Agile for as long as Agile has existed: the assumption that teams could ship small and then iterate never bore out in practice. Calling out flaws during theory formation is cheap & therefore encouraged; calling out flaws with the finished product leads to expensive changes, and so it is systemically discouraged.

This creates an effect known as learned helplessness. Employees become conditioned to keep their mouths shut. Everyone knows that they are expected to sit down and ship — even if they can plainly see that what’s being delivered is steering the product away from the company’s stated goals.

Now AI is here to intensify this effect. Agents are eye-wateringly expensive and only getting pricier as frontier model providers aim for IPOs. Our bosses expect the return on that investment to be similarly sky-high. They don’t want to hear any doubts — and rank-and-file employees are scared to speak up. One is expected to produce good news, no matter what they actually see:

But reality can’t be held at bay forever. There is always a feedback loop. And if you can’t run a fast feedback loop, you will run the slow one: increasing your costs with every line of code you ship until the company is no longer able to function at all.

That feedback loop is less OODA, and more FAFO.

— Pavel at the Product Picnic