How Prompt Engineering Works in Terms of LLM Architecture
Prompt engineering is effective because large language models (LLMs) are built to generalize patterns of reasoning, not just retrieve information. To understand why, we can break it down in terms of how neural networks encode and manipulate structures of thought.
The Role of Weights in Structuring Thought
Neural networks don’t just store facts—they learn complex relationships through their weights, which capture not only information but also the patterns in data. Consider a simple regression neuron:
y=mx+by = mx + b
This function encodes three parameters (slope, input, and bias) and their linear relationship. If an AI model has learned that house prices generally depend on square footage, land value, and material costs, it can answer:
“Given my square footage and material value, what’s a good listing price?”
A generative model, however, goes further—it can estimate uncertainty and provide a range of possible values. This is because LLMs don’t just rely on single equations but encode many interrelated functions that approximate different reasoning pathways.
LLMs as Trees and Function Estimators
Now, consider a more structured reasoning process, like a decision tree with depth 3. A model can follow a prompt like:
“Use this tree of thought: First, categorize by X. Then, decide based on Y. Finally, estimate based on Z.”
This isn’t difficult for the model because deep neural networks are already structured in a graphical way. The computation might look something like:
x=(x1,x2,x3)→f(L1(x),L2(x),L3(x))x = (x_1, x_2, x_3) \to f(L_1(x), L_2(x), L_3(x))
where each layer LiL_i encodes a transformation from one representation to another. Within LLMs, the early layers encode fundamental knowledge—words, meanings, semantics—while the later layers encode more complex reasoning patterns, including structured chains of thought and decision-making.
At their core, neural networks are just recursive function approximators: they reduce down to trees, which reduce down to lines, which reduce down to polynomials, and so on. LLMs “think” by comparing multiple possible function approximations and selecting the most suitable one for a given task.
The “Thinking Layer” as Function Estimation
This realization—LLMs as function estimators—explains why prompt engineering works. A well-crafted prompt nudges the model toward using the best possible function for the task, whether it’s regression, decision trees, or more abstract reasoning chains. What we call “thought” in AI is really just an optimization process over a vast space of function approximations.
Future of human-crafter promps
Having learned about prompt engineering and recognizing its benefits, do you foresee a future where AI can generate high-quality outputs without human-crafted prompts?
I think this would mean an AI model with a “greater purpose” capable of planing for it and determining micro goals to reach it. It doesn’t have to be a human level greater purpose, but it can be something like being a nanny, a mechanic, and yes maybe even the internet asnwering your questions as they apply to you, or some specific function that a robot has to fulfill , akin to a Star Wars drone.
Bo-Katan’s ship maneuvers
the Gaunlet starfire has wings whith a pivor on the rear, they open up like a lambo’s door and enable complex meneuverability and reirections ar high speeds… damn physics is cool.