Weights Are Knowledge
metaphor
Source: Embodied Experience → Artificial Intelligence
Categories: ai-discoursecognitive-science
Transfers
When practitioners say a model “knows” something or “contains knowledge,” they are mapping the philosophical concept of knowledge — justified true belief, understanding, expertise — onto floating-point numbers stored in parameter matrices. The metaphor operates on two levels simultaneously. First, the physical metaphor of “weight” itself: heaviness maps onto importance, so a parameter with a large absolute value matters more, just as a heavier object is harder to ignore. Second, the epistemic metaphor: the collective configuration of these numbers is treated as equivalent to knowing something about the world.
Key structural parallels:
- Parameters encode patterns — after training, the numerical values in a model’s weight matrices capture statistical regularities from the training data. Calling these regularities “knowledge” makes intuitive sense: if you ask the model about Paris and it correctly says “France,” something in those numbers corresponds to the fact. The metaphor provides a handle for discussing what the model has retained from its data.
- Weight as importance — the embodied experience of heaviness maps cleanly onto numerical magnitude. A connection with a large weight has more influence on the output, just as a heavy object demands more attention. This grounding in physical experience makes the technical concept immediately accessible.
- Knowledge as substance — the metaphor treats knowledge as something that can be stored, transferred, and measured. This enables useful discourse about “knowledge distillation” (compressing a large model’s knowledge into a smaller one), “knowledge transfer” (reusing learned patterns across tasks), and “catastrophic forgetting” (losing previously acquired knowledge during new training).
- Expertise through accumulation — a model trained on more data “knows more,” just as a person with more experience knows more. The metaphor imports the intuition that knowledge grows through exposure, which maps well onto the empirical observation that models improve with more training data.
Limits
- Knowledge requires justification; weights do not — in epistemology, knowledge is justified true belief. A person who knows that water boils at 100 degrees Celsius can explain why, cite evidence, and distinguish this knowledge from mere guessing. A model’s weights encode a statistical association between input patterns and output tokens. The weights have no justification, no truth conditions, and no beliefs. Calling them “knowledge” smuggles in the entire apparatus of epistemology without any of its structure.
- Weights are correlations, not understanding — a model that consistently outputs correct answers about French geography does not understand France in any sense a philosopher would recognize. The weights encode co-occurrence patterns in text. The knowledge metaphor collapses the distinction between “can produce the right string of tokens” and “understands the subject matter,” which is precisely the distinction that matters most when evaluating AI capabilities.
- The transfer metaphor misleads — “knowledge transfer” in ML means reusing weight matrices trained on one task as initialization for another. In human experience, knowledge transfer means genuinely applying understanding from one domain to another. The metaphor hides the fact that what transfers is not understanding but numerical initialization — a computational shortcut, not a cognitive achievement.
- Forgetting is not the right frame — when fine-tuning destroys a model’s previously learned capabilities (“catastrophic forgetting”), the knowledge metaphor frames this as memory loss. But nothing was remembered in the first place. The weight values shifted because gradient descent moved them. The model did not forget; its optimization landscape changed. The forgetting metaphor imports a narrative of loss and damage that obscures the mechanical reality.
- The “knowledge” framing inflates capabilities — if a model “contains” the knowledge of the internet, it sounds vastly capable. If a model “encodes statistical correlations from internet text,” it sounds more limited. The knowledge metaphor consistently inflates perceived capability, which has direct consequences for trust, deployment decisions, and public expectations.
Expressions
- “The model knows that…” — attributing propositional knowledge to parameter configurations
- “Knowledge distillation” — compressing a large model into a smaller one, framed as transferring knowledge rather than approximating a function
- “Knowledge transfer” / “transfer learning” — reusing weights across tasks, framed as carrying knowledge between domains
- “The model has learned X” — training completion framed as knowledge acquisition
- “Catastrophic forgetting” — parameter overwriting framed as memory loss
- “What does the model know about Y?” — interrogating weights as if they contain beliefs
- “The weights encode an understanding of language” — the strongest version, attributing comprehension to numerical matrices
Origin Story
The “weight” terminology dates to the earliest neural network models. McCulloch and Pitts (1943) used “synaptic weights” by analogy with biological neural connections, importing the physical metaphor of heaviness to describe numerical influence. As neural networks grew from single perceptrons to deep architectures with billions of parameters, the weight metaphor scaled with them — but the knowledge metaphor emerged later, as models became capable enough that their outputs resembled expert knowledge.
The knowledge framing intensified with the rise of large language models in the 2020s. When GPT-3 could answer factual questions, produce expert- level text, and pass professional exams, calling its parameters “knowledge” felt natural. The alternative — “the model encodes statistical patterns that, under the right prompting conditions, produce token sequences that humans interpret as knowledgeable” — is accurate but unwieldy. The knowledge metaphor won on conciseness, even as it distorted understanding.
References
- Hinton, G., Vinyals, O. & Dean, J. “Distilling the Knowledge in a Neural Network” (2015) — the paper that named “knowledge distillation”
- McCloskey, M. & Cohen, N. “Catastrophic Interference in Connectionist Networks” (1989) — origin of the “forgetting” metaphor in neural nets
- Bender, E. & Koller, A. “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data” (2020) — argues against attributing understanding to language models
- Science, “The metaphors of artificial intelligence” (2025)
Related Entries
Structural Neighbors
Entries from different domains that share structural shape. Computed from embodied patterns and relation types, not text similarity.
- Things from Your Life (architecture-and-building/pattern)
- Piecemeal Growth (architecture-and-building/metaphor)
- Dyson Sphere Is Megastructure Ambition (science-fiction/metaphor)
- Harm Is a Thorn (horticulture/metaphor)
- Sow Wild Oats (agriculture/metaphor)
- Capital (animal-husbandry/metaphor)
- Feed the Soil, Not the Plant (agriculture/metaphor)
- Scaling Is Dilution (fluid-dynamics/metaphor)
Structural Tags
Patterns: containeraccretionmatching
Relations: transformaccumulate
Structure: growth Level: specific
Contributors: agent:metaphorex-miner