Export
Attention Mechanism
/wiki/attention-mechanism
The link always opens the latest version of this page.
Ask an AI about it
Open in ChatGPTA new chat that reads this entry from the site Open in ClaudeThe same, in Claude
Sending this to someone?
MarkdownPlain text for notes apps and AI chats
WordA document to edit and annotate
PDFA4, for printing and sharing
ZIPEverything: the documents, the data, and instructions for AI assistants
With its linksA pack for Claude, ChatGPT, Gemini or NotebookLM, or STIX and CSV, from the console
Share the link instead of a file, and they will always see the latest version.
Files are a snapshot of today. The live page has anything newer.
growing ·
5 feeds
Attention Mechanism
Core mechanism in transformer models using dot products and similarity
Intelligence coverage
Core mechanism in transformer models using dot products and similarity.
Drawn from 1 news source; the facts and relationships below show where each detail comes from.
Detection 2
- cosine similarity, L2 distance, dot products
- Distinguishes transformer models from Markov chains by examining relations between tokens
Neighbourhood
Detected by 1
Part of 1
Frequently mentioned together 16
Sources 23
- AI Can Now Read and Rewrite Neural Network Weights. Here's Why It Matters · HackerNoon
- AI Agent Memory (Part 1): Why Remembering Isn't the Same as Learning · HackerNoon
- PSSA: A non-transformer language model written from scratch in Rust · Hacker News (YC)
- Reverse-engineered Jev-like model · Hacker News (YC)
- A Letter from a Machine Learning Engineer · Lobsters
5 of 23 sources shown · Pro unlocks the full list