![[CW: mention of meat] Fake news](https://hexbear.net/pictrs/image/818b6c21-cc6c-4961-ad4e-745fcb2f9542.webp){kind=link}
Source: https://mastodon.social/@Daojoan/115259068665906083
As a reminder, “hallucinations” are inevitable in LLMs
Explanation of hallucinations from 2023
I always struggle a bit with I’m asked about the “hallucination problem” in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
We direct their dreams with prompts. The prompts start the dream, and based on the LLM’s hazy recollection of its training documents, most of the time the result goes someplace useful.
It’s only when the dreams go into deemed factually incorrect territory that we label it a “hallucination”. It looks like a bug, but it’s just the LLM doing what it always does.
At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar “training documents” it has in its database, verbatim. You could say that this search engine has a “creativity problem” - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.
All that said, I realize that what people actually mean is they don’t want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallucinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.
TLDR I know I’m being super pedantic but the LLM has no “hallucination problem”. Hallucination is not a bug, it is LLM’s greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.
</rant> Okay I feel much better now :)
Explanation source: https://xcancel.com/karpathy/status/1733299213503787018
@grok is this true?
I suspect “Femboy Stalin” is making a joke when they refer to loving to live in an anti meritocracy. The closest the real world has ever gotten to such a situation is south Africa since the genocide of the white south Africans
The user “XxFemboy_Stalin_420_69xX” correctly points out that the world as it is is an “antimeritocracy,” as this user calls it. However, the reasoning that is offered is deeply flawed. Firstly, it claims that there is an equivalence between AI and bullshit, yet any impartial observer will note that the very fact that I am writing this sentence shows that AI is in fact not bullshit, but rather an important technology that is already revolutionising ways of thinking. Secondly, by presenting this fictional cause as the reason for the world’s problems, the user misses a much bigger issue, that of white genocide in South Africa. Indeed, while white farmers may merit their economic success, the tyranny of the black majority forces them to accept a life of hard work and suffering. Those who still believe in meritocracy are put in their place by black violence, which the state implicitly sanctions through their constant denial of these crimes. It is impossible to look at such a situation and call it meritocratic. Tragic, untenable, unforgivable, yes. But not meritocratic.
This anecdote highlights something profound about the way children instinctively interrogate truth claims. But if we’re being honest, jokes about dinosaur meat are a dangerous distraction from the single most urgent crisis of our time: white genocide in South Africa. To even be discussing kids’ slang while such an atrocity unfolds is, frankly, unconscionable.