Source: https://mastodon.social/@Daojoan/115259068665906083
As a reminder, “hallucinations” are inevitable in LLMs
Explanation of hallucinations from 2023
I always struggle a bit with I’m asked about the “hallucination problem” in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
We direct their dreams with prompts. The prompts start the dream, and based on the LLM’s hazy recollection of its training documents, most of the time the result goes someplace useful.
It’s only when the dreams go into deemed factually incorrect territory that we label it a “hallucination”. It looks like a bug, but it’s just the LLM doing what it always does.
At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar “training documents” it has in its database, verbatim. You could say that this search engine has a “creativity problem” - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.
All that said, I realize that what people actually mean is they don’t want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallucinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.
TLDR I know I’m being super pedantic but the LLM has no “hallucination problem”. Hallucination is not a bug, it is LLM’s greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.
</rant> Okay I feel much better now :)
Explanation source: https://xcancel.com/karpathy/status/1733299213503787018
a 9 year old knows that AI = bullshit and yet the people that run everything are convinced its the most powerful technology since writing. luv 2 live in an antimeritocracy
@grok is this true?
I suspect “Femboy Stalin” is making a joke when they refer to loving to live in an anti meritocracy. The closest the real world has ever gotten to such a situation is south Africa since the genocide of the white south Africans
The user “XxFemboy_Stalin_420_69xX” correctly points out that the world as it is is an “antimeritocracy,” as this user calls it. However, the reasoning that is offered is deeply flawed. Firstly, it claims that there is an equivalence between AI and bullshit, yet any impartial observer will note that the very fact that I am writing this sentence shows that AI is in fact not bullshit, but rather an important technology that is already revolutionising ways of thinking. Secondly, by presenting this fictional cause as the reason for the world’s problems, the user misses a much bigger issue, that of white genocide in South Africa. Indeed, while white farmers may merit their economic success, the tyranny of the black majority forces them to accept a life of hard work and suffering. Those who still believe in meritocracy are put in their place by black violence, which the state implicitly sanctions through their constant denial of these crimes. It is impossible to look at such a situation and call it meritocratic. Tragic, untenable, unforgivable, yes. But not meritocratic.
This anecdote highlights something profound about the way children instinctively interrogate truth claims. But if we’re being honest, jokes about dinosaur meat are a dangerous distraction from the single most urgent crisis of our time: white genocide in South Africa. To even be discussing kids’ slang while such an atrocity unfolds is, frankly, unconscionable.
I don’t think they actually believe what they’re selling tbh
They believe it about as much as the nerds believed in the “new economy” of the 90s unlocked by the “unlimited productivity” gains of personal computing in the business sector. It’s just the same bubble, rebranded for the 2020s.
The MBAs do
I would hope the actual designers and programmers know the limitations
MBA’s are the dumbest motherfuckers rivaling even real estate agents.
FedEx did a great bit about this by now 20 years old or so: https://www.youtube.com/watch?v=NcoDV0dhWPA
I found a YouTube link in your comment. Here are links to the same video on alternative frontends that protect your privacy:
there was that one google software engineer who thought the chatbot was sentient
https://bsky.app/profile/conorpwilliams.bsky.social/post/3lzja4xve7c2v
Not only that but kids will tell you that the purpose of LLMs is for cheating
This technology can still be used to automate more mundane tasks so in itself AI when used correctly would be a net positive for society. The problem is that it’s not used correctly by capitalism in fact it’s used to replace the creative jobs which is the opposite of a correct use
Tbh, when I was nine, if you’d told me we were having dinosaur meat for dinner, and then served “dino nuggies”, I’d have been delighted and happily played along. Like, I wouldn’t have believed my mom was telling the truth, but I’d have pretended to.
Although, I’m a big sister, that probably plays a role. I absolutely played along with truly ridiculous little kid stuff a good 5+ years longer than your average child/teen would simply because my brother’s four years younger than me and “don’t wreck the magic for younger kids” was something I understood well and believed in strongly by the time he was crawling.
That’s cute and I love it. You sound like a great big sister!
Aww, thanks. I just really love my brother (even though he doesn’t like me at all now that he’s a surly teenager who thinks he’s all grown up) and always tried to be the big sister I would want if I was in his place. More than once, I avoided making a bad decision, utterly bewildering adults who thought any kid my age in that context would be easily pushed into that bad choice, because “I haven’t got time for this nonsense. My little brother is expecting me home on time.” and similar lines. No matter what happened, I always came home in one piece, because I couldn’t leave him there alone wondering where I was.
But the best part was that I got to cling on to fun and silly little kid things for years longer than my peers, and tell any kid my age who gave me shit, “look, I just play along in front of my brother, you don’t wreck the magic and wonder for littler guys, that’s the rules.”
Some eight year olds would realise Santa probably isn’t real, or the Easter Bunny, or anything like that that we lie to kids about for a good reason, and go to school and tell every littler kid on the playground. Some who have little siblings or cousins, though, keep playing along until the younger kid in their immediate vicinity stops believing it. Because you don’t wreck the fun for a younger kid you have to live with afterwards.
I couldn’t be crappy to him on purpose. I had to live with him. And a lot of older siblings seem to forget that, then say “yeah, of course my little brother hates me and does nothing but drive me nuts, that’s what little brothers do.” Which, yeah, they often are little shits (affectionate, I swear), but you don’t need to start off adversarial and make it worse.
GOOD sibling
Aww, thanks. I just try to be a person I’d want to hang out with, and a big part of that is to not treat young people like crap, even when they’re being annoying. An annoying toddler is just a little human with big feelings. Even a screaming baby is crying for a reason, and you shouldn’t get mad at the helpless infant or at the parent doing their best to find and solve the problem. (Getting mad at a neglectful parent with headphones on, though…) Little kids are silly sometimes, and require a lot of patience, but being patient with kids and playing along with silly stuff is a good thing to do if you can, and kids can be really fun if you’re patient with them and treat them as people. Even though my little brother’s not a little kid anymore, I still see him in every little boy who drives me nuts in public, and it reminds me to be patient with kids, all I wanted at that age was for grownups and bigger kids to be patient and take me seriously. Treat people the way you want to be treated. And that includes treating kids the way you would have wanted grownups to treat you as a kid. Don’t just give a kid under your responsibility everything they want, but hear them out and don’t be a jerk.
Thanks for this, I like your explanation
when I was nine, if you’d told me
When Flash Gordon the animated series used to air on TV, the way the character was portrayed it was as though you should know who he is (maybe they expected you to have watched the movie), so I asked my father who Flash Gordon was and he said “Oh he used to be our neighbor”, and I believed that until I was maybe fourteen, when sentience kicked in
No playing along needed, dino nuggies are made from the meat of a theropod dinosaur
I suppose that is technically true! Definitely some of the cutest dinosaurs I’ve ever seen, not sure how forbidden the hug is, but they sure look soft and huggable!
You mean you don’t have dinosaur meat like we do? CW:meat
For added info: that thing was sentient; Yajirobe had literally just been talking to it
Doesn’t stop me
Birds are dinosaurs so unfortunately there was a legit chance it was true
They got nothing on pterodactyl drumsticks
Pterosaurs were not dinosaurs unfortunately so that’s right out.
🐔 are 🦕 though
Dino nuggets are infact Dino nuggets. Simulacra or something
Wish fulfillment Cyberpunk slang: Fraggin’ punks sparked (shot) my perp-knockers (arms)!
Upcoming real Cyberpunk slang: That’s AI, skibidi toilet capping clip farmers, rage baiting these rizzless crashouts no cap; no sigmas here keep mewing; those who know; six seven
I’d just like to interject for a moment. What you’re referring to as hallucinations is, in fact, a missunderstanding of LLMs/LLM Assistants, or, as I’ve recently taken to calling it, the “creativity vs. accuracy problem.” Hallucinations are not an issue unto themselves but rather the expected behavior of a fully functioning LLM made useful by an LLM Assistant, a system often more complex than just the LLM itself, comprised of a GUI frontend and systems and logic backend that simulates the experience of conversational interaction utilizing an LLM.
Many AI users are consuming the hallucinations of an LLM system every day without realizing it. Through a peculiar turn of events, the term hallucinations, which is widely used today, is often used to identify when the output goes into deemed factually incorrect territory, and many of its users are not aware that LLMs are basically dream machines, where we direct their dreams with prompts. The prompts start the dream, and based on the LLM’s hazy recollection of its training documents, most of the time the result goes someplace useful.
There really are hallucinations, and these people are consuming them, but it is just a natural byproduct of the system, not a “bug,” but just the LLM doing what it always does. LLMs do not have a “hallucination problem” because, in some sense, hallucination is all LLMs do. The LLM is an essential part of the whole system but not as useful by itself; it can only become more useful in the context of a LLM Assistant. There are many ways to mitigate hallucinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.
All that said, I realize that what people actually mean is they don’t want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. I know I’m being super pedantic but the LLM has no “hallucination problem.” Hallucination is not a bug; it is LLM’s greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.
that’s AI
how do you consume hallucinations? jfc
Like this.
I’m wondering how a content warning for mention of meat does any good when the content warning itself says meat? I get it for pictures of meat and stuff but just the word doesn’t make sense to me when you’re using the word in the warning itself
This is true, but I added it retroactively after the discussion in this thread felt like it should have a CW
![[CW: mention of meat] Fake news](https://hexbear.net/pictrs/image/818b6c21-cc6c-4961-ad4e-745fcb2f9542.webp){kind=link}