blame [they/them]

  • 0 Posts
  • 486 Comments
Joined 10 months ago
cake
Cake day: August 17th, 2024

help-circle






  • There’s AI The Brand that is the next thing silicon valley corps and investors are hoping is a repeat of the smartphone, and there is the much more ordinary machine learning and neural networks that have been used in a wide variety of industrial and consumer applications for decades. Those things don’t tell you how right and how smart you are though.

    Let’s not gloss over AI in games which may not actually involve machine learning at all.



  • when people here call it a text extrusion machine thats literally what it is. In fact it doesnt even look at text, it looks at tokens. And there are a limited number of tokens (llama uses a vocabulary size of about 32k i think). It takes all of the previously entered input and output, turns it into tokens, and then each token “attends” (is multiplied by with some coefficient) to each other token. Then it all goes through more gigantic layers of matrix multiplication and at the end you have the statistically most likely next token. Then it does the whole thing again recursively until it reaches what it decides is the end of the output. It may also not decide and would need to be cut off.

    So its not really looking at the game. It is in a way but it doesnt really know the rules, its just producing the next most likely token which is not necrssarily the next best move or even next correct move.