Utopia Talk / Politics / More GPT stuff (Nim)

Seb Member	Tue May 16 08:34:16 Nim: This sort of summarises what I mean RE limitations of LLM. I'm not sure I entirely agree with the argument because the words probability of concurrence in a sentence *do* convey meaning. So the fact a word has no meaning except because we internally assign them meaning doesn't imply that the collection of all words and their probabilistic relation to each other doesn't encode meaning. However the implication is that the models ability to assign new meaning to things is limited. I think. Or at least that's what I think now. https://jonayre.uk/blog/2023/04/20/the-problem-with-large-language-models/ Mods - delete the thread with a typo
Nimatzo iChihuaha	Tue May 16 09:52:18 Thanks, interesting read, nothing I disagree with. They are not a mind, they can not experience the world, I touched on this with my last post regarding inspiration, they can't see or hear to draw inspiration from the world or even remember any conversation outside of that conversation. Those are massive deficits for a "mind" to have. Some of which the article mentions in the broader context of the limitations. Does that makes sense? Does an LLM need to store things in a memory like we do to do things with it later?
Seb Member	Tue May 16 11:32:23 It needs to have it in context. There is stuff people are doing with embedding - they can essentially turn a lot of tokens into a vector in the parameter space of the model. The point is that to improve it's "understanding" you need to add parameters or change weightings. "meaningness" is, essentially, the cosine of two vectors between two sets of tokens in that multi-dimensional parameter space. There is a good explainer somewhere that illustrates the concept - I'll dig it out. But the gist of it was imagine you have "Big" "small", "Child", "adult" You can put big and small at opposite ends of one axis with that axis representing size. And you can put Adult above big and child above small. And in between Adult and Big you can have "Dog" and "child and small" you can put puppy. The vertical axis represents some dimension representing "humanness" and the horizontal axis is "size/age"... so you have lots of these parameters that represent relation between words and thus capture the conceptual meaning of them. BUT that's all set at training time. So it is difficult to "add" a new concept or meaning to the model outside of training in an abstract way. BUT if your language model is rich enough the weightings of different parameters can be used to capture the concept in natural language to some degree. This is what I liken to Khannemans system 1/2 thinking. The first one is like heuristic, instinctive and deep understanding - as we might understand how a ball should fly through the air. The latter allows us to mechanically work through things we have no intuitive understanding of heuristically by encapsulating it in symbolic reasoning where can better understand the relationships between symbols and operators. But you are limited that way. It's like - maybe a really powerful LLM could derive a proof for fermats last theorem, now that it has been solved. But it wouldn't be able to come up with the hypothesis fermat did, and it wouldn't be able to develop a proof because it can't imagine/think very far forward unless there is a lot of encapsulation of that kind of thought in the training data such that strings of words with that meaning are strongly weighted; and it can then only break things down into steps working backwards.
Nimatzo iChihuaha	Wed May 17 03:55:34 Pretty good explanation, I am doing homework on this stuff and trying to learn more, the system 1/2 thinking is a great example. Ok, so these things are not strictly working according to human thinking process, yet we are temped, impressive as they can be to compare the outcomes to humans at different points of development, I mean I don't see how we get away from that since we are the blueprint for what we are trying to build. It strikes me that we are in an uncanny valley, maybe not The uncanny valley, but the first meaningful one. I was thinking about this the other day and my brain conjured up some vision of what a P-zombie AI would be, it was not a pleasant feeling. I don’t know that I would ever feel comfortable with such an entity nor that anyone should be. In the near term, I have become interested in how these things could be personalized to become an extension of us individually, a personal and local AI, trained by your interactions with it, to become your assistant and personal guide through life, to teach and educate you. That does come with its own set of problem, the usual security and privacy concerns. Beyond that, how early would you give a child their own AI and how/if that would further negatively impact human social interactions. There is this idea floating around that this could protect us from all the nasty things other AI augmented humans can send our way to manipulate, scam and troll us. This application would bring all the near-term regulatory concerns I share with you under the magnifying glass.
williamthebastard Member	Wed May 17 04:35:58 "convey meaning. So the fact a word has no meaning except because we internally assign them meaning doesn't imply that the collection of all words and their probabilistic relation to each other doesn't encode meaning." Connotations. A huge part of human language. If someone opens a window and I say, "Brrr its cold", what did I actually say? I actually said, "Bleeding hell, close that window" and we both heard it, unless the other guy is tone deaf
williamthebastard Member	Wed May 17 04:37:40 The same connotations people use when never mentioning nazism in connection with Soros but making it abundently clear to everyone anyway
Seb Member	Wed May 17 05:05:19 WtB: This is the basis of Chomsky's (incorrect) criticism of LLMs - and the "time flies like an arrow" example above. The way an LLM encodes information means that it "knows" that this combination of words means that "flies" correlates with words like "soars" not "insects" and "like" corelates with a token "as if" not "adores". I think there is a real question as whether human understanding *is* anything different to such associations. Certainly humans associate these things with observed phenomenon of which language is a layer of abstraction. BUT that does not preclude meaning/understanding - in the same way a mathematical description of particle physics we cannot observe is 'understood' entirely in the abstract or through relation to inaccurate heuristics (waves and particles) we can observe directly. I do think, at least at current preformance, that an LLM alone isn't going to "understand" enough about the real world to enact a plan - or be a genuine sentience. I do believe a sophisticated one could act out what the gestalt latent space of the internet thinks an AGI should behave like and do the things it thinks an AGI would do and may be able to execute those plans if they are straightforward enough to reduce to simple steps and not require it to develop a complex hypothesis.
Seb Member	Wed May 17 05:12:59 Nim: Yeah, I think we are in an uncanny valley here - and that is one of the potential risks/abuses of the problem. Is it ethical or legal/ought to be legal to try and get people (who are already good at anthopomorphising things) to anthropomorphise their phone OS, because it makes them then trust recommendations (i.e. bespoke adverts)? I would argue stridently this is abusive - even more so than attention hacking. However I can also say that it might be a very reasonable thing to do to help care for people with dementia. But for use with a toddler, arguably child abuse. To me these are far more pressing things to work on than stopping skynet - at least for now. And the idea we end up with an infosphere that is a perpetual arms race between AI based semantic spam filters and semantic spambots is terrifying - but probably inevitable without really strong regulations. That really will give those that control tech companies the whip hand over the electorate, or the govt. Either is terrible for democracy and freedom.
Seb Member	Wed May 17 05:16:30 Neal Stephenson's book "Fall, or Dodge in Hell" has a section describing a near future America that extrapolates what a society with a truly chaotic infosphere looks like. Those that can afford proper curation of information, and crazy folks that are never able to create a stable sensible worldview because they are constantly being red-pilled from birth. It probably wouldn't be that stark - but you just know that this sort of thing will be the goal for global powers propaganda, subversion and psyops campaigns; and that the hyperpartisan approach to political campaigning will see these as powerful and totally legit tools for campaigning rather than turning the weapons of autocracy loose on those they seek to represent.
Seb Member	Wed May 17 05:18:54 On the subject of neal stephenson, those that have read diamond age will think that AI as a child minder is infact a cool and totally legit thing to do. A way of getting a "A Young Lady's Illustrated Primer" to give your kids a competitive edge.
Seb Member	Wed May 17 05:19:38 "Human's are such social animals they are working on technology to avoid having to socialise with each other" - as it was put.
Seb Member	Wed May 17 08:20:13 http://twi...?t=OXfbSz1j3vviyz3yAsltZQ&s=19 Posted with no particular comment
williamthebastard Member	Wed May 17 08:45:18 Linguists categorize meaningness into different boxes, the whole Suassureian signifier/signified stuff, but meaningness is often wholly abstract and culturally variable, such as pearl necklaces containing meaningness such as wealth, old age, woman, etc etc
williamthebastard Member	Wed May 17 08:55:56 I took some classes in semiotics but I dont remember much. Fairly interesting. Chomsky is generally considered to have rather stiff and rigidly deterministic theories, a bit of a Dawkins in that field.
williamthebastard Member	Wed May 17 09:05:17 "Electricity runs through the wire" - this is a metaphor. Electricity doesnt run, but even if we've never heard the metaphor before, we understand its meaning. Dont know if a machine can create meaning in the same way
williamthebastard Member	Wed May 17 09:21:46 "Clutch their pearls" is a fun example pertaining to the pearl necklace of how we create meaning. From creating meaningness pertaining to pearl necklaces regarding age, status, etc., we can further create new meaningness until it means, for example, "People who pretend to be outraged"
Seb Member	Wed May 17 09:46:16 WtB: I think LLMs can create meaning but they can do so strictly and only in relation to the existing body of language. Like the first person to say "clutched their pearls" in reference to anxious high society ladies grabbing their pearl necklace in fear and horror when confronted by something shocking - it doesn't have the visual / social context to work off. It would only be able to approach it textually, until someone had explained via a piece of text like the above (either in training data or as input, that then creates a weighting between the vector in the parameter representing "clutching pearls" and the vector corresponding to "performative shock and horror"
williamthebastard Member	Wed May 17 09:50:38 Yeah, it can only browes millions of similar examples and extract the most common instances, and also has some relevance logarithms going, but we can create meaning in a way that I think is different. I may never have heard you say that electricity runs through a cable before, indeed, in Swedish that sounds very strange, but I can immediately decode the info youre sending me and create the meaning you want me to get out of that metaphor
williamthebastard Member	Wed May 17 09:51:30 an LLM may arrive at the same result but not through interpreting a metaphor from the physical world
williamthebastard Member	Wed May 17 09:58:11 Dr. Bowman: "HAL, run those engines before we all die!" Hal: "?" Dr. Bowman: "For gods sake HAL, get those engines running before we all die!" Hal: "Huh?" Dr. Bowman: "HAL! Start those goddam engines!" HAL: "Oh, start the engines, ok, right away Sir"
Seb Member	Wed May 17 10:01:46 WtB: "it can only browes millions of similar examples" No, that's not how it works. It's pretty good at getting metaphor provided that those concepts exist. It would totally get "electricity runs through a wire" - in the same way it would never confuse "time flies like an arrow" to mean "a creature called a time fly has a preference for arrows". It would get "clutching pearls" if there was lots of textual records regarding clutching of pearls being a reaction indicating shock; and that being a common trope in hammy films. If you invented a metaphor for the first time though, which relied on likening something to a behaviour that itself had not been previously described in text to date - then it would fall over.
williamthebastard Member	Wed May 17 10:04:36 "It would get "clutching pearls" if there was lots of textual records regarding clutching of pearls being a reaction indicating shock" And it also has to get that pearls meaning a certain kind of people otherise it falls flat, and that this is sarcastically intended, else it falls flat. I can work that out and create meaning from it, such as the sarcastic aspect. But I dont know what AI can do, havent kept up
williamthebastard Member	Wed May 17 10:06:13 "If you invented a metaphor for the first time though, which relied on likening something to a behaviour that itself had not been previously described in text to date - then it would fall over." Exactly my point, A human can create meaning from it though. Water running makes sense to a Swede the first time he hears it even though it sounds strange in Swedish, he can place it in a logical model fetched from the physical world
williamthebastard Member	Wed May 17 10:22:18 Completely random signifiers such as pearl necklaces can be more easily transferred, since theyre learned, but I can make the connection to sarcasm through these various bits of circumstantial information whereas I think AI would have to be taught that clutching pearls is often a derogatory comment
Seb Member	Wed May 17 10:59:12 WtB: "And it also has to get that pearls meaning a certain kind of people otherise it falls flat, and that this is sarcastically intended, else it falls flat." It can do that pretty well - one of the more amazing things about GPT4 that blew my mind a bit. "he can place it in a logical model fetched from the physical world" Yeah, ok, then we are on the same page. There is *some* evidence that GPT4 has developed a fuzzy world model from language. For example there was a puzzle that was put to GPT3: "I have a thimble, into which I put a diamond. I put the timble in a coffee cup on the kitchen counter. I take the coffee cup to the kitchen table. I put it down upside down. I take the coffee cup to the desk in my office. Thinking step by step, which room is the diamond in?" GPT3 would answer something like: The diamond is in the thimble, which is in the cup. Because you took the cup to the desk in your office, and the thimble is in the cup, and the diamond is in the thimble, the diamond is in your office. GPT 4 answered: The diamond is in the kitchen. When you put the cup down upside down, the thimble and diamond would have fallen out of the cup. So at some point in adding increased parameters, GPT4s deep learning algorithm developed an abstraction from test that understood a bit about gravity. But understand, it has no intuitive understanding in the way you or I does, but it does know from text that turning things upside down causes things to fall out and not be inside the things that turned upside down. It would be interesting to see if understood the idea of open and closed objects. But this is the point I was meaning RE AlphaGo: AlphaGo could innovate strategies in GO because it started off with hardwired knowledge of boards, position on the board, pieces etc. GPT4 knows nothing of chess other than there are strings like White: Ke4 Black: Ba2 It is not entirely inconceivable given enough text it would understand that certain moves have 0% probability of following previous moves and effectively model the rules equivalent to understanding there was a board etc. And it is also not entirely inconceivable given enough games it would be able to pick a winning strategy. But it akin to attempting to "solve" chess by brute force approaches rather than having an understanding the relative merits of a particular move and how they correlate to success or play part of a broader strategy. Now there ARE other AI models that are better at that. There was a very very intersting piece about wiring up an LLM AI to an AI "strategy engine" for diplomacy. It performed extremely well and other diplomacy players had a hard time telling which one of them was an AI. The strategy engine told the LLM what moves it wanted to make, the LLM part handled communication with the players, and fed back via the game model to the strategy AI what it evaluated the other players moves would be - allowing the strategy AI to determine what moves it would make. It was an extremely impressive and slightly scary example of the long term trend. But again, the important point is the game has a well defined, well known, perfectly knowable model. Reality doesn't. You can't derive psychology from physics. You can't event derive chemistry from physics that well. You need SO MANY MODELS to create something that would be more dangerous than an organisation of humans and be a skynet. That's my thinking anyway. BUT you can certainly create incredibly powerful and dangerous things. I could totally believe well before skynet you can get a p-zombie LLM that instructs a low category lab to synthesis an RNA string for a highly infectious disease if somehow badly prompted deliberately or accidentally into role playing an evil AI.
Seb Member	Wed May 17 11:02:33 BTW an LLM that was also coupled to an image classifier *would* understand "water running". Indeed, that's precisely how things like midjourney and Dali work. And might then be able to understand "novel" metaphors.
williamthebastard Member	Wed May 17 11:13:21 Yeah, but how did it relate water to running? Through being taught? If an Englishman directly translates that to Swedish, "vattnet springer", any Swede will think that sounds comical but immediately gets the metaphoric content. Personally, I think AI's inability to experience the physical world might be one of the frightening things about it. Also, working out sarcasm means Im working out that the author means the exact opposite of his words. That sounds like a pretty tricky feat for AI, but I suppose a mountain of algorithms can address a lot of things
williamthebastard Member	Wed May 17 11:20:14 Someone should try "vattnet springer" in an art AI app that does swedish, I dont know any links/can't be bothered heh
Seb Member	Wed May 17 11:45:45 WtB: Hmm. Ok fair point - it would have been explicitly taught that "water running" has that meaning then. I'm pretty sure though if you trained an LLM in Swedish language, and image tokens, it would end up with an understanding that run means motion and water has motion and so what a "running river" meant flowing rather than donning jogging pants. It's not an algorithm that understands sarcasm, just context. It is as good at working out that a phrase is sarcastic within a block of text with no other context as a human is.
Seb Member	Wed May 17 11:46:50 The problem with just trying it now is LLMs probably translate on the fly. It knows that the Swedish word for running is the same as the english.
Nimatzo iChihuaha	Wed May 17 12:21:44 Curiously ”run” and the proper translation for the water example in Swedish ”rinner” are from the same root. Running, Rinner. And rinner (flowing) and springer (running) are related terms. You can use ”springer” (running) to describe the behavior of water in Swedish, we say ”vatten springer ur en källa”, meaning water springs out of a well.
Nimatzo iChihuaha	Wed May 17 12:22:42 Seb "because it makes them then trust recommendations (i.e. bespoke adverts)? I would argue stridently this is abusive - even more so than attention hacking." "Those that can afford proper curation of information" Yea, this is awful. In my mind perhaps the best way to describe this personal AI is like a spirit animal and guide in the digital realm, like truly an intimate extension of your mind, I don't know if is possible right now to train an AI in your own image, but the idea that it would tailor ads for me, even better than algos are doing atm, sounds terrible. Yet we can all imagine how we would end up with a paid premium version of such a powerful tool and an ad based one. The latter is probably a concept so bad, that it shouldn't be allowed to happen.
	show deleted posts

Your Name:
Your Password:
Your Message: