AIs gunning for our precious freelancers

[Raphael]

Stephen Wolfram has a very interesting recent article on Doug Lenat and CYC which I think captures the ideas of the time fairly well. (Not that I’d know, of course; I wasn’t alive then.)

Hm. Wolfram writes:

I consider it a magnificent experiment—that if nothing else ultimately served to demonstrate the importance of building frameworks beyond logic alone in usefully representing and reasoning about the world.

Personally, I still think that the Universe, at its most basic and fundamental level, is logical. It's just that many of the most interesting things in the Universe happen at some distance from the most basic and fundamental level.

[malloc]

Extrapolating the current trajectory of the technology... no.

So unlike pretty much every other technology in history, this one has reached a dead end in improvement right after being introduced? Airplanes went from barely flying to fighting in WWI in the space of one decade. Firearms went from missing the broad side of a barn to hitting targets a kilometer away. Yet you argue that generative AI uniquely among technologies permits no analogous advances in its power. What about all the artists and writers now who say that AI is destroying their livelihood? Are they actually industry plants hyping the technology?

(I have decided to stop referring to them as AIs, given they are demonstrably not intelligent)

If creating artwork and literature requires no intelligence then what the hell does? It sounds like you are setting the threshold for intelligence incredibly high to the point that most humans would hardly qualify.

This is a manifestation of the alignment problem, namely that we can't be sure we know what exactly the system is optimising for, because what we think we told it to optimise for is typically not the actual optimum of whatever instructions we gave. There are countless examples of alignment failures, and right now the best we can hope for is that the misalignment is still nonetheless useful.

Indeed, all the more reason not to trust this technology. We cannot meaningfully control it, only supplicate it for solutions and hope for the best. We are no longer using tools to gain mastery over our environment but praying to inscrutable dei ex machina with no idea what to expect or how to explain their decisions. It blows my mind that these techies know the problem exists and even have a name for it yet refuse to question the course they have laid for us. Imagine you get on the bus and the driver decides to barrel through uncharted territory with no idea where they are going. You would undoubtedly demand that they turn around or at least let the passengers off the bus. The course the tech industry has chosen is not only uncharted but fundamentally unchartable by their own admission and yet they refuse to reconsider their decision. Stay the course even if it takes us through R'lyeh...

[Raphael]

I can't believe I'm getting drawn into an Eddiethread...

(I have decided to stop referring to them as AIs, given they are demonstrably not intelligent)

If creating artwork and literature requires no intelligence then what the hell does? It sounds like you are setting the threshold for intelligence incredibly high to the point that most humans would hardly qualify.

Forget about today's supposed "AI". Captive elephants have painted paintings. Fractal creation software from decades ago was able to create graphics which someone who wouldn't know where they came from might well describe as art. A very simple software, running on 1980s mainframes (or perhaps even 1980s PCs), which would be simply programmed to generate random shapes, colors, and patterns, might well generate stuff that would look like quite impressive abstract art to some people.

After reading your post, I used a tool on my computer that visualizes how much space the files in different directories take up on my hard drive to create this:

composition.jpg (36.83 KiB) Viewed 1863 times

Made by something not even today's techbros would call an "AI", but arguably it doesn't look that different from some of Piet Mondrian's paintings. Is that simple visualization tool intelligent, too?

[Raphael]

Serious question, completely unrelated to the current ongoing discussion, but still on the topic of AI/generative software:

Could you use generative software to take any random song and create a sound file that sounds like a recording of a large stadium full of passionate sports fans singing that song?

[Travis B.]

Serious question, completely unrelated to the current ongoing discussion, but still on the topic of AI/generative software:

Could you use generative software to take any random song and create a sound file that sounds like a recording of a large stadium full of passionate sports fans singing that song?

I bet you could, but you don't need generative AI to do it. An inverse discrete Fourier transform combined with some elementary signal processing (i.e. filtering out frequencies that the human ear cannot hear) and then feeding the chosen frequencies into a piece of software which takes samples of people singing in a stadium and adjusts them accordingly would do it.

[zompist]

In a nutshell: ‘expert systems’, ‘fifth-generation computing’, and all the other attempts to build an AI by explicitly teaching computers about how to reason (à la ELIZA). Stephen Wolfram has a very interesting recent article on Doug Lenat and CYC which I think captures the ideas of the time fairly well. (Not that I’d know, of course; I wasn’t alive then.)

The article was interesting because I didn't know a guy named Wolfram was behind Wolfram Alpha.

Expert systems produced some neat tools (including Wolfram Alpha), but I don't think they ever got people very excited, or attracted the sort of money that's gone into LLMs. I suspect the reason is that no one was able to get any of them to actually communicate in English.

I just tried this in WA, which encourages English input. It can answer "what's the maximum prime number under 1000". But for "minimal path from a point on land to a point in the water", it took me as asking for the nutritional value of 1 carat (2 mg) of water. I've written about this before— if you try for an explicit rules-based approach to NLP, you end up in a fractal explosion of rules. LLMs solved that problem, at the cost of us being able to understand the rules, or the machine really understanding anything but language.

Or to put it more cynically, CEOs couldn't get enthused over attempts to model human reason, because they don't see the point of human reason. But LLMs understand language without understanding the world, and CEOs get that, because that's what they do.

[bradrn]

Serious question, completely unrelated to the current ongoing discussion, but still on the topic of AI/generative software:

Could you use generative software to take any random song and create a sound file that sounds like a recording of a large stadium full of passionate sports fans singing that song?

Almost certainly. But it’s much easier to simply use a chorus and reverberation effect in the digital audio software of your choice, for nearly as good results. (‘Chorus’ takes multiple copies of your sound and offsets them slightly; ‘reverberation’ does a convolution with your impulse response function.)

In a nutshell: ‘expert systems’, ‘fifth-generation computing’, and all the other attempts to build an AI by explicitly teaching computers about how to reason (à la ELIZA). Stephen Wolfram has a very interesting recent article on Doug Lenat and CYC which I think captures the ideas of the time fairly well. (Not that I’d know, of course; I wasn’t alive then.)

The article was interesting because I didn't know a guy named Wolfram was behind Wolfram Alpha.

Oh yes. He has a habit of naming everything after himself, and for that matter seems incapable of writing an article without talking up his own achievements. He’s done good work, though, and writes well.

Expert systems produced some neat tools (including Wolfram Alpha), but I don't think they ever got people very excited, or attracted the sort of money that's gone into LLMs.

Quoth the Wiki:

[Japan’s Fifth Generation Computer Systems project] ran from 1982 to 1994, spending a little less than ¥57 billion (about US$320 million) total.

In today’s money, I believe that would be around one trillion USD.

Or you can just read Hofstadter’s Gödel, Escher, Bach, which very excitedly talks about systems which are capable of reasoning about blocks stacked on top of each other, and predicts that any system intelligent enough to play chess will be intelligent enough to refuse to do so. It’s a great book, but totally missed the mark when it came to AI — just like most efforts of that era.

Or to put it more cynically, CEOs couldn't get enthused over attempts to model human reason, because they don't see the point of human reason. But LLMs understand language without understanding the world, and CEOs get that, because that's what they do.

I think this is too cynical. On the one hand, people genuinely got excited about expert systems. On the other hand, I think there is definitely a sense in which LLMs have ‘understanding’ of the world, though of course it depends on how you define ‘understanding’.

[zompist]

In today’s money, I believe that would be around one trillion USD.

That's more money than I would have expected. Not sure it paid off though.

Or you can just read Hofstadter’s Gödel, Escher, Bach, which very excitedly talks about systems which are capable of reasoning about blocks stacked on top of each other, and predicts that any system intelligent enough to play chess will be intelligent enough to refuse to do so. It’s a great book, but totally missed the mark when it came to AI — just like most efforts of that era.

I read it long ago-- I was an AI nerd. Hofstadter was like the Wired magazine of the 1970s, endlessly enthusiastic about any movement toward AI but not (so far as I can see) expecting it would happen any time soon. I don't think anyone thought that projects at that time "were AI", just that eventually they might lead to it.

I think this is too cynical. On the one hand, people genuinely got excited about expert systems. On the other hand, I think there is definitely a sense in which LLMs have ‘understanding’ of the world, though of course it depends on how you define ‘understanding’.

And I think people who should know better forget get fooled, just because LLMs can "talk". It's amazing what a huge corpus of inputs can do, but it's also pitifully easy to get an LLM to show that it really has no internal understanding. We are simply not used to interacting with things that sound smart but aren't.

I'm sure we've talked about this before-- it is possible to analyze an LLM, if you have access to it, and see what connections it's making. I think talk of understanding is premature until that work has been done.

It's pretty much the reverse of expert systems: they were good at real-world knowledge (carefully fed to them) but bad at language; LLMs are the reverse.

I do feel that LLMs are a huge step toward AI-- a hundred times more than earlier attempts. And I suspect that human brains are more like LLMs than we really like to think. On the other hand, AI people have a long, long history of overestimating their progress, and assuming that the next step is a couple of years away.

[Travis B.]

To me, just a few years ago I would not have expected AI to get nearly as far as the LLM's of late have shown themselves to be. Sure, LLM's definitely have their limitations, but they still are very impressive in what they can do. If you had asked me a year ago how long it would take to reach the point that LLM's have already reached, I would have said "decades".

[bradrn]

Or you can just read Hofstadter’s Gödel, Escher, Bach, which very excitedly talks about systems which are capable of reasoning about blocks stacked on top of each other, and predicts that any system intelligent enough to play chess will be intelligent enough to refuse to do so. It’s a great book, but totally missed the mark when it came to AI — just like most efforts of that era.

I read it long ago-- I was an AI nerd. Hofstadter was like the Wired magazine of the 1970s, endlessly enthusiastic about any movement toward AI but not (so far as I can see) expecting it would happen any time soon. I don't think anyone thought that projects at that time "were AI", just that eventually they might lead to it.

Interesting. I won’t argue, seeing as this was before my time.

We are simply not used to interacting with things that sound smart but aren't.

I dunno, sometimes I feel that quite a lot of people fall into this category…

(/s, I hope)

I do feel that LLMs are a huge step toward AI-- a hundred times more than earlier attempts. And I suspect that human brains are more like LLMs than we really like to think. On the other hand, AI people have a long, long history of overestimating their progress, and assuming that the next step is a couple of years away.

This is an excellent summary of what my position is too. (Especially that second sentence.)

[KathTheDragon]

Extrapolating the current trajectory of the technology... no.

So unlike pretty much every other technology in history, this one has reached a dead end in improvement right after being introduced? Airplanes went from barely flying to fighting in WWI in the space of one decade. Firearms went from missing the broad side of a barn to hitting targets a kilometer away. Yet you argue that generative AI uniquely among technologies permits no analogous advances in its power. What about all the artists and writers now who say that AI is destroying their livelihood? Are they actually industry plants hyping the technology?

Try again when you've learned to make good-faith responses. This is a gross mischaracterisation of what I actually said, and if I did not know you I'd believe you were doing it on purpose.

(I have decided to stop referring to them as AIs, given they are demonstrably not intelligent)

If creating artwork and literature requires no intelligence then what the hell does? It sounds like you are setting the threshold for intelligence incredibly high to the point that most humans would hardly qualify.

No, you're the one attributing too much to generative networks. They are fundamentally not creating anything new. Any creativity they may allegedly display is actually being derived from the human(s) prompting them. To reiterate: they are merely knowledgeable and competent. Do not mistake this for intelligence.

[Ares Land]

So unlike pretty much every other technology in history, this one has reached a dead end in improvement right after being introduced?

You raise an interesting point. There have been dead ends in technology before. I didn't know about fifth generation computing, but it certainly was one. Arguably, there are plenty in computing; one I can think of is LISP machines. Oversimplifying a bit, most early cars were electric; the idea of electric cars wasn't revived until very recently.
Fission power, while still in use of course, turned out less useful than initially planned. (Nobody went for nuclear rockets; I recently read a fascinating piece on nuclear airplanes.)

One problem I see with LLMs is that they provide plausible answers, but not necessarily accurate ones; and that problem is pretty much deadly when it comes to practical applications. I don't know if there's any easy way to fix this; judging from descriptions of the technology it doesn't look like we can even fix it right now.

I do feel that LLMs are a huge step toward AI-- a hundred times more than earlier attempts. And I suspect that human brains are more like LLMs than we really like to think.

I believe they're going to be huge in linguistics. It seems very likely human brains process language in a very similar way.

I can't believe I'm getting drawn into an Eddiethread...

Oh, I disagree with malloc, but lots of people are worrying along similar lines. I've had similar debates a few times IRL.

[alice]

Oh, I disagree with malloc, but lots of people are worrying along similar lines. I've had similar debates a few times IRL.

It's time to come "out of the AI closet", as it were: the ZBB member you know as "alice" was actually replaced by an LLM several months ago, and none of you have noticed!

[KathTheDragon]

I do feel that LLMs are a huge step toward AI-- a hundred times more than earlier attempts. And I suspect that human brains are more like LLMs than we really like to think.

I believe they're going to be huge in linguistics. It seems very likely human brains process language in a very similar way.

I'm inclined to disagree, though formulating my instincts into text is proving a challenge. Pertinent notions are that LLMs chunk text with no regard for natural boundaries, they have no notion of semantics at all beyond which tokens occur near each other, and their idea of context is just to memorise everything until they run out of storage.

[zompist]

There have been dead ends in technology before. I didn't know about fifth generation computing, but it certainly was one. Arguably, there are plenty in computing; one I can think of is LISP machines.

That reminds me of a story I read somewhere-- I think on Joel Spolsky's blog, but I can't find it now-- about a guy who did web development in LISP. Everyone thought he was insane, but he could do things far faster than they could.

I don't think LISP was a dead end; it was just too geeky even for geeks.

[bradrn]

There have been dead ends in technology before. I didn't know about fifth generation computing, but it certainly was one. Arguably, there are plenty in computing; one I can think of is LISP machines.

That reminds me of a story I read somewhere-- I think on Joel Spolsky's blog, but I can't find it now-- about a guy who did web development in LISP. Everyone thought he was insane, but he could do things far faster than they could.

You might be thinking of Paul Graham, and his famous essay Beating the Averages.

I don't think LISP was a dead end; it was just too geeky even for geeks.

It’s not a dead end at all. Firstly, Lisp (no capitals!) is not a single language but a fairly large family of languages. The most popular these days is probably Clojure, which runs on the JVM — I regularly chat online with someone who writes Clojure for a living. Racket and the various Schemes are also alive and kicking; Common Lisp has stagnated somewhat, but still has as many as five or six viable and actively developed implementation. Even Emacs Lisp is pretty lively, and I use it nearly every day. In fact, one could easily consider Emacs to be the successor of the Lisp Machines of yore. And that’s not even counting all the many, many other Lisp-inspired languages…

[Raphael]

Expert systems produced some neat tools (including Wolfram Alpha), but I don't think they ever got people very excited, or attracted the sort of money that's gone into LLMs.

Quoth the Wiki:

[Japan’s Fifth Generation Computer Systems project] ran from 1982 to 1994, spending a little less than ¥57 billion (about US$320 million) total.

In today’s money, I believe that would be around one trillion USD.

What? Are you sure about that? Less than half a billion 1980s US dollars would be a trillion of today's US dollars? That would make an ordinary 1980s billionaire the equivalent of a multiple trillionaire today! It would mean that the richest people of the 1980s owned a lot more money than the richest people of today! Sounds hard to believe to me.

[bradrn]

Expert systems produced some neat tools (including Wolfram Alpha), but I don't think they ever got people very excited, or attracted the sort of money that's gone into LLMs.

Quoth the Wiki:

[Japan’s Fifth Generation Computer Systems project] ran from 1982 to 1994, spending a little less than ¥57 billion (about US$320 million) total.

In today’s money, I believe that would be around one trillion USD.

What? Are you sure about that?

No, not really. Let me double-check…

…ah, sorry, I meant one billion dollars. Or, more specifically, US$1,013,690,362 and 69 cents. All I can say is that I miscounted the number of digits.

[zompist]

That reminds me of a story I read somewhere-- I think on Joel Spolsky's blog, but I can't find it now-- about a guy who did web development in LISP. Everyone thought he was insane, but he could do things far faster than they could.

You might be thinking of Paul Graham, and his famous essay Beating the Averages.

That's it! Thanks, that was fun to re-read.

[malloc]

One problem I see with LLMs is that they provide plausible answers, but not necessarily accurate ones; and that problem is pretty much deadly when it comes to practical applications. I don't know if there's any easy way to fix this; judging from descriptions of the technology it doesn't look like we can even fix it right now.

But why do we even such things? We already have human writers, a venerable and cherished profession. It seems like everyone here and in the tech industry sees a huge problem with humans writing their own texts and so forth but nobody has explained why. Automation makes sense for boring and dangerous tasks but art and scholarship hardly falls into that category. People die in collapsing mines but nobody dies from writing a film script or drawing a portrait.