The voice modality plays a huge role in how impressive it seems.

When GPT-2/3/3.5/4 came out, it was fairly easy to see the progression from reading model outputs that it was just getting better and better at text. Which was pretty amazing but in a very intellectual way, since reading is typically a very "intellectual" "front-brain" type of activity.

But this voice stuff really does make it much more emotional. I don't know about you, but the first time I used GPT's voice mode I notice that I felt something -- very un-intellectually, very un-cerebral -- like, the feeling that there is a spirit embodying the computer. Of course with LLM's there always is a spirit embodying the computer (or, there never is, depending on your philosophical beliefs).

The Suno demos that popped up recently should have clued us all in that this kind of emotional range was possible with these models. This announcement is not so much a step function in model capabilities, but it is a step function in HCI. People are just not used to their interactions with a computer be emotional like this. I'm excited and concerned in equal parts that many people won't be truly prepared for what is coming. It's on the horizon, having an AI companion, that really truly makes you feel things.

Us nerds who habitually read text have had that since roughly GPT-3, but now the door has been blown open.

Honestly, as someone who has been using this functionality almost daily for months now, the times that break immersion the most by far is when it does human-like things, such as clearing its throat, pandering, or attaching emotions to its responses.

Very excited about faster response times, auto interrupt, cheaper api, and voice api — but the “emotional range” is actually disappointing to me. hopefully it doesn’t impact the default experience too much, or the memory features get good enough that I can stop it from trying to pretend to be a human

Speech is a lot more than just the words being conveyed.

Tone, Emphasis, Speed, Accent are all very important parts of how humans communicate verbally.

Before today, voice mode was strictly your audio>text then text>audio. All that information destroyed.

Now the same model takes in audio tokens and spits back out audio tokens directly.

Watch this demo, it's the best example of the kind of thing that would be flat out impossible with the previous setup.

https://www.youtube.com/live/DQacCB9tDaw?si=2LzQwlS8FHfot7Jy

Flat out impossible? If you mean “without clicking anything”, sure, but you could interrupt with your thumb, exit chat to send images and go back (maybe video too, I’ve never had any need), and honestly the 2-3 second response time never once bothered me.

I’m very excited about all these updates and it’s really cool tech, but all I’m seeing is quality of life improvements and some cool engineering.

That’s not necessarily a bad thing. Not everything has to be magic or revolutionary to be a cool update

Did you even watch the video ? It's just baffling how I have to spell this out.

Skip to 11:50 or watch the very first demo with the breathing. None of that is possible with TTS and STT. You can't ask old voice mode to slow down or modulate tone or anything like that because it's just working with text.

Yes I watched the demo. True those things were not possible, so if that’s what’s blowing you away then fair enough I guess. For me that doesn’t impact at all anything have ever used voice for or probably will ever use voice for.

I’ve voice chatted with ChatGPT for hundreds of hours and never once thought “can you modulate your tone please?”, so those improvements are a far cry from magic or revolutionary imho. Again, that’s not to say they aren’t cool tech, forward advancements, or impressive —- but magic or revolutionary are pretty high bars.

To each their own though.

Few people are going to say "modulate your tone" in a vacuum sure but that doesn't mean that ability along with being able to manipulate all other aspects of speech isn't an incredible advance that is going to be very useful.

Language learning, audiobook narration that is far more involved, you could probably generate an audio drama, actual voice acting, even just not needing to get all my words in before it prompts the model with the transcribed text, conversation that doesn't feel like someone is reading a script.

And that's just voice.

This is the kind of interaction that's possible now. https://www.youtube.com/watch?v=_nSmkyDNulk

And no, thumbing the pause button, sending an image and going back does not even begin to compare in usability.

Great leaps in usability are a revolution in itself. GPT-3 existed for years so why did ChatGPT explode when it did? You think it was intelligence? No. It was the usability of the chat interface.

Kinda stretching the definition of interactive there.

How so? You don’t have to press the mic button after every sentence. You press the headphone button and speak like you normally would and it speaks back once you stop talking.

How much more “interactive” could it be?

Right but this works differently.

Yeah the product itself is only incrementally better (lower latency responses + can look at a camera feed, both great improvements but nothing mindblowing or "magical"), but I think the big difference is that this thing is available for free users now.