The new wave of generative text models is impressive. It’s also an opportunity to think about the purpose of reading (or listening to) information. Can generative models satisfy what I want from these activities? What do I want, really?
I like interviews. I like reading and listening to interviews with authors or thinkers I like. When I listen, I think I’m generally looking for ways to understand their ideas better. If I’ve read their writing, then I’ve probably heard the distilled version of the argument. The cleanest and the best they could come up with. That’s great, but sometimes it can be so clean it’s hard to find a way in. Informal interviews, clumsier and with more digressions, can provide another way in, the messiness can be welcoming. It also helps remind me the author is a person, lessening the gulf between us. We’re all just people trying to figure things out I think, there’s comfort and closeness in that for me.
An example generated interview is Joe Rogan interviewing Steve Jobs by podast.ai. It’s mostly plausible to me on both an information and voice level. Is it interesting in content, though? I don’t think so. The model is trained on past Steve Jobs communications, so I know what I’m hearing is mostly just a remix of things he said (and that I could read or listen to) elsewhere.
Aren’t most actual podcast interviews functionally similar remixes, though? Reiterations of points already made elsewhere. I think so. But something feels different, and I think figuring out what the difference is might be the most interesting thing to do here.
I think what I’m really trying to do when I listen to an interview is understand the interviewee’s mental model of the subject (possibly of the world). Not just the info they’re sharing, but an idea of how they processed info to get to that point. I almost imagine this as a puzzle where you’re gradually filling in pieces: some of the pieces are directly related to the main content, some of them are seemingly random but somehow revealing asides. It may be you have most of the pieces filled in from reading their work, but interviews may be how you get some of the less central pieces, that get cut from their writing.
I’m much less confident I’m going to get those puzzle pieces from generated interviews. Generated views are only going to be retreads. There’s some nuance here – it’s possible the generated interview hits upon a rephrase that sparks an understanding in me I wouldn’t have gotten from the original material. That feels possible but unlikely to me. The bigger claim would be that the models are actually assembling or approximating the person’s mental model, and so they can reveral more of it through generation. I don’t feel like they’re there yet, I don’t know if they ever will be.
Generated interviews also miss out (probably) on the specificity of an interview happening at a certain time and place. In real interviews you might get a sense of what they’re like when they’re having a bad day or a great one. You might also be able to trace the evolution or rejection of their ideas from different periods. My guess is that current models are grab-bagging from material across time-periods and moods.
There’s something about the generated material being potentially endless – it makes it feel less valuable. If I’m searching for insight, it also makes me feel like I’m better going to the source, that my odds of finding it are much better there. Then maybe I use the generated material as background noise to keep me in that thought space longer? Maybe? I do use podcasts that way sometimes, but doing it with a generated one feels more like a waste of time to me at the moment.
Another use I could see but am currently skeptical of the value of: using the generative model to ask the questions yourself. This is an opportunity existing interviews don’t provide. I still feel like it would be a retread, more like a search of their past answers than a genuine conversation. Still possibly useful though.
Grant Custer is a designer-programmer interested in alternative interfaces.
You can see work and inspiration in progress on my Feed and my alternative interface experiments on Constraint Systems. I’m happy to talk on Twitter, email: grantcuster at gmail dot com, or Mastodon. You can see a full list of projects on my Index.