Main Topic: The emerging role of the prompt engineer in the AI industry, particularly in text-to-image models.
Key Points:
Participants:
Takeaways:
Additional Insights:
With every new technology, some jobs are lost while others are gained. People often focus on the former, but in this episode we chose to highlight the latter – a highly creative role that emerges alongside AI: the prompt engineer.
Until AI can close the loop of its own, each tool still requires a set of prompts. Just like a composer feeds an instrument the notes to play, a prompt engineer feeds an AI a map of what to produce. And if we know anything from music it’s that composing great music takes great skill!
In this episode we explore the emerging importance of prompting with Guy Parsons, the early learnings of how to do it effectively, and where this field might be going.
Will the prompt engineer be more like the highly sought after DevOps engineer, or a proficiency like Excel that you find on every resume? Listen in to hear Guy’s take.
Interested in the prompt competition? Email us at [email protected].
Resources:
DALL-E 2 Prompt Book: https://dallery.gallery/the-dalle-2-prompt-book/
Find Guy on Twitter: https://twitter.com/GuyP
Guy’s combining image experiment: https://twitter.com/GuyP/status/1612880405207580672
Guy’s amorphous prompt experiment: https://twitter.com/GuyP/status/1608475973300948993
Guy’s space duck: https://twitter.com/GuyP/status/1601342688225525761
Prompt base: https://promptbase.com/
Lexica: https://lexica.art/
Topics Covered:
0:00 - Introduction
01:49 - DALL-E 2 Prompt Book
05:29 - Parallel skills
06:51 - 80/20 prompting
10:16 - New ways of prompting
13:44 - Pulling the AI slot machine
18:09 - Comparing models
21:04 - Requested features
26:34 - Learning with AI
27:58 - Practical use cases
32:08 - A top 1% prompt engineer
36:17 - The most popular images
Stay Updated:
Find us on Twitter: https://twitter.com/a16z
Find us on LinkedIn: https://www.linkedin.com/company/a16z
Subscribe on your favorite podcast app: https://a16z.simplecast.com/
Follow our host: https://twitter.com/stephsmithio
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. For more details please see a16z.com/disclosures.
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Podcast on Spotify
Listen to the a16z Podcast on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Host
If you think about the next layer, it's still quite hard to describe things with words. Designers, when they do work for clients, it's one of their pet peeves because clients don't like it, but they can't explain why. With every new technology, some jobs are lost while others are gained. And while people often focus on the former, in this episode we're highlighting the latter. A highly creative role that emerges alongside AI, the prompt engineer.
Until AI can close the loop of its own, each tool still requires a set of prompts. And just like a composer feeds an instrument a set of notes to play, a prompt engineer feeds the AI a map of what to produce. And if we know anything from music, it's that composing great music takes great skill. So in this episode, we dive into the emerging importance of prompting, the early learnings and how to do it effectively, and also where this field might be heading. And we do so with Guy Parsons. Guy has been an early mover in the text image AI space, having written the Dolly 2 prompt book in July of last year. So will the prompt engineer be more like the highly sought after DevOps engineer or a proficiency like Excel that you find on every resume? Listen in to hear Guy's take.
By the way, we're thinking of running a prompt competition coming up. So if you think you have what it takes, email us at podpitches at a16nc.com with the subject, prompt engineer. As a reminder, the content here is for informational purposes only. Should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. For more details, please see A16Z.com slash disclosures.
Guy, welcome to the show.
Guy Parsons
Thank you for having me. I'm excited to be here.
Host
When we originally reached out to you, it was around six months ago, and you had just written something called your prompt book. Why don't you give everyone a little bit of an idea of what that prompt book was, what it is now, and also what prompted you to want to write it in the first place?
Guy Parsons
This was in the initial, the heyday of DALL-E 2, which was OpenAI's text-to-image model. When it came out, they rolled it out to a few test people at a time. They were super cautious about how it might be misused, how it could end up having a backlash, all these kinds of things, which then only increased the sense of people wanting to get their hands on this thing. Because at the time, this was pre things you might think of now, a stable diffusion, mid-journey kind of predated those by some small margin and seemed way ahead of anything people have tried using before.
So yeah, if you've used a text-to-image AI, by now you know it's basically a text box. And it all comes down to what you type in. It doesn't have buttons and all the kind of controls you might expect when you log in to something like Photoshop. So the question then becomes, like a lot of people, once mine goes blank, or you don't actually know the name or the words of what you're trying to type in, right? If you've actually been to art school or you're up on your art history or your design language, then you've probably got a head start on everyone else. But on places like Twitter and Reddit, there are people posting these amazing images. But because of the nature of social media, it's all lost.
So I started trying to like collect these cool examples and these cool terms people were using to create these like amazing visual effects. So I started putting everything in a essentially like a slide deck. By the time I'd copied and pasted all these cool things, I've seen there as 80, 100 slides long, something like that. So then I rather grandly called it a book and shared it online. And it's just a jumping off point for people to realize the kind of stuff at the time that these tools were just about becoming capable of. Obviously, now the capabilities are even more advanced.
Host
And we'll get into that because within six months, it's crazy to see how these tools, the way people are using these tools, how that's all changed in a matter of, again, just six months. It feels like yesterday when we didn't even have access to this. But this idea that these are tools and just like any other tool, person A versus person B may not get the same result. They may not have the same understanding of how to leverage the tool. And so before we get into maybe the tips and tricks that you've learned, I just want to give the audience a broad sense of how much time you've spent within the bowels of mid-journey, DALL-E, stable diffusion. If you could give an estimate, how much time do you think you've spent mastering this idea of prompting?
Guy Parsons
I wouldn't say I'm a master in any sense. It's so engaging and interesting to experiment with these goals. In the last six months, sure, a couple of hundred hours... What I really admire is people that are using these tools to create this like real body of work where they can really like pursue a direction to discover what's possible. I think I saw a thread where, you know, I think it's a mid-journey, you can get it to tell you how many prompts you've ever done. And there are people in there, thousands, hundreds of thousands.
Host
Yeah, and I appreciate how humble you are, but I think it's one of those scenarios where, again, we're six months in. You know, a parallel is when there's a new coding language and then you see people write job descriptions for developers looking for someone with five years experience when that particular language has only been around for six months or a year. And so, yes, I don't think anyone could definitively say they're an expert in prompt engineering, partially because it's only been around for so long. But I do think you've at least shared a lot more than the average person. And given your experience with these tools, I'm curious if you see a parallel skill set where you can kind of compare prompt engineering to learning to code. Is it similar to being able to storytell effectively? Is it similar to being able to process numbers in an Excel sheet? Like, is there a parallel skill set where it reminds you of, you know, something you've done before?
Guy Parsons
I think there was an era, I don't know if we're still in it, where there was a certain category of person who could consider themselves like good at Googling stuff. Do you know that? Do you know that kind of like, oh, file type this. And there's this big debate over whether, especially in text to image, you know, is there really like any artistry to it? For me, I'm not so sure because I'm no artist, but there's definitely something. It's always about discovering an image that's already out there. You've just got to find the words that summon it forth as if you're kind of navigating like an infinite Pinterest of things that haven't quite existed until you manifested them.
Host
Well, I mean, to that point, like we have so much information online. I feel like that is a skill set even before these AI tools. I used to work on a product called Trends and that really was about using the right tools like subreddit stats or Ahrefs or other data sets online and learning to parse them and learning to surface what other people find interesting. But let's get into the nitty gritty. Like you wrote this prompt book, you've been playing around with these tools for quite some time. Are there certain learnings, maybe the 80-20 approach of becoming a good prompt engineer in terms of things that you think are really valuable to understand? Maybe it's the prompt length. Maybe it's using certain modifiers within your prompt. Maybe it's just like a framework for thinking about prompting. Is there anything that's surface that you think would be really valuable to someone who's just starting out with prompting?
Guy Parsons
Oh, yeah. Like I think if you've never used one before, like the best way to explain how they work at the moment, which is again, always shifting and something else we can talk about is to always like describe something as if it already exists. So imagine that it's an image in some kind of downloadable clip art library or a photography gallery and you know someone's written underneath oh this is a fine example of a early modern photography shot and those are the kind of descriptions that you're trying to kind of mimic to tell these tools what you're looking for and it also gives it like a natural sense of why these tools are bad at some things and the kind of problems that don't really work because if there's like let's say some archive image of some women celebrating on the steps of a church in 1972.
It was that kind of caption, where they never go, the woman on the left is wearing a yellow hat, the woman on the right is wearing, you know, they just don't go into that because you can see it. So ironically, they often describe very generally what the image is about, but not like how you would draw it step by step. And that's why these tools are less good at saying like, I want this thing over here, and then that thing next to it and then something on top and that thing should be much bigger because that's in the real life. That's not how images are described in language.
Host
Yeah, that's a really good point though because that's how these AIs were trained, right? So I think DALL-E trained on 600 and less million images and they're using that alt text. again, that descriptor. And I've never thought about it that way, but actually training yourself to become a good prompter by reviewing the inputs to the tool, which I've never done this before, but I can imagine someone literally going online and reading the alt text on different images and going, ah, this is how this was described. This is how an AI might interpret my future prompt.
Guy Parsons
And I think to your point also, it's something that I've learned from my very limited set of prompting is just the level of detail that you need with your prompt where when I first started, I like you know monkey wearing a hat And you know there you don even realize until you start prompting the many iterations that could come from that Like you have one image in your head but then you get back all of these different results And then you end up looking on different prompt search engines or libraries and seeing what other people are doing. You're like, this prompt is like 200 words. I would have never thought to do that. And I think there's something to be said. Like, I think the longer they are, there's definitely diminishing returns. but sometimes using a lot of related almost synonymous terms just like chucking in like loads of like you know detailed techie like photography language is all kind of pushing it in the direction of like wow this really sounds like a kind of a real fancy photo.
Host
As I went through your prompt book there were so many different ways that you could describe a shot you could say a different camera angle you could say a time period as you just spoke to you could say a specific type of artistry or even a specific artist. I know there's some controversy around using specific artists' work to train your new images, but let's look forward to today. I feel like as we talked about six months later, these tools have evolved a lot. Are there any different ways that you can prompt today or leverage these tools that didn't exist six months ago that are really important and maybe extending the way that you can use them?
Guy Parsons
100%. So the main one, and these things are like changing all the time, right? So but now there's increasingly tools where you can prompt with an image. Again, that's almost like an entire new field of exploration, because it's not combining the image with your words in the way you would expect something like Photoshop to do it, like it's not collaging them together. It's almost describing to itself this like source image with words. and then doing the same with like a second image or maybe some additional text you supply and then being like, okay, now I'm going to make a new picture that somehow represents both these things.
So the results can be really surprising, really unexpected, probably quite difficult to control. But then you potentially have interesting opportunities like, okay, I can make a load of kind of abstract stuff using my brand colors or something that's important to me, photos of me, who knows. And then, yeah, and then I'm going to use that and kind of multiply that visual base with custom other prompts. And then everything will have this kind of lightness. And then, of course, like the big thing that happened since the days of the prompt book and so on was, of course, that huge spike in interest in selfies, right?
Like the lenses and the profile picture.ai and there were like a dozen of them, which was just prompting with your face, being like, yeah, I want to see more of this guy because it's me, obviously. And then within the image to image space, you've now got other startups that are doing interesting things where, okay, give us 10 core images and now we'll generate you like infinite versions of that based on like the modifiers that you want to see. So there's all kinds of that's a really interesting space that's going to probably power like the next generation of how people, especially consumers, interact with these products.
Host
Yeah. One way that maybe you could put it is that when we first got access to these tools, you were really starting from scratch. You didn't even have the prompt libraries available to you. you were just like, okay, I have this image in my head. But today, you not only have those libraries, you also have images that you can input. So you're not starting from scratch. You have a baseline of, as you said, maybe it's brand colors, maybe it's a certain style. And instead of having to articulate that yourself, you can just say, hey, here's what I want. But to your point, sometimes it's hard to control, right?
Because you're trying to say something to the AI. You're trying to say, I want this output. You don't always get it. And so something I want to ask you about is how you've learned to rein that in to really, you know, on the whole, get a higher throughput of images that you want versus images you don't want over time, because these AIs, they are a little bit of a black box, right? You can't understand every little piece that went from your input to your output. And so you can't like fine tune it in the same ways as maybe some other skills that we've learned in the past. And so how have you learned to actually become a better prompt engineer given that black box nature?
Guy Parsons
I mean, I think another aspect is there's also like a random element. So if you and I both type in the same thing, it's not going to make the same picture because it kind of starts from this random cloud of noise. And your cloud of noise is different to mine. And then it's slowly turning these clouds more and more into something that looks like orangutan in a tuxedo. But we're going to end up with different things. So that's really frustrating when you're like testing things, because was it good? Or did you just get lucky? Or alternatively, if you're not seeing what you're expected, should you just hit it again?
And again? And then when you see someone else has made something really cool, did they do something really clever? Or is it like a persistence thing? I have found myself in that exact spot where I have an idea for what I want. It's not something that is super important where I need to nail it. So I just need it close enough. And I'm getting these results and they're getting a little closer and closer and closer. But I have found myself in that spot where I'm just like, let's just generate it again. Like if I do this enough times, I'll eventually get to something that's workable. So do you have any thoughts there in terms of like how you don't end up in that spot where you're just like hoping for a better image?
You're kind of like pulling the AI slot machine, if you will?
Host
No, I mean, I think unless you kind of have evidence, I think it's why some of these other tools and communities are so important, where you see lots of other people's work, is if you can see someone else has done it, you can also see the prompt they use and work out how they did it. But even if not, then you're like, okay, I can get there. Also, you run into these things where you would think it's the most simple thing. And then you're like, it doesn't know what a hot dog is. It just doesn't understand the rules of physically what can and can't that look like. And you're like trying and it's like now the sausage is a right angle. The bun has ears because it's starting to throw in some like dashing like, you know, aesthetic. And then you're like minus minus no, no dashing. That is kind of the limitation of weather technology is at the moment, which is it's amazing until you're trying to do something very specific. And especially if you want to do something very specific, this also to a very high professional standard.
Well, I'm glad you even mentioned the negative queries. That's something I think a lot of people don't know is that you can say, hey, AI, I don't want this. It doesn't always manage to still generate what you're looking for. But there's also almost like these glitches. One of them that is kind of infamous now is hands, right? So you can generate these beautiful images of these Instagram looking models and you can put them in all these different backgrounds. And you're like, wow, this is amazing. And then it's always like, well, look at the hands, which is kind of funny. I feel like it's like the perfect manifestation of how technology always is like much better in one direction when it's invented. But there's always like these things that need to be iterated on. And so are there other things worth knowing about, whether it's these negative prompts, whether it's these glitches that are still in the matrix? What would you call out from your, again, many hours of being deep in these tools?
Guy Parsons
I think it depends on the model. One example is when DALL-E came out, and it's still the case as part, I know, it's not very good at understanding that it's drawing things in a square. If you're drawing a person, it's often going to have like its feet and its head cut off because it's seeing those in portrait photos. But one thing you could do with DALL-E is you can actually upload like an image to like do variations of. And if you upload an image that's just like a little white border, then it knows that nothing can go there. That kind of encourages it, forces it to kind of think inside the box, if you will.
But then, of course, you have now tools like Midjourney who've been iterating on their text to image model a lot more aggressively than OpenAI, who understandably, I think maybe have some other things in the cooker, you know, which have grown that into the model itself. So when you type things in, it knows it's a square. And actually, it would sometimes do quite clever things in order to fit it in that space. So if you ask for kind of like a group selfie of three people on something like DALL-E, that's going to be cut off at the end because you're used to seeing someone taking like disposable camera photo.
Whereas Midgen is clever enough to know that one of them kind of needs to be standing behind the other or like leaning in from the side. So it's kind of clever how they've managed to like solve that composition problem within the AI, which then, you know, the prompt engineering thing, I think, is just understanding the possibilities and the limitations of where you are at the moment. Meanwhile, there's these other people who are doing some very technically serious work to kind of make those limitations kind of no longer relevant.
Host
Yeah. Well, I'm glad you brought up the differences between these different tools. So if we talk about just stable diffusion, mid-journey, and DALL-E, I feel like are three that a lot of people are familiar with. Yeah. Would you liken the ability to prompt within each of these more like the difference between Excel and Google Sheets, where if you know how to use Excel, you really can drop right into Google Sheets and it's relatively straightforward. You might have to switch up your shortcuts a little bit or learn one little thing here and there. But for the most part, you can again drop from one to the other. Or would you liken them more to learning to speak different languages? It's not that different. I think the principles are like very similar. And then the nuances of each are slightly different. So I think now if you went from DALL-E to MidJourney, it would be like amazing. And then if you went back in the other direction, you'd be like, it doesn't do what I want. But that's because Midjourney is doing so much of the heavy lifting to help you make something really good.
Guy Parsons
If you are using the tools to create some like very specific effect imagine that I guess yeah like a very complicated Excel formula that would not have the exact same output in the other if you know i mean because they trained on like a different set of images stable diffusion i think some five billion for the the what things look like learning and then like a smaller set of like 12 million for the what does nice look like and then the fine tuning that's happened on the top and how they've optimized it in the later phases with a technical element that escapes me slightly you know they have made different creative decisions there. And it's maybe like driving a different car. Like if you like floor the accelerator in various different cars, some are going to take off, some are going to trundle along. That's a good analogy.
Host
Do you also find that, I mean, we've talked already about this idea where sometimes it's pretty easy to get to that 80%, but then that final 20%, the real refinement to get to exactly what you pictured in your head or exactly what you want and didn't picture in your head sometimes requires another tool. And so have you found, I've heard some people are using Facetune or different AIs to take it to the final level, or I guess you could also use in-painting and out-painting a little more discreetly. So how have you found the relationship of maybe one tool to the suite of other tools that exist out there?
Guy Parsons
I think there's lots of exciting crossovers, but actually I kind of think it's a big opportunity for the Photoshop's of this world, because those are tools that presuppose you have some kind of original image to manipulate. Whereas now there's a huge amount of raw, but maybe not perfect material that for people to work with. There's lots of things also that I've been trying to do in prompting that actually more easily achieved in other tools. So you can, you know, spend ages trying to get this kind of vintage film look. But if you're like an Instagram influencer, which I'm sure you are, who isn't? But there's loads of there's loads of iPhone apps, right, that are out there just to like give all your photos that kind of like dreamy vintage film look. Yeah. I mean, I think back in July when you first wrote your prompt book, you had a requested feature list for DALL-E 2. But are there things that are on your new list of, hey, these tools are great, but they're missing XYZ or they're lacking in these areas? This would be top of my list to see improved on.
I think we're going to see more models come out. I mean, the fact that stable diffusion is open source means that lots of other things are going to be built on top of that. And I think it's going be really exciting to see some of the directions that people take that in either kind of on an individual sort of prosumer level people building their own models to create their own stuff more likely some bigger organizations training it for specific purposes the whole challenge and the whole opportunity i think at the moment is like how do you go beyond the text box how do you go beyond this like blank rectangle to create something that is more user-friendly that's more inspiring that's more how people think because on the one hand if you're not an artist the ability to describe things with words is definitely a big step forward but if you think about the next layer is it's still quite hard to describe things with words designers when they do work for clients like it's one of their pet peeves because clients don't like it but they can't explain why or what they want different they're like oh i want it to be more do you know what i mean like more and they're like i don't know i don't know what that means which is basically the position these you know ai models are in so could you see like a conversational interface can you do the generations fast enough that you're always showing people multiple options possible new directions almost like in a sort of multi-dimensional space where it's like do you want to take it more this way or more this way you know part of the prompt book is i didn't know what metaphysical painting or kodachrome all these things were but those at least have names but there's probably other aesthetics right other styles that we don't have what actually words for it's like you know that kind of gritty but like modern gritty like almost like shiny gritty like the grit has a shine on it and probably i can make you a mood board of that and you'd be like oh yeah like that's a thing but there's no word for it so if you can create ways of unleashing the inexplicable the undefinable that's the exciting thing about vision art is to express things or moods or things that you can't quite put into words.
Host
I totally have my mind spinning, thinking of different ideas. A couple of them that came to mind, one of them is just a better onboarding experience, but one where you're guiding the new prompter to understand how all these things might fit together to your point. Like, try this. Oh, look at what you got here. Oh, did you notice how when you use these two prompts together, this one kind of overshadows the other? Maybe there's a third word, that's a synonym of this. And I think you've kind of done this on your own by just going through and prompting like crazy, going through these different prompt libraries and trying to sort through the signal from the noise. But I do think any one of these models or maybe the UI built on top could have just a much better onboarding experience so that people come into the tool, to your point, with just a better understanding of like what they should be paying attention to.
And then I also, in terms of these visual styles, I mean, it reminds me of a lot of Instagram influencers for a period of time were selling these filters because they had figured out the precise tuning of every little variable, which sounds easy, but I had tried to do it myself. I never managed to create good Lightroom filters, but people had and they would sell them. And so I wonder if you'll see the same thing where maybe someone creates kind of like a zip file of a mood board and then they train the AI in some way that does make it, I guess, play nice with that particular concept that you can't distill necessarily into a single term.
Guy Parsons
Yeah, because you had that breakthrough. Someone did a paper on it. I think it's always kind of what led to that selfie craze, which was that you don't need to put you, photos and stuff in that original 600 million training data or wait for the next time we do that again for it to teach it what you look like there's this kind of embedding trick where you can show it like a bunch of pairs of you and then you can refer to you and it knows how to kind of recreate that and there was also an interesting thing in the same paper but hasn't really been used or like commercialized in the same way which is to do that with style so rather than show it yeah this is what this person looks like it's like this is what the style of blah blah blah is called here it is and then off you go which obviously has all kinds of potentially shady legal qualifications. But let's assume this is all lovely art we've made ourselves.
Host
Well, no, I mean, to the idea of honing in a style, I do wish there was a version of the product where I could go and like we've talked about, maybe upload certain brand images or certain brand colors, and then have it iterate with me where it shows me a bunch of images. And I say, it's okay, but I want a little more of this color. And then we keep doing that to the point where I get a bunch of images where I'm like, yes, this is the style. You can lock that in. You lock it into a variable that you can then plug into future prompts. I've definitely seen there's some people out there that have managed to lock in a particular look. And now every blog post they have is always the same kind of thing. And that's pretty cool. But we haven't seen that always built into the foundation models yet as a way of interacting with it. And then there are some startups like Scenario, which is doing it for game assets. And then Leonardo, which is like more multi-purpose, I think, or is just positioning itself that way, which is again all about, can you like control things down to like consistent look?
Yeah. Well, I'm glad you brought up the differences between these different tools. So if we talk about just stable diffusion, mid-journey, and DALL-E, I feel like are three that a lot of people are familiar with. Yeah. Would you liken the ability to prompt within each of these more like the difference between Excel and Google Sheets, where if you know how to use Excel, you really can drop right into Google Sheets and it's relatively straightforward. You might have to switch up your shortcuts a little bit or learn one little thing here and there. But for the most part, you can again drop from one to the other. Or would you liken them more to learning to speak different languages? It's not that different. I think the principles are like very similar. And then the nuances of each are slightly different. So I think now if you went from DALL-E to MidJourney, it would be like amazing. And then if you went back in the other direction, you'd be like, it doesn't do what I want. But that's because Midjourney is doing so much of the heavy lifting to help you make something really good.
Guy Parsons
If you are using the tools to create some like very specific effect imagine that I guess yeah like a very complicated Excel formula that would not have the exact same output in the other if you know i mean because they trained on like a different set of images stable diffusion i think some five billion for the the what things look like learning and then like a smaller set of like 12 million for the what does nice look like and then the fine tuning that's happened on the top and how they've optimized it in the later phases with a technical element that escapes me slightly you know they have made different creative decisions there. And it's maybe like driving a different car. Like if you like floor the accelerator in various different cars, some are going to take off, some are going to trundle along. That's a good analogy.
Host
Do you also find that, I mean, we've talked already about this idea where sometimes it's pretty easy to get to that 80%, but then that final 20%, the real refinement to get to exactly what you pictured in your head or exactly what you want and didn't picture in your head sometimes requires another tool. And so have you found, I've heard some people are using Facetune or different AIs to take it to the final level, or I guess you could also use in-painting and out-painting a little more discreetly. So how have you found the relationship of maybe one tool to the suite of other tools that exist out there?
Guy Parsons
I think there's lots of exciting crossovers, but actually I kind of think it's a big opportunity for the Photoshop's of this world, because those are tools that presuppose you have some kind of original image to manipulate. Whereas now there's a huge amount of raw, but maybe not perfect material that for people to work with. There's lots of things also that I've been trying to do in prompting that actually more easily achieved in other tools. So you can, you know, spend ages trying to get this kind of vintage film look. But if you're like an Instagram influencer, which I'm sure you are, who isn't? But there's loads of there's loads of iPhone apps, right, that are out there just to like give all your photos that kind of like dreamy vintage film look. Yeah. I mean, I think back in July when you first wrote your prompt book, you had a requested feature list for DALL-E 2. But are there things that are on your new list of, hey, these tools are great, but they're missing XYZ or they're lacking in these areas? This would be top of my list to see improved on.
I think we're going to see more models come out. I mean, the fact that stable diffusion is open source means that lots of other things are going to be built on top of that. And I think it's going be really exciting to see some of the directions that people take that in either kind of on an individual sort of prosumer level people building their own models to create their own stuff more likely some bigger organizations training it for specific purposes the whole challenge and the whole opportunity i think at the moment is like how do you go beyond the text box how do you go beyond this like blank rectangle to create something that is more user-friendly that's more inspiring that's more how people think because on the one hand if you're not an artist the ability to describe things with words is definitely a big step forward but if you think about the next layer is it's still quite hard to describe things with words designers when they do work for clients like it's one of their pet peeves because clients don't like it but they can't explain why or what they want different they're like oh i want it to be more do you know what i mean like more and they're like i don't know i don't know what that means which is basically the position these you know ai models are in so could you see like a conversational interface can you do the generations fast enough that you're always showing people multiple options possible new directions almost like in a sort of multi-dimensional space where it's like do you want to take it more this way or more this way you know part of the prompt book is i didn't know what metaphysical painting or kodachrome all these things were but those at least have names but there's probably other aesthetics right other styles that we don't have what actually words for it's like you know that kind of gritty but like modern gritty like almost like shiny gritty like the grit has a shine on it and probably i can make you a mood board of that and you'd be like oh yeah like that's a thing but there's no word for it so if you can create ways of unleashing the inexplicable the undefinable that's the exciting thing about vision art is to express things or moods or things that you can't quite put into words.
Host
I totally have my mind spinning, thinking of different ideas. A couple of them that came to mind, one of them is just a better onboarding experience, but one where you're guiding the new prompter