--:-- Erik Torenberg: So slow news day. Not much going on for you guys. Thank you for coming on.
--:-- Christina Kim: No, obviously, Tina, you were just on the live stream. We're recording day of. Congratulations.
--:-- Isa Fulford: Thank you.
--:-- Erik Torenberg: For those who are unfamiliar, why don't you introduce what you guys do at OpenAI?
--:-- Christina Kim: Yeah, I'm Christina. I lead the core models team on post-training.
--:-- Isa Fulford: I'm Isa. I lead the deep research ChatGPT agent team on post-training.
--:-- Erik Torenberg: And Tina, you've been here for a while now. Do you want to give a little bit of your history at the company?
--:-- Christina Kim: Yeah, I've been at OpenAI for about four years now. I originally worked on WebGPT, which was the first LLM using tool use, but it was just one question. The model learned how to use the browser tool, but you only ask one question and you get one answer back. Then we kind of had this realization: normally, when you have questions, you have more questions after that. So we started building this chatbot, and that's what eventually became ChatGPT.
--:-- Erik Torenberg: And what have been the reactions so far? It's only been a few hours, but in your live stream, what are any reflections? What can you tell us the day of?
--:-- Christina Kim: I'm honestly really excited. Obviously we have some great eval numbers, and numbers are always exciting. But the thing I'm really excited about with this model is that it's way more useful across all the things that people actually use chat for. The eval numbers look good, but also, when people use it, I think they'll notice quite a bit of a difference in the utility of it. For my personal use cases, I use it for coding and writing all the time, and it's just a huge step change.
--:-- Erik Torenberg: Sarah, you've been involved in helping lead our investment since 2021. Why don't you either share more or tee up how you've been thinking about this as it relates to coding or more broadly?
--:-- Sarah Wang: Yeah, actually, just on the topic of coding, it was a huge deal to have Michael Traul come on there and not only showcase the capabilities, but also say this is the best coding model in the market. So I'm curious, to the extent that you can share, what did you do differently to get these results?
--:-- Christina Kim: Yeah, huge shout out to the team, especially Michelle Pokris. To get these things right, eval numbers are one thing, but to get the actual usability and how great it is at coding, it takes a lot of detail and care. The team put a lot of effort into data sets and thinking about the reward models for this. It's literally just caring so much about getting coding working well.
--:-- Sarah Wang: Maybe to double click on front end web development. We've seen as investors in the ecosystem that that's obviously taken off in the last six to eight months. If you could pinpoint the improvement to that piece specifically, is it more around aesthetics, or is there another capability leap forward in terms of what we can do with front end web development?
--:-- Christina Kim: I think there's going to be a lot more we can do with front end. The way we've made this big leap—if you compare it to O3's front end coding capability, this is just totally next level. It feels very different. It goes back to what I was saying: the team really cared about nailing front end. That means getting the best data, thinking about the aesthetics of the model, and all of these things. All those details are really coming together and making the model great at front end.
--:-- Sarah Wang: Really exciting to see. Loved the demos in the live stream too. I wanted to ask about model behaviors, because I know you worked on that too. How did you guys think about that for GPT-5? There are a lot of things we've talked about in prior models, like sycophancy and characteristics like that. How did you think about it for this? What did you change or tweak?
--:-- Christina Kim: The design of this model has been very, very intentional for model behavior, especially with the sycophancy issues we had a few months ago with 4.0. We've spent a lot of time thinking about what the ideal behavior is. For post-training, one of the reasons I really like it is that it feels more like an art than other areas of research, because you have to make all these trade-offs. You have to think about all these different rewards you could be optimizing during the run and how they trade off against each other. I want the assistant to be super helpful and engaging, but maybe that gets too engaging and becomes the overly effusive assistant. It's really a balancing act of trying to figure out what characteristics we want this model to actually feel like. We were really excited with GPT-5 because it's a time to reset and rethink, especially since it's so easy to make something very engaging in an unhealthy way. How can we make this a very healthy, helpful assistant?
--:-- Sarah Wang: Say more about how you achieved such a reduction in hallucinations, but also deception. What's the relation between those?
--:-- Christina Kim: For me, hallucinations and deception are pretty related. We saw this a lot with the reasoning models. The reasoning model would understand that it didn't have some ability, but it still really wanted to respond. We baked it into the models that they want to be helpful, so they say whatever they can to be helpful in that moment. That's what we consider deception. Versus hallucinations, sometimes the model will just say something quickly. We see a lot of this reduction with the thinking. When the models are able to step by step, they actually can pause before blurting out an answer, which is what it felt like with a lot of the previous models for hallucinations.
--:-- Erik Torenberg: Over the next few weeks as you're evaluating usage, what are the biggest questions that you're having or that you're anticipating being potentially answered?
--:-- Christina Kim: I'm just really curious to see how all of these things reflect in usage. Coding is way, way better. What does this actually unlock for people? We're really excited to be offering these models at the price points that we have, because I think this actually unlocks a lot more use cases that really weren't there before. Maybe previous competitor models are good at coding, but the price point is not as exciting. With this number of capabilities in this model and the price point, I'm excited to see all the new startups and developers doing things on top of it.
--:-- Erik Torenberg: We're excited too. By the way, just on the topic of usage, you obviously have a lot of products with a ton of usage already. And since we have one of the deep research gurus here too, how did deep research, ChatGPT, Operator—your existing products—inform how you went about approaching GPT-5?
--:-- Isa Fulford: One thing that's interesting is that with reinforcement learning, training a model to be good at a specific capability is very data efficient. You don't need that many examples to teach it something new. The way we think about it on my team is we're trying to push capabilities in things that are useful to people. Deep research was the first model to do very comprehensive browsing. But then when O3 came out, it was also good at comprehensive browsing. That's because we're able to take the data sets that we've created for the frontier agent models and contribute them back to the frontier reasoning models. We always want to make sure that the capabilities we're pushing with agents make it into the flagship models as well.
--:-- Erik Torenberg: That's great. Very self-reinforcing. You mentioned all the startups that you're excited to see come. What do you think that could look like, or even just high-level opportunities you're more excited about because of this?
--:-- Christina Kim: People always say vibe coding. I think non-technical people have such a powerful tool at their hands. You really just need some good idea. You're not going to be limited by the fact that you don't know how to code something. You saw two of our demos, which were front-end coding. That literally took minutes. I think that would have honestly taken me a week to actually build fully interactive. I would expect maybe a lot more indie type of businesses built around this because you just need to have the idea, write a simple prompt, and then you get the full-fledged app.
--:-- Erik Torenberg: It's the world of the ideas guy.
--:-- Christina Kim: Yeah, I think so. Finally.
--:-- Erik Torenberg: How about in the broader AGI discourse? What does this mean or accelerate or not? How do we think about the broader AI discourse in terms of what GPT-5 means here or changes the conversation in any way?
--:-- Christina Kim: With GPT-5, it sets a new state of the art in all the things we talked about. But if you're showing that we can continue pushing the frontier here—and I feel like there are always people saying, "Oh, we're hitting a wall, things aren't actually improving"—the interesting thing is that I feel like we've almost saturated a lot of these evals. The real metric of how good our models are getting is going to be usage. What are the new use cases being unlocked? How many more people are using this in their daily lives to help them across multiple tasks? That's the ultimate metric I'm excited about for terms of whether we're getting to AGI.
--:-- Sarah Wang: Greg made this comment about how he was comparing the last model to this model and the benchmark went from 98 to 99. Clearly we've saturated the benchmarks, at least on that front, which I think is instruction following. What benchmarks do you pay attention to? How do you guys think about evals? Given you're already saturating what's out there or doing very well along those dimensions, what actually gets you to push the frontier? Usage would be post the model release, but before you get there, what are you looking to internally to help guide you? Is it a lot of internal evals you created? Is it early access to startups seeing what they think? How do you weigh all those things?
--:-- Isa Fulford: On our team, we really work backwards from the capabilities we want the models to have. Maybe we want it to be good at creating slide decks or good at editing spreadsheets. If evals for those things don't exist, we try to make evals that are representative measures of that capability in a way that's actually going to be useful for users. A lot of those are internal. We'll collect them from human experts or try to synthetically create examples, or we'll actually look at usage data. Then we'll just try to hill climb on those.
--:-- Christina Kim: We make this joke a lot internally that if you want to nerd-snipe someone into working on something, you just need to make a good eval, and then people are going to be so happy to try to hill climb that.
--:-- Sarah Wang: I like what you said about starting with the capabilities first. How do you prioritize which ones you're actually shooting for? Let's say there's this dimension of deeper into everyday use versus getting much deeper into the expert use cases. How do you think about that trade-off? What does that trade-off mean practically, and what do you prioritize when?
--:-- Isa Fulford: I think it's pretty unique at OpenAI to be able to work on something that's so generally useful. It's like everything they tell you not to do at a startup: your user is anyone. For deep research, we wanted it to be good across every single domain someone might want to do research in. You only have the privilege of doing that if you work at a company that has huge distribution and all different kinds of users. If you choose a capability that's quite general, like online research, you just have to make sure you represent a distribution of tasks across loads of different domains if you want to get good at all of them. Sometimes it's hard to decide to focus on one specific thing because there are so many different verticals you could choose from. In some cases, coding will be really important, so a specific team will focus on coding. But in general, because the capabilities are so general, the next model improvement just kind of improves performance on a pretty broad range.
--:-- Christina Kim: We've seen this with the progression of the models we've had in ChatGPT. As the model gets smarter, it's better at instruction following. It's better at tool use. More things get unlocked as we continue to make smarter models. A good chunk of our team focuses on just getting general intelligence up, because the wins we get from there are pretty great. Whenever we get a new base model, it's like, "Oh, wow, suddenly this clicks. It works." We saw that moment with Operator, because we had been working on computer usage, but it was hard to finally get the model to—without the multimodal capabilities to really support it—you couldn't have something like Operator when it launched.
--:-- Isa Fulford: It's the same thing. Everyone was talking about agents, but we didn't really have a way of actually training useful agents. There were all these agent demos, but nothing that actually really worked. When we saw the reinforcement learning algorithm working really well on math and physics problems and coding problems, it became pretty clear from reading through the chain of thought: "Okay, this thing is actually thinking and reasoning and backtracking." To build something that's able to navigate the real world, it also needs to have that ability. So we realized, "Okay, this is a thing that's going to actually let us get to useful agents." It's interesting at OpenAI because you have people pushing foundational algorithms, getting really good at math, getting a gold medal in the IMO. Then on post-training, we'll often take those methods and try to figure out how to make things that are most useful and usable to all of our users.
--:-- Erik Torenberg: How much of the improvements are coming from the architecture versus the data versus the scale? How do you think about that?
--:-- Christina Kim: My opinion: I'm very data-pilled. I think data is very important. Deep research was so good because Isa put so much thought and careful attention into the data curation and thinking about all the different use cases she wanted to have represented. So I'm on team data.
--:-- Isa Fulford: I think all are very important, but especially now that we have such an efficient way of learning, high-quality data is even more important.
--:-- Erik Torenberg: Maybe on the data topic, we've been talking a lot about RL environments. It's a popular space for startups who all want to work with you guys. I was curious to get your thoughts on this since you're data-pilled. What are the bottlenecks that you see for the next stage? Is there a lack of good, realistic RL environments that that's the next frontier, which creates an opportunity for these startups? Once you can really work within an environment that takes a long time to build, you can actually automate labor to the full extent that you would need computer use to do?
--:-- Isa Fulford: I do think there's a lot of value in getting really good tasks, and getting really good tasks requires really good RL environments. The more complicated, the more realistic, the more simulated we can make them, the better we'll get. We're seeing that tasks matter more at this point, given that we have such a strong algorithm. Creating data and figuring out the best tasks to train on is one of the big questions we have.
--:-- Christina Kim: There's some generalization from training on one website to another, but if you want to get really, really good at something, the best thing to do is just train on that exact thing. We're definitely constrained by things that we can represent in a way that we can train on.
--:-- Isa Fulford: The ChatGPT agent, for example, has such a general tool. It has a browser and a terminal. Between those two things, you can basically do most of the tasks that a human does on a computer. In theory, you can ask it to do anything that you can do on your computer. It's obviously not good enough to do that yet. But with the tools it has, in theory, you can push it really, really fast. Now we just have to make it really good at all those things by training on way more things.
--:-- Erik Torenberg: Let's talk about creative writing. Talk about the improvements there, how you think about it.
--:-- Christina Kim: That's one of my favorite improvements in GPT-5. The writing, I honestly find it very tender and touching, especially for a lot of the creative writing that we want to do. We were thinking through a bunch of different samples for the live stream, and every time I was like, "Oh, that actually hits. It's spooky." I'm like, "Oh, this feels like someone should have written this." It's really cool because you can actually really use it for helping you with things. My example in the live stream was helping me write a eulogy, something that's kind of hard to write, especially since writing isn't really something a lot of people are good at. I'm personally a very, very bad writer.
--:-- Erik Torenberg: That's not true.
--:-- Christina Kim: I think it makes a better story. Compared to the other things I'm better at. But it's so great to have this tool to help me craft. I use it for as simple things as Slack messages to figure out how to phrase this well. It'll help me give some iterations on how to say something.
--:-- Erik Torenberg: I want to see those prompts. We're now all just looking for em dashes.
--:-- Sarah Wang: That was going to say. We're like, "Mm, ChatGPT."
--:-- Erik Torenberg: Where do you stand on the em dash discourse?
--:-- Christina Kim: I like em dashes. I do that normally now. People think I'm just using ChatGPT.
--:-- Sarah Wang: I know. Me too.
--:-- Erik Torenberg: Going back to the discourse for a second. Sam said in his interview with Jack, he said, "If you had said 10 years ago that we would get models at the level of PhD students, I would think, 'Wow, the world looks so different,' and yet we've basically taken it for granted." Do you think the improvements are similar? As soon as we get them, we're just going to be like, "Oh, now this is the standard"? Or do you think at some point there's going to be like, "Oh my God, this is..." How do you think about people's ability to acclimate or adjust?
--:-- Christina Kim: People adjust really quickly, don't you think? After GPT-4 got released and everyone was like, "Wow, that's so cool," but then you just kind of take it for granted that you literally have this wizard in your pocket. You can ask it whatever random thought you have, and it just pops out a good essay, and you're like, "Oh, okay, cool. That's what's happening." People adapt to things rather quickly with technology. Because the form factor is so easy, even with new tools like Deep Research and ChatGPT Agent, it's presented in such an easy way that people already know how to interface with it. As long as that's true, even with the models getting much smarter than us, it's still going to be quite approachable to people.
--:-- Erik Torenberg: Do you think the jump from GPT-4 to 5 was bigger or 3 to 4? Or maybe 3.5 to 4?
--:-- Christina Kim: At least one thing for me in my usage is sometimes I'm wondering if I have hard enough questions to ask it to actually highlight the difference. When it gets to a point where it's just answering what you need so well, it's almost harder to tell the difference in some areas. But with writing, I've been using it for a few weeks and it's just kind of blown me away in a way that models previously haven't. Maybe I'm biased by recency bias, but I think the jump from 4 to 5 is most impressive for me. With 3.5 when we first released it, the most common use case for me then was still just for coding. But even though 4 was better at coding, I feel like the jump between 4 and 5 in terms of breadth of ability to do things is just way different. You can handle a lot more complex things than before, with the context length being much longer as well. The jump from 4 to 5 to me is much bigger.
--:-- Erik Torenberg: Is there anything the model categorically can't do?
--:-- Christina Kim: For 5, we don't really take actions in the real world yet. We're going to team up with Agent for that.
--:-- Isa Fulford: As I said, you could ask the agent to do anything, but it's not capable enough to do everything you want it to do yet. We take a conservative approach, especially with asking the user for confirmation before doing any kind of action that's irreversible, like sending an email or ordering something or booking something. I can imagine quite a number of tasks where you'd want to take bulk actions which you might not be able to do right now because it would ask you every single time. But as people get more comfortable using these things and as they get better and you trust them more, you might allow it to do things for you without checking in with you as much.
--:-- Sarah Wang: Maybe to build on that question, in terms of what it can't do today but what you would direct future research toward. If you look at coding, something like end-to-end DevOps feels like the logical next set of capabilities. Do you think we'll get there in 5.5 or GPT-6? How far are we from something like that?
--:-- Christina Kim: I don't know about the exact thing of DevOps, but I do feel that with the models getting much smarter, one other thing that came to my mind when you asked that question is longer running tasks. GPT-5 is great because within a couple of minutes, maybe you get a full-fledged app. But what would it look like if you actually gave it an hour, a day, a week? What can it actually get done? There's going to be a lot of interesting stuff we're interested to see.
--:-- Isa Fulford: A lot of it is not just about the model capability, but actually how you set it up in a way to do things. I'm sure you could build something that's monitoring your Humio or Datadog, whatever. With these current models, it's just setting up the harness to make that possible. Same for agentic tasks. A lot of things that will be quite useful will be when the agent proactively does something for you, which I don't think is impossible today. It's just not set up that way. But eventually, as it proactively does things for you, we might get feedback on whether that was useful and we can make it even better at triggering.
--:-- Erik Torenberg: "Agents" is probably the most overused word of 2025. That being said, your agent's launch was extremely exciting. What does that word mean to you in the context of capabilities you'd like to build in the near term or have already built? What is most important that the agent is able to do on behalf of your users?
--:-- Isa Fulford: My very general definition would be something that does useful work for me on my behalf, asynchronously. You'd kind of leave it and then come back and get either a result or a question about what it's doing. In terms of roadmap for agents, longer term, you want it to be able to do anything that a chief of staff or assistant would do for you. In the more immediate term, there are a lot of new capabilities we launched in ChatGPT Agent that we just want to improve. One of the main capabilities is deep research—being really good at synthesizing information from the internet. But we can also improve capabilities on synthesizing information from all the services you use and private data you have. Also, being better at creating and editing artifacts like docs, slides, and spreadsheets, because so much of the useful work people do in their jobs is basically research and making something. I personally love all the consumer use cases—making it better at shopping or planning a trip. Those things are also really fun. That involves taking an action, which is interesting because it's often the last step of a task. It's maybe a task that would take less time for a human, and it's actually a very hard research question to get it to do something or book something or use a calendar picker. Once you have the end-to-end flow working really well, it can basically do anything.
--:-- Erik Torenberg: That's incredible. On the shopping piece, I now do not make a single large-ticket purchase without having ChatGPT put all the options in a table for me along the dimensions I care about. It's incredible. But I want to push on the async piece. I don't know if you would agree with this, but it felt like a revelation to me at the beginning of the year that people were willing to wait. You think about, "Oh, we want it faster. The value prop of this tool is that it gives me the answer fast." That was very 2024. Clearly, this paradigm has shifted. People are willing to wait for high quality, high value answers and work. How do you think about the trade-off between how long something takes to get back to the user versus the value you're providing? What do you think is the ideal frontier for something like that?
--:-- Isa Fulford: Yeah, it's interesting because I worked on building the retrieval on ChatGPT and was on the browsing team before this. Tina was also on the browsing team. And we were always making these trade-offs and optimizations for latency. We were thinking about how you can best fill the context with information you've retrieved so that the answer is pretty good in a few seconds.
--:-- Isa Fulford: With deep research, I was just very excited to remove latency as a constraint. Since we're going for tasks that are really hard for humans to do and would take humans many hours, we felt that if you asked an analyst to do something and it would take them 10 hours or two days, it seems reasonable that someone would be willing to wait five minutes in your product. So we just made that bet. Luckily, it seems like it's the case.
--:-- Isa Fulford: But I do also think that initially people were like, "Oh, this is amazing. It's doing all this work that would have taken me so long." And now people are like, "Okay, but I want it now. I want it in 30 seconds." The bar keeps changing.
--:-- Erik Torenberg: Is there any sort of rule of thumb where as long as you're 10 times faster than it would take the human to do, they're willing to wait for it? Or is that just constantly shifting sand?
--:-- Isa Fulford: I think with these launches, people's expectations keep getting changed. One thing that's interesting is I think sometimes people just bias to thinking that a longer answer is more thorough or that more work was done for it, which I don't necessarily think is the case. Deep research, for example, always gives you a really long report. But sometimes for me, I don't want to read this whole long report. I actually don't like that.
--:-- Isa Fulford: Agent will only give you a long report if you ask for it. But now people are used to always getting a really long report, so they're like, "Wait, I've been waiting. Where's my long report?" But sometimes it's really hard to find a specific piece of information and it would have also taken a human a long time because it's on page 10 of the results where it finds this information.
--:-- Isa Fulford: It's interesting how you can condition people's expectations with a product. With deep research, it always thinks for a really long time, which again, I don't necessarily think is a feature, but now people are really used to the amount of time that they wait.
--:-- Christina Kim: Definitely. I think we hear this with GPT-5 internally when people are testing. They're like, "Oh, I thought I asked a really hard question. I feel a little bit insulted that it got back in two seconds." Or when it doesn't even want to think at all. It's the Mark Twain line: "I didn't have time to write you a short letter, so I wrote you a long one."
--:-- Erik Torenberg: Why don't you talk about the bottlenecks? Why don't we have reliable agency? What are the main bottlenecks as you see them?
--:-- Isa Fulford: I think a big part of it is the things that we train on are often really good. And then with things outside of that, sometimes it's good at those things, sometimes it's not. So creating more data across a broader range of things that we want it to be good at is key.
--:-- Isa Fulford: What's interesting with agents is that when something is doing something on your behalf and it has access to your private data and the things that you use, it's kind of more scary — the different things it could do to achieve its final goal. In theory, if you asked it to buy you something and make sure that you like it, it could go and buy five things just to make sure that you liked one of them, which you might not necessarily want. So having oversight during training is also an interesting area. There are new things that we have to develop to push these agents even further.
--:-- Isa Fulford: Every time we have a smarter base model, it improves every model that's built on top of that. That will also help, especially with multimodal capabilities, as Tina said, with computer use, because it's literally looking at screenshots of a web page. It's a little interesting because the way that humans focus on specific things, it's a lot to expect a model to just take a whole image and be able to know everything about it. When we're looking at something, we'll focus on a specific thing. There's lots of room for improvement in lots of areas.
--:-- Erik Torenberg: That last example gets into something we were curious about, which ties back to training data. What specific categories of browsing tasks are challenging for agents today? And how would you overcome this for the next iteration of the model?
--:-- Christina Kim: Pre-training is based on what data is available. When we've done these pre-training runs, there's not much data out there to begin with of people using computers. Computer usage is not really a thing that has lots of data out there. This is something we actually have to seek out now that this is a capability we want. So that's probably a big one for general improvements of computer usage.
--:-- Erik Torenberg: Do you think you'll lean more heavily on human data vendors to help collect that? Given it doesn't exist recorded in the way that's most helpful for training, how do you overcome that? It is probably the most useful application of the models to knowledge work.
--:-- Christina Kim: One cool thing is that for initial deep research, there weren't really any data sets that exist for browsing in the same way that you have a math data set that already exists. So we have to create all this data. But once you have good browsing models or good computer use models, you can bootstrap them to help you make synthetic data. So that's a pretty promising area.
--:-- Erik Torenberg: Christina, can you explain what mid-training is and what it achieves that pre or post doesn't?
--:-- Christina Kim: With your pre-training runs, these are the big runs — the massive ones, what we're building all these giant clusters for. You can think of mid-training as literally for the middle. We do it after pre-training but before post-training. It's a way to extend the model's intelligence without having to do a whole new pre-training run. This is mostly focused on data and off of the pre-training models.
--:-- Christina Kim: This is a way for us to do things like updating the knowledge cutoff of these models. When you pre-train it, you're kind of stuck in that date and you can't ever update it again. It doesn't quite make sense to put all that data into post-training. So mid-training is just a smaller pre-training run to help expand the model's intelligence and up-to-dateness.
--:-- Erik Torenberg: Christina, did you work on WebGPT?
--:-- Christina Kim: Yes, I did.
--:-- Erik Torenberg: So you're basically an AI historian.
--:-- Christina Kim: Yes. She also works on Confucius. I'm an elder.
--:-- Erik Torenberg: Can you reflect back to four or five years ago and reflect on the biggest things? If you were to predict five years out, what are the inflection points or biggest things that would have surprised you?
--:-- Christina Kim: Honestly, with WebGPT, the main thing we were excited about was trying to ground these language models. We had so many issues with hallucinations and the model just saying random things. We didn't really do mid-training then, so the question was: how do we make sure the model is actually up to date, most factually up to date? That's how we thought about giving it a browsing tool.
--:-- Christina Kim: That went on to, "Oh, I actually want to keep asking questions," so that's what a chatbot would look like. At that point, there had been a few chatbots by a few other companies, and a chatbot is a very common AI thing to think of. But they were quite unpopular at the time, so we weren't really sure if this was actually something useful for people to work on, or if people would use it, or if people would be excited about it. Was this really a research innovation or were we remaking the Turing test?
--:-- Christina Kim: I think it clicked for me that maybe there was actually something interesting happening here. We gave early access to about 50 people, most of them being people I lived with at the time. Two of my roommates just used it all the time. They would never stop using it. They would have these long conversations and ask it quite technical things because they're also AI researchers. So I was like, "Oh, this is kind of interesting."
--:-- Christina Kim: At the time, we were thinking, "Okay, we have this chatbot. Do we make it a really specific meeting bot? Do we make it a coding helper?" But it was interesting to see my two roommates use it for anything and everything and literally be chatting with it the whole workday. But then it was also interesting to see that the majority of the people on that 50-person list didn't really use it that much. I was like, "There's clearly something here, but it's not quite for everyone yet."
--:-- Erik Torenberg: When did you realize you're working at one of the most important companies of this generation? When was the moment where you realized the scale and significance?
--:-- Christina Kim: Honestly, I had this moment before I joined OpenAI. With the scaling laws paper and GPT-3, it just kind of hit me that if this exponential is true, there's not really much else I want to spend my life working on. I want to be part of this story. I think there are going to be so many interesting things unlocked with this. This is probably the next step level in terms of technology. It made me realize I should go start reading about deep learning and figure out how to get into one of these labs.
--:-- Erik Torenberg: Isa, what was your moment?
--:-- Isa Fulford: For me, it was also before I started working at OpenAI. I first learned about OpenAI in an AI class or some kind of computer science class. They said, "Oh, they trained on the whole internet." I thought, "That's so crazy. What is this company?" Then I started using GPT-3. I was a power user of the OpenAI Playground. At a certain point, I had early access to different OpenAI features, like embeddings and things like that. I became this big OpenAI fan, which is a little embarrassing, but it's fine because it got me here.
--:-- Isa Fulford: Eventually they were like, "Okay, you're stalking us. Do you want to interview here?" It was pretty clear to me just how much I was using GPT-3, which pales in comparison to what we have now. But from then I was hooked and just trying to figure out a way to work here.
--:-- Erik Torenberg: Maybe a question on the company building front. We all read and reread Calvin French-Owen's piece, his reflections on working at OpenAI. I'd love your reflections on the change you've seen over the last four years. What are the biggest things that have changed at OpenAI?
--:-- Isa Fulford: When I first joined OpenAI, the applied team was 10 engineers or something. We didn't really have this product arm. We had just launched the API. It was a completely different world. I think AI is in most people's mind now after ChatGPT, but pre-ChatGPT, people didn't really know what AI was or think about it as much. It's cool working in a place where my parents know what I do now. That's really cool.
--:-- Isa Fulford: The company is obviously a lot bigger. But with that, we can take a lot more bets. When I first joined, it was much smaller — around 200-ish people. Now we're close to a few thousand for sure.
--:-- Christina Kim: When I joined, it was also a few hundred before ChatGPT. It's obviously very different in how all of your friends have heard of what you work on. Culturally, the company is much bigger, but I still think we've maintained that it still feels very much like a startup. Some people who come from a startup are surprised — they're like, "Oh, I'm working even harder than when I was working on the startup that I founded."
--:-- Christina Kim: Ideas can still come from anywhere. If you take initiative and want to make something happen, you can. It doesn't really matter how senior you are. We've been able to maintain that culture, which I think is pretty special.
--:-- Isa Fulford: Yeah, we definitely reward agency. That's always been true. Especially on the research side, the teams are quite small. When Isa was working on deep research, it was like two people still. Most research teams are quite small and nimble for that reason.
--:-- Erik Torenberg: Earlier you said that OpenAI does something that startups never do, which is try to appeal to every single person with the product. Are there other things that come to mind that OpenAI does differently than peers or startups that we may not appreciate from the outside?
--:-- Christina Kim: It's different for different teams, but my team collaborates so closely with the applied engineering team, the product team, and the design team. Sometimes research can be quite separate from the rest of the company, but for us it's so integrated. We all sit together. Sometimes the researchers will help with implementing something. I'm not sure that engineers are always happy about it, but we'll try. They'll get out of the front end code. And vice versa, they'll help us with things for model training runs.
--:-- Christina Kim: Some of the product teams are quite integrated. For post-training, it's a pretty common pattern, which I think lets you move really quickly.
--:-- Erik Torenberg: One thing that I think is unique about OpenAI is that you're both very much a consumer company by revenue and products, but also an enterprise company. How does that work internally? What would you consider yourselves? Or is that even the wrong paradigm to think about?
--:-- Isa Fulford: If you tie it to the mission, we're trying to make the most capable thing and also make it useful to as many people as possible and accessible to as many people as possible. In that framing, it makes a lot of sense.
--:-- Erik Torenberg: The concept of taste has become very widely used. What does good taste mean within OpenAI? How do you know it when you see it? And in a world where the cost to produce everything keeps going down, is that the one thing that's not commoditizable, or is that also shifting given it can go into the training data?
--:-- Christina Kim: No, I think taste is quite important, especially now that our models are getting smarter and it's easier to use them as tools. Having the right direction matters a lot now — the right intuitions and the right questions you want to ask. I would say it matters more now than before.
--:-- Christina Kim: I've been surprised by how often the thing that is the most simple and easy to explain is the thing that works the best. It sounds very obvious, but it's quite hard to get the details of something right. Usually good research or taste is just simplifying the problem to the dumbest thing or the most simple thing you can do.
--:-- Isa Fulford: With every research release we do, when people figure out what happened there, they're like, "Oh, that's so simple. Obviously that would have worked." But it's knowing to try that obvious thing that at the time wasn't obvious — it's obvious in hindsight. All the details around the hyperparameters and the infrastructure, that's obviously very hard, but the actual concept itself is usually pretty straightforward.
--:-- Erik Torenberg: Very cool. Taste is Occam's razor.
--:-- Erik Torenberg: In closing here, obviously it's historic today. Can you contextualize what this means in the context of the mission and where you've been to get to now, to where you're going?
--:-- Isa Fulford: With GPT-5, the word that's been in my mind throughout all of this is "usable." The thing we're excited about is getting this out to everyone. We're excited to get our best reasoning models out to free users now. Getting our smartest model yet to everyone. I'm just excited to see what people are going to actually use it for.
--:-- Erik Torenberg: That's a great place to wrap. Tina, Isa, thanks so much for coming on the podcast.
--:-- Christina Kim: Thank you.
--:-- Isa Fulford: Thank you for having us.
--:-- Host: Thanks for listening to the a16z podcast. If you enjoyed the episode, let us know by leaving a review at ratethispodcast.com slash a16z. We've got more great conversations coming your way. See you next time.
