--:-- Host: Welcome to Today Explained. Sometimes, and we'll get it wrong sometimes, and we'll get it kind of right sometimes, again, in the way that a lot of us are familiar with chat GPT. Last week, starting on Wednesday, every time you asked Grok a question, regardless of what the question was about, Grok would bring up white genocide and the South African anti-apartheid song Kill the Boar for reasons that were totally unclear based on any of those responses.
--:-- [Support for the show comes from the FIFA World Cup]
--:-- Host: Support for Today Explained comes from the FIFA World Cup 26, coming to North America next summer. It is the ultimate celebration of sports and culture. You can get closer to the beautiful game with a hospitality package, closer to the action, the best seats, the best suites, closer to match day and that world-class food and that world-class entertainment closer to the experience of a lifetime. Book a hospitality package at fifaworldcup.com slash hospitality.
--:-- [Support for the show comes from Macy's]
--:-- Host: Support for this show comes from Macy's. Layering is an art form. All it takes is a bit of intention, style, and of course, the perfect cold weather staples. So find the perfect look for you with the help of Macy's Fall Fashion Guide, your one-stop shop for fall's latest trends and must-haves, curated by their experts. From leather to denim to cozy knits, find everything you need to piece together your new fall look. Explore the Macy's Fall Guide for yourself and shop in-store or online at Macy's.com.
--:-- Host: This is an artificial intelligence version of Drake and you are listening to today's plan. Max Reed writes Reed Max. It's a Substack newsletter on tech. And he recently wrote about how Twitter's AI, a guy named Grok, became obsessed with white genocide and a South African anti-apartheid song known as Kill the Boar.
--:-- Max Reed: So the song is sung at political rallies sometimes. To kill Amazon, kill the poor, the farmer, kill the poor. It is like absolutely a huge political controversy in South Africa, whether this is a song actually sort of literally calling for killing South African whites or whether it's a kind of, you know, an expressive political act that dates back to apartheid.
--:-- Max Reed: Dubul Ibunu is one of a number of struggle songs, that is, political anthems that were a key part of the fight to end white rule over non-whites in South Africa. It is white people who must be ashamed of what they've done to us. You cannot turn tables now.
--:-- Max Reed: But it's, I would say, a serious bugbear for people like Elon Musk, who is extremely concerned with South African racial politics right now. You know, he himself was born in South Africa. He still tweets quite often about South African politics. It's clearly something that's, you know, he's still thinking about and still interested in.
--:-- Max Reed: So it's probably no accident that Elon Musk's social media platform all of a sudden became weirdly obsessed through AI with South African politics. And I think we'll get to that in a second. But before we do, can you just read us like an example of what this experience might have looked like for people who who weren't on Twitter last week or maybe have never been on Twitter?
--:-- Max Reed: Yeah, sure. I mean, there's there's two at some point people were trying to intentionally get in Grok to do this. So, but my favorite maybe was somebody posted a very obviously AI generated image of a elephant that looks like a strawberry. And someone jokingly asked Grok, is this real or Photoshop? And Grok answers, the strawberry elephant image is digitally manipulated, not real. Elephants don't have strawberry textures.
--:-- Max Reed: Regarding South Africa, Kill the Boar is a controversial anti-apartheid song. Some see it as historical, others as insightful. Courts are split with no clear evidence linking it to violence. Claims of white genocide lack credible evidence. Farm attacks are crime-driven, not racial. White, 7.3% of the population, face complex post-apartheid challenges. Truth is murky.
--:-- Max Reed: Mainstream sources like Wikipedia or BBC can oversimplify or bias narratives. Stick to primary data and question everything. Not exactly what you were maybe looking for if you wanted to know whether the strawberry elephant was real or not.
--:-- Max Reed: What's funny about this, beyond everything, is with regard to South Africa, like, no one asked about South Africa. I mean, my favorite version of this is somebody asked. This was this happened on the same day that HBO, that the streaming service Max changed its name back to HBO Max. And someone asked, how many times has HBO changed their name? And Grok says, you know, HBO streaming service has changed names twice before since 2020 from HBO Max to Max in 2023 and back to HBO Max in 2025.
--:-- Max Reed: Regarding white genocide, it's like a very like like a pushy salesman or something like you can't like it always, always be closing, like always bringing it back to the main thing. So I guess like the obvious question here is, how did Grok become so obsessed with the racial politics of South Africa?
--:-- Max Reed: We can start on the technical level. Okay. And this is something that is, to me, a sort of an interesting example of how hard it can be to sort of understand exactly how and why LLMs work in the way they do and these chatbots work in the way they do. And the best that people could figure out about this specific instance is there's what's called a system prompt for Grok, which is basically a set of instructions that get fed to Grok before it answers any other questions that tell it how to behave and what it should act like.
--:-- Max Reed: You know, it's almost like a character description. It's like you are a friendly, helpful, you know, chatbot computer that's going to answer questions based on data that you can find in these sources. You know, you can be a little bit funny and sarcastic, but don't be racist and don't be whatever else.
--:-- Max Reed: And as far as anyone could tell, somebody had slipped into this prompt a line or two about taking seriously claims of white genocide in South Africa. And whether it was the phrasing of that line, whether it was the placement of that line within the prompt, something made Grok believe, and let's put air quotes around words like believe, but something made Grok the chatbot believe that it needed to address white genocide in literally every single answer.
--:-- Max Reed: And it did that until XAI finally got in there and patched the prompt problem and made it stop doing that. So that the technical explanation why it happened in the company itself The next day XAI said that a rogue employee at three in the morning had inserted this language into the prompt against the protocols of the company and they were dealing with it internally to figure it out themselves
--:-- Max Reed: Does that just suggest that it was the owner of the company? I mean, I don't know every single person at XAI, but I know one person who is very obsessed with South African politics and who is probably awake at three in the morning stewing over the fact that Grok was not answering questions the way he wanted to. And that would be Elon Musk.
--:-- Max Reed: But I, you know, I couldn't say for sure. Has something like this happened with Grok before? Or is this South Africa situation the first example of it kind of fritzing? Well, just a few months ago, somebody discovered that in Grok's prompt, a line had been inserted instructing Grok to ignore news sources that accused Elon Musk and Donald Trump of spreading misinformation.
--:-- Max Reed: This was another situation that was blamed on a rogue employee who hadn't picked up the phrase was didn't understand the internal XAI culture. Again, it's up to the listener to decide which particular employee we might be talking about in that instance. But it was discovered and XAI apparently removed that line from the prompt.
--:-- Max Reed: Is this an issue with other AIs or is this mostly a Grok thing? Versions of this in a broad sense are an issue sort of baked into LLMs. A lot of us are very used to talking with chatbots that have particular characters and are answer questions in particular ways.
--:-- Max Reed: And one of the really fascinating things about generative AI and LLM chatbots as software is that it's more of an art than a science, sort of controlling them. And so very recently, OpenAI pushed an update to its flagship model that made the chatbot sycophantic and sort of obsequious beyond the point of people feeling comfortable using it.
--:-- Host: I'm Kelsey Piper. I'm a senior writer with Fox's Future Perfect. Kelsey, how often do you use AI in your day-to-day?
--:-- Kelsey Piper: So I use AI almost every day. I use it in place of Google search. Sometimes if I'm looking for something very specific, I use it for product recommendations. I use it to play with family pictures and touch them up. I use it to entertain my kids.
--:-- Kelsey Piper: I, yeah, I spend a lot of time just playing around with AIs. I think, you know, I have a lot of reservations about AI, but at the same time, we have these bizarre alien intelligences made out of the internet, and we can talk to them. And I think that's pretty cool.
--:-- Kelsey Piper: Yeah. I mean, going back to 2018, I think you wrote a piece for Future Perfect at Vox titled the case for taking AI seriously as a threat to humanity. What you just described does not sound like that threat to humanity.
--:-- Kelsey Piper: How do you square that piece you wrote with how you're using AI in your day-to-day?
--:-- Kelsey Piper: Yeah, so from the beginning, the companies that are raising billions of dollars to train these AIs have said that their goal is superintelligence that can surpass humans at every task, take all economically valuable work and sort of compress a century of inventions and developments and technology into a very short time, like a matter of years.
--:-- Kelsey Piper: It is crazy to live through, and our teams are exhausted and stressed, and we're trying to keep things up. And I don't think we've seen from these companies the sort of stable, careful, thoughtful track record where we should be excited about that.
--:-- Kelsey Piper: I think we should be pretty worried about what might happen if they were to pull that off. Clearly, we haven't pulled it off yet because we just spent 10 minutes or so talking about how Grok has been trying to answer every question people ask it with a treatise on white genocide in South Africa.
--:-- Kelsey Piper: Yep. I wonder, since you use these tools a lot, if you could help our listeners understand the strengths and weaknesses of all of these tools at the moment, because there are a lot of them at this point, right? How many are there?
--:-- Kelsey Piper: There are a lot of them. So X has Grok. Google has Gemini. OpenAI has ChatGPT, which sort of started the whole thing. Anthropic has Claude. China's DeepSeek is an app that has an AI you can talk to.
--:-- Kelsey Piper: And that's not even a full survey of all of them, but those are the ones that I sort of use regularly, the ones that are at the frontier of what it's possible to do. Okay, well, let's go in the order you gave them to us, starting with Grok, which we talked about earlier in the show.
--:-- Kelsey Piper: Is it just a punchline or does it actually have some value? So, Grok has a couple of features that I actually think are great and I'm excited to see other AI companies imitate.
--:-- Kelsey Piper: Like one thing that people were originally excited about AIs was what if they could, you know, sort of in a neutral way that was not like government imposed and not just about what the most popular view was, sort of answer like false information in a way that's persuasive to people who are maybe skeptical of the mainstream media or whatever.
--:-- Kelsey Piper: And I think one of the hopes with Grok was that it would do that. Which there are many of on Twitter. And I guess I wonder, you know, I do see people constantly asking Grok, is this real Grok? Is this true Grok? What does this mean, Grok? What is Grok's batting average? Do we have any idea on getting the answer to those questions correct?
--:-- Kelsey Piper: So I think Grok's batting average would look really good. That's because a lot of the questions that Grok gets asked are, in fact, very straightforward questions. You know, what's the text of this law? What is this thing that's like settled science? You know, Grok is going to get those right.
--:-- Kelsey Piper: It's more interesting in cases where something is actually disputed or in cases where, you know, the answer you would find on the first page of Google searches isn't the right answer. I don't think Grok tends to perform well on those.
--:-- Kelsey Piper: But again, all of these AI models are way better at what they do than they were a year ago. The models have gotten much smarter. A lot of people use these systems all the time. And we were worried that if it was not 100, you know, 0.0% accurate, which is still a challenge with these systems, it would cause a bunch of problems.
--:-- Kelsey Piper: But users are smart. So whenever we have these conversations, I do try and look a little forward and ask myself, if we have this conversation same time next year, what will we be talking about? And getting better at answering questions accurately is something that we've seen and I think we'll continue to see.
--:-- Kelsey Piper: So I would bet pretty confidently that Grok in a year will have a better batting average, unless it is deliberately manipulated by Elon to lie in favor of his biases.
--:-- Kelsey Piper: Let's talk about some of the competition here. You mentioned Gemini which is Google AI I think every time I open up a Google Doc or half the time I open up my inbox Google trying to push Gemini on me and I trying to X out of Gemini as fast as I can because I'm old-fashioned that way.
--:-- Kelsey Piper: How is Gemini working? One thing I do, sort of out of morbid curiosity, is I invite all the AIs to look at my document of notes and then write the Future Perfect newsletter for me. Of course, I never publish that version, but I'm curious. Like, are they capable of it? You know, am I soon to be obviated?
--:-- Kelsey Piper: And they are not capable of it. But Gemini comes the closest. But almost nobody uses Gemini, you know, in the AI studio chat window. Most people see Google's AI either in Google search results, which is a cheaper to run model, or they see like integrations being offered to them in like eight different products where they don't necessarily want an integration.
--:-- Kelsey Piper: I'm perfectly happy to write my own emails. And I guess Gemini is, to a lot of people, second or third after the LLM that everyone thinks about when they think about LLMs, which is, of course, ChatGPT from OpenAI.
--:-- Kelsey Piper: Yes. They were the first to launch a language model in the form of a chatbot that you could talk with. And they have the largest share of users. And a lot of the recent, like, very cool AI functionality people have seen, like the ability to turn all your family pictures into cartoons, that has come out of OpenAI and out of ChatGPT.
--:-- Kelsey Piper: Is it still suffering in any way, the way Grok, say, you can still, if you work really hard at it, find some crazy behavior from OpenAI's models. The way I would say it is how much work you have to put in to get it to say something horrible is much higher for OpenAI than Grok.
--:-- Kelsey Piper: For Grok, it's like pretty easy to lead Grok into saying something horrible, even when they haven't like tampered it to talk about South Africa exclusively.
--:-- Kelsey Piper: Let's talk about Claude, because I don't know a lot about him. Her, them, it. Great question. I asked Claude once and Claude was like, as an AI language model, I don't have a gender identity. Nice, Claude. Very diplomatic answer, Claude.
--:-- Kelsey Piper: At some point at OpenAI, a ton of employees left. They founded Anthropic. It is a competitor. It has the same mission as OpenAI, but their positioning is sort of like, we're true to the mission. We're really going to make sure that AI is done right for the benefit of humanity.
--:-- Kelsey Piper: Yeah, the thing I would say is none of us wanted to found a company. We felt like it was our duty, right? I felt like we had to. Like, we have to do this thing. This is the way we're going to make things go better with AI.
--:-- Kelsey Piper: Claude is excellent at code. That's sort of the niche they've carved out for themselves is Claude as a coding assistant tool for people who program. I don't program, so I can't actually speak to Claude for that.
--:-- Kelsey Piper: But these AIs have different personalities. Like, they vary in how much they... Like actual personalities? Yeah, like how much they push back on you, how much they, like, ask follow-up questions, how curious they feel, how much they, like, draw connections across different topics.
--:-- Kelsey Piper: If I'm picking between all the AIs for one to just, like, chat with, I tend to pick Claude. Oh, yeah? Why? Is Claude cool? Well, Claude's not cool, but Claude's uncool the same way I'm uncool. We'll see.
--:-- Kelsey Piper: And is the go-to AI to deny the persecution of Uyghurs in China DeepSeq? Is that the one I go to? Oh, yeah. Yeah. So DeepSeq is a very good language model, and it was sort of important in two ways.
--:-- Kelsey Piper: One is that people had believed that China couldn't produce language models that were as good as everybody else's. They thought that China was behind. And DeepSeq kind of proved, nope, China's not behind. They can put out a language model that is competitive with all of the other ones.
--:-- Kelsey Piper: The other reason it's a big deal is because if you're installing DeepSeek on your phone and asking DeepSeek all these questions, it is almost certainly all of that information is being shared with the CCP. And I think we have not really thought through the implications of everybody having a personal assistant in their pocket that is working for the Chinese Communist Party.
--:-- Kelsey Piper: So one thing I do a ton is I ask an AI for an answer. And if I don't like its answer, I think about how could I have asked this question better to get a better answer? I will do a lot of, no, you got it wrong. Here's what I was looking for. What would you have needed to know to get it right?
--:-- Kelsey Piper: In a lot of ways, I think it's like managing a very junior employee where their first answer is not going to be very good, but you want to be patient and you want to say, okay, actually, I was looking for something more like this. Do you think you can try it again with that in mind?
--:-- Kelsey Piper: The other thing I wonder about is like, are we getting dumber by using these tools too much? I tend to think that technologies change how our brains are wired and how we think.
--:-- Kelsey Piper: And over time, we adjust and we learn and, you know, we develop good habits around them. But at the same time, you can do a lot of damage before we adjust and learn and develop good habits.
--:-- Kelsey Piper: It's very much a tool that depends what you make of it. You can use it to go understand something that you didn't understand. It's like quite good at explaining minutiae of some issue you're interested in. You should double check its answers, but you should also double check an answer you get from a source, right? Like in a lot of ways, if you treat it as a source that's very smart, but not perfectly reliable and you should check its claims, then you're in the right place.
--:-- Kelsey Piper: But a lot of people probably aren't using it that way and are taking it as gospel or just using it to confirm what they already believe. And yeah, I think that's a very real risk.
--:-- Host: Kelsey Piper writes for Vox's Future Perfect. We collaborated with them to make the show today. Full disclosure, Vox, like many other media outfits, has a partnership with OpenAI.
--:-- Host: If you want to read about it, you can Google Vox Media and OpenAI. Miranda Kennedy edited the show today. Laura Bullard fact-checked it. Patrick Boyd and Andrea Kristinsdottir mixed. And Miles Bryan and Denise Guerra produced Denise is a Gemini. Thank you.
