China’s new DeepSeek AI model, which reportedly matches GPT-4’s performance at 1/45th the cost, has rattled the AI hardware market and contributed to a 20% dip in Nvidia’s stock price. Investor-technologist Jeffrey Emanuel argues that DeepSeek’s efficiency gains aren’t the only story, as his viral 12,000-word article “The Short Case for Nvidia Stock” also catalyzed the market’s panic. In this episode, we explore how these converging factors could unbundle Nvidia’s once-unassailable lead and drastically reshape AI compute economics.
------ BANKLESS SPONSOR TOOLS:
🪙FRAX | SELF SUFFICIENT DeFi https://bankless.cc/Frax
🦄UNISWAP | BUG BOUNTY PROGRAM https://bankless.cc/Uniswap-Bug-Bounty
⚖️ARBITRUM | SCALING ETHEREUM https://bankless.cc/Arbitrum
🛞MANTLE | MODULAR LAYER 2 NETWORK https://bankless.cc/Mantle
🌐CELO | BUILD TOGETHER AND PROSPER https://bankless.cc/Celo
🎮RONIN | THE FUTURE OF WEB3 GAMING https://bankless.cc/Ronin
----- ✨ Mint the episode on Zora ✨ https://zora.co/collect/base:0x4be6cd4d402fed49eb2de95fbc8e737e8ffd3e7f/23?referrer=0x077Fe9e96Aa9b20Bd36F1C6290f54F8717C5674E
------ TIMESTAMPS 00:00 Start 05:20 Intro To Jeff 07:27 The Issue With DeepSeek 16:00 Inference Compute Explained 24:51 Nvidia's Competition 29:14 DeepSeek's Impact On Valuations 36:46 Criticism Around Jeff's Thesis 44:17 DeepSeek's 45x Upgrade 51:30 The Transformer Explained 01:03:15 Why Is Everyone Shocked? 01:12:07 Synthetic Data 01:21:30 Why Jeff's Article Went Viral
------ Not financial or tax advice. See our investment disclosures here: https://www.bankless.com/disclosures
No summary for this episode yet.
I basically was trying to help my organic search ranking of my little YouTube tool, and then it's like, in the process, I may have inadvertently contributed to $2 trillion getting wiped off global epiphanies because, you know, the fact is, all of the news headlines came out saying, the stock market crash, because deep sea. I'd like to point out that the deep sea V3 technical paper, it came out December 27th, okay? That's a month ago. Even the newer model, the R1 model that does the chain of thought, that paper came out a week ago, and people were all over that. So why suddenly on Monday did everything crash?
And I'd like to think it's because I'm pretty sure it is that I wrote this article in a way that sort of speaks to hedge fund managers so they can understand it. And I published it, like, in the middle, you know, the night on Friday, and then it started taking off, and then it got shared by Charmeth, who has, you know, whatever, 1.8 million, right? And Tomah, then it's been viewed over two million times. The fall or off account has two and a half million, and then like the Y Combinator, Gary Tan, and the Y Combinator account, between them, they have millions of followers. And not only did they share it, but they were like very effusive in their praise about, this is really smart.
And that went crazy. Everyone is talking about this new deep seek AI model from China that is reportedly 45 times more cost efficient than US-based AI models and charges 95% less money to use than chat GBT. As a result, NVIDIA is down 20% wiping out $600 billion in market value and both open AI and Meta's AI labs are scrambling to discover how a relatively unheard of Chinese AI lab was able to outperform their very expensive models with a Chinese grown model that just cost $6 million to train. The guest on the show today is Jeffrey Immanuel, who actually thinks that this part of the story, the deep seek AI model part, is over indexed on.
And it's actually a confluence of other factors that is contributing to the unbundling of NVIDIA's market share. And it's not the release of deep seek that triggered the 20% drawdown, but instead a 12,000 word article that he wrote on his blog that quickly went from just a few handful of readers to over two million readers over the weekend that actually coincided with the 20% drop in NVIDIA price when the market opened on Monday. In this episode, Jeffrey and I go through his article and reasoning behind why NVIDIA is under threat of getting unbundled by other chip suppliers. In addition to deep seeks impact upon the entire resource supply chain of training and inference around LLM models.
Let's go ahead and get right into this episode with the Jeffrey. But first, a moment to talk about some of these fantastic sponsors that make the show possible. Are you ready to swap smarter? You to swap apps are simple secure and seamless tools that crypto users trust. The Uniswap protocol has processed more than $2.5 trillion in all-time swap volume, proving it's the go-to liquidity hub for swaps. With support for growing numbers of chains, including Ethereum, Maynet, Base, Arbitron Polygon, ZKC, Uniswap apps are built for a multi-chain world. Uniswap seeks your transactions across its web interface, mobile apps, and Chrome browser extensions.
So you're never tied to one device. And with self-custody for your funds and MED protection, Uniswap keeps your crypto secure while you swap anywhere any time. Connect your wallet and swap smarter today with the Uniswap web app, or download the Uniswap wallet, available now in iOS, Android, and Chrome. Uniswap, the simple secure way to swap in a multi-chain world. With over $1.5 billion in TBL, the MED protocol is home to MED. The fourth largest ETH liquid-staking token, offering one of the highest APRs among the top 10 LSTs. And now, CME takes things even further. This restaked version captures multiple yields across CARAC, eigenlayer, symbiotic, and many more, making CME the most efficient and most composable LRT solution on the market.
Metamorphosis, Season 1 dropped $7.7 million in Cook Rewards to METH holders. Season 2 is currently ongoing, allowing users to earn staking, restaking, and AVS yields, plus rewards in Cook, METH protocol's governance token, and more. Don't miss out on the opportunity to stake, restake, and shape the future of MED protocol with Cook. Participate today at meeth.mantle.xyz. What if the future of Web 3 gaming wasn't just a fantasy, but something you could explore today? Ronin, the blockchain already trusted by millions of players and creators, is opening its doors to a new era of innovation starting February 12th.
For players and investors, Ronin is a home to a thriving ecosystem of games, NFTs, and live projects like Axi and Pixels. With its permission less expansion, the platform is about to unleash new opportunities in gaming, DeFi, AI agents, and more. Sign up for the Ronin Wallet now to join 17 million others exploring the ecosystem. And for developers, Ronin is your platform to build grow and scale. With fast transactions, low fees, and proven infrastructure, it's optimized for creativity at scale. Start building on the testnet today and prepare to launch your ideas, whether it's games, meme coins, or an entirely new Web 3 experience.
Ronin's millions of active users in Wallet means tapping into a thriving ecosystem of 3 million monthly active addresses ready to explore your creations. Sign up for Ronin Wallet at wallet.Roninchain.com and explore the possibilities, whether your player, investor, or builder, the future of Web 3 starts on Ronin. Bankless Nation, very excited to introduce Jeffrey Immanuel. He is both an investor and a technologist. He, however, is a very specific flavor of both of those things. On the tech side, he is deeply informed about the research advances that come out of major AI labs like OpenAI, Meta Google.
And on the investing side, he plays in the markets as a value investor, one who dares to go short at times. Jeffrey released an article on his blog called The Short Case for NVIDIA Stock, which has been echoing across the tech industry as this new deep seek model has fired a shot all the way from China across the battle of the US AI industry and has left the US based AI companies scrambling, reeling both tradfi and crypto markets as everyone learns to digest deep seeks impact upon the world. Jeffrey, welcome to Bankless. Thanks for having me. Jeffrey, I really enjoyed your article. I want to kind of start with the punchline.
I want to read one of the last paragraphs in your article that I really felt summed up the entire digestion of everyone's analysis on how the new deep seek model has impacted the market. So this is actually the second to last paragraph in your article. You wrote perhaps the most devastating to NVIDIA's is deep seeks recent efficiency breakthrough, achieving comparable model performance at approximately 45th the compute cost. This suggests the entire industry has been massively over provisioning compute resources combined with the emergence of more efficient inference architectures through chain of thought models.
The aggregate demand for compute could be significantly lower than current projections assume the economics here are compelling when deep seek can match GPT-4 level performance while charging 95% less for API calls. It suggests either NVIDIA's customers are burning cash unnecessarily or margins must come down dramatically. To me, Jeffrey, that was the punchline for I think what everyone felt on the market's Monday when NVIDIA stock fell 17%. To me, I'm summing this up as there is a tug of war between hardware and software and with the emergence of deep seek, the software side of this tug of war got a very large W.
That's my interpretation. That's my analysis. Check me on that. How do you feel about that kind of conclusion? You know, it's funny because the deep seek is the part that everybody's the most focused on. But I actually think the whole shirt thesis still works pretty well without that for all the other reasons that we can discuss. The one issue with the deep seek is that it's funny. There's the St. Jevin's paradox, which is like nobody was talking about this until suddenly now everybody's saying Jevin's every other word. It's something that comes from energy economics, which is like you think you make things more energy efficient.
Great. We're going to use less energy. But then what ends up happening is that the price of energy goes down and everybody wants to use more energy and so it actually increases demand for energy. Everyone's saying now that this deep seek thing is wrong because of the of Jevans. I am sympathetic to that, to a degree, but it's not always so clear and it's not like the Jevin stuff happens immediately. There's often what causes booms and busts is these temporary dislocations between anticipated demand and realized demand. Really, what I think people miss is that the big decisions about CapEx come down to a couple people like Mark Zuckerberg and a lot of it is gut feel like Masayoshi's son is this the good time to just push on the accelerator and I think someone like Zach has this take a set back and say listen I know my guys are really smart but maybe the answer is not necessarily to spend another $3 billion on Nvidia chips that are very expensive and where I mean literally like they're paying 40 grand for the GPU that's costing Nvidia maybe what 3,500 bucks to make.
So they're putting a lot of money in Nvidia's pocket and maybe they can pump the brakes just a little bit and then see if they can still because they projected that they needed a certain amount of chips for their forecasted demand. So if they can and the deep seek stuff is all public so they can look at the technical report. They can start making these changes themselves internally theoretically for at least for the next generation of models they're training and as a result maybe they can kind of because I think there is still some skepticism on Wall Street that like are they going to see a return on this money because it's not like anyone's paying to use all this meta AI stuff yet.
And so I think it's it's a little it's it's I'm not convinced that the Jeff oh yeah well Jeff it's like okay let's see if that's actually the case but then you know really separately from that like I was saying even if you remove deep seek entirely I believe that Nvidia predictor and I want to clarify I'm such a bull on it I'm about as bullish like 99th percentile on AI as anyone you will ever meet I live in the AI feature all day every day I have three cloud accounts I'm like you know use it I'm using this stuff nonstop all day every day so I'm a huge blue like but in video as a company they you know this is just goes back to my sort of training and investing is that you see you see this over and over again that with the one exception of a regulatory like enforcement monopoly you do not have companies just get to print infinite profits without you know with with you know triple digit revenue growth with 90% gross margins you don't get to see and without having everyone in their brother trying to figure out a way to beat them and and that's what's happening and so you look at you know these companies Cerberus and Groc with the queue like these these companies already have extremely compelling hardware that you know largely does get around the Nvidia mode for at least for inference and you know in the case of Cerberus for I think for training too and you know there's all these other sort of and I mean the other thing is like you know normal companies of the scale of Nvidia tend to have extremely diversified revenue sources whereas Nvidia all the high margin data center revenues coming from like you know five hyper scalers or something like it's it very much power law distribution and I just it's funny because when I started writing the article which I started writing because you know my friend who's a hedge fund guy asked me about it on Friday and I just started writing about it after as I was explaining it to him I realized like I should just write this up and it's funny because it started out as you know if I was forced to make the shortcase for Nvidia here's what it would be and by the time I had finished I was like shit this actually is this short just from because I wasn't like I knew there was a lot of custom silicon in the works but it was kind of eye opening to me that every single hyper scaler customer is literally making their own custom silicon in some cases for both training and inference so it's like Amazon, Microsoft, OpenAI, Meta it's like they're all doing this and it's like you don't like and as soon as they get this stuff to work the other thing that's so important to remember is it doesn't necessarily have to be better than Nvidia stuff right because Nvidia is charging 10x what it costs them so if you can make it yourself for you know 1x what it costs then you can cut the price by 50% to your end customers you'll still make a huge margin and what matters to you is as a hyper scaler is you know how many requests you can handle to your APIs and stuff per dollar you know you don't care if you need more chips like it's fun as long as you don't have to pay these inflated prices for them and so I think all these and look I mean there are other parts of the thesis we can talk about but I actually think all of that stuff is should be just as much of a focal point as the deep seek news yeah maybe to go back and trace over your article I see your article in two parts it's the moat of Nvidia and how it's being unbundled at the margins by the various set of companies some of which you just mentioned and some of these moats are the fast GPU interconnect Nvidia has had this amazing ability to make their GPUs talk to each other with extreme bandwidths as if they are one big unit like one big GPU and that is getting unbundled by another company that is just making very large GPUs that reduces the need.
Well not GPUs they're making custom spent like it's not really a GPU it's like this weird mega chip like I mean it's funny because like the H100 is considered like an absolute unit when it comes to chipsize because it's like this massive freaking package but then the service thing is like they literally took an entire 300 millimeter wafer and just made the entire thing one enormous I mean these chips are extremely expensive to me and but yeah you don't need to worry about wiring things together it's are all on the same wafer right and I actually just want to point out too that even Nvidia didn't make that technology they bought melanoxis Israeli company the doubles of the side I think they had 10,000 employees by the time they bought melanox for seven billion dollars and that brought in like another you know about the same so it was a big really smart thing I mean if they hadn't bought that company like they would not be in the dominant position they are today with data center stuff but yeah I mean everyone has been sort of relying on oh yeah but what about interconnect even if like even if AMD could get their act together and come out with a decent driver and come up with some alternative to CUDA they don't have the interconnect so you can't use it for this and you hear that argument a lot and I think well you know you're starting to see on the training side this company's cerebrus with the wafer scale chip but then also you know the the other big news that started before deep seek was you know the O1 model from open AI and that is sort of unlike this other new scaling which is about inference time compute which is like it used to be almost all the you know processing power was needed on the training side and then the inference was pretty fast but nowadays with these models that do chain of thought you know the more they compute at the time you give them a request the better the answer they can give and so people are now saying whoa so actually most of the compute might be on the inference side but the inference side is a very different you know computer problem that can be so if you want like right now they use the same GPUs for training and inference okay like can we just quickly define training and yeah training is is like actually making the model like you have like a zillion you know gigs worth of data of like text from the internet Wikipedia blah blah blah broken up into these tokens deep seek used 15 trillion of them and and then you take thousands of GPUs and you basically learn how to condense all that data down to like you know 99 percent less base you know in the weights and then in the process you basically the models learn these like coherent model of the world and how to understand things because it's the only way to compress stuff that much without losing all the information is to understand it whereas inferences you already have a trained model and now I want to ask it you know to write me an essay or to do a logic problem for me and so the inference is a very different product like you don't need to have insane you don't need thousands of GPUs to do it because you've already got to train model you just need a couple GPUs maybe and you can get the answers and so just to really trace that over one more time and like training is like chat GPT open AI creating their products yes creating their models that you know I go on to chat GPT and then when I type in a query query I am doing inference and so it's really there's a weight here of just like a ton of compute upfront to make the model once and then hopefully like a little amount amount of compute to run inference on it which is just the daily requests and like in theory there's like a trade-off here between how much compute you do initially to train the model and hopefully that just makes all future inferences as efficient as possible but there's a computer just makes the model smarter smarter yeah and that way you get better answers but with the what changed recently it used to be that basically all the inferences you know use this sort of moderately you know or a fixed compute budget like it but now it's like open-ended now like it you know O1 is like their flagship model from open AI if you pay $20 a month for chat GPT plus you can use O1 you know certain number of requests per per week if you pay 10 times as much $200 a month for chat GPT pro which I do and I recommend to anyone who uses this stuff a lot it's got O1 pro it's the same model as regular O1 the only difference is that it takes like much longer to respond because while it's doing inference it's using up far more of these intermediate token logic tokens as it were this chain of thought which is sort of like the scratch pad of its internal thinking process and then it gives you an answer but the answer is better like your code will work the very first time everything that you won't have any kind of mistakes in your essay or whatever and so over this one more time just so like it's the same it's the same pro the $200 a month version and the $20 a month version is the same model but there's this extra step this extra layer of things happening where the pro version is running that same model over and over and over again in chunks and it is able to go back and trace over previous work to like check its work before it actually gives you an output right you're saying that just because of this additional layer it's just like they just do it for longer they just it's like it's because you basically it's a dial you say how much money do I want to spend generating tokens before I give the final answer and with pro it's like it would not be economical for them to use the amount of tokens that they use for pro for the plus in fact Sam Altman said that you know it's funny because everyone on hacker news was like and in the industry you know all these developers are like $200 a month get real how could that make sense and Sam Altman came out later and said believe it or not we're actually losing money charging $200 a month because people are using it and it just uses like insane amounts of compute and so it really flips you know the equation on in terms of how much compute is being used for inference versus training and then this isn't really relevant because like I said with you know the NVIDIA GPUs they use you buy an H100 GPU data center GPU for 40 grand from NVIDIA you're going to use the same GPU to train the model and do inference on it but this company GROC with the Q this if people everyone gets confused because GROC with the can not the Twitter GROC is the Twitter GROC exactly but GROC with the Q should be better known because this company is really I mean they've got unbelievable technology they basically said we're not going to try to solve training at all we only care about inference and so if you want to optimize the entire stack for inference only how might you approach that and the result of that is that they can do inference from you know like a standard model like the llama 3.
3 70 billion which until the deep sea came out it was the sort of leading edge open source model and you know if you if you get a fancy desktop computer with one let's say NVIDIA 4090 GPU which you can get for under a thousand dollars now you could get I don't know maybe 40 tokens per second which is actually like good enough that you could use that as your sort of home version of chat gbt that works pretty well but when you try it on GROC and you and anyone can try this for free you just sign up with your Google account and you can do inference from this model and it's like insane it's like instead of like 40 or 50 tokens per second it's like 1800 per second and so you click and you just boom there's your answer and it's like whoa that's pretty interesting and so even though the GROC hardware costs like millions of dollars for one server if you have enough demand that you can just keep it busy all the time it's actually much cheaper to use and most importantly you're not giving your money to NVIDIA you're giving it to GROC you know so it's and so it's just an example of how people manage to you know like if you're trying to assault like a castle that has a big mode instead of trying to cross the mode and get you know shut up by arrows why don't you like dig a hole under the motor do you know what catapult to go over you find creative ways to get around it and that's that's what's happening is everybody's been focused on well of frontal assaults not going to work and it's like okay but there are other ways to see a castle and that's what you're seeing is that all the ingenuity of the market of like because the and the reason is because the price is so big that if you can you know you two can make your company worth a trillion dollars if you can take a big piece of this pie whereas that was not true in 2016 it was like a backwater you know and so it and it just the wheels take a lot of time like if you want to make your own custom even if you're Amazon with infinite money to spend if you want to make your own chips you know what do you know about making silicon first you have to you know poach or hire the really brilliant people and then it's going to take them probably two three years to design a really good chip and then you're going to have to try to you know come with giant sacks of cash to TSMC and try to convince them to give you like volumes at their fabs because they're like already just being you know inundated with money from Nvidia and Apple and stuff and it takes a while to get ramped up but eventually the chips start coming out and you know the irony of it is like again it's like even though you know none of these custom silken chips are going to be as good as the Nvidia chips the sort of way they're made is pretty similar in that they're they're all both going to be using TSMC as the fab they're and they're both using the same machines from this Dutch company ASML that actually like does the lithography so it's like yeah they won't have the same brilliant design maybe but again they don't need that's the thing people miss it doesn't need to be as good it could be it could be one fifth is good and it still makes sense to for Amazon to use it because they don't have to pay 90% gross margin to Nvidia because Nvidia has been tried it has the luxury of having very high margins and what that creates is like well if your product is 90% as good but you only take 10% of the margins then all of a sudden you're solving like a lot of market I'm saying when your margins are so high like just to put things into perspective companies that sell chips like in the semiconductor industry it's like generally not such a great industry it's very subject to boom and bust cycles of like over capacity and you know and so if you look at another area like memory DRAM which you know everyone has it in their phones and their computers you know you might think on the surface that this should be like this great business because there's only basically three companies in the world that do it it's like a micron Samsung and SK high next I mean there used to be like 15 memory companies but they all like either went bust or merged and so you would think you'd think it would be this all adoptive thing with great pricing and margins but if you look at the history of it over the last 10 15 years it's like it's very cyclical and at the very peak when supply demand mismatch is really out of whack and they can charge really high prices they make like a 60% gross margin and then but if you take the average over the cycle it's closer to like 20% and at the bottom of the cycle gross margins actually turn negative okay and so then you look at Nvidia and you're like you have a 90 plus percent gross margin on data they're overall gross margins more like 75% because they make much lower margins on the consumer stuff like for playing video games and that's because they have competition from AMD you know like that's what happens in a competitive market and so but my point is that when your margins are that high it doesn't need to be 90% is good it could be literally like 40% is good and it stills a no brain or for Amazon to switch as many loads over to their own thing because it's like you know it's like when you buy like a handbag from Hermes for you know 40 grand how much do you think it cost them to me even though that it's made by hand by some French guy like you know it's probably only like it's like two three thousand bucks tops and then they're charging you 40 thousand dollars for the end it's like very similar margins for the the GPUs from Nvidia and so it's like you don't you what matters is like and and the users don't care they're they're submitting requests they want to use a model a lot of 3.
370 but they don't care if an Nvidia card is doing the inference on it and so Amazon is going to you know Amazon made their own CPUs called like a graviton and they are very aggressive with the pricing of that to try to switch people over from if you normally use like an Intel or AMD CPU try using one of our things and you'll save a lot of money and I you're going to see that work and they're going to try to push people over to their product because by making it more you know they're going to basically split the savings you know with the customers and I think that and and so all that stuff you know it's like death by a million cuts like the combination of the compare competition from these different areas and then of course it's like AMD does compete with them effectively in consumer stuff but they've been completely absent in this whole data center AI stuff which is you know it's as crazy like I mean they're going to be writing like business school case studies about how they squandered trillion dollar opportunity you can't get too mad at them because they also like managed to kill Intel like so it's like it's not it's not like they're not good too and it's so funny because Lisa Sue the CEO of AMD is like first cousins with Jensen Huang from in the video I did not know that yeah which is so like how how good are these genes in this family but um so yeah I mean but but if they can get their act together and it's so funny because it's like they're so out of it like I don't I just don't understand it but like they're literally like people like George Hots the the guy who's famous for jail breaking the iPhone and all this stuff he's like literally by himself without any help from writing his own stack of that's like you know we'll be able to make these GPUs usable for for doing at least you know some training and and inference and so you might see even AMD coming up as as a as a real competitor and yeah yeah so yeah going back to tracing over like the broad strokes of your article I kind of break it out into two parts two halves there is the unbundling of Nvidia's moat in the hardware side of things via hardware competitors as as you've kind of just traced over but then also the deep seek side of things is a rebalancing of the value of software and in algorithm design maybe is one way to put it maybe you can take us to the second half of that equation where I how how did deep seek really impact people's understanding of the value of software and its impact on the value of hardware well so you know when you say like what is the software side of the thesis it's not it actually has very little deep seek what it has to do with is one of the the sort of biggest source of Nvidia's mode has been because you know AMD has quite reasonably good you know chips so the reason is that Nvidia basically was a very forward thinking and when they noticed that like this deep learning stuff was really taking off and like back on with 2012 and they so they really figured out that we need to make it easy to use our chips for this sort of thing and so they have this system called kuda which because you have to understand like these GPUs are insanely complicated I mean in the old days you'd have one CPU with one core now CPUs are pretty complicated like I have a CPU in my computer has 32 cores but these Nvidia GPUs are like that's like thousands of cores and so it's a whole deal they have lots of cores and so it's like very like if you were to try to like write code naively to like take your problem and break it up and send it to thousands of cores and reassemble it like no one can do that you know basically and so instead you describe the problem using these much more abstract high level concepts and then kuda turns that into hyper optimized code that runs really really well on Nvidia GPUs but not on it is a Nvidia built software package to allow developers to you to use Nvidia GPUs to their to their best degree possible without without being like yeah Einstein like they can be very smart but I mean it's it's kind of like a driver is it a no it's it's it's like a framework framework yeah the driver is a sort of separate layer of but it allows the power of it in Nvidia GPUs to be expressed to more people without them having to be yeah it's like the difference between writing code and Python versus writing code in like like assembler like like which is the lowest level you know and so and then it's actually so kuda is even and most people actually don't even write kuda directly most people use machine learning frameworks like you know it used to be TensorFlow but it's been sort of totally replaced by something called PyTorch which was sponsored by meta and so that's what most researchers use as PyTorch which lets them think like in terms of the math and you know as a researcher say oh I have this loss function I have this optimizer and everything's like modular and plug and play and then you write high level Python code which is like very very high level and then internally PyTorch can then run that on kuda and then run it on a GPU from Nvidia very very very officially but if you have an AMD driver AMD GPU it's not as easy to have your stuff run really really fast writing using like PyTorch and stuff and so and a lot of people were saying that it doesn't matter what anyone else does in terms of chips if they don't have kuda you know it's game over and there's like a two I think big assaults on that which is that you're seeing the rise of these sort of even more high level frameworks for expressing highly parallelized programming and so you have this one MLX is is one it's another one called Triton and these are gaining you know momentum and then for that it's like kuda is just one they they can't you can write your stuff in MLX and and then basically run it on an Nvidia GPU really really fast but you could also make another you know compilation target of MLX that could run on a completely different chip like the one you know the Amazon is making internally the you know training chip and so and it's also very high level language so maybe it makes you know instead of writing you know targeting kuda maybe you should target MLX or Triton and then you can also get run it on using kuda but you could also run it using these other things and then you're not locked into using the really expensive Nvidia chips so that's one assault and then the other one I think is this idea that and I haven't heard a lot of people talk about this but one thing I'll take like I use LLMs all the time for programming and they're just stunningly good at that now but what they're really really good at is if you already have a working prototype of code in Python or JavaScript or whatever and so it can really understand what it is you're trying to do they're unbelievably good at porting that to another language so for you you you have this Python algorithm and you want to turn it into like rust or go laying they do that unbelievably well like maybe not on the first shop but you know with a couple iterations you can get it all working and so what that made me realize is that you know because the part of with the kuda thing it's like about it's become a lingua franca like everyone who's good at this kind of programming knows it and so they think in terms of kuda concepts it's just the fastest way for them to express these algorithms and so I was thinking that like they could write their code in kuda like they normally do but then instead of using it on a video GPU they could use it almost as like what like it's called a specification language where it's just for documenting the algorithm in a very efficient elegant way and then they could feed that into an LLM and say all right now port this into this other framework which will work really well with you know AMD GPUs or with you know Cerberus or something and I think you really you really explain this well in the article when you illustrated the there's like a job market for kuda engineers yeah and it's insular to the rest of like you know engineering jobs engineers out there it's very special engineer it's like there's this own independent like vertical of job markets and like the cost for these engineers and the way that you illustrate in the article is like well that those walls break down uh and all of a sudden there's just like not really the same monopoly around kuda it's not that it's I think they'll still use kuda but the question is like can they use kuda but then not use an Nvidia GPU right which is where the moat comes from right and you get at least part of its value from yeah yeah and now you did bring up a point about like so the deep seek in a sense is software because by writing smarter training software they did reduce the demand but I say that's sort of separate that's like kind of orthogonal if you will to this other stuff which again it's like so if you took away the deep seek part of it I mean you can see the big threats to the moat software and hardware houses now let me just say I just saw but right before you start talking somebody said here's why this you know my thesis is all wrong and they're saying that well the problem is that TSMC which is Taiwan semiconductor which builds all these chips and they're basically the only ones that can do I mean not the only ones because like Samsung can also make pretty good chips but like for the most yeah they make all the Nvidia stuff and all most of the Apple stuff and you know but by the way I want to point out again it's like yes it would be best to do something in a four nanometer process node which is the smallest you can do but you know you could use like a bigger like an older process node and your chips won't be as fast and they won't be as energy efficient but you've got a lot of room wiggle room because you just you don't need it to be as good you just need to be cheap but so anyway but the objection to my thesis is that these guys are book solid even if you came to them with like you know giant bags of money they they're book solid and and the reason is because the manufacturer is book solid they're they're wrapped up they have too much orders yeah for the next couple years they don't care how much money you give then because they they're they're the all book solid and they can't you know can't just instantly make a new although I will say like you know Taiwan semi built a fab in Arizona and there's all this you know stuff about oh it's taking them so long and they can't hire good people but you know what they finally did get it all up and running and they could literally if there was enough money in there to do it they could just copy paste the blueprints get another big chunk of land and like just replicate what they just did again and they could do that like and it wouldn't take like that it wouldn't take that long and so in any case but so so that's the objections that so even if everything I said is true these companies are bracing rock and the hyperscalers like amazon and you know google and blah blah blah did meta and so that they won't even be able to make these chips in enough volumes that it's going to dent in video and my response to that is like okay your analysis is essentially conceding that this is a highly sort of transitory circumstance here that like they just very temporarily going to have this advantage and then as soon as the additional capacity comes online or opens up then there's going to be this massive flood of of alternative supply which is going to pressure market share potential you know if the even if the pie grows the market share is going to go down but most importantly it's like it doesn't take you know something there's some stuff that has nothing to do with technology that's just basic you know economic industrial finance kind of thinking about how do markets work and the difference between having basically a monopoly and having even one or two competitors is like the margins really can fall quickly because it's like you know if you have two office buildings that are like you know 98 percent occupied nobody's you know it's a race to the bottom to try to cut your rents but like if both of them start losing tenants and you know every day that goes by and this floor is empty is just they're losing money and so there's just race to the bottom where they just and there's this critical threshold where you know once let's say the occupancy rate in a market for office product you know dips below let's say 80 percent the rents it's very non-linear you know like if occupancy falls another 5 percent rents are going to fall a hell of a lot more than 5 percent to make the market clear and I think you'll see that the margins can fall very very quickly once they're the real competitors and then the question is okay again this is not about technology this is about how do you rationally value a stock and I mean one of my favorite I mentioned in my piece that you know I once want a prize from this value investor club website for a short idea this was like more than 10 years ago but I'll quickly tell you the story of it because I think it's so relevant here which is that this was a company called petrol logistics pdh was the ticker and they they were a company that just had a single plant that took propane and turned it into propylene which is this and it's through this like random you know it's basically because the shell play happened all this like I don't have to get into all the details suffice it to say they were earning unbelievably high spread much much higher than historical or like when they built the plant what they ever expected to earn they're earning so much that their profit in one year from running this plant was like 80% of the cost of building a new plant and it's not like rocket science to build one of these plants you can just go to a big construction company like backtell and say I want to I want a conversion plan for propane to propylene and they have off the shelf blueprints they'll make it for you guaranteed in a couple years and sure enough this company was earning these higher turns and people were putting a big multiple on the earnings because they're like look at this the earnings have gone up so much and but you could tell that all these other plants were already under construction and you actually knew approximately when those plants would come online and so you could basically figure out all right even if I grant you that they're going to continue earning these massive margins it's going to start stepping down in like a year and then in 18 months it's really going to step down and in 24 months it's going to be right back to the so if I want to value this as let's say what is the present value of the future cash flows discounted because of the time value of money I can do that I can say big big profits this year a little bit less profits next year and then after that normal profits and add up the discounted cash flows and you realize like you can't put a big multiple on earnings that are not sustainable and and right now like so so if you tell me that oh well but you're wrong because in videos and to keep earning these huge profits for the next two or three years it's like dude you're putting like a 30 40 x multiple on that that's essentially implying that it's going to sustain at this rate like indefinitely and that's just not how you know you should think about this value of a stock and so it's really this is why I wanted to say like and a lot of the jettens stuff it's like yeah I am bullish on the aggregate like the amount of total demand for inference is going to skyrocket the pie is going to grow that's totally separate question from will and video be able to continue growing revenues triple digit percentage year over year at these insanely high margins that's a completely separate bit and you need to answer that question if you want to feel comfortable putting such a high multiple on that earnings stream you have to know that it's going to sustain and it seems actually quite likely that it won't sustain I do want to dive headfirst into the deep seek efficiency gains part of this this conversation I think that's a kind of way we should go next one thing that you wrote in your article you said this sum total of all of these innovations these are innovations referring to the lab that made deep seek when layered together has led to the 45 x efficiency improvement numbers that have been tossed around online and I'm perfectly willing to believe that these are in the right ballpark maybe you can just like explain the significance of this new chat GPT like model deep seek and that and how it got to be 45 x more efficient and what that efficiency what 45 x efficiency means when it comes to the industries that are like are the supply chain to create like the usage of these models sure so look I mean it's funny like in in the west it's like we have this sort of resource curse you know almost of like we have too much money it's easier almost to just throw money at a problem to try to be like really clever and so you know the joke or the sort of parallel I make is like when you look at people's houses in Saudi Arabia they're not very energy efficient and that's because they get subsidized power because they have unlimited energy there and so there's no yeah there's no point in wasting all this extra construction costs on double playing glass and blah blah blah and it's a similar thing like meta and google and they just have so much operating cash flow hitting you know every quarter of like fuck it let's just let's just hire more yeah yeah let's pay our people five million a year and let's or whatever a million a year and that's and let's just you know send over to jensen another three billion and whereas you know this China it's they're not getting paid that much that's for sure and you know they and they do have these export control now I know a lot of people say oh they don't they're smuggling them in through Singapore I'm sure that's happening but like smuggling chips chips yeah because say first of all under Biden they made it they have basically have like a slightly crippled version of of the Nvidia GPU just for the China market or export market that's not as good as the h100 but but then also I mean what people point to you which I think makes a lot of sense is like something between 15 and 20% of Nvidia's revenue comes from the tiny nation state of Singapore it's like really they're using that many GPUs there and it's like because everyone knows that they're somehow getting laundered and smuggled into China and so the question is we don't even know how many Nvidia GPUs are in China and and so we don't really know how many deep seek used but the point is they don't have as many as we do and it's not as easy for them to get them and so maybe the the punchline you're making is like that Tony Stark Ironman meme of Tony Stark was able to build this in a cave exactly and that's China they don't have an abundance of capital they don't have an abundance of chips they have some chips but they don't have the ability to and by the way they're quickly that's a whole other story but like they hired like you know they poached some of the smartest guys at TSMC to make their national champion and Smith or whatever it's called and they're obviously not there yet but like they made a pretty good Huawei CPU and I wouldn't be surprised if I mean that's the other giant you know wild card that nobody's really taking into account like don't count out they got some of the smartest people from Taiwan semi over there and it's like they'll buy the machines from ASML too and you know right so but anyway what I wanted to say is that you know their engineers are a necessities mother of invention but also you know in the west we tend to have this sort of bifurcation in the market where you're either in the AI like research track in which case you have a PhD and you've written these papers and you're like a guy like does stuff on the whiteboard or whatever and often these people are not they're not very good engineers like there's a joke that these researchers are actually horrible at programming they're good at math horrible at writing optimized code it's not obviously not universally true there are some people are great at both but so what happens is usually the researchers think at these high level and then they make like a prototype and then they hand it over to these people who are more engineers like high performance optimization guys people like John Carmack or Jeff Dean at Google who you know they're not going to invent the new optimizer or like you know some new loss function for AI models but if you give them an algorithm they know how to make it run really fast you know on a computer and and so it's sort of like they the way we do it is this sort of two-step process in the west where the researchers designed the thing and prototype it handed off to the engineering department that says all right we have this algorithm how can we make it go fast the deep seat guys are unbelievable at books so it's like instead of having it be two teams working one than the other it's like they kind of inverted it and they started out with let's start out first with how can we saturate every ounce of performance on these GPUs so that nothing is wasted because it's like it almost doesn't matter how fast the GPU can calculate if it's waiting to get data that it needs to do the calculation then it's just sitting there idle okay and there's a lot of this interconnect right there's a lot of talking to each other and so normally you have to dedicate a big chunk of your processing power just to handling that communication overhead so they did a lot of really clever work with like making the communication stuff as efficient as possible so there's very little overhead so they basically started with rather than say how do I make this algorithm go fast they said how can I make a really really fast algorithm that'll that'll really run these GPUs as much as I can and then sort of design a smart training system based on that so they sort of inverted things and so there's just this collection of sort of optimization tricks and by the way I want to point out like many of these ideas were not invented by them many of them were actually published by American and other researchers like Noam Shazir who could just got rehired by Google for a zillion dollars they bought his startup just to get him because he's so he's like that smart I mean and and but it's it's it's it's it's implementing them in a clever way and so I'll just get you just a couple examples of this it's like so you know the all this whole chat should be tea thing really exploded because there is this model design called the transformer which came out in 2017 this is probably the most cited paper in history now it's called attention is all you need and this kind of it combined a sort of regular neural nets that we've been using for a while with something called like the attention mechanism which is this very clever way of like kind of contextualizing the information so that like instead of always processing it the same way it it depends on its context and you sort of automatically learn how to think about that context and storing all that data while you're training is like one of the major things that use up memory and the memory it's very important because you can't use like the system memory on a computer you have to do everything on the the what's called the V-RAM the memory that's very fast memory on the GPU itself right and that's pretty limited and so if you can save on the amount of memory you're using that's huge because not only can you do more with fewer GPUs but you're also not transferring as much data because it's just smaller and so anyway these these there's something called KV key value caches and indices that you need to keep in memory while you're training a transformer model and they came up with this incredibly smart I mean this is like probably the coolest thing in the whole paper the deep seek V3 technical paper is that they realize that you know really it's very wasteful how it's done normally that you're storing way more data than you need to that the sort of only like some very small subset of that data actually is meaningful and in fact by storing more than you need to you're almost like overfitting to like noise and it's not necessary and so they maybe a simple way to explain this for listeners who want some extra help with that is it's just maybe closer to how your brain works with attention where when you're applying attention somewhere you're not thinking about every single thing under the Sun all at once you're kind of focusing on what's necessary and I think maybe you can't go too far with the answer from a more rising like attention attention in this context means a very specific thing and says not I don't think it's going to help people maybe I can't remember if I heard this in your article maybe a different one but it's like if a house has you know 20 different rooms and lights are on in every single room even though a person is only in one room and this new model only keeps the lights on for the specific room that the person is in at that one given time it's some some loose broad stroke pattern like that I mean it's basically like instead of just not easily storing this massive amount of like key value data that shows you like it's basically like if you have like the word job it's very different if you say nice job versus I just got a new job or you know like you know you're going to be able to handle that job for me like um don't and and it's like knowing so so like the word job has like a certain representation in the model but that representation has to be altered depending on its context that's what about attention's been and that means that for every word every token you have to really have to store lots of different things depending on the context and that's why it takes up so much memory and they're able to store that in a very efficient way using you know basically like just storing this sort of subset of the data in a compressed representation that so that's one thing they did that saved a lot of a thing the another thing they did that's very smart is what's called like multi token predictions so like usually these models they predict the next token the next word basically based on the preceding tokens or words and um you know one at a time and this sort of kind of is this bottleneck and they're like well make what if we try to do let's say two or three at a time now the problem with that is you can't really predict the second token without knowing what the next token is and so you can how can you start with the second token until you know the first token but you can do what's called the speculative decoding and so but your speculative decoding might be wrong in which case you wasted your time computing that second token but what they did is they got very good at guessing what that second one would be such that 95% of the time they get it right and so they basically just from that you can sort of double your throughput on inference because and and but by the way that's part of the reason why they're able to charge so little for their API is because that's about inference cost and so they said that one trick let them almost double throughput for no additional cost but just by so so that's a very clever trick they did and then they did another very clever trick with just the um you know the these models are basically just a gigantic list of numbers if you will call the parameters of the model and they figured out a way um to store those parameters in a much more kind of compressed form and like kind of normally the way these models are trained as they they use more precision kind of you can think of it almost as like more decimal places of accuracy but it's not actually how it works but it's just sort of close enough to understand conceptually and uh and then often what they do is once they then train the model that way to make it so that it can run on a cheaper GPU they do what's called quantization where they sort of then kind of truncate and round off the numbers a little bit and what but that does hurt the accuracy or the not the the quality of the intelligence of the model and what the deep seek guys did is they managed to instead of having to train at a higher precision and then quantize to a lower precision at the end they managed to figure out how to mostly do the entire process end to end using the the um smaller representation and again it's like it's one of these things where efficiency gains are they they pay for themselves so many times because it's like not only do you use less memory but the calculations go faster and um you know and then you don't need to do as much uh inter GPU communication because there's less data and so it's like it's these efficiency gains paid off in multiple multiple different ways and so that that's another thing they did I mean there's just there's just this whole laundry list of like little tricks and optimizations they did that when you add them all together and and if they're not additive right they're multiplicative like each thing you know if it double if this thing doubles it and this one increases it by 40% and this one doubles it also you're multiplying those multipliers if you will and that's how you can get this very big number like 45 times which by the way we we don't really know you know we don't know for sure they could have lied about the number of GPU hours they use um you know they one thing is clear though that they're they are charging 95% less for inference so if they're losing money on that or they really can do at least the inference part much cheaper than you know um we can hear in the west because uh yeah that 95% less money for inference I think is really the the sticker shot the shocking number that is sending companies like meta in open AI reeling like Sam Altman actually meta was up I think because it's actually look on the one hand it's bad for meta in that they have spent so many billions of dollars on GPUs and they paying so much money to their team to like come up with the llama models and stuff like that and then it sort of does make them look a little like foolish when these guys are able to beat them at their own game for you know on a shoe string right but at the same time what they really care about is how much does it cost them to serve AI to all of their billions of users around the world and so it's actually good for them if they can cut their cost 95% that's great who is bad for is open AI and anthropic because it's going to put more pressure on on their pricing like they're not going right now um right now open AI charges a fortune for the oh one model API and even you know GPT 40 is much more expensive and so they're going to probably have to respond by cutting you know their API prices significantly and that's where they get their profit from right well they don't actually have profits both companies are deeply unprofitable at the you know consolidated level and I actually suspect even at the sort of you know incremental marginal level they're not all that profitable because they're prioritizing revenue growth above sort of everything else I don't think it's a case where they lose money on every unit you know sold on the margin um you know any fast growing company is going to post consolidated losses just because they're always spending on growth and the new models so but so the real question is like if what if they what if open AI and anthropic completely stop trying to do R&D and making the new model and just try to milk the business for for money what they have now would they be able to eat out of profit and I think yeah the answer is probably yes but um but if they have to cut their pricing by 80% then it's very unclear you know so that that's where it starts to be pretty well the Arbitram portal is your one stop hub to entering the Ethereum ecosystem with over 800 apps Arbitram offers something for everyone dive into the epicenter of DeFi where advanced trading lending and staking platforms are redefining how we interact with money explore Arbitram's rapidly growing gaming hub from immersed role-playing games fast-paced fantasy MMOs to casual luck battle mobile games move assets effortlessly between chains and access the ecosystem with ease via Arbitram's expansive network of bridges and on-rits step into Arbitram's flourishing NFT and creator space where artists collectors and social converge and support your favorite streamers all on chain find new and trending apps and learn how to earn rewards across the Arbitram ecosystem with limited time campaigns from your favorite projects empower your future with Arbitram visit portal.
arbitram.io to find out what's next on your web3 journey fellow is transitioning from a mobile first EVM compatible layer one blockchain to a high performance Ethereum layer two built on OP stack with eigenDA and one block finality all happening soon with a hard fork with over 600 million total transactions 12 million weekly transactions and 750,000 daily active users sell those meteoric rise would place it among one of the top layer two built for the real world and optimized for fast low cost global payments as the home of the stablecoins sell a host 13 native stablecoins across seven different currencies including native USDT on opera mini pay and with over 4 million users in Africa alone in November stablecoin volumes hit 6.
8 billion dollars made for seamless on-chain FX trading plus users can pay gas with the ERC 20 tokens like USDT and USDC and send crypto to phone numbers in seconds but why should you care about cello's transition to a layer two layer two's unified ethereum L1's fragmented by becoming a layer two cello leads the way for other EVM compatible layer ones to follow follow cello on x and witness the great cello happening where cello cuts its inflation in half as it enters its layer two era and continuing its environmental leadership so Jeffrey I just want to kind of zoom out and summit everything we have this new model this deep seek model which is 45 times more efficient than than you know chat GPT or other competitive models that's cause a repricing in video because people think like oh wow 45 times more efficient we just need much less hardware in order to make that outcome happen is just we're getting more from less hardware and so maybe we've been overpricing a hardware and that's that's what has shocked the market with repricing of in video and then also now open AI Sam Altman are getting a squeeze because deep seek is charging 95% less money for inference requests but my broad question to you is like well isn't this the expected outcome like AI and AI technology is on a very steep curve and we're seeing you know breakthrough efficiency gains across the complete tech stack whether it's hardware or the models where we've always known like AI is going to accelerate very very quickly and isn't this just what this looks like isn't this kind of the expected outcome here like of course we're going to get more efficient that's how technology works like why is everyone surprised I mean it's clearly not the expected outcome big because the stock wouldn't have moved so much I mean it is a expected outcome for me which is why I wrote my article but and I think the answer is that everyone does expect progress progress on the hardware front that every year the chips are going to get faster and bigger progress on the algorithmic front that you're going to come up with a better way to train the models or do inference that's going to make things faster I mean when these elements first really came out a couple years ago they had a much more limited context window like the amount of text you could put into them that has gone off dramatically and originally everyone thought that that was going to be really hard to make that they able to go up because they thought it was going to dramatically increase the amount of memory but people came up with really brilliant inventions to you know new algorithms to make it faster and so people do expect some level of algorithmic improvement some level of hardware improvement every year but they expected to be you know like a Moore's law type progression where it's somewhat predictable and what really catches people off guard our step function changes where overnight it goes no so like if if the news was that they tripled efficiency you know that would be for you know I mean can you imagine like if you made an air conditioner that was three times more energy efficient like you crushed them you would get huge market share tripling something it's like in any normal you know if you had something with triple the mileage for your car like that would do great but like we've become so sort of used to it in technology that it's like that you know but 45 times okay now we're talking here that's that's really crazy and so when that happens overnight in a way that people didn't anticipate that's when you get the sort of shocking thing and you know the thing is like you know there's this expression of being price to perfection like the invidious share price it only looked reasonable to people who extrapolated these curves out and like you have to be very careful when you extrapolate revenue growth that has been going at 120% year over year and again it's not just revenues it's about the margins and they were basically saying that the margins would maintain and the revenues would remain would keep growing at this incredible rate and as a result yeah sure this is and that's why every single every single investment they basically had a strong buy on Nvidia all of them they all got caught completely off sides with this thing they were all scrambling honestly to read my article and they're like you know I got inbound requests from some investment banks to like help because nobody even wants to talk to their analysts about this they want to talk experts and so they're scrambling to find experts not that I'm even an expert but compared to the equity analyst at the sales side apparently I am and so it was not expected at all like that that would happen like step function change like this and that's what just is like the the body blow to the stock is that hey this thing was pricing in you know clear skies and then it's like all of a sudden it's like oh they're actually are these threats and then again it's not just the deep seek it's like the people were ignoring a lot of these other threats and I don't know why because they're literally these are people where their full-time job is to cover Nvidia for Goldman Sachs and Morgan Stanley and I don't know what the hell they're doing that they didn't you know how come they weren't talking about you know the the competitive threats to Kuda are like what you know cerebris and grog and maybe they mentioned it but they certainly didn't figure out that this actually is going to be really important and with the step function change it's not just a step function improvement because it's also a step function improvement in a slightly different direction than what the market was thinking correct it's not it's not we aren't just skipping ahead on more it's like it's also going in a different direction well it's additive to everything else it's like you are going to have faster chips next year you are going to have more chips next year you know like you are going to have other algorithmic improvements on the margin but on top of that now every big AI lab in the world is going to be you know the llama team at meta the anthropic guys you better believe Zuck has brought these guys into his office and said we need to use every one of the tricks these guys are using for llama for yeah so like as a consumer of AI products it's great if you're not exposed to you know in video if you don't have open AI equity private equity if you if you are just a consumer you're stoked oh like the products coming down the pipeline are going to be sick in a very short order oh and not only just that but from a standpoint of like you'll be able to run this shit on your own computer like you get like a thousand dollar Mac laptop you're going to be able to have like AGI on your computer and tap privately and it's like it's the most miraculous thing ever I mean nobody would have believed this even a few years ago like is that why Apple is up on the week because I think I saw Apple being like up three or four percent when in video is down on 20 percent Apple is one of the guys that actually it's so funny because Amazon and Microsoft and OpenAI they're all like trumpeting in these big press releases about their custom chips that they're making and you know Apple's so different Apple's like so secretive and it's like but you know they have like one of the best silicon teams in the world but they only announce something if it's like they're ready to sell it to consumers but they're making chips internally for their own uses like no one even freaking knows about it and all the people who do know about it are like signed up with NDAs like and no one talks about it and for all we know they have pretty fucking awesome chips already and and so and but they're essentially like users of of AI you know so it's good for them it means that they'll be able to use some of these tricks to make some of these models in fact there's an app you can get I think it's called a Apollo on the App Store that lets you download these models and if you have like an iPhone 16 Pro or something you can just run this thing and you could be on an airplane or whatever with no internet in a bunker somewhere and have essentially you know not quite a GI on tap but like you know certainly like smarter than most college students on a lot of topics and and it's wild to see it go you know you could go into airplane mode and be like asking you at all these questions about you know chemistry and physics and history and it'll give you really good responses at a reasonable you know pace and so yeah it's good for Apple it's good for Apple I think it's ultimately good for meta which is why meta stock wasn't down right um you know so it's not it's not it's it's not a bad thing right it's just bad it's as far as as it gets a recalibration but you know I do think it was excessive for one in one day that they'll you know add two trillion of capital wiped out but it's it's I'm not I'm not saying that you should be buying the dip in Nvidia though because I think it did get ahead of itself and it could still look it could fall to two trillion and you know two trillion and still a lot of money okay like this is a company that earned you know like five billion dollars like you know a few years ago like so that's still you know quite quite a quite a big valuation um Jeffrey there's one last conversation before I let you go is the conversation of synthetic data and this this I think comes from just having stronger um and better models is creates this notion of synthetic data and this is also like part of the equation of like the rebalancing of how people value things can you just walk us through this synthetic data conversation what what is synthetic data what do different and stronger models have to do with synthetic data and what does it mean for the overall x apply chain of AI well I'm not so sure that it I mean I think it's an important concept I'm not sure how much it applies to sort of those things it's what it is is that um you know when you're training these models the pre-training that actually makes the model smart it's it's partly a function of how much compute you apply you know how many GPUs and how fast they are but it's also the amount and quality of the data that you're training on these like you know when deep seek says we use 15 trillion tokens in our trainings set that's what they're talking about and the thing is it's like there's only so much data that's of high enough quality that you didn't want to use it to train a model out there like if you take all of Wikipedia I don't know how many tokens that is but it's like not that many you know it's like it's like it's less than you know it's measured in the low single digit billions that not not even close actually um I'm sorry many billions yeah but it's like if you take all the books out there we're talking really like it's just like a couple trillion like if you talked about like all the newspapers that have ever been written it's a couple trillion yeah but what you're saying is the the quality data that's out there is a process processable amount of data we're running out of data we're running out of data yes that that people write smart books like you know they're not writing you know the the books fast enough basically does keep supplying us with more and more data and so that's a big wall that we've been facing like how are we going to keep improving the models if we're not going to be able to scale up the data that they're using and people say oh but you could just take every YouTube video but it's like have you seen most YouTube videos it's not going to make your model smarter you know it's going to make it dumber no and so but there's an exception to this rule and so the exception now so synthetic data is using an LLM to generate text and then turn around and training a new model on that text and so that sounds like very circular like how it's like me trying to teach myself in a room without a book or anything just talking to myself and I'm going to teach myself like how might how's that supposed to work like in terms of getting new information isn't that in a sense almost getting high on your own supply that it's not going to help you and that is sort of true if let's say you're talking about like you know the history of the Peloponnesium War or something you're not going to get anything new by just regurgitate in your own output the exception to all that is if you're talking about like logic math computer programs because in those things the big difference is that you can verify that what you said is correct so you know just like the rules of chess are very simple but it's like almost unlimited complexity of the possible chess games it's the same thing like there are so many possible simple python programs that are like 100 lines or less that you know we've only ever seen a tiny subset of them so you could come up with a you could say oh I want to make a python program that does xyz generate a candidate and then test and be like well when I run it did I get that output and if you did you know that the program's right and so now you can say okay well let me add that to the training set and it wasn't in the training set originally but it is correct and good and so what you can do is you could start exploring the world of like all possible math theorems and working out you know all these math proofs verify that they're right and then add them to the training set and in that way you could basically come up with lots of data that's known to be super high quality and that's why these models are getting better at logic and math at a much faster rate than they're getting better at anything else because you could sort of just keep cranking and getting this synthetic training data and then the scaling just can just keep going forever and so that's why it's sort of funny that you know which jobs are you know most at risk from AI you know I think a lot of people thought it was like you know well you're still going to need people are like really really smart quantitative stuff and it's like I got news for you that's like the thing that they're going to become superhuman at before anything else because like you're still going to want to read the history book by like a really smart human before you read you know the AI's history book but like the AI mathematician might you know you pretty good you know two years from now like three years from right Jeffrey we heavily talked about your article which I'll have a linked in the show notes and people want to go read that for themselves first hands but also just tell us a little bit more about you like where you come from what you do what else you're working on sure so in my day job for the last couple of years as I found it and CEO of pastel network which is actually a crypto project PSL is our ticker we trade on on a few exchanges like MEXC and gate and so and we started out as sort of NFT platform it's an interesting project it's based on the Bitcoin core proof of work concept but with all this additional layers to it and but sort of in the last year we've we've done a big pivot to decentralized AI inference and so I've written just a tremendous amount of code in the last year to essentially let you do inference across all sorts of modalities all sorts of providers of AI models including totally uncensored models and you don't have to docks yourself by giving an email address and a credit card and your IP address you can just pay with crypto and it's also donimous and it's decentralized it's going it's all the inference is being handled by these super notes that anyone can start and themselves and and you can even I mean the the example I like to joke about is you can use one of these uncensored versions of the Lama models and say like how do I make method home and then you'll actually just tell you exactly the recipe whereas you know good luck trying that on chat GPT or plus is this what you call this the sovereign AI sector yeah yeah but it's really sort of decentralized I mean part of the thinking of it was that you know for me is not necessarily on the consumer level like chat GPT although I did make something like that it's like if you go to inference top pastel dot network you can try it all in a browser and you could do inference across all these models but it's also that to make it sort of an API that if you have another crypto project like let's say you have a prediction market and you want to make it so anyone can in a decentralized way create their own prediction event but you want to have some rules around that like you don't want people to make like assassination markets where they're predicting that somebody's in a dive by a certain date so you need to have some kind of moderation you don't want to necessarily have it be that there's a moderator who has like powered it delete stuff right because that how's that decentralized so but so the I think the better way to implement something like that is to have an LLN do it in a totally impartial way where we have this prompt that says you're not allowed to do an event about you know with involving any of these subjects and then you have the user wants to create their prediction event they have to describe what is being predicted at the time they're trying to create that event in the system it's going to show it to an LLN the LLN is going to say yes or no and based on that we're going to say no you can't do this you have to change it now if you have this prediction market that's decentralized you can't really go and use open AI or cloud for this because that requires an API key hooked up to a credit card which means that's not decentralized it can't work like that it has to actually be decentralized so that's the idea of pastels that they could use pastel and they can save with a straight face in all honesty that this is decentralized right down the line that that this is fully decentralized and that it can never be shut down by just turning off this one API key or credit card or you know and so that that's the basic idea and then I have some other you know side projects like my YouTube transcripts optimizer which is where I publish people are very confused that why is this not on medium or something or a sub stack and I'm like sorry it's still funny because I basically was trying to you know help my organic search ranking of my little YouTube tool which you know I've generated like thousand dollars or something of revenue from it and and and then it's like in the process I may have inadvertently contributed to two trillion dollars getting wiped off global equity because you know the fact is I really look I don't want people to say that I'm some mega little maniac here but the fact of the matter is all of the news headlines came out saying the stock market crash because deep sea and it's like I'd like to point out that the deep sea V3 technical paper that talked about the efficiency gains it came out December 27th okay that's a month ago and all these famous people like Andre Karpathy were all over this talking about this weeks ago on Twitter even the newer model the R1 model that does the chain of thought that paper came out a week ago and people were all over that so why suddenly on Monday did everything crash and I'd like to think it's because I'm pretty sure it is that I wrote this article in a way that sort of speaks to hedge fund managers it so they can understand it and I published it like in the middle you know the night on friday and then it started taking off and then it got shared by charmeth who has you know whatever 1.
8 million right and to moth and it's been viewed over two million times the Valorov account has two and a half million and then like the y-combinator Gary Tan and the y-combinator account between them they have millions of followers and not only did they share it but they were like very effusive in their praise about this is really smart and that went crazy and I can tell you I have been inundated by you know requests from huge funds that want to talk to me about this and I believe that it did in fact you know as crazy as it sounds precipitate the decline obviously I didn't cause it it was caused by the underlying situation but in terms of highlighting it it didn't come from the you know investment banks and and and and and I think part of the problem is just people like are talking in different circles like like the they're not like the people who are buying and video you know with you know billions of dollars at a big fund are not reading the technical papers and they're not even necessarily reading the tweets from Andre Carpathi you know they're they're just relying on sort of this consensus of of of of of where things are going and all it took was sort of a really in-depth explanation that made sense to them and they were like holy shit I didn't know this and you know what I say one other funny thing is I have because it's running on my own blog I have Google Analytics I can see real time like who's you know not cool but where they are and it's so funny because I just when it started going far at first I was so thrilled that 50 people were reading it at once and then it was like before I knew it was like 1500 people at any given moment and it's a 60 minute reads 12,000 words so it's not sure and and but at first it was like mostly guys in New York because that's where all the hedge ones are and but then I noticed right before I fell asleep on Saturday night that you know where the biggest place where people are reading it was San Jose and I'm like who that sounds like Nvidia is based right and then because there were like hundreds of people from San Jose reading the thing at the same time and as of yesterday is when I last checked over 2000 people from San Jose read my article and you know the funny thing about Nvidia is that they've gone up so much that something like 80% of the employees are having more than like 10 million dollars with the stock and you know it's like the main thing that they talk about with their spouse and friends of like man I have a lot of the stock should I keep staying on for this ride or and they understand the technology but maybe they don't understand how to value a company and they read this and this thing started passing around like a wildfire and I was like oh my god if it's you Jensen's reading this too and and I think there's a lot of stock that's sort of never hit the market because it was awarded as artists use and options to these people and it only takes a little bit of that to on the margin to start to cause me a balance and so I wouldn't be surprised if a lot of that cell pressure came from Nvidia employees but also it's like these big hedge funds like control a lot of there's a lot of fast money players and they suddenly got spooked and it's like so it's wild to think about that you know it could have actually been this sort of you know the Reichstag fire if you will of setting off this whole course of events but I actually do believe without it being so you know I mean I'm sure there are people who say no about this other guy wrote this and this other guy wrote this and I was like yeah but my thing went pretty freaking viral and from the right and other people's stuff cited yours your article sure or you know maybe they didn't I mean I saw the guy Ben Thompson from stature it's sort of sounded like he paraphrased my thing without giving me any credit but whatever you know it's um but but I just think it's really funny how like there's headline stories today from the New York Times and the Wall Street Journal that both said you know they're always trying to assign causality to stuff and they said it was caused by this and it was like not really that because it's a ludicrous concept that the 45x efficiency gains that was known a month ago so you have to explain why there was a one-month lag okay whereas like this is very understandable that this spread like wildfire from thought leaders like to moth and and the vol like I mean the vol is like put on such a pedestal by the VC guys and you know and the VC day and the tech hedge fund guys look up to the VC guys like like andryson are what's in all these guys and and the white combinator guys they're the experts right and then you have those guys saying that this is a great article and it's like well okay and so of course that can very quickly convince and it's not like you have to convince everyone you just have to convince you know the guys that like co-tube were managing $70 billion that they should maybe like sell a little bit to get in front of this and you're you know that's what you need and so um anyway I emailed both of the journalist at least you know you should be aware you may have gotten the causality a little wrong anyway well Jeffrey it's an honor to have the original source of the information on the podcast it's great to have you as a guest and perhaps as these AI wars and video chip wars USA we didn't even get a chance to talk about USA versus China but as this progresses maybe we can get you back on to keep on commentating yeah really appreciate you coming on thanks a lot bankless nation you guys know the deal crypto is risky you can lose what you put in uh but it sounds like the traditional market is also risky too but we're headed west this is the frontier it's not for everyone but we're glad to be here with us on the bankless journey thanks a lot