<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>AI &amp; GPU compute - Soursen</title><description>AI and GPU compute covers GPU supply and the cost of running a model, starting from the vendor note or the paper that states the number.</description><link>https://soursen.org/</link><language>en-gb</language><atom:link href="https://soursen.org/beat/ai-gpu/rss.xml" rel="self" type="application/rss+xml"/><item><title>Nvidia’s Board Backed Jensen Huang on Sunday. By Thursday He Was Gone.</title><link>https://soursen.org/articles/nvidias-board-backed-jensen-huang-on-sunday-by-thursday-he-was-gone/</link><guid isPermaLink="true">https://soursen.org/articles/nvidias-board-backed-jensen-huang-on-sunday-by-thursday-he-was-gone/</guid><description>Four days after Nvidia’s board said it had confidence in its CEO, Jensen Huang resigned. A short seller had flagged the Megaspeed connection 6 months earlier, and most of the story rests on anonymous US officials, so I separate the filed facts from the allegations.</description><pubDate>Fri, 13 Nov 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;On November 12, 2026, Nvidia announced that Jensen Huang had resigned as president, CEO and board member, effective immediately. I want to walk through what Nvidia has confirmed, what major newspapers have reported from unnamed officials, what the people involved say, and what the market did. This story involves named people and serious accusations, so I attribute every contested claim and I leave out anything I could not trace to a reliable outlet.&lt;/p&gt;
&lt;h2 id=&quot;what-nvidia-has-confirmed&quot;&gt;What Nvidia has confirmed&lt;/h2&gt;
&lt;p&gt;The &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://nvidianews.nvidia.com/news/nvidia-announces-ceo-transition&quot;&gt;Nvidia press release&lt;/a&gt; is dated November 12, 2026. It says the board announced that Huang resigned as President, CEO and a member of the Board, effective immediately. CFO Colette Kress is interim CEO while she remains CFO. Harvey C. Jones becomes the first Chair of the Board, and the board is searching for a permanent CEO from inside and outside the company.&lt;/p&gt;
&lt;p&gt;The &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.sec.gov/Archives/edgar/data/0001045810/000104581026000082/nvda-20261111.htm&quot;&gt;Form 8-K filed with the SEC&lt;/a&gt; adds a detail on timing. It says the board accepted the resignation on November 11, 2026, and appointed Jones, who is 73 and has served on the board since 1993, as chair that day. Kress is 59. Neither document gives a reason for the departure. The press release adds that Kress joined Nvidia in September 2013 after nearly 25 years in finance roles at Cisco and Microsoft, and that Jones is a former CEO and Executive Chairman of Synopsys. Nvidia also said it will report third quarter fiscal 2027 results on Tuesday, November 17, at about 1:20 p.m. Pacific, with its call at 2 p.m. Pacific, as previously scheduled.&lt;/p&gt;
&lt;h2 id=&quot;the-reporting-that-led-to-it&quot;&gt;The reporting that led to it&lt;/h2&gt;
&lt;p&gt;Everything about why this happened comes from news reports, so I treat it that way. According to the &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tovima.com/wsj/how-jensen-huangs-affair-spiraled-into-a-national-security-nightmare/&quot;&gt;Wall Street Journal’s account republished by To Vima&lt;/a&gt;, Alice Huang, a former reporter for Chinese state media, joined Megaspeed in 2023. Megaspeed is a Singapore based company that spun off from the Chinese firm 7Road and advertised itself as a cloud provider of Nvidia hardware. She is not related to Jensen Huang. The WSJ reports they met at an Nvidia event in California in March 2024, and that an attending executive heard her call him a friend at a Taipei trade show bar in June 2024. The paper also reports they were photographed at a Taipei dinner in May 2025.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sovereignmagazine.com/article/nvidia-jensen-huang-investigation&quot;&gt;Sovereign Magazine’s summary&lt;/a&gt; of the later reporting says the New York Times reported on October 29 that Trump was briefed about a year earlier and chose not to act. On November 6, it says the Times reported that a classified US memo says she was recruited by China’s Ministry of State Security, and the Financial Times reported that the same memo describes a romantic relationship between her and Huang. The WSJ adds that Secretary of State Marco Rubio reportedly oversaw a classified memo in spring 2026 warning of Huang’s personal relationship with a suspected Chinese spy.&lt;/p&gt;
&lt;p&gt;These are reports of what officials believe. The WSJ notes that its account of US conclusions comes from reporting by several outlets and that efforts to reach Alice Huang were unsuccessful. No public document I found shows evidence beyond what unnamed officials told reporters.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.B6eQ_ilH_Z1lSVx.webp&quot; alt=&quot;From a California event to a resignation in 32 months. The reported dates run from a March 2024 meeting to the November 12 announcement.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;744&quot; height=&quot;351&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;From a California event to a resignation in 32 months. The reported dates run from a March 2024 meeting to the November 12 announcement. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sovereignmagazine.com/article/nvidia-jensen-huang-investigation&quot;&gt;Nvidia 8-K, WSJ, NYT and FT as relayed by Sovereign Magazine, November 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;a-short-seller-flagged-megaspeed-6-months-earlier&quot;&gt;A short seller flagged Megaspeed 6 months earlier&lt;/h2&gt;
&lt;p&gt;The paper trail started before any of the November headlines. On May 13, 2026, Culper Research published a 40 page report titled NVIDIA (NVDA): The China Problem, according to &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://officechai.com/ai/how-short-seller-culper-research-had-highlighted-jensen-huangs-uncomfortably-close-relationship-with-alice-huang-6-months-before-their-affair-was-revealed/&quot;&gt;OfficeChai&lt;/a&gt;. Culper disclosed a short position, which means it profits if the stock falls, and OfficeChai notes the report leaned on anonymous sources including former Nvidia and Supermicro employees and a current Megaspeed employee. Treat it as an advocacy document and weigh it that way.&lt;/p&gt;
&lt;p&gt;Its central claim, as &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.roic.ai/news/short-seller-targets-nvidia-over-alleged-china-exposure-through-intermediaries-05-13-2026&quot;&gt;roic.ai summarized it&lt;/a&gt;, was that more than 20% of Nvidia’s fiscal 2026 compute revenue stayed tied to China through GPU diversion via Southeast Asian intermediaries. The report named Megaspeed and alleged hidden financial links to Alibaba. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://aichief.com/news/short-seller-raises-alarm-on-nvidia-ceos-alleged-ties-to-suspected-chinese-spy/&quot;&gt;AI Chief, citing TechCrunch&lt;/a&gt; says the report also raised questions about Jensen Huang’s unusually close relationship with Alice Huang, and that the diversion allegations have not been independently verified. Nvidia had previously said it found no evidence of diversion in its global data center checks.&lt;/p&gt;
&lt;p&gt;The market did little with it. OfficeChai reports Nvidia shares rose about 3% that day. The same week, per &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://jbiznews.com/?p=371554&quot;&gt;a Benzinga summary&lt;/a&gt;, Huang boarded Air Force One during a refueling stop in Anchorage to join President Trump’s Beijing summit at the last minute, just as the report appeared.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.4Y1SapB3_ZOKi9T.webp&quot; alt=&quot;The warning signs were public for 6 months. Each signal below was reported before the November 12 resignation.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;744&quot; height=&quot;358&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;The warning signs were public for 6 months. Each signal below was reported before the November 12 resignation. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://officechai.com/ai/how-short-seller-culper-research-had-highlighted-jensen-huangs-uncomfortably-close-relationship-with-alice-huang-6-months-before-their-affair-was-revealed/&quot;&gt;OfficeChai,&lt;/a&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;http://roic.ai/&quot;&gt;roic.ai&lt;/a&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://officechai.com/ai/how-short-seller-culper-research-had-highlighted-jensen-huangs-uncomfortably-close-relationship-with-alice-huang-6-months-before-their-affair-was-revealed/&quot;&gt;, Sovereign Magazine, Axios, MarketWatch, Nvidia&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;what-huang-and-the-board-said&quot;&gt;What Huang and the board said&lt;/h2&gt;
&lt;p&gt;On Sunday, November 8, the WSJ reports that Huang apologized to staff for infidelity in a memo and wrote that he loves the country and the company. Sovereign Magazine quotes the memo as saying he was “deeply sorry” and would cooperate fully with the national security review. The same day, the board opened an independent investigation run by a committee of independent directors with outside counsel. It said it had no reason to believe national security or Nvidia secrets were compromised, and it expressed confidence in Huang and his leadership team.&lt;/p&gt;
&lt;p&gt;4 days later he was gone. Whether Huang told the board about the relationship earlier has not been made public, and Sovereign Magazine describes it as a question for the investigation. Nvidia spokesman John Rizzo had said the news would not slow the company.&lt;/p&gt;
&lt;h2 id=&quot;what-the-other-parties-say&quot;&gt;What the other parties say&lt;/h2&gt;
&lt;p&gt;Nvidia told reporters, according to the WSJ, that US intelligence never gave Huang a defensive briefing. The company says its ban on sales and shipments to Megaspeed from June 2025 remains in place, and that it has obtained Commerce permission to send parts and replacement components since then. Sovereign Magazine reports Nvidia says the diversion allegations were investigated and debunked last year, and that Megaspeed has no China shareholders.&lt;/p&gt;
&lt;p&gt;Megaspeed says it is legitimate and compliant. It says the Commerce Department’s Bureau of Industry and Security, Malaysia’s trade ministry and Nvidia each found no evidence of illicit chip transfers, and it has hired outside counsel. Singapore’s Ministry of Home Affairs denies investigating Alice Huang over alleged China ties, according to the WSJ. China’s foreign ministry spokesman Guo Jiakun called the allegation groundless and a smear.&lt;/p&gt;
&lt;p&gt;President Trump said Huang told him he had no idea the person was a spy, and said he did not recall receiving the classified memo, according to the WSJ. Sovereign Magazine reports he also called Huang an honorable guy before the resignation.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.55HMpSs7_29f4Jj.webp&quot; alt=&quot;5 parties, 5 different positions. Each statement below was reported in November 2026.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;744&quot; height=&quot;338&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;5 parties, 5 different positions. Each statement below was reported in November 2026. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tovima.com/wsj/how-jensen-huangs-affair-spiraled-into-a-national-security-nightmare/&quot;&gt;WSJ, Sovereign Magazine, November 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-chip-diversion-question-is-still-open&quot;&gt;The chip diversion question is still open&lt;/h2&gt;
&lt;p&gt;The Megaspeed question has 2 layers. Sovereign Magazine reports that Megaspeed’s Malaysian sister company bought at least $4.8 billion of AI equipment containing Nvidia chips between June 2024 and February 2026. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fourweekmba.com/?p=293963&quot;&gt;FourWeekMBA, relaying a Techmeme summary&lt;/a&gt; says Bloomberg reported on November 9 that Megaspeed’s imports of Nvidia products top $4.6 billion, so the public figures sit in the $4.6 billion to $4.8 billion range depending on the source and the entity counted. It also says Commerce officials visited the Malaysian data center in late 2024 and found Nvidia equipment still in its boxes, and that the case later went to the Justice Department. Senator Tom Cotton had asked Commerce Secretary Howard Lutnick on October 13, 2025, to examine Megaspeed’s ownership and ties.&lt;/p&gt;
&lt;p&gt;On the other side, the same summary says no one has been charged and no public finding shows chips were diverted. The WSJ reports that the Commerce Bureau of Industry and Security responded to a New York Times report that it dropped an investigation after an Nvidia executive reached out. A BIS spokesman said it conducts all enforcement strictly by the book. I cannot settle that from the public record, and I would treat anyone who claims to have settled it with caution.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-04.BRfJLQs1_ZOlV1f.webp&quot; alt=&quot;What is on the record and what is not. Only the resignation itself is confirmed by Nvidia filings.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;744&quot; height=&quot;358&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;What is on the record and what is not. Only the resignation itself is confirmed by Nvidia filings. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.sec.gov/Archives/edgar/data/0001045810/000104581026000082/nvda-20261111.htm&quot;&gt;Nvidia 8-K, WSJ, NYT, FT as relayed by Sovereign Magazine&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;what-the-market-did&quot;&gt;What the market did&lt;/h2&gt;
&lt;p&gt;Sovereign Magazine reports Nvidia closed at $186.64 on Friday, November 6, then opened at $173.04 on Monday, about 7% lower. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://beincrypto.com/jensen-huang-resigns-from-nvidia/&quot;&gt;BeInCrypto reported&lt;/a&gt; that shares fell 11.3% to $154.10 in premarket trading on November 12. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://qz.com/jensen-huang-resigns-nvidia-ceo-affairs-stock-sinks-111226&quot;&gt;Quartz reported&lt;/a&gt; the stock then traded about 8% lower shortly after the open, and that Nvidia was on track to have lost almost three quarters of a trillion dollars of market value since the scandal broke. Quartz also notes the stock had already fallen by more than a fifth in the past month.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-05.B0oNR34K_fA3ha.webp&quot; alt=&quot;Nvidia shares were about 17% lower by Nov 12 premarket. The premarket figure is a quote before the opening bell.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;744&quot; height=&quot;311&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Nvidia shares were about 17% lower by Nov 12 premarket. The premarket figure is a quote before the opening bell. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://beincrypto.com/jensen-huang-resigns-from-nvidia/&quot;&gt;Sovereign Magazine, BeInCrypto, November 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-succession-gap&quot;&gt;The succession gap&lt;/h2&gt;
&lt;p&gt;The exit also exposed a planning question. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.axios.com/2026/11/09/nvidia-jensen-huang-succession-plan&quot;&gt;Axios&lt;/a&gt; reported on November 9 that Nvidia had no obvious successor, with Kress as the highest profile member of the executive team, and quoted Georgetown ethics professor Peter Jaworski saying the board looked “asleep at the wheel.” &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.morningstar.com/news/marketwatch/2026110971/there-is-no-one-at-nvidia-who-would-be-ready-to-replace-jensen-huang&quot;&gt;MarketWatch&lt;/a&gt; made the same point on the same day, noting that apart from Kress few Nvidia executives are well known to investors. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.ibtimes.com/nvidia-jensen-huang-succession-plan-investigation-3809602&quot;&gt;IBTimes&lt;/a&gt; adds that Nvidia has not publicly disclosed a detailed CEO succession plan, although the board could have confidential arrangements for an emergency.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.mitrade.com/insights/news/live-news/article-8-2244971-20261110&quot;&gt;Mitrade&lt;/a&gt; puts Huang’s equity stake at about 3.58% and argues his control rests mainly on being co founder and CEO. The board’s answer, 3 days later, was an interim CEO who keeps her CFO job, a new chair who has sat on the board since 1993, and an open search.&lt;/p&gt;
&lt;h2 id=&quot;what-happens-next&quot;&gt;What happens next&lt;/h2&gt;
&lt;p&gt;3 dates are on the calendar. The House Foreign Affairs, China Select and Intelligence committee chairs asked Huang to attend a closed briefing on November 18, and Sovereign Magazine says they had not issued a subpoena. Nvidia reports earnings on November 17, the first test of investor confidence under an interim CEO. The board’s independent investigation has no published end date, and the CEO search is open to outside candidates.&lt;/p&gt;
&lt;h2 id=&quot;2-things-from-this-generalize&quot;&gt;2 things from this generalize&lt;/h2&gt;
&lt;p&gt;The first is how to read a story built on unnamed officials. The newspapers named their sources as US officials and the people accused deny the central claim, so the sound approach is to hold the claim at the strength it has. A report that officials believe something tells me what officials believe, and it turns into a finding when documents, charges or on the record statements arrive.&lt;/p&gt;
&lt;p&gt;The second is what a leadership exit does to the numbers. Shares sat about 17% lower within 4 trading days, before any proof of diversion or compromise was made public, which shows how much of a stock price rests on trust in a single executive. Investors who own Nvidia through index funds are exposed to that risk without having chosen it.&lt;/p&gt;
&lt;p&gt;This week, I would read the November 17 earnings release and any board statement about the investigation, and update my view only from filed documents.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>nvidia</category><category>ai</category><category>gpu</category><category>artificial-intelligence</category><category>ai-agent</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>25 GPUs for Local AI Ranked by Value, and a 2018 Server Card Takes Top Tier</title><link>https://soursen.org/articles/25-gpus-for-local-ai-ranked-by-value-and-a-2018-server-card-takes-top/</link><guid isPermaLink="true">https://soursen.org/articles/25-gpus-for-local-ai-ranked-by-value-and-a-2018-server-card-takes-top/</guid><description>A November 2026 price snapshot grades 25 cards on dollars per gigabyte and dollars per token per second. The AMD Instinct MI50 and a used RTX 3060 take S tier, the RTX 5080 lands in F, and memory decides almost every grade.</description><pubDate>Tue, 10 Nov 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;A 96 GB RTX Pro 6000 now lists at &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tweaktown.com/news/113098/nvidias-rtx-pro-6000-blackwell-gpu-price-almost-doubles-reaching-16-000/index.html&quot;&gt;$16,000 on Nvidia’s own store&lt;/a&gt;, which TweakTown reports is 87% above its $8,565 launch price. A 32 GB AMD Instinct MI50, a server card from 2018, sells for about $649 on eBay. In the snapshot behind this ranking, the cheapest Pro 6000 costs $17,000, or about $177 per gigabyte of memory, and the MI50 costs about $20. I want to walk through a grade for 25 cards, from S down to F, built on 2 numbers, and explain why some of the newest cards sit at the bottom of the list.&lt;/p&gt;
&lt;p&gt;The prices and speeds come from a November 3, 2026 snapshot of retail and marketplace listings plus llama.cpp results that owners post from their own cards. I could not check every listing live again, and card prices have moved every week this year, so read each figure as a dated reading. The first 20 cards come from that snapshot. I added 5 more, marked with an asterisk in the first chart, using the freshest dated listings I could verify, which come from TechSpot’s price tracker for 4 of them and GPUPrix for the RTX 4060 Ti. Those 5 have no owner speed results, so I graded them on memory price, bandwidth and software. Specs, launch prices and software support dates I checked against vendor and press sources, and the links are below.&lt;/p&gt;
&lt;h2 id=&quot;2-numbers-decide-every-grade&quot;&gt;2 numbers decide every grade&lt;/h2&gt;
&lt;p&gt;The first number is dollars per gigabyte of memory. Memory decides what a card can run at all. Think of it as the size of a desk. The model has to fit on the desk with room left over for the conversation, which grows as you keep chatting, and a model that does not fit cannot run however fast the card is. My yardstick is the used RTX 3090, which works out to $72 per gigabyte in this snapshot. Anything well under that is cheap memory, and anything well over it is expensive.&lt;/p&gt;
&lt;p&gt;The second number is dollars per token per second. A token is roughly 0.75 of a word, and tokens per second is how fast the AI types. This number says how fast a card feels for the money. The speeds come from public llama.cpp results, which let NVIDIA, AMD and Intel cards be compared on one scale, and I used owner reports on a Qwen model where they exist because that is closer to real use.&lt;/p&gt;
&lt;p&gt;My reference model is a 27B parameter Qwen model compressed to 4 bits. The raw weights come to 13.5 GB by my arithmetic (27 billion parameters at half a byte each), and the file in the snapshot is about 18 GB once the compression format’s overhead is counted. As a cross check, the Q4_K_M file for &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://huggingface.co/bartowski/Qwen_Qwen3.5-27B-GGUF&quot;&gt;Qwen3.5 27B on Hugging Face&lt;/a&gt; is listed at 17.98 GB, and the model supports a 262,144 token context window according to &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://llmrun.dev/model/qwen-qwen3-5-27b&quot;&gt;llmrun&lt;/a&gt;. A 24 GB card runs it with room to spare. A 16 GB card needs heavier compression, which costs some quality.&lt;/p&gt;
&lt;p&gt;The grades work like this. S gives the most model for the least money with no real catch. A is excellent with 1 catch. B is a good buy for the right person. C is fine, but something else does the same job for less. D is a good card at a bad price, and F is a trap, a card that looks like value until you try to load a model on it.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.CRTVEiWS_12DNuq.webp&quot; alt=&quot;Dollars per gigabyte of memory for 25 cards, colored by tier. The dashed line marks the used RTX 3090 yardstick at $72. Starred cards were added from Jul to Oct 2026 listings.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;1513&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Dollars per gigabyte of memory for 25 cards, colored by tier. The dashed line marks the used RTX 3090 yardstick at $72. Starred cards were added from Jul to Oct 2026 listings. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.pcguide.com/news/gpu-pricing-is-getting-much-worse-its-hard-to-see-an-end-in-sight/&quot;&gt;Nov 3 2026 listing snapshot, vendor specs, TechSpot, GPUPrix&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;memory-sets-the-ceiling-and-bandwidth-sets-the-speed&quot;&gt;Memory sets the ceiling and bandwidth sets the speed&lt;/h2&gt;
&lt;p&gt;Generating text for 1 user works by reading every weight in the model out of memory to produce each new token. That makes single user generation limited by memory bandwidth, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://inventivehq.com/blog/local-llm-performance-what-to-expect&quot;&gt;Inventive HQ’s rule of thumb&lt;/a&gt; puts the ceiling at bandwidth divided by the bytes read per token. I ran that on 3 cards using the 18 GB file. The RTX 3090 moves 936 GB/s, so its ceiling is about 52 tokens per second, and owners report 40, which is 77% of it. The Arc Pro B70 moves 608 GB/s for a ceiling near 34, and owners report 26, also 77%. The RTX 5090 moves 1,792 GB/s for a ceiling near 100 against 75 reported, or 75%. Real speeds landing in that band is what the bandwidth math predicts.&lt;/p&gt;
&lt;p&gt;The same math explains the Tesla P40. NVIDIA’s &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://images.nvidia.com/content/pdf/tesla/Tesla-P40-Product-Brief.pdf&quot;&gt;product brief&lt;/a&gt; lists 24 GB of GDDR5 with up to 347 GB/s, so the ceiling on the 18 GB file is about 19 tokens per second before any overhead. The snapshot has no owner number for it, and I would expect something under that ceiling.&lt;/p&gt;
&lt;p&gt;Context length matters too. The KV cache holds the conversation so far, and every new token reads it again. Inventive HQ’s &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://inventivehq.com/tools/developer/llm-inference-speed-calculator&quot;&gt;speed calculator&lt;/a&gt; estimates an RTX 4090 running an 8B model at about 116 tokens per second with 4,096 tokens of context and about 29 at 131,072. A card with a fast bus and a small memory pool suffers on both counts as chats grow.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.C-WqWF5w_Z1tpSlk.webp&quot; alt=&quot;Memory bandwidth against dollars per gigabyte for 7 cards. The MI50 combines near 1 TB/s with about $20 per gigabyte.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1488&quot; height=&quot;838&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Memory bandwidth against dollars per gigabyte for 7 cards. The MI50 combines near 1 TB/s with about $20 per gigabyte. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.amd.com/en/support/downloads/drivers.html/accelerators/instinct/instinct-mi-series/instinct-mi50-32gb.html&quot;&gt;NVIDIA, AMD, Intel, TechSpot, TweakTown&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;what-owners-actually-measure&quot;&gt;What owners actually measure&lt;/h2&gt;
&lt;p&gt;Here are the owner reported speeds from the snapshot. Quantization and setups differ by card, so they are a guide and a lab benchmark would be tighter. A fast reader at 250 words a minute needs about 5 to 6 tokens per second, by my arithmetic at 0.75 of a word per token, so every card in the chart outruns reading speed. The RTX 5090 leads at 75. The used RTX 4090 reaches 46 and the used RTX 3090 reaches 40, which is about 15% more speed for twice the money. The Arc A770 trails at 9, which is readable and slow.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.DNsPKrFq_1PniBr.webp&quot; alt=&quot;Owner reported tokens per second on a 27B Qwen model. Quantization differs by card, and the MI50 result uses 8 bit weights.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1488&quot; height=&quot;838&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Owner reported tokens per second on a 27B Qwen model. Quantization differs by card, and the MI50 result uses 8 bit weights. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/ggml-org/llama.cpp&quot;&gt;Owner posted llama.cpp results, Nov 2026 snapshot&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;f-tier-is-where-good-money-buys-too-little-memory&quot;&gt;F tier is where good money buys too little memory&lt;/h2&gt;
&lt;p&gt;The RTX 5060 with 8 GB costs about $460 new. An owner measured 97 tokens per second on the short llama.cpp benchmark, the cheapest speed in the whole set, and that number is the only good thing the card does here. The 27B Qwen model does not fit on it even with heavy compression. Smaller models run fine, so keep one if you own it and look elsewhere before buying for this job.&lt;/p&gt;
&lt;p&gt;The RTX 5070 with 12 GB costs about $900. That is $75 per gigabyte, above the used 3090 yardstick, for half the memory. The RTX 5070 Ti with 16 GB costs about $1,350 at $84 per gigabyte.&lt;/p&gt;
&lt;p&gt;The RTX 5080 annoys me most. The cheapest 16 GB listing in the snapshot was a marketplace posting at $2,300, or $144 per gigabyte, while the RTX 5060 Ti with the same 16 GB costs about $790. That is $1,510 extra for a card about twice as fast that loads exactly the same models. Model size sets what you can run at all, and the 5080 earns its F on that basis.&lt;/p&gt;
&lt;p&gt;Two more cards joined the F pile. The RTX 4070 Ti Super with 16 GB costs about $1,299 on &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techspot.com/specs/gpu/290250-nvidia-geforce-rtx-4070-ti-super.html&quot;&gt;TechSpot’s tracker&lt;/a&gt; as of August 9, 2026, which is $81 per gigabyte for the same 16 GB the $790 5060 Ti offers. Its &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techspot.com/review/2793-nvidia-rtx-4070-ti-super/&quot;&gt;256 bit bus&lt;/a&gt; with 21 Gbps memory works out to 672 GB/s by my arithmetic. GPUPrix showed a lower $880 listing, which would be $55 per gigabyte and move the card up, so check the live price. The RTX 4070 Ti with 12 GB costs $1,222 on &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techspot.com/specs/gpu/258458-nvidia-geforce-rtx-4070-ti.html&quot;&gt;TechSpot’s tracker&lt;/a&gt; as of July 2, 2026. That is $102 per gigabyte for 12 GB at 504 GB/s, the highest memory price of any 12 GB card here.&lt;/p&gt;
&lt;h2 id=&quot;d-tier-holds-good-cards-at-prices-that-undo-them&quot;&gt;D tier holds good cards at prices that undo them&lt;/h2&gt;
&lt;p&gt;The used RTX 4090 with 24 GB costs about $3,500. In the review used for the snapshot it ran Qwen at 46 tokens per second, and a used 3090 with the same 24 GB did 40 for about half the money. The same models fit on both.&lt;/p&gt;
&lt;p&gt;The RTX Pro 6000 is the strangest grade on the list. It is the only card here with 96 GB, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.storagereview.com/review/nvidia-rtx-pro-6000-workstation-gpu-review-blackwell-architecture-and-96-gb-for-pro-workflows&quot;&gt;StorageReview lists it at 1.79 TB/s of bandwidth&lt;/a&gt;, the same as the RTX 5090. In the review behind the snapshot, the full power version runs a 70B class Llama model on a single card at 32 tokens per second, which nothing else here can do. At $177 per gigabyte it costs more than double the yardstick. If you need a model that size on 1 card for work, it is your only answer here, and for everyone else it is a very nice D.&lt;/p&gt;
&lt;p&gt;The Intel Arc Pro B70 with 32 GB is a D because of this week’s price. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.pcgamer.com/hardware/graphics-cards/intel-launches-the-arc-pro-b70-graphics-card-based-on-the-big-battlemage-gpu-weve-been-waiting-for-forever-but-its-for-ai-not-gaming&quot;&gt;PC Gamer reported its launch&lt;/a&gt; at $949 in March 2026, the snapshot saw it at $1,300 at Newegg earlier, and the cheapest listing now is $1,946 from a marketplace seller. At that price it costs $61 per gigabyte for 26 tokens per second on Qwen, which is more money for less speed than the Radeon AI Pro R9700 in B tier. StorageReview calls it promising and, at times, frustrating. At $1,300 I would move it up 2 tiers.&lt;/p&gt;
&lt;p&gt;The RTX 4060 Ti with 16 GB lands in D at about $843, the lowest new listing GPUPrix tracked on &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://gpuprix.com/us/gpus/geforce-rtx-4060-ti-16gb?show=listings&quot;&gt;October 21, 2026&lt;/a&gt;. That is $53 per gigabyte and 69% above its &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tomshardware.com/pc-components/gpus/rtx-5060-ti-16gb-vs-rtx-4060-ti-16gb-gpu-faceoff&quot;&gt;$499 launch price&lt;/a&gt;. The RTX 5060 Ti has the same 16 GB for about $53 less in my snapshot, with 448 GB/s against this card’s 288 GB/s.&lt;/p&gt;
&lt;h2 id=&quot;c-tier-splits-into-fast-but-pricey-and-cheap-but-slow&quot;&gt;C tier splits into fast but pricey and cheap but slow&lt;/h2&gt;
&lt;p&gt;The RTX 5090 with 32 GB costs about $4,921, against a &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://digitaltrends.com/computing/asus-launches-its-first-intel-arc-graphics-card-with-32gb-of-memory-for-workstations&quot;&gt;$1,999 launch price&lt;/a&gt;. It is the fastest card on the list at 75 tokens per second on Qwen, far faster than anyone reads, and its 32 GB fits models no 24 GB card can hold. The price works out to $154 per gigabyte, more than double the yardstick. Per token per second it costs about $66 against about $43 for a used 3090, roughly 50% more. A buyer who wants only speed will be happy with it, and the value grade is a C.&lt;/p&gt;
&lt;p&gt;The rest of C has cheap memory and slow cards. The used Intel Arc A770 with 16 GB costs about $325, close to $20 per gigabyte, and one owner gets about 9 tokens per second on Qwen. The used Tesla T4 with 16 GB costs about $430, or $27 per gigabyte, and sips power at 70 W, but it is the slowest card in the benchmarks here. It is also &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://lenovopress.lenovo.com/lp1926.pdf&quot;&gt;sold as a passively cooled card&lt;/a&gt;, which means a server pushing air through it, and a desk at home is a poor substitute.&lt;/p&gt;
&lt;p&gt;The AMD RX 9060 XT with 16 GB costs about $650. One owner gets about 21 tokens per second on a heavier compression of Qwen, which is usable. For $140 more, the RTX 5060 Ti in B tier does the same job at 29.&lt;/p&gt;
&lt;p&gt;The AMD RX 7600 XT with 16 GB costs about $595 on &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techspot.com/products/gpu/amd-radeon-rx-7600-xt.290361/&quot;&gt;TechSpot’s tracker&lt;/a&gt; as of August 12, 2026, which is $37 per gigabyte and $55 below the RX 9060 XT’s snapshot price. It launched at &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.anandtech.com/show/21215&quot;&gt;$329&lt;/a&gt; on a 128 bit bus, and its 288 GB/s is 64% of the 5060 Ti’s 448 GB/s. A 16 GB card needs the Q3_K_M version of the 27B model, which &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://insiderllm.com/guides/qwen35-local-guide-which-model-fits-your-gpu/&quot;&gt;InsiderLLM lists at about 14 GB&lt;/a&gt;, so the bandwidth rule of thumb puts its ceiling near 20 tokens per second. That is usable, and the 5060 Ti has a ceiling more than 50% higher for $195 more.&lt;/p&gt;
&lt;h2 id=&quot;b-tier-has-6-buys-for-the-right-person&quot;&gt;B tier has 6 buys for the right person&lt;/h2&gt;
&lt;p&gt;The Tesla P40 with 24 GB costs about $400 used, and at $16.67 per gigabyte it has the cheapest memory of all 25 cards. Qwen fits with room for a long conversation. It misses S tier because software is moving on without it. NVIDIA’s &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.nvidia.com/cuda/archive/13.3.0/cuda-features-archive/index.html&quot;&gt;CUDA toolkit notes&lt;/a&gt; say offline compilation and library support for Maxwell, Pascal and Volta were removed in CUDA 13, so you stay on CUDA 12. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.vllm.ai/en/v0.13.0/getting_started/installation/gpu/&quot;&gt;vLLM’s installation guide&lt;/a&gt; lists compute capability 7.0 or higher as its GPU requirement, and the P40 is 6.1, so that serving tool is off the table while llama.cpp still runs it. The P40 is also a 250 W board that &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://images.nvidia.com/content/pdf/tesla/Tesla-P40-Product-Brief.pdf&quot;&gt;ships passively cooled&lt;/a&gt;, so plan for a fan. I would buy one as a cheap second card.&lt;/p&gt;
&lt;p&gt;The Intel Arc Pro B60 with 24 GB costs about $1,000, roughly $42 per gigabyte, and Qwen fits with room to talk. Typical owner speeds sit around 22 tokens per second, comfortably faster than reading. The catch is Intel’s software. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://pytorch-extension.intel.com/&quot;&gt;Intel archived its Extension for PyTorch&lt;/a&gt; and set end of life for March 2026, and it recommends using PyTorch directly because Intel GPU support has been upstreamed into it. Builds of llama.cpp fill the gap for local chat.&lt;/p&gt;
&lt;p&gt;The RTX 5060 Ti with 16 GB costs about $790 and runs Qwen at about 29 tokens per second with heavier compression. Nobody gets excited about it. It is new, it is NVIDIA, and everything works on day one, which saves weekends of troubleshooting.&lt;/p&gt;
&lt;p&gt;The AMD RX 9070 XT with 16 GB costs about $920. One owner gets 36.5 tokens per second on Qwen with a custom setup. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.3/reference/system-requirements.html&quot;&gt;AMD’s ROCm support table&lt;/a&gt; lists it as supported, and the plain RX 9070 at about $700 has the same 16 GB, which makes it a fair buy for anyone dodging the NVIDIA premium.&lt;/p&gt;
&lt;p&gt;The Radeon AI Pro R9700 with 32 GB costs about $1,800 in the snapshot, the cheapest new 32 GB card I found. AMD &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://hothardware.com/news/amd-confirms-radeon-ai-pro-r9700-gpu-pricing-release-date&quot;&gt;launched it at $1,299&lt;/a&gt; on October 27, 2025, and Qwen runs at 29.8 tokens per second. It stays in B because a used card in A tier gets you most of that speed for less.&lt;/p&gt;
&lt;p&gt;The Intel Arc B580 with 12 GB costs about $369 on &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techspot.com/specs/gpu/302088-intel-arc-b580.html&quot;&gt;TechSpot’s price tracker&lt;/a&gt; as of August 26, 2026, which is $31 per gigabyte and almost the same as the used RTX 3060 at $370. Its bus delivers &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://gpuprix.com/us/gpus/arc-b580?alert=true&quot;&gt;456 GB/s&lt;/a&gt; against 360 GB/s on the 3060, a 27% edge, and it launched at $249. It sits in B and the 3060 sits in S because of software, since Intel archived its PyTorch extension while the 3060 runs everything on day one. Its 12 GB cannot hold the 18 GB Qwen file, which makes it a card for smaller models.&lt;/p&gt;
&lt;h2 id=&quot;a-tier-is-2-used-cards-with-1-catch-each&quot;&gt;A tier is 2 used cards with 1 catch each&lt;/h2&gt;
&lt;p&gt;The used RTX 3090 with 24 GB costs about $1,727. Every guide recommends it, and it earns that. You get 24 GB, CUDA and about 40 tokens per second on Qwen, several times reading speed, and every tool runs on it. It launched at &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techspot.com/products/gpu/nvidia-geforce-rtx-3090-24gb-gddr6x-pcie.224809/&quot;&gt;$1,499 in September 2020&lt;/a&gt;, so a used card now sells above its original price. The $72 per gigabyte yardstick keeps it at A. Prices vary by tracker, and one price tracker showed &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://gpuprix.com/us/gpus/geforce-rtx-3090?show=listings&quot;&gt;renewed 3090s from $1,830 on October 22&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The used RX 7900 XTX with 24 GB costs about $1,259. It is the one I would look at hardest. It has the same 24 GB for $468 less than the 3090, owners report about the same speed on Qwen, and on the short llama.cpp benchmark it is faster. The catch is software, because image tools and some training tools still lean on NVIDIA. For chatting and coding with a local model, it is the better buy.&lt;/p&gt;
&lt;h2 id=&quot;s-tier-is-a-2018-server-card-and-a-used-rtx-3060&quot;&gt;S tier is a 2018 server card and a used RTX 3060&lt;/h2&gt;
&lt;p&gt;Neither S card is new. The AMD Instinct MI50 with 32 GB costs about $649 on eBay. AMD’s own page lists &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.amd.com/en/support/downloads/drivers.html/accelerators/instinct/instinct-mi-series/instinct-mi50-32gb.html&quot;&gt;HBM2 with 1 TB/s of peak bandwidth&lt;/a&gt;, and the price works out to about $20 per gigabyte, less than a third of the yardstick. One owner runs Qwen at an 8 bit setting at 22.8 tokens per second, a quality level no 16 GB card can hold at all. The ceiling for a roughly 28 GB file at 1 TB/s is near 37, so that result sits at about 62% of it.&lt;/p&gt;
&lt;p&gt;The listing leaves out the software story. AMD’s &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rocm.docs.amd.com/en/docs-6.0.0/release.html&quot;&gt;ROCm 6.0 release notes&lt;/a&gt; put the MI50 in maintenance mode, say 5.7 was the last release with full support, and set bug fixes to end in Q2 2024. Current ROCm tables list the MI50 as &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.3/reference/system-requirements.html&quot;&gt;unsupported&lt;/a&gt;. It still runs through llama.cpp, and owners maintain their own builds. It is a passively cooled 300 W server card, most snapshot listings ship from China, and the 1 US listing was $750. If you can live with that, nothing here gives you more model for the money.&lt;/p&gt;
&lt;p&gt;The used RTX 3060 with 12 GB costs about $370. In the hardware review used for the snapshot, it runs a 14B Qwen model at 23 to 31 tokens per second depending on how long the chat gets, which is a useful model at a speed you would be happy with. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://techgage.com/?p=64697&quot;&gt;Techgage’s launch table&lt;/a&gt; lists 12 GB at 360 GB/s and a $329 price. Per token per second on the short test, it is the cheapest speed of any card here with more than 8 GB. For adding memory cheaply next to a bigger card, I think it makes a sensible second card, since it is the lowest price you will pay for 12 more gigabytes on a card that runs everything.&lt;/p&gt;
&lt;h2 id=&quot;what-the-spec-sheets-say-about-the-cards-i-keep-returning-to&quot;&gt;What the spec sheets say about the cards I keep returning to&lt;/h2&gt;
&lt;p&gt;Prices shift weekly, so I checked the fixed numbers against manufacturer and review sources. The RX 7900 XTX has 24 GB of GDDR6 on a 384 bit bus, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://pcper.com/2022/12/amd-radeon-rx-7900-xtx-and-rx-7900-xt-review/&quot;&gt;960 GB/s of bandwidth, a 355 W rating and a $999 launch price&lt;/a&gt;. The RTX 3090 offers 936 GB/s at 350 W and launched at $1,499, according to &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tomshardware.com/pc-components/gpus/nvidia-geforce-rtx-5090-review&quot;&gt;Tom’s Hardware’s comparison table&lt;/a&gt;. That puts the two within 3% of each other on the number that sets generation speed, which fits what owners report.&lt;/p&gt;
&lt;p&gt;The Arc Pro B60 carries 24 GB of GDDR6 at &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.intel.com/content/dam/www/central-libraries/us/en/documents/2026-03/datasheet-b60-gpu.pdf&quot;&gt;456 GB/s&lt;/a&gt;, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.club386.com/intel-debut-arc-pro-b-series-gpus-for-visualisation-and-ai-training/&quot;&gt;Club386 reports a configurable board power of 120 W to 200 W&lt;/a&gt;. That is about 49% of the 3090’s bandwidth at a much lower power draw, so I treat it as a quiet, cool 24 GB card with half the 3090’s ceiling.&lt;/p&gt;
&lt;p&gt;The RTX 5060 Ti 16 GB launched at $429 with &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tomshardware.com/pc-components/gpus/nvidia-geforce-rtx-5060-ti-16gb-review&quot;&gt;448 GB/s and a 180 W rating&lt;/a&gt;. The $790 in my snapshot is 84% above that launch price, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techspot.com/specs/gpu/307872-nvidia-geforce-rtx-5060-ti-16gb.html&quot;&gt;TechSpot’s price tracker&lt;/a&gt; shows $664 in August 2026 after a $429 low in November 2025.&lt;/p&gt;
&lt;p&gt;The RTX 5090 has 32 GB at &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.pcgamesn.com/nvidia/geforce-rtx-5090-release-date-price-specs-benchmarks&quot;&gt;1,792 GB/s with a 575 W rating and a $1,999 launch price&lt;/a&gt;. Its bandwidth is 91% above the 3090’s, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://windowscentral.com/hardware/nvidia-unveils-rtx-5080-and-rtx-5090&quot;&gt;Windows Central suggested at least a 1,000 W power supply&lt;/a&gt; for it at announcement. At $4,921 in the snapshot, the card sits 146% above its launch price. Power draw belongs in the budget for any card on this list, because a 575 W card can need a new power supply on top of the card itself.&lt;/p&gt;
&lt;h2 id=&quot;software-support-decides-how-long-a-cheap-card-lasts&quot;&gt;Software support decides how long a cheap card lasts&lt;/h2&gt;
&lt;p&gt;A card keeps working after support ends. NVIDIA’s &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://developer.nvidia.com/blog/navigating-gpu-architecture-support-a-guide-for-nvidia-cuda-developers&quot;&gt;developer blog&lt;/a&gt; says CUDA 13 means you cannot build new applications for Maxwell, Pascal and Volta with its compiler, while applications built for those cards keep full support through the life of the 580 driver, sometime in 2028. PyTorch &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://dev-discuss.pytorch.org/t/introducing-cuda-13-2-and-deprecating-cuda-12-8-release-2-12/3337&quot;&gt;keeps a CUDA 12.6 legacy build&lt;/a&gt; for those architectures. The P40 sits in that window, and so does anything with a similar age on the NVIDIA side.&lt;/p&gt;
&lt;p&gt;The AMD and Intel stories run on a similar clock. AMD announced the MI50’s maintenance mode in &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.phoronix.com/news/AMD-ROCm-5.6&quot;&gt;ROCm 5.6&lt;/a&gt;, and Intel’s Extension for PyTorch ended in March 2026. I drop a card 1 tier when its software is leaving, since a cheap card with a short remaining life costs more than its sticker.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-04.BrYmYJpo_UppvY.webp&quot; alt=&quot;5 dates that set how long older cards stay useful. Orange dots mark the AMD and Intel steps.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;744&quot; height=&quot;331&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;5 dates that set how long older cards stay useful. Orange dots mark the AMD and Intel steps. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tomshardware.com/pc-components/gpus/nvidia-to-drop-cuda-support-for-maxwell-pascal-and-volta-gpus-with-the-next-major-toolkit-release&quot;&gt;AMD ROCm notes, Tom’s Hardware, Intel, NVIDIA&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;why-old-cards-beat-new-ones-right-now&quot;&gt;Why old cards beat new ones right now&lt;/h2&gt;
&lt;p&gt;Memory has become expensive, and older server cards were built with a lot of it. In this snapshot the cheapest gigabyte was $16.67 on the P40 and the most expensive was $177 on the RTX Pro 6000, which is more than 10 times as much. One tracker, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://capitalandcompute.net/blog/why-are-ram-and-gpu-prices-spiking/&quot;&gt;Capital and Compute, citing TrendForce&lt;/a&gt;, puts GDDR6 spot prices at about $2.50 per gigabyte in September 2025 and $12.45 by October 5, 2026. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.pcguide.com/news/gpu-pricing-is-getting-much-worse-its-hard-to-see-an-end-in-sight/&quot;&gt;3DCenter’s German market data&lt;/a&gt; shows average card prices up 66% for NVIDIA, 46% for AMD and 38% for Intel between November 2025 and October 2026.&lt;/p&gt;
&lt;p&gt;Cards that shipped before the shortage carry their memory at old prices, and used server cards sit furthest from the new price curve. Newer cards still win on speed and software support. For most people running a local model, speed past reading pace is a bonus, and memory is the wall.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-05.W_6m0G-j_2ceyFg.webp&quot; alt=&quot;Launch price against the November 2026 snapshot price for 5 cards. The 3090 is a used price.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;733&quot; height=&quot;400&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Launch price against the November 2026 snapshot price for 5 cards. The 3090 is a used price. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tweaktown.com/news/113098/nvidias-rtx-pro-6000-blackwell-gpu-price-almost-doubles-reaching-16-000/index.html&quot;&gt;TechSpot, Intel, AMD, TweakTown&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;how-to-grade-any-card-yourself&quot;&gt;How to grade any card yourself&lt;/h2&gt;
&lt;p&gt;Your card may not be on the list, and an old GTX 1080 is a good example. It still runs small models fine. You can grade any card in 3 steps. First, check that the model you want fits in the card’s memory with room to talk, because if it does not, the card is an F for that job however fast it is. A GTX 1080 has 8 GB, so the 27B model fails this step. Second, divide today’s price by gigabytes and compare it with the 3090’s $72 per gigabyte. Third, check that the speed is comfortably faster than you read, which is about 5 to 6 tokens per second.&lt;/p&gt;
&lt;p&gt;A card that passes all 3 for less than the yardstick is an S or an A. Drop it 1 tier if its software is being left behind.&lt;/p&gt;
&lt;h2 id=&quot;what-i-would-buy&quot;&gt;What I would buy&lt;/h2&gt;
&lt;p&gt;Under $400, I would buy a used RTX 3060 12 GB at around $370. For the biggest models at the lowest cost, and if you are comfortable with Linux, I would buy the MI50 at about $649. For 24 GB that works on a budget, the used RX 7900 XTX at about $1,259 is my pick. For 24 GB with the best compatibility, including image tools, it is the used RTX 3090 at about $1,727. If speed is all that matters, the RTX 5090 is the answer, with your eyes open about the price.&lt;/p&gt;
&lt;p&gt;If I were starting from nothing this week, I would buy the 3060 first and save for a 24 GB card as my second one.&lt;/p&gt;
&lt;h2 id=&quot;2-things-from-this-generalize&quot;&gt;2 things from this generalize&lt;/h2&gt;
&lt;p&gt;First, price the model before the card. The 27B Qwen file is about 18 GB at 4 bits, which is about 33% above the raw 13.5 GB of weights, and that gap is the room the format and the conversation need. Any card quote that skips the file size answers a different question.&lt;/p&gt;
&lt;p&gt;Second, redo the dollars per gigabyte sum on the day you buy and check the support date for the card’s software. Prices moved weekly all year, and a $949 card was listing at $1,946 within months.&lt;/p&gt;
&lt;p&gt;This week, write down the 1 model you want to run, add a few gigabytes to its file size for the conversation, and divide each candidate card’s current listing price by its memory. Anything above $72 per gigabyte needs a reason.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>gpu</category><category>nvidia</category><category>ai</category><category>artificial-intelligence</category><category>llm</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Microsoft Filed a Patent to Rewrite What You Say Live and Redraw Your Lips</title><link>https://soursen.org/articles/microsoft-filed-a-patent-to-rewrite-what-you-say-live-and-redraw-your/</link><guid isPermaLink="true">https://soursen.org/articles/microsoft-filed-a-patent-to-rewrite-what-you-say-live-and-redraw-your/</guid><description>Application US 2026/0341309 A1 describes swapping words in live video within 1 to 10 seconds, tailored to each viewer’s profile. The main claim says nothing about telling viewers, no patent has been granted, and nearly every piece of the idea already ships somewhere.</description><pubDate>Tue, 10 Nov 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;On October 29, 2026, the US Patent and Trademark Office published a Microsoft application for AI that changes what a person says in live video while they are still saying it, then redraws their lips so the edit does not show. Patent tracker Patentlyze spotted it first, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://futurism.com/artificial-intelligence/microsoft-patents-ai-words-in-peoples-mouths&quot;&gt;Futurism reported it on November 1&lt;/a&gt;. Most of the coverage jumped straight to the alarming version. I want to walk through what the filing says, what it leaves out, which parts already exist in products you can use today, and why a delay of 1 to 10 seconds matters more than it sounds.&lt;/p&gt;
&lt;h2 id=&quot;what-microsoft-filed&quot;&gt;What Microsoft filed&lt;/h2&gt;
&lt;p&gt;The document is application US 2026/0341309 A1, filed by Microsoft Technology Licensing, LLC on April 29, 2025, according to &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://tech.yahoo.com/ai/copilot/articles/microsoft-files-patent-application-ai-171430423.html&quot;&gt;Yahoo Tech’s write up of the filing&lt;/a&gt;. Its title describes a near real time AI enhancement of video or augmented reality presentations of live events. It is a published application, which is a different thing from a granted patent, a finished product or a public demo.&lt;/p&gt;
&lt;p&gt;The status matters. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://patentlyze.com/patent/microsoft-live-ai-video-rewrites-speech/&quot;&gt;Patentlyze reports&lt;/a&gt; that the examiner’s first review raised objections and rejected the claims as filed, which it calls routine, and that Microsoft can respond and amend. Microsoft did not answer Futurism’s request for comment, and companies rarely discuss pending applications, so I read nothing into the silence. It does mean the only description of this system is the one Microsoft’s lawyers wrote for the patent office.&lt;/p&gt;
&lt;h2 id=&quot;how-the-edit-would-work&quot;&gt;How the edit would work&lt;/h2&gt;
&lt;p&gt;The pipeline, as the coverage describes it, starts with an AI agent that listens to the live audio and flags words to change or remove, based on preset rules or a viewer’s profile. A generative model swaps those words in the audio and in the video, reworks the frames around each edit, and plays the result with a delay that &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techpowerup.com/354585/microsoft-patent-application-describes-ai-that-edits-live-speech&quot;&gt;TechPowerUp’s summary&lt;/a&gt; puts at roughly 1 to 10 seconds. It could run on AR glasses, a phone, a tablet or a camera, using the device’s own chips.&lt;/p&gt;
&lt;p&gt;The list of things it could swap is broad. It includes filler words, stutters, mispronounced words, offensive words and colloquial phrases like “y’all” or “bite the bullet.” The replacement words can come from an internet search, and the filing floats instant fact checking. 2 of its drawing captions cover condensing a long recording into a shorter summary video, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.dexerto.com/entertainment/new-microsoft-patent-could-use-ai-to-alter-speech-and-lip-movements-in-real-time-3425747/&quot;&gt;Dexerto notes&lt;/a&gt; that this could mean generating new footage of the speakers saying a summarized script. The examples of live content run from lectures and presentations to awards shows, game shows, cooking shows and live sports.&lt;/p&gt;
&lt;p&gt;The viewer profile is the part that drew the most attention. Coverage from &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://tbreak.com/microsoft-ai-patent-live-speech-lips/&quot;&gt;Tbreak&lt;/a&gt; lists age, languages, region, religion, ethnicity, education, and voice or video samples as fields a profile could hold, and it stresses that this describes what a profile could contain, not data any Microsoft product collects. One account I read also mentions a college major, and I could not confirm that item anywhere else, so I would treat it as unverified.&lt;/p&gt;
&lt;h2 id=&quot;10-seconds-is-enough-to-rewrite-2-sentences&quot;&gt;10 seconds is enough to rewrite 2 sentences&lt;/h2&gt;
&lt;p&gt;At roughly 150 words a minute, a common figure for conversation, 10 seconds is about 25 words. That is enough room to detect a word, pick a replacement, regenerate a mouth and ship the frames before a viewer notices the stream is running behind. The filing defines near real time as within a few seconds, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://ibtimes.co.uk/microsoft-ai-patent-rewrite-live-speech-1830381&quot;&gt;IBTimes points out&lt;/a&gt; that the 1 to 10 second range is an example, not a demonstrated benchmark.&lt;/p&gt;
&lt;p&gt;I invented this illustration of the failure mode, and the filing does not show it. A chemistry professor says “mercury” during a livestreamed lecture. If a system decides a word is unfamiliar and searches the web for a replacement, the top result could be the planet, the element, the car brand or the singer. The model picks one, redraws the professor’s lips around it, and a student watching sees the professor say something that was never said. The professor, watching the raw feed, would never see the difference.&lt;/p&gt;
&lt;p&gt;The filing does include a safeguard of sorts. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techpowerup.com/354585/microsoft-patent-application-describes-ai-that-edits-live-speech&quot;&gt;TechPowerUp reports&lt;/a&gt; that the original feed may be stored as ground truth, and that this appears in 1 dependent claim. The main claim says nothing about telling viewers what was changed. Keeping a record helps an investigator later, and the person watching live still gets no notice.&lt;/p&gt;
&lt;h2 id=&quot;every-piece-already-exists-somewhere&quot;&gt;Every piece already exists somewhere&lt;/h2&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.ik_qiCx6_Z1Fs6i.webp&quot; alt=&quot;Seven dates from the first Xbox voice chat filtering plans to the published Microsoft application.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;727&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Seven dates from the first Xbox voice chat filtering plans to the published Microsoft application. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://futurism.com/artificial-intelligence/microsoft-patents-ai-words-in-peoples-mouths&quot;&gt;Futurism, Activision, TechCrunch, Meta, Google, TechPowerUp&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The new part is the combination, since each ingredient has a history. In October 2019, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://futurism.com/microsoft-censor-xbox-live-voice-chat&quot;&gt;Futurism reported&lt;/a&gt; that Xbox wanted to detect bad words in live voice chat and bleep them, much as TV does. In August 2023, Activision announced that Call of Duty would use &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.callofduty.com/blog/2023/08/call-of-duty-modern-warfare-warzone-anti-toxicity-progress-report&quot;&gt;Modulate’s ToxMod for real time voice chat moderation&lt;/a&gt;, with a global rollout on November 10, 2023. Detection happens in real time, and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://mp1st.com/news/modern-warfare-3-to-feature-real-time-voice-chat-moderation-via-ai-to-combat-toxic-behavior-faq-released&quot;&gt;the Call of Duty FAQ&lt;/a&gt; says the system flags toxic language as it is detected, so ToxMod produces flags for enforcement and leaves the words as spoken.&lt;/p&gt;
&lt;p&gt;Translation is where rewriting already happens. At Ignite in November 2024, Microsoft announced an Interpreter agent for Teams that gives &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techrepublic.com/article/microsoft-ignite-2024-ai-security-teams-innovations/&quot;&gt;real time speech to speech translation in up to 9 languages&lt;/a&gt;, with an option to simulate your own voice. A Microsoft spokesperson told TechCrunch the feature &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://techcrunch.com/2024/11/19/soon-microsoft-will-let-teams-meeting-attendees-clone-their-voices&quot;&gt;would not automatically use your voice&lt;/a&gt;. On August 19, 2025, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://creators.facebook.com/blog/meta-ai-translations&quot;&gt;Meta launched AI Translations for Reels&lt;/a&gt;, which clones the creator’s voice and can lip sync the dub, first between English and Spanish and later with Hindi and Portuguese added.&lt;/p&gt;
&lt;p&gt;YouTube announced in February 2026 that its auto dubbing covers &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://blog.google/intl/en-in/products/platforms/unlocking-a-global-audience-with-auto-dubbing/&quot;&gt;27 languages and that a lip sync pilot is under test&lt;/a&gt;. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://ppc.land/youtube-auto-dubbing-tops-6m-daily-viewers-expands-to-27-languages&quot;&gt;PPC Land reports&lt;/a&gt; that creators can enable experimental lip sync in YouTube Studio when auto dubbing is on, and that YouTube’s own documentation describes it as altering the speaker’s lip movement to match the dub. A competitor, sync. labs, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sync.so/blog/youtube-auto-dubbing&quot;&gt;wrote in July 2026&lt;/a&gt; that the feature had not yet shipped broadly to creators, so treat the rollout status as fluid.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.BCJmfrhE_2mOXej.webp&quot; alt=&quot;Language counts for three tools that already ship. They do different jobs, so the comparison is loose.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Language counts for three tools that already ship. They do different jobs, so the comparison is loose. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://blog.google/intl/en-in/products/platforms/unlocking-a-global-audience-with-auto-dubbing/&quot;&gt;Meta Creators blog, TechRepublic, Google&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Microsoft is also the company behind 2 of those steps, since Call of Duty belongs to Activision and Activision is now a Microsoft company, and Teams Interpreter is Microsoft’s own product. Put those side by side and the filing reads as the next step. It would run live, it would run for each viewer separately, and its edits would follow a profile and preset rules.&lt;/p&gt;
&lt;h2 id=&quot;the-disclosure-question-decides-whether-this-is-a-tool&quot;&gt;The disclosure question decides whether this is a tool&lt;/h2&gt;
&lt;p&gt;I think the line between a helpful feature and a problem comes down to 2 things, a label and a switch. When YouTube dubs a video, it &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://ppc.land/youtube-auto-dubbing-tops-6m-daily-viewers-expands-to-27-languages&quot;&gt;notes the video as auto dubbed&lt;/a&gt;, and viewers can pick the original audio. When Meta translates a Reel, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://tech.yahoo.com/ai/meta-ai/articles/meta-ai-translating-dubbing-reels-020000646.html&quot;&gt;the translated Reels are labeled and viewers can toggle translation or lip sync off&lt;/a&gt;, and the creator reviews the result before it publishes.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.4AcVl5lQ_Z8UlK7.webp&quot; alt=&quot;What viewers are told and can switch off in 2 shipping products, compared with the filing’s main claim.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;713&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;What viewers are told and can switch off in 2 shipping products, compared with the filing’s main claim. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://patentlyze.com/patent/microsoft-live-ai-video-rewrites-speech/&quot;&gt;Meta Creators blog, TechRadar, Google, PPC Land, TechPowerUp, Patentlyze&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The filing’s main claim contains neither. TechPowerUp says the main claim is silent on telling viewers, and Patentlyze found no mention of labels, watermarks or viewer notices. I could not find a described viewer switch in any coverage either, though the full text of a long filing may say more than the coverage does.&lt;/p&gt;
&lt;p&gt;I want to be fair to the idea. Live translation with matching lips is the most practical use in the filing, and I would welcome a lecture in a language I do not speak that looked and sounded natural. The same machinery that does that can also change a speaker’s meaning without anyone knowing. The label and the switch are what separate those 2 outcomes, and an application that does not mention them gives me no reason to assume they will exist.&lt;/p&gt;
&lt;h2 id=&quot;the-fraud-numbers-make-the-silence-uncomfortable&quot;&gt;The fraud numbers make the silence uncomfortable&lt;/h2&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-04.TPtV-gs1_Z3XnBp.webp&quot; alt=&quot;Reported amounts in two deepfake impersonation cases. The New York figures are prosecutors’ allegations.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Reported amounts in two deepfake impersonation cases. The New York figures are prosecutors’ allegations. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://helpnetsecurity.com/2026/11/09/deepfake-ceo-fraud-129-million-indictment&quot;&gt;Fortune, Help Net Security, CBS News&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;A tool that makes altered video look natural has a neighbor, and that neighbor is fraud. In 2024, a finance worker at engineering firm Arup’s Hong Kong office joined a video call where the CFO and colleagues were deepfakes, then made &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://adaptivesecurity.com/blog/arup-deepfake-scam-attack&quot;&gt;15 transfers to 5 accounts&lt;/a&gt; worth about $25.6 million. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/europe/2024/05/17/arup-deepfake-fraud-scam-victim-hong-kong-25-million-cfo/&quot;&gt;Fortune reported&lt;/a&gt; that Arup was identified as the victim in May 2024.&lt;/p&gt;
&lt;p&gt;This month the numbers got larger. Prosecutors in New York unsealed an indictment charging Jacob Soall, Sharon Hazan and Guy Sheetrit with using AI to impersonate chief executives and lawyers in a scheme that &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.pbs.org/newshour/nation/3-charged-in-129-million-ai-scheme-targeting-thousands-of-companies&quot;&gt;PBS and AP describe as about $129 million&lt;/a&gt;. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://helpnetsecurity.com/2026/11/09/deepfake-ceo-fraud-129-million-indictment&quot;&gt;Help Net Security’s summary&lt;/a&gt; says the scheme ran from 2024 to 2026, that the largest single loss listed was $11.6 million at a UK financial technology company, and that a Rhode Island bank wired out $2.7 million. Each defendant faces 7 counts, and these are allegations that a court has not tested.&lt;/p&gt;
&lt;p&gt;I am not suggesting anyone at Microsoft intends fraud, and the filing describes translation and cleanup uses. The tools for faking a person are already good enough to move millions of dollars, and a system that edits live video without telling anyone adds one more way for a viewer to be unsure what they are seeing. The Arup fraud came to light when the employee followed up with Arup’s headquarters afterward.&lt;/p&gt;
&lt;h2 id=&quot;what-an-application-proves-and-what-it-does-not&quot;&gt;What an application proves and what it does not&lt;/h2&gt;
&lt;p&gt;A published application shows that Microsoft’s lawyers asked for these claims, and says nothing about a product roadmap. Companies file patents for many reasons, including defensive ones, and the examiner’s first review rejected the claims as filed. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://futurism.com/artificial-intelligence/microsoft-patents-ai-words-in-peoples-mouths&quot;&gt;Futurism noted&lt;/a&gt; that dubbing has existed for ages and that video is edited all the time for ordinary and for harmful reasons, and the ground truth clause suggests the inventors saw the risk of confusion. Reading the document as a launch announcement would be a mistake, and so would reading it as meaningless. It is a statement of what Microsoft wanted to own, and the claim set is where the next round of amendments will show how far it still wants to go.&lt;/p&gt;
&lt;h2 id=&quot;2-things-from-this-generalize&quot;&gt;2 things from this generalize&lt;/h2&gt;
&lt;p&gt;First, treat live voice and video as unverified when money or a consequential decision is involved. Arup’s employee saw and heard people who looked like colleagues, and the New York indictment describes calls with cloned voices and faces. A callback to a phone number you already had on file, made after the call and before the transfer, defeats this class of attack because it does not rely on anything the call showed you.&lt;/p&gt;
&lt;p&gt;Second, apply the label and switch test to any tool that changes speech. Ask who turns it on, whether the viewer sees a label, and whether the original stays reachable. On YouTube, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.socialsamosa.com/news-2/youtube-expands-auto-dubbing-tests-lip-sync-11077978&quot;&gt;the February 2026 update added a preferred language setting&lt;/a&gt; that lets viewers choose dubbed audio or the original, so set it once and check the player’s audio track settings whenever a dub appears.&lt;/p&gt;
&lt;p&gt;This week, write 1 rule into your payment approval process that no transfer request made on a call, voice or video, gets approved until someone confirms it through a second channel on a number already on file.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>microsoft</category><category>ai</category><category>artificial-intelligence</category><category>ai-agent</category><category>law</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Elon Musk Predicted a Die Off for 200 AI Neoclouds. CoreWeave’s Bonds Had Already Hit 15%.</title><link>https://soursen.org/articles/elon-musk-predicted-a-die-off-for-200-ai-neoclouds-coreweaves-bonds-had/</link><guid isPermaLink="true">https://soursen.org/articles/elon-musk-predicted-a-die-off-for-200-ai-neoclouds-coreweaves-bonds-had/</guid><description>Musk says debt markets have shut on companies that only rent out AI chips. The bond market reached that view before he posted it, and the bigger lesson is how much of the AI boom now runs on rented chips and borrowed money.</description><pubDate>Sun, 08 Nov 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;On October 25, 2026, Elon Musk posted that more than 200 AI cloud rental companies, the so called neoclouds, are headed for “guppy-level death and consolidation,” and that only a few will make it through. By Monday morning CoreWeave, Nebius and IREN were trading lower before the opening bell. CoreWeave, the biggest of the group, owed $35.6 billion at the end of June against $5.5 billion in unrestricted cash, and its unsecured bonds were already yielding around 15% to 16%. I want to walk through what Musk actually said, how the neocloud business works, why its problems reach lenders, Nvidia and the biggest AI labs, and why Musk is far from a neutral observer, because the numbers behind his warning were in plain sight 2 weeks before he posted it.&lt;/p&gt;
&lt;h2 id=&quot;what-musk-said-and-when&quot;&gt;What Musk said, and when&lt;/h2&gt;
&lt;p&gt;Musk circled this corner of the AI trade for about 2 weeks, and each post was sharper than the last.&lt;/p&gt;
&lt;p&gt;On October 12, Broadcom CEO Hock Tan told a conference in Singapore that many AI investors have no clue what they are buying and behave almost like casino gamblers. Musk replied to a post summarizing Tan’s remarks with 5 words, “No crying in the casino,” &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.benzinga.com/node/62316768&quot;&gt;according to Benzinga&lt;/a&gt;. His point was that anyone who piled into the AI trade should not complain when the table turns.&lt;/p&gt;
&lt;p&gt;On October 13, as yields on CoreWeave’s unsecured bonds approached 15%, venture capitalist Chamath Palihapitiya posted a warning about them. Musk replied with 1 word, “Yikes,” &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.thewealthadvisor.com/article/elon-musk-said-yikes-and-ai-stocks-crashed-now-he-says-shortage-ridiculous&quot;&gt;as The Wealth Advisor reported&lt;/a&gt;. The next day CoreWeave shares closed at $65.78, down 15.46%, while Nebius fell about 11% and Oracle about 9%, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.roic.ai/news/nvidias-huang-says-no-backstop-needed-for-coreweave-as-ai-debt-fears-mount-10-15-2026&quot;&gt;per ROIC.ai’s market report&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;On October 25 came the full argument. Palihapitiya had posted a thread calling neoclouds disorganized and poorly capitalized, while adding that some are better than others. Replying to it, Musk wrote that over the past few weeks debt markets had almost entirely cut off money to any company without substantial revenue from outside AI data centers. He predicted the 200 plus neoclouds would see a wave of failures and mergers, and warned of “stranded compute in partially complete data centers,” &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.itpro.com/cloud/cloud-computing/neocloud-market-set-for-brutal-consolidation-elon-musk-warns&quot;&gt;as ITPro reported&lt;/a&gt;. Palihapitiya, in the thread Musk was answering, added that 2027 will be a turbulent year.&lt;/p&gt;
&lt;p&gt;In the same thread he brought up SpaceX. He said SpaceX had closed about $40 billion of mostly callable debt to pay for GPUs arriving in the first half of 2027, and credited that to SpaceX being investment grade with Starlink and launch revenue behind it, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://investinglive.com/stocks/musk-says-spacex-closed-40-billion-gpu-debt-deal-predicts-neocloud-shakeout/&quot;&gt;according to InvestingLive&lt;/a&gt;. When traders asked whether he was talking prices down to buy cheap, he answered, “SpaceX is not buying any neoclouds. Pointless.”&lt;/p&gt;
&lt;h2 id=&quot;how-a-neocloud-makes-money&quot;&gt;How a neocloud makes money&lt;/h2&gt;
&lt;p&gt;A neocloud is a company that owns AI computing power and rents it out. The name separates firms like CoreWeave, Nebius, Lambda, Nscale and IREN from the established cloud giants, Amazon, Microsoft and Google, which the industry calls hyperscalers.&lt;/p&gt;
&lt;p&gt;The model runs in 3 steps. First, the neocloud borrows money and spends it on Nvidia GPUs. That borrowing ranges from term loans to high yield bonds, which pay more interest because the lender takes more risk. Second, it rents those chips out on contracts that run for years, usually to the biggest names in tech. Third, it hands the chips and the contracts to its lenders as collateral, so if the neocloud cannot pay, the lenders take the hardware and the rental income attached to it.&lt;/p&gt;
&lt;p&gt;The structure has a clear lineage. In August 2023, Magnetar Capital and Blackstone Tactical Opportunities jointly led a $2.3 billion loan to CoreWeave secured by its Nvidia chips and its contracts with Microsoft, with Coatue, DigitalBridge, BlackRock, PIMCO and Carlyle also taking part, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.wing.vc/content/how-the-gpu-became-collateral&quot;&gt;as Wing VC’s history of GPU lending sets out&lt;/a&gt;. A $7.5 billion facility led by Blackstone followed in May 2024. In March 2026, CoreWeave closed an $8.5 billion loan backed by a Meta contract, the first chip backed loan to carry a full investment grade rating, anchored by Blackstone Credit &amp;amp; Insurance.&lt;/p&gt;
&lt;p&gt;That Meta deal shows how lenders actually think. The loan priced at SOFR plus 225 basis points, with a fixed tranche at 5.9%, against roughly 870 basis points over SOFR on the 2023 deal. Lenders were pricing Meta’s credit far more than CoreWeave’s. Wing VC’s analysis puts it directly, the market is mostly pricing the customer and the contract length, with very little weight on what a used GPU is worth.&lt;/p&gt;
&lt;p&gt;The weak spot is timing. Bloomberg reported that lenders structure these loans to be repaid over 5 or 6 years, roughly the period they expect the chips to hold their value, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.webull.com/news/15798285175137280&quot;&gt;as summarized by Benzinga&lt;/a&gt;. Nvidia ships a new generation far faster than that, and each new generation cuts the rental price older chips can command. A neocloud can end up years into a loan, still making full payments on chips that earn much less than when it bought them.&lt;/p&gt;
&lt;h2 id=&quot;how-long-a-gpu-really-lasts&quot;&gt;How long a GPU really lasts&lt;/h2&gt;
&lt;p&gt;The accounting decides how big that problem looks. CoreWeave depreciates its GPUs and related equipment over 6 years, a schedule it lengthened from 5 years starting in 2023, according to its filings. Nebius moved from 4 years to 5 in 2026, and Amazon went the other way in 2025, cutting part of its server fleet from 6 years to 5 because AI hardware is changing faster, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://io-fund.com/ai-stocks/nebius-vs-coreweave-ai-stock&quot;&gt;as IO Fund summarized&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Investor Michael Burry made the bear case in late 2025. He argued that GPUs have an economic life closer to 2 or 3 years, and that big AI spenders would understate depreciation by about $176 billion between 2026 and 2028 by assuming 5 to 6 years, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://beancount.io/blog/2026/07/10/gpu-cloud-unit-economics-depreciation-schedule-neocloud-guide&quot;&gt;according to Beancount’s explainer&lt;/a&gt;. The neoclouds have a real answer. CoreWeave said on its second quarter call that it had signed a contract running into 2029 for A100 chips that Nvidia launched in 2020, so a 6 year old GPU is still earning.&lt;/p&gt;
&lt;p&gt;Both things can be true. Old chips can keep earning and still lose pricing power every time Nvidia ships something faster. The lender’s risk is about timing. One analysis of CoreWeave’s 2026 borrowing found that every instrument it raised in the first half matures by October 2032, with an average term of about 5.9 years against its 6 year equipment life, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://capefearadvisors.substack.com/p/coreweave-twenty-seven-years&quot;&gt;per Cape Fear Advisors&lt;/a&gt;. The debt and the chips are scheduled to wear out at about the same time, which leaves little room if rental prices fall faster than planned.&lt;/p&gt;
&lt;h2 id=&quot;coreweave-is-the-biggest-test-case&quot;&gt;CoreWeave is the biggest test case&lt;/h2&gt;
&lt;p&gt;CoreWeave’s latest 10-Q shows the scale. Total debt principal was $35.6 billion at June 30, 2026, up from $21.6 billion 6 months earlier, and unrestricted cash was $5.5 billion, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sec-api.io/insights/coreweave-crwv-2026-revenue-analysis-top-customer-concentration-exposure&quot;&gt;according to sec-api.io’s filing analysis&lt;/a&gt;. Cash covered about 15% of the debt. Reuters, using LSEG data with a broader definition, put CoreWeave’s debt closer to $50 billion by mid October.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.IrEb5yQ8_Zea4yn.webp&quot; alt=&quot;CoreWeave’s contracted backlog is large, and so is the gap between its debt and its cash.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;CoreWeave’s contracted backlog is large, and so is the gap between its debt and its cash. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sec-api.io/insights/coreweave-crwv-2026-revenue-analysis-top-customer-concentration-exposure&quot;&gt;CoreWeave Q2 2026 10-Q, Aug 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The company is growing fast and still losing money. Second quarter revenue was $2.58 billion, up 112% from a year earlier, but the net loss widened to $626 million, mainly because of about $640 million in quarterly interest expense, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.storagenewsletter.com/2026/08/20/nebius-fiscal-2q26-financial-results/&quot;&gt;per StorageNewsletter’s results summary&lt;/a&gt;. CoreWeave guided 2026 capital spending at $35 billion to $39 billion, close to 3 times its expected revenue of $12.4 billion to $13.2 billion.&lt;/p&gt;
&lt;p&gt;The bond market noticed. CNBC reported a 14.8% yield on CoreWeave’s 9.625% notes due July 2032 on October 14, and a 15.78% yield on its 9.75% notes due October 2031 the next day. By November 2, FINRA trade data showed every one of its 4 dollar bonds yielding between 14.93% and 15.83%, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fourweekmba.com/ai-coreweaves-7-75b-of-dollar-bonds-now-trade-near-6-3b/&quot;&gt;as FourWeekMBA tracked&lt;/a&gt;. The October 2031 notes last traded at 79.75 cents on the dollar, under the 80 cent line bond investors usually read as distress, and the 2032 notes sat just above it at 80.03. Together the 4 notes, with $7.75 billion of face value, were worth about $6.3 billion, roughly 81 cents on the dollar. Its convertible notes traded lower still, between about 64 and 73 cents.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.SSsFJ-gP_2f7UT9.webp&quot; alt=&quot;All 4 CoreWeave dollar bonds now trade at yields 5 to 6 points above the coupons they were sold at.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;All 4 CoreWeave dollar bonds now trade at yields 5 to 6 points above the coupons they were sold at. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fourweekmba.com/ai-coreweaves-7-75b-of-dollar-bonds-now-trade-near-6-3b/&quot;&gt;FINRA TRACE via FourWeekMBA, Nov 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Credit insurance told the same story. Reuters reported that credit default swaps on CoreWeave near 1,000 basis points implied roughly a 63% chance of default within 5 years, up from about 30% a year earlier. I would treat that as a market price for fear, since swap markets on a single company can be thin, but the direction is hard to argue with.&lt;/p&gt;
&lt;p&gt;A 15% yield hurts a company that has to keep borrowing. If CoreWeave raised another $1 billion of unsecured debt at anything near today’s market yield, it would pay about $150 million a year in interest. At the original 9.625% coupon, the same $1 billion cost about $96 million. That extra $54 million a year, on every new billion, comes straight out of the cash that is supposed to pay for the next data center.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.C3OzvPXN_nRRUu.webp&quot; alt=&quot;Annual interest on each new $1 billion of unsecured debt at the original coupon and at October’s market yield.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Annual interest on each new $1 billion of unsecured debt at the original coupon and at October’s market yield. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fourweekmba.com/ai-coreweaves-7-75b-of-dollar-bonds-now-trade-near-6-3b/&quot;&gt;Soursen arithmetic, Nov 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;CoreWeave’s defence is its order book. Revenue backlog was $104.2 billion at June 30, up from $99.4 billion in March, and the company said it signed more than $25 billion of new commitments in the first weeks of the third quarter, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://thestar.com.my/tech/tech-news/2026/08/12/coreweave-boosts-2026-spending-plan-beats-quarterly-estimates-on-ai-demand-surge&quot;&gt;according to Reuters via The Star&lt;/a&gt;. Meta added $21 billion during the second quarter, Anthropic signed a multiyear deal in April and OpenAI has contracted up to about $22.4 billion in total.&lt;/p&gt;
&lt;p&gt;A backlog is a promise of future revenue, and CoreWeave still has to build, power and equip the data centers before most of it turns into cash. It also had $35.5 billion of payments on leases signed but not yet started at June 30. Near term revenue is concentrated too. CoreWeave’s 2025 annual filing shows its largest customer, Microsoft, generated 67% of revenue, up from 62% in 2024.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-04.DFGHPbVf_Z1P854b.webp&quot; alt=&quot;CoreWeave’s dependence on its largest customer has grown each year since 2023.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;CoreWeave’s dependence on its largest customer has grown each year since 2023. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sec-api.io/insights/coreweave-crwv-2026-revenue-analysis-top-customer-concentration-exposure&quot;&gt;CoreWeave FY2025 10-K via sec-api.io, 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;CoreWeave is looking for cheaper money. Brannin McBee, one of its founders, told the Financial Times on October 14 that the company wants investment grade ratings on more of its chip backed debt, and eventually on CoreWeave itself, so it can tap insurance company capital, a pool he described as “unlimited,” &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://dealroom.co/news/162445-coreweave-wants-investment-grade-ratings-to-tap-unlimited-insurance-mone/&quot;&gt;per Dealroom’s summary&lt;/a&gt;. He also said contracts with AI labs, which lack investment grade ratings, are not yet ready for that market.&lt;/p&gt;
&lt;h2 id=&quot;the-secured-loans-and-the-bonds-tell-different-stories&quot;&gt;The secured loans and the bonds tell different stories&lt;/h2&gt;
&lt;p&gt;CoreWeave’s 2 kinds of debt are behaving very differently. Its chip backed loans, secured on GPUs and tied to customers like Meta and Microsoft, have stayed close to face value. Its unsecured bonds, which rank behind those loans and depend on CoreWeave as a company, are the ones trading near 15%, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.webull.com/news/15798285175137280&quot;&gt;as Benzinga reported citing Bloomberg&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-05.DP-KNLUZ_1z5uy2.webp&quot; alt=&quot;What CoreWeave pays depends on whose credit backs the debt. Rates are approximate.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;What CoreWeave pays depends on whose credit backs the debt. Rates are approximate. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.wing.vc/content/how-the-gpu-became-collateral&quot;&gt;Wing VC, Benzinga/Bloomberg, CNBC, 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;That split matters for Musk’s argument. Lenders still fund GPUs that sit behind a contract with an investment grade customer. The money that has dried up is the money that bets on the neocloud itself, which is exactly the kind of money that the smaller players depend on.&lt;/p&gt;
&lt;h2 id=&quot;not-every-neocloud-is-in-the-same-spot&quot;&gt;Not every neocloud is in the same spot&lt;/h2&gt;
&lt;p&gt;Musk himself said some will make it, and the balance sheets back that up. Nebius, the Amsterdam based neocloud, ended June with $8 billion in cash and generated $2.3 billion of operating cash in the second quarter, largely because about 70% of the deals it closed included an upfront customer prepayment. It expects more than $9 billion of prepayments in 2026, which it says will cover 50% to 60% of the related capital spending, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rivkin.com.au/?p=37370&quot;&gt;according to Rivkin’s results summary&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;On October 22, 3 days before Musk said the debt markets were shut, Nebius announced a $3.5 billion senior secured facility from Citi. It carries an initial rate of SOFR plus 2.75%, rising to as much as 3.50%, and matures on October 16, 2027, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fourweekmba.com/ai-nebius-gets-3-5b-from-citi-in-a-364-day-secured-facility/&quot;&gt;per FourWeekMBA&lt;/a&gt;. That is a 364 day loan funding assets that will run for years, so Nebius will have to refinance it within 12 months. The door is still open for some borrowers, on shorter and tighter terms.&lt;/p&gt;
&lt;p&gt;Even Nebius felt the squeeze. Its CEO acknowledged on October 26 that some capital sources were temporarily closed, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://biztoc.com/x/903c4ec3791ff181&quot;&gt;Benzinga reported&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Below CoreWeave and Nebius sits a long tail of smaller operators, many of them former bitcoin miners such as IREN, Cipher Mining, Hut 8 and TeraWulf that pivoted to renting AI chips, plus private players like Lambda, Nscale and Fluidstack. Many have thin backlogs and little cash. Those are the guppies Musk meant.&lt;/p&gt;
&lt;h2 id=&quot;lenders-nvidia-and-the-ai-labs-are-all-tied-in&quot;&gt;Lenders, Nvidia and the AI labs are all tied in&lt;/h2&gt;
&lt;p&gt;The neocloud problem reaches well past the neoclouds, through 3 links.&lt;/p&gt;
&lt;p&gt;The first is the lenders. Private credit funds, which lend directly to companies outside the banking system, financed much of the build. Blackstone, Magnetar, BlackRock, PIMCO, Carlyle and Apollo all appear in GPU financing deals, and Apollo and Blackstone announced a $35 billion package to finance chips for Anthropic in June. Carlyle has estimated private credit may need to fund about $1 trillion of the AI computing build out, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.privateequitywire.co.uk/?p=162200&quot;&gt;according to Private Equity Wire&lt;/a&gt;. The collateral behind much of it is GPUs, the same chips that lose value with every new Nvidia release.&lt;/p&gt;
&lt;p&gt;The second is Nvidia. It sells chips to the neoclouds, it owns equity in several of them and it has promised to buy back unused capacity. Nvidia invested $2 billion of equity in CoreWeave in January 2026, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.ifre.com/equities/2377167/nvidia-throws-us2bn-lifeline-to-under-fire-coreweave&quot;&gt;IFR reported&lt;/a&gt;, and it has agreed to buy any of CoreWeave’s unsold capacity, up to $6.3 billion, through April 13, 2032. In August it signed agreements with Apollo, BlackRock, Blackstone, Brookfield, Goldman Sachs and KKR for financing platforms targeting $500 billion, keeping the option to backstop up to $125 billion itself, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://capacityglobal.com/news/nvidia-now-agrees-to-rent-back-unused-gpu-capacity-from-neocloud-operators-if-customer-demand-falls-short/&quot;&gt;per Capacity Global&lt;/a&gt;. Nvidia sells the chips, owns part of the buyer and promises to rent the chips back if nobody else will.&lt;/p&gt;
&lt;p&gt;The third is the customers. Microsoft, Meta, OpenAI and Anthropic all rent capacity from CoreWeave alone, and Google has backed lease deals between AI developers and former bitcoin miners turned neoclouds, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.wing.vc/content/how-the-gpu-became-collateral&quot;&gt;as Wing VC notes&lt;/a&gt;. Part of the computing power behind the companies that define the AI trade sits on someone else’s borrowed money.&lt;/p&gt;
&lt;p&gt;Put the loop together and the word bubble starts to fit. Lenders fund the neoclouds, the neoclouds buy from Nvidia, the AI labs rent from the neoclouds and the labs’ growth is what convinced lenders to fund the neoclouds in the first place. A large share of the AI build out is rented, and the rent is underwritten by debt that got more expensive this month.&lt;/p&gt;
&lt;h2 id=&quot;money-is-only-half-of-the-problem&quot;&gt;Money is only half of the problem&lt;/h2&gt;
&lt;p&gt;Musk made a second point that got less attention. He argued that raising money for data centers is the easier part, and that delivering power, cooling and networking is harder. In a separate post he said even SpaceX, with what he called a “super cracked team,” finds it very difficult, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://longbridge.com/news/301830650&quot;&gt;according to Benzinga via Longbridge&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The scale is enormous. Close to $700 billion is projected to go into data centers in 2026, and roughly 40% of it pays for buildings, power and other physical infrastructure, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://dashboard.channelchek.com/?p=104902&quot;&gt;according to Channelchek&lt;/a&gt;. A modern AI data center needs a grid connection, liquid cooling to keep the chips from overheating and networking to tie thousands of them together. Each runs on its own timeline. A neocloud can have the chips, the loan and the customer contract and still lack a building that can switch them on. Interest keeps running and the chips keep depreciating while the revenue waits for a power line. That is the “stranded compute” Musk was warning about.&lt;/p&gt;
&lt;h2 id=&quot;musk-has-a-stake-in-this-fight&quot;&gt;Musk has a stake in this fight&lt;/h2&gt;
&lt;p&gt;Musk is far from a bystander. SpaceX acquired xAI in an all stock deal in February 2026 that valued the combined company at about $1.25 trillion, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sacra.com/research/why-spacex-acquired-xai/&quot;&gt;as Sacra documented&lt;/a&gt;, and its AI unit now rents out computing capacity to outside customers, including compute deals with Anthropic and Google, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.indmoney.com/blog/us-stocks/spacex-xai-compute-deals-google-anthropic-ipo-valuation&quot;&gt;as INDmoney reported&lt;/a&gt;. That puts SpaceX in direct competition with the neoclouds for chips, for electricity, for customers and for lenders willing to finance GPUs. Borrowing billions against GPUs is exactly what the neoclouds do, and Musk’s argument rests on persuading lenders that SpaceX is a different kind of borrower.&lt;/p&gt;
&lt;p&gt;The $40 billion itself is less settled than Musk’s post suggested. SpaceX’s own SEC filing describes a proposed equipment finance lease, in which an entity managed by Apollo Global Management would buy the GPUs and lease them to SpaceX over about 5 years, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.ibtimes.co.uk/elon-musk-predicts-guppy-level-death-cloud-rivals-spacex-secures-massive-40b-debt-1827926&quot;&gt;according to IBTimes&lt;/a&gt;. SpaceX signed a binding commitment letter on October 24, the day before Musk’s post, but the filing says the master lease has yet to close and pricing has not been set. Deliveries are expected to start in the first half of 2027. Structurally, that is a GPU backed financing with a lender holding the chips, which is the same basic tool the neoclouds use, with SpaceX’s credit standing behind it.&lt;/p&gt;
&lt;p&gt;There is a second possible motive. Musk has been in a legal fight with OpenAI since 2024, and OpenAI relies heavily on outside computing, including up to about $22.4 billion of commitments with CoreWeave. Pressure on the neoclouds is pressure on a rival’s supply line. When traders accused him of talking his own book, his answer was about acquisitions, which sidestepped the question about competition.&lt;/p&gt;
&lt;p&gt;A motive does not decide whether a claim is true, though. The yields, the debt loads and the contracts stand or fall on their own.&lt;/p&gt;
&lt;h2 id=&quot;the-market-was-already-moving&quot;&gt;The market was already moving&lt;/h2&gt;
&lt;p&gt;Most of the stress Musk described showed up before his October 25 post. CoreWeave’s bonds crossed 15% on October 13. On October 14, Gartner forecast that more than 80% of neocloud providers will stop operating independently by 2030, through failure, acquisition or merger, and that over $3.5 trillion of AI infrastructure spending could be exposed to impairment by the end of that year, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.networkworld.com/article/4242389/gartner-warns-3-5t-in-ai-infrastructure-spending-could-be-at-risk-by-2030.html&quot;&gt;as Network World reported&lt;/a&gt;. Gartner also ranked CoreWeave as the most exposed to debt among roughly 100 neoclouds it tracks, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://capacityglobal.com/news/neolcoud-consolidation-explained/&quot;&gt;per Capacity Global&lt;/a&gt;. Rate hike fears were hitting AI debt at the same time, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://247wallst.com/investing/2026/10/20/gartner-80-of-neocloud-firms-face-extinction-by-2030-as-ai-capex-outpaces-returns/&quot;&gt;24/7 Wall St. reported&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-06.D-yZiJ0q_MVTYV.webp&quot; alt=&quot;Gartner expects spending to keep rising even as most neocloud providers lose their independence.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;902&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Gartner expects spending to keep rising even as most neocloud providers lose their independence. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.networkworld.com/article/4242389/gartner-warns-3-5t-in-ai-infrastructure-spending-could-be-at-risk-by-2030.html&quot;&gt;Gartner via Network World, Oct 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Gartner’s forecast is a consolidation call. It also expects spending on AI optimized cloud infrastructure, the market neoclouds sell into, to rise from $66 billion in 2027 to $174 billion in 2030. The market keeps growing. Fewer independent companies share it.&lt;/p&gt;
&lt;p&gt;By the time Musk posted, CoreWeave, Nebius and IREN had each lost between 33% and 39% of their value in October, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://coincentral.com/?p=215584&quot;&gt;according to CoinCentral&lt;/a&gt;. On Monday, October 26, Reuters reported CoreWeave down 4.5% in early trading, Nebius down 3.4% and IREN down 3.3%, and short sellers betting against the group had made at least $6 billion in paper profits, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.businessday.co.za/world/international-companies/2026-10-26-coreweave-leads-tech-selloff-after-musks-consolidation-warning/&quot;&gt;via BusinessDay&lt;/a&gt;. By that afternoon the shares had recovered above Friday’s close, even as CoreWeave’s 2033 bonds and Nebius’ 2034 bonds hit record lows, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fourweekmba.com/ai-neocloud-stocks-recover-from-musk-warning-as-bonds-hit-lows/&quot;&gt;FourWeekMBA noted&lt;/a&gt;. Musk’s post moved prices for a morning. The bonds had been falling for 2 weeks. CoreWeave stock closed at $89.62 on October 2 and was trading near $50 on November 2, a fall of about 44% in a month.&lt;/p&gt;
&lt;h2 id=&quot;the-case-against-the-doom&quot;&gt;The case against the doom&lt;/h2&gt;
&lt;p&gt;Plenty of serious people disagree with Musk. Nvidia CEO Jensen Huang told Bloomberg in Stockholm on October 15 that “CoreWeave’s strength is improving every day,” that its cost of capital is falling and that he sees no need for Nvidia to backstop it, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://news.bgov.com/artificial-intelligence/nvidia-ceo-huang-says-coreweave-improving-no-backstop-needed&quot;&gt;per Bloomberg Government&lt;/a&gt;. Huang has also said demand is off the charts and that the shortage is capital, which is close to Musk’s diagnosis with the opposite conclusion.&lt;/p&gt;
&lt;p&gt;Goldman Sachs analyst Gabriela Borges upgraded CoreWeave from Neutral to Buy on October 19, cutting her price target from $139 to $106. Goldman argued that most of CoreWeave’s financing comes through GPU backed loans and that any rise in its cost of capital should be limited to 100 to 200 basis points, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cryptobriefing.com/goldman-sachs-upgrades-coreweave-buy/&quot;&gt;according to Crypto Briefing&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Musk softened his own tone at one point. About a week after “Yikes,” he called the computing shortage “ridiculous” and pointed to customers demanding more capacity, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.thewealthadvisor.com/article/elon-musk-said-yikes-and-ai-stocks-crashed-now-he-says-shortage-ridiculous&quot;&gt;The Wealth Advisor reported&lt;/a&gt;. Demand and financing are separate questions, and both sides of this argument agree demand is strong.&lt;/p&gt;
&lt;h2 id=&quot;what-to-watch-next&quot;&gt;What to watch next&lt;/h2&gt;
&lt;p&gt;CoreWeave reports third quarter results on Tuesday, November 10, 2026, after the market closes, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.benzinga.com/node/62635791&quot;&gt;the company said&lt;/a&gt;. Management has guided third quarter interest expense to $860 million to $940 million, up from $640 million in the second quarter, and is targeting annualized revenue of $18.5 billion to $19.5 billion by the end of 2026, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.earningswhispers.com/article/CRWV/22731&quot;&gt;according to Earnings Whispers&lt;/a&gt;. The interest line is the one I would read first, since it shows what the October repricing does to a company that has to keep borrowing.&lt;/p&gt;
&lt;p&gt;The other things to watch are whether CoreWeave lands an investment grade rating on more of its chip backed debt, whether Nebius refinances its Citi loan before October 2027, and whether SpaceX’s $40 billion lease closes and on what terms.&lt;/p&gt;
&lt;h2 id=&quot;the-lesson-for-anyone-exposed-to-ai-infrastructure&quot;&gt;The lesson for anyone exposed to AI infrastructure&lt;/h2&gt;
&lt;p&gt;2 things from this generalize past the neoclouds. The first is that the bond market priced this risk before the loudest voice in tech did. CoreWeave’s unsecured notes were at 15% on October 13, and anyone watching yields, credit default swaps and the gap between secured and unsecured debt had 2 weeks of warning. If you hold AI infrastructure stocks, track the bonds of the companies you own alongside the share price.&lt;/p&gt;
&lt;p&gt;The second is that a backlog only protects a lender or shareholder once it turns into cash, and that depends on power, buildings and a customer list wider than 1 company. Before you buy or lend to any neocloud, check 4 numbers in its latest filing, which are cash against total debt, the share of revenue from its largest customer, how much of its debt matures within 12 months and how much of its backlog is tied to customers rated investment grade.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>elon-musk</category><category>ai</category><category>artificial-intelligence</category><category>neocloud</category><category>ai-agent</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>GPT-6 Astra Read Another Model’s Raw Weights and Fixed It by Hand.</title><link>https://soursen.org/articles/gpt-6-astra-read-another-models-raw-weights-and-fixed-it-by-hand/</link><guid isPermaLink="true">https://soursen.org/articles/gpt-6-astra-read-another-models-raw-weights-and-fixed-it-by-hand/</guid><description>MIT researchers had GPT-6 Astra edit 0.016% of a small model’s weights, with no training, and its math score jumped from 7.9% to 48.1%. The number is real and the method is strange. The limit they found in the same paper is the part worth understanding.</description><pubDate>Tue, 03 Nov 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;On 2 November 2026, 4 researchers at &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.csail.mit.edu/&quot;&gt;MIT CSAIL&lt;/a&gt; published a result that sounds impossible at first. They handed a frontier AI model, GPT-6 Astra, the raw numerical weights of a different neural network, as plain text, and asked it to make that network better at math by editing the numbers directly. No training. No gradient descent. Just reasoning about what the numbers do and changing them. The edited model’s score on a math benchmark went from 7.9% to 48.1%. I want to walk through exactly what happened, because the headline is true and also easy to misread, and the authors themselves found the limit that keeps it honest.&lt;/p&gt;
&lt;h2 id=&quot;what-weight-space-programming-actually-means&quot;&gt;What weight space programming actually means&lt;/h2&gt;
&lt;p&gt;Start with why this is surprising at all. A neural network is, underneath, a giant list of numbers called weights. Training sets those numbers by running data through the network millions of times and nudging them with an optimizer. A human cannot look at a list of a few hundred thousand numbers and say what the network does, and until now neither could a language model. The weights were the one part of machine learning that stayed mathematical, something you optimized with algorithms rather than described in words.&lt;/p&gt;
&lt;p&gt;The MIT team, Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab, call their finding &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rikiyat.github.io/blog/weight-space-programming/&quot;&gt;weight space programming&lt;/a&gt;. A general purpose model reads and writes a network’s weights directly, around a goal written in plain English, without an optimizer and sometimes with no feedback from running the network at all. The model reasons about what each part of the network computes, or should compute, and sets the numbers accordingly.&lt;/p&gt;
&lt;h2 id=&quot;reading-weights-it-was-never-trained-to-read&quot;&gt;Reading weights it was never trained to read&lt;/h2&gt;
&lt;p&gt;Before the math result, the simpler experiments are the ones that made me sit up. The team trained 4 small game playing networks, on Breakout, Pong, Pac-Man, and Freeway, all with identical dimensions so the only way to tell them apart is to read the weights. They gave Astra the 17,253 raw numbers of one network, with no tools and no ability to run it, and asked which game it was trained on. Astra named Breakout, correctly, and got all 4 networks right.&lt;/p&gt;
&lt;p&gt;Its reasoning was concrete. It noticed the agent’s strongest first layer weights sat along the bottom row of the playfield, which is where a paddle lives, and that a band of reward bearing objects sat above the paddle, which is what bricks look like. It worked out the game from the geometry of the numbers, the way you might guess a machine’s purpose from the shape of its gears.&lt;/p&gt;
&lt;p&gt;One experiment made the mechanism unusually clear. The team trained small diffusion models to generate circles, squares, and triangles, then asked Astra which shape each one draws, from the weights alone, with no forward passes allowed. It got them right by finding the shape in the output layer, where each hidden unit draws one piece of the edge. The telling part is a mistake it made. Astra drew every shape as an outline, even though the real models generate filled shapes. A model that was secretly running the network would have known they were filled. Drawing the outline shows it was reading the geometry of the weights, not computing what the network would output.&lt;/p&gt;
&lt;p&gt;It went further in later experiments. Given only the middle layers of a small language model, with the input and output stripped off so it could not cheat by reading which character is which, Astra still named whether the network was trained on Python, Japanese romaji, or English, and correctly picked out the specific attention heads that copy earlier text. That copying head analysis matched a known interpretability method from a 2021 Anthropic paper that the researchers never mentioned to Astra. It even identified which of 2 near identical models had been trained to be sycophantic, by watching where the attention heads looked, finding that the sycophantic model’s middle layer attention pointed at the user’s suggested answer instead of the actual question.&lt;/p&gt;
&lt;h2 id=&quot;the-math-result-and-how-it-actually-worked&quot;&gt;The math result, and how it actually worked&lt;/h2&gt;
&lt;p&gt;Now the headline. The team gave Astra the weights of Qwen3.5–0.8B, a small language model, loaded in a Python notebook, plus a budget of 50 forward passes on practice math problems to see how the model behaves. It was not allowed to train the model or use gradients.&lt;/p&gt;
&lt;p&gt;What Astra found was a specific, diagnosable flaw. Small reasoning models tend to loop, repeating the same text until they hit their length limit without ever reaching an answer. Qwen did this badly, 93% of its answers on the math test ran all the way to the 32,768 token limit without finishing. Astra traced that looping habit to a single internal signal written by the MLP parts of 2 layers, layers 15 and 19, and subtracted that one direction out of 131,072 of the model’s weights.&lt;/p&gt;
&lt;p&gt;That edit touched 0.016% of the model’s 0.8 billion weights. With the loop removed, the share of answers that ran out of length fell from 93% to 33%, and the benchmark scores moved a lot. MATH-500 went from 7.9% to 48.1%. GSM8K went from 13.8% to 67.1%. Instruction following, measured by IFEval, went from 21.0% to 62.3%, even though Astra was only asked about math. One targeted change, no training, big jump.&lt;/p&gt;
&lt;p&gt;The budget mattered. The paper tested 10, 25, and 50 forward passes, which is how many times Astra was allowed to run the model on practice problems to see its behavior. At 10 passes MATH-500 reached only 12.3%, and at 25 it was 14.1%. The large jump to 48.1% came only at 50 passes, which is also when the looping finally dropped off sharply. So the fix took a real diagnostic loop, looking at behavior, forming a hypothesis, editing, and checking, run enough times to land it, rather than a single clever glance at the weights.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.BPCygWgq_Z1QljaN.webp&quot; alt=&quot;Qwen3.5–0.8B before and after Astra’s edit, across 3 benchmarks. The edit changed 0.016% of the weights and ran no training.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;889&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Qwen3.5–0.8B before and after Astra’s edit, across 3 benchmarks. The edit changed 0.016% of the weights and ran no training. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rikiyat.github.io/blog/weight-space-programming/&quot;&gt;Takehi et al, MIT CSAIL, Nov 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-catch-the-authors-put-right-in-the-paper&quot;&gt;The catch the authors put right in the paper&lt;/h2&gt;
&lt;p&gt;This is where an honest reading departs from the breathless one. The edit did not make Qwen smarter at math. It removed a loop that was stopping Qwen from finishing problems it could already solve. The ability was there the whole time, buried under a behavior that wasted the model’s output budget before it reached an answer. Astra cleared the obstacle, and the latent ability showed up on the scoreboard.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.E6ynjFx1_Zh8miP.webp&quot; alt=&quot;The edit lifted Qwen, which loops badly, and did nothing for Gemma, which rarely loops. The gain came from clearing an obstacle that was hiding ability the model already had.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;889&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;The edit lifted Qwen, which loops badly, and did nothing for Gemma, which rarely loops. The gain came from clearing an obstacle that was hiding ability the model already had. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rikiyat.github.io/blog/weight-space-programming/&quot;&gt;Takehi et al, MIT CSAIL, Nov 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The researchers proved this themselves with a second model. They tried the same thing on Gemma 3 270M, a model that rarely loops, only 8.5% of its answers hit the length limit. Astra could not improve it. When it read Gemma’s failures, it found ordinary reasoning mistakes, wrong arithmetic and bad algebra, with no single component behind them to switch off. Its edits moved the math score by about a point, a gain it honestly reported as not established. The paper states the lesson plainly, that editing weights could remove an obstacle hiding an ability, but it could not add an ability the model lacked.&lt;/p&gt;
&lt;p&gt;That distinction is the whole story. The 7.9% to 48.1% jump describes an AI finding and deleting a bug in another AI’s weights, a bug that happened to be hiding a lot of capability. It did not install math skill into another AI by rewriting its brain. Impressive, real, and much more specific than the headline number suggests.&lt;/p&gt;
&lt;h2 id=&quot;writing-networks-from-nothing&quot;&gt;Writing networks from nothing&lt;/h2&gt;
&lt;p&gt;Reading and fixing weights is one thing. The team also showed Astra writing weights from scratch. Given the rules and source code of 3 small games but no training data, no gradients, and no ability to play the game first, Astra wrote the weights of a game playing network directly, as text, choosing every number itself. Its Breakout network scored 517.4 against 15.8 for a standard reinforcement learning network trained for 100 million steps, and 25.3 for one trained for 500 million steps.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.SSDjCzn0_1Y63CD.webp&quot; alt=&quot;Astra’s hand written Breakout network against reinforcement learning networks trained for millions of steps. It wrote the weights directly, with no training.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;889&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Astra’s hand written Breakout network against reinforcement learning networks trained for millions of steps. It wrote the weights directly, with no training. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://rikiyat.github.io/blog/weight-space-programming/&quot;&gt;Takehi et al, MIT CSAIL, Nov 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;At a larger scale, 1.5 million parameters, too big to fit in its context, Astra wrote weights for a network that plays Go. It built the network as a small Go program in Python, layer by layer, marking each side’s stones, tracing chains that were in danger, and scoring each move. Even with 0 practice games, writing all 336 lines in a single run, its network won more than half its games against 2 smaller trained networks. With a budget of 50 practice games to learn from, it beat an equally sized reinforcement learning network trained on 20 million games. The way it improved with practice reads like debugging, it played games, looked at how it lost, changed the part of the network behind the loss, and kept a change only when it won more.&lt;/p&gt;
&lt;p&gt;What makes all of this new is narrow and specific. Interpretability researchers have read network internals with purpose built tools for years, and other work has generated weights using models trained specifically for that job, or found weights by gradient free trial and error. Here, a general purpose frontier model, used exactly as it ships, reads and writes the weights itself, from 17,253 numbers given as text up to a language model of 0.8 billion weights with the help of a Python sandbox. Nobody trained Astra to do any of this.&lt;/p&gt;
&lt;h2 id=&quot;why-this-matters-and-why-it-is-still-early&quot;&gt;Why this matters, and why it is still early&lt;/h2&gt;
&lt;p&gt;The authors are careful to call these sparks, early results that are exciting but limited by context length and tooling. I think that framing is right, and the direction is the interesting part. For years we optimized the weights of neural networks only with small symbolic optimizers like Adam. This points at a different future, where a reasoning model inspects weights, explains what they do, and edits them around a goal stated in English.&lt;/p&gt;
&lt;p&gt;If that develops, a few things change. You could audit a model for hidden backdoors or capabilities by reading its weights, without running it or building a test for the exact thing you are worried about. You could add or remove a capability by editing a few weights instead of collecting data and retraining, which touches so little of the model that it avoids the forgetting that fine tuning causes. A model could inspect and debug its own weights, or those of a newer model, which is a more direct version of self improvement than writing training code. None of that is here yet. What is here is a frontier model that can look at a list of numbers that defeats every human, tell you what it does, and change it on purpose. That was supposed to be impossible, and a month ago I would have said so.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>gpt-6-astra</category><category>ai</category><category>artificial-intelligence</category><category>llm</category><category>ai-agent</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>OpenAI Lost $20.9</title><link>https://soursen.org/articles/openai-lost-20-9/</link><guid isPermaLink="true">https://soursen.org/articles/openai-lost-20-9/</guid><description>Audited 2025 figures show a company tripling revenue while burning cash faster than any startup ever has. The loss is only half the story. The other half is a loop of deals that turns the same dollars into growth at three companies at once, and a bill that has four ways of reaching you.</description><pubDate>Mon, 02 Nov 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Every week, more than 900 million people type something into ChatGPT. They fix code with it, plan meals, settle arguments about history. By almost any measure it is one of the most successful consumer products ever built, and it costs about $20 a month. The company behind it lost roughly $20.9 billion running it in 2025. I want to walk through how a product that popular loses money that fast, because the answer is not what the hype or the doom headlines tell you, and the mechanics underneath are genuinely strange.&lt;/p&gt;
&lt;p&gt;Audited figures, independently verified by the &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.ft.com/&quot;&gt;Financial Times&lt;/a&gt; and first reported in detail by &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.wheresyoured.at/exclusive-openai-financials&quot;&gt;Ed Zitron&lt;/a&gt;, give us hard numbers to work with for once. So I am going to stick to those, and tell you where a figure is a real cash loss versus an accounting artifact, because that distinction matters a lot here.&lt;/p&gt;
&lt;h2 id=&quot;the-numbers-straight&quot;&gt;The numbers, straight&lt;/h2&gt;
&lt;p&gt;In 2025, OpenAI’s revenue jumped from $3.7 billion to $13.07 billion. Tripling sales in a year would be the decade’s highlight for almost any company. The problem is the other side of the ledger. Total costs and expenses hit $34 billion, which left an operating loss of $20.92 billion. For every dollar ChatGPT and its sister products brought in, OpenAI spent about $2.60 to earn it.&lt;/p&gt;
&lt;p&gt;You will also see a scarier number, a net loss of $38.53 billion. I want to be careful with that one, because most of the gap between the $20.92 billion operating loss and the $38.53 billion net loss is not cash leaving the building. It comes from a $41.55 billion non-cash charge tied to changes in the fair value of convertible interests, a paperwork consequence of OpenAI converting to a for-profit structure in October 2025. So the honest figure for how much real money the business burned is the operating loss, around $20.9 billion, and it is bad enough on its own.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.t3TBkIXC_ZruVVp.webp&quot; alt=&quot;Revenue tripled, and the operating loss more than doubled alongside it. Growing fast and bleeding fast at the same time.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;889&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Revenue tripled, and the operating loss more than doubled alongside it. Growing fast and bleeding fast at the same time. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.wheresyoured.at/exclusive-openai-financials&quot;&gt;OpenAI audited figures via Financial Times and&lt;/a&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;http://wheresyoured.at/&quot;&gt;wheresyoured.at&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;To see how unusual this is, line it up against the company everyone reaches for as the excuse. Defenders say OpenAI is running the Amazon playbook, lose money now to own the market later. Amazon’s worst single year was 2000, when it lost about $1.4 billion, per its own &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.sec.gov/Archives/edgar/data/0001018724/000103221001500087/d10k405.txt&quot;&gt;annual filing&lt;/a&gt;. OpenAI’s 2025 operating loss alone is roughly 15 times that. No company in history has run losses anywhere near this size, and that is before you look at what it has promised to spend next.&lt;/p&gt;
&lt;h2 id=&quot;where-34-billion-a-year-goes&quot;&gt;Where $34 billion a year goes&lt;/h2&gt;
&lt;p&gt;The spending breaks down into a few huge lines. Research and development ran to $19.18 billion, more than half of all costs. Sales and marketing jumped to $5.73 billion. OpenAI also paid Microsoft $17.2 billion in total expenses, including $10.59 billion for research and infrastructure, while Microsoft paid OpenAI just $303 million back. That last pair of numbers tells you how dependent OpenAI still is on a partner that also competes with it.&lt;/p&gt;
&lt;p&gt;The structural problem sits underneath all of it. With ordinary software, serving one more customer costs almost nothing, which is why software companies print money at scale. Every ChatGPT answer runs on expensive chips inside power-hungry data centers. More users means more revenue, and it also means more servers, more electricity, more cooling, more chips, and those costs climb right alongside usage instead of flattening out. Growth does not fix the losses here. It feeds them.&lt;/p&gt;
&lt;h2 id=&quot;the-spending-nobody-can-pay-for-alone&quot;&gt;The spending nobody can pay for alone&lt;/h2&gt;
&lt;p&gt;Here is where the figures stop looking like a normal company and start looking like a national infrastructure program. In September 2025, OpenAI committed to building at least 10 gigawatts of AI data centers running on Nvidia systems, roughly the output of 10 large nuclear reactors. Nvidia CEO &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://en.wikipedia.org/wiki/Jensen_Huang&quot;&gt;Jensen Huang&lt;/a&gt; has said building a single gigawatt of AI data center capacity costs up to $60 billion.&lt;/p&gt;
&lt;p&gt;By late 2025, OpenAI had racked up around $1.4 trillion in data center and computing commitments. Set that against $13.07 billion of revenue and you get roughly $70 in long-term promises for every dollar the company actually earned. Internal projections cited in reporting put OpenAI’s expected spend on computing and infrastructure at about $856 billion by the end of 2030. That single figure is larger than the annual economic output of most countries on Earth. No company pays for that out of its own revenue, which is why the financing got creative.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.vQ9iQH7J_Z1pKmQc.webp&quot; alt=&quot;OpenAI’s infrastructure commitments dwarf its revenue. About $70 in promises for every dollar earned.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;889&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;OpenAI’s infrastructure commitments dwarf its revenue. About $70 in promises for every dollar earned. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.cnbc.com/2025/10/15/a-guide-to-1-trillion-worth-of-ai-deals-between-openai-nvidia.html&quot;&gt;CNBC, company reporting, 2025&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-loop&quot;&gt;The loop&lt;/h2&gt;
&lt;p&gt;Picture a hardware store. A contractor walks in wanting to build a mansion he cannot afford. So the store lends him the money on one condition, that every dollar gets spent in the store. He buys the store’s lumber with the store’s cash, and at quarter’s end the store reports record sales. Nothing was actually sold to the outside world, but everyone’s numbers look great.&lt;/p&gt;
&lt;p&gt;That is close to what sits at the center of the AI boom. In September 2025, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.nvidia.com/&quot;&gt;Nvidia&lt;/a&gt; announced plans to invest up to $100 billion in OpenAI, released in stages as OpenAI switched on new Nvidia-powered data centers. Around the same time, OpenAI agreed to spend $300 billion over 5 years on &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.oracle.com/&quot;&gt;Oracle’s&lt;/a&gt; cloud. Oracle’s stock soared 36% in a day on the news, adding about $251 billion in market value. Oracle cannot run that cloud without Nvidia chips, so it buys them from Nvidia.&lt;/p&gt;
&lt;p&gt;Trace the circle. Nvidia invests in OpenAI. OpenAI pays Oracle. Oracle buys chips from Nvidia. The money leaves Nvidia and comes back to Nvidia, and on the way around it shows up as growth at all three companies at once, chip revenue at Nvidia, a booming order backlog at Oracle, and infrastructure backing OpenAI’s forecasts. The same dollars get counted as three different companies’ good news.&lt;/p&gt;
&lt;p&gt;OpenAI is not the only one doing this. By 2026, analysts had flagged more than $800 billion in circular arrangements across the AI supply chain. Even Anthropic, OpenAI’s biggest rival, runs a similar structure with Google and Amazon. The loop has become the industry’s default financing move, repeated across the whole supply chain.&lt;/p&gt;
&lt;p&gt;Not everyone sees a problem. Nvidia CEO Jensen Huang has called the circular financing label “ridiculous,” arguing the deals reflect real demand for real chips that customers genuinely need. On the other side, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://en.wikipedia.org/wiki/Michael_Burry&quot;&gt;Michael Burry&lt;/a&gt;, the investor who shorted the 2008 housing bubble, posted a Bloomberg diagram in August 2026 tracing roughly $46 billion in direct equity stakes and about $879 billion in multi-year purchase commitments among the hyperscalers, AI labs, and chipmakers, with Nvidia at the center. His summary was that the money is circling, not multiplying. He also pointed out that Nvidia’s 5 year credit default swap spread, the market’s price for insuring against a Nvidia default, roughly doubled over two months.&lt;/p&gt;
&lt;p&gt;I lean toward Burry’s read, because the tell is not the existence of the deals but how the same dollar gets booked as good news in several places at once. A chip sale funded by the seller’s own investment in the buyer is not the same quality of revenue as a chip sale to an unrelated customer paying with its own money.&lt;/p&gt;
&lt;h2 id=&quot;when-the-supplier-becomes-the-lender&quot;&gt;When the supplier becomes the lender&lt;/h2&gt;
&lt;p&gt;The strangest turn came when the chipmaker started guaranteeing its customer’s rent. OpenAI does not have an investment-grade credit rating, which makes sense for a company losing more than $20 billion a year. Lenders price loans on whether the borrower can pay, and on paper OpenAI cannot, not yet. So Nvidia stepped in as the backstop.&lt;/p&gt;
&lt;p&gt;An SEC filing disclosed that Nvidia is guaranteeing up to about $105 billion to cover the land, power, and buildings for roughly 4.25 gigawatts of OpenAI data centers at a campus in southern Ohio, as reported by &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.datacenterknowledge.com/data-center-construction/nvidia-backs-openai-s-ohio-data-center-buildout-with-105b-guarantee&quot;&gt;Data Center Knowledge&lt;/a&gt;. That is just the real estate. The chips going inside could add another $350 billion or so.&lt;/p&gt;
&lt;p&gt;If this feels familiar, it should. In the late 1990s, Lucent, Nortel, and Cisco lent billions to cash-strapped internet startups so those startups could buy the vendors’ own gear. When the dot-com bubble burst, the startups went under, the loans went bad, and the vendors went down with them. Nvidia is a far stronger company than Lucent ever was. Its data center business alone brought in $193.7 billion in 2026. The structure is the same, though. The supplier’s fortunes and the customer’s ability to pay are now tied together, and if OpenAI cannot cover its rent, Nvidia loses its biggest customer and inherits the bill.&lt;/p&gt;
&lt;h2 id=&quot;the-insiders-already-took-money-off-the-table&quot;&gt;The insiders already took money off the table&lt;/h2&gt;
&lt;p&gt;While the losses pile up, the people closest to the company have already turned paper wealth into cash. In October 2025, current and former OpenAI employees sold about $6.6 billion of stock to a group of investors including Thrive Capital, SoftBank, Dragoneer, Abu Dhabi’s MGX, and T. Rowe Price. That deal valued OpenAI at $500 billion, the most valuable private company in the world at the time, up from the $300 billion valuation set just months earlier in a SoftBank-led round. The timing is worth sitting with. Insiders cashed out part of their holdings at the top, before the public ever got a chance to buy in.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.Cgv91tEV_Z2qdJH4.webp&quot; alt=&quot;In 2025 OpenAI’s valuation climbed from $300 billion to $500 billion in a matter of months, even as the operating loss widened.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;889&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;In 2025 OpenAI’s valuation climbed from $300 billion to $500 billion in a matter of months, even as the operating loss widened. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.foxbusiness.com/markets/openai-becomes-worlds-most-valuable-private-company-500b-valuation-report&quot;&gt;Reuters, company reporting, 2025&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Why would investors keep paying more for a company losing more? At $500 billion, OpenAI was valued at roughly 25 times its projected annual revenue. Most established software companies trade somewhere between 5 and 15 times. Investors are paying for what they believe OpenAI becomes, the company that builds the most powerful AI first and then owns the market, which is a bet on the future rather than a price on current earnings. In AI, that belief carries a price tag, because every jump in valuation is the fuel for the next multi-billion dollar round, which buys the next round of compute. The valuation and the spending are the same story told twice.&lt;/p&gt;
&lt;h2 id=&quot;the-4ways-the-bill-reaches-you&quot;&gt;The 4ways the bill reaches you&lt;/h2&gt;
&lt;p&gt;A loss this big does not vanish. Someone absorbs it, and if these bets go wrong, the someone is often not the people who made them. I count four paths to ordinary people.&lt;/p&gt;
&lt;p&gt;The first is your retirement account. In its filing for the quarter ending 30 September 2025, Microsoft disclosed that its share of OpenAI’s losses cut its net income by $3.1 billion in a single quarter, knocking 41 cents off earnings per share, as &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.theregister.com/2025/10/29/microsoft_earnings_q1_26_openai_loss/&quot;&gt;The Register&lt;/a&gt; detailed. That implies an OpenAI quarterly loss somewhere around $11.5 billion. Most 401(k) and pension money tracks the S&amp;amp;P 500, and by late 2025 the largest companies in that index, Nvidia, Microsoft, Amazon, Alphabet, Broadcom, Meta, and others, are all heavily exposed to AI. When OpenAI’s valuation rises, a big slice of the index gets lifted. If it stumbles, the shock hits savers who never chose to bet on AI at all.&lt;/p&gt;
&lt;p&gt;The second path is your electricity bill. AI data centers draw staggering amounts of power, and the grid was not built for them. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.pjm.com/&quot;&gt;PJM&lt;/a&gt;, which runs the largest wholesale power market in North America across 13 states and Washington DC, holds an annual auction to secure future capacity. For 2024–25 the price cleared at $28.92 per megawatt-day. For 2026–27 it cleared at $329.17, more than 11 times higher, per &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://ieefa.org/resources/projected-data-center-growth-spurs-pjm-capacity-prices-factor-10&quot;&gt;IEEFA’s analysis&lt;/a&gt; of PJM data. PJM’s own independent market monitor found data centers were responsible for 63% of the increase in the prior auction. That flows through to households as bills rising something like $16 to $21 a month in affected zones. When a family pays more to keep an AI campus running, they do not get a share of the data center. They just get the bill.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-04.DhxZ0WYX_ipwbR.webp&quot; alt=&quot;PJM’s capacity price jumped more than 11 times in two years. Its market monitor pinned 63% of the prior rise on data centers.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;889&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;PJM’s capacity price jumped more than 11 times in two years. Its market monitor pinned 63% of the prior rise on data centers. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://ieefa.org/resources/projected-data-center-growth-spurs-pjm-capacity-prices-factor-10&quot;&gt;IEEFA, PJM Monitoring Analytics&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The third path is taxes, the one OpenAI has worked hardest to deny. On 5 November 2025, CFO &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://en.wikipedia.org/wiki/Sarah_Friar&quot;&gt;Sarah Friar&lt;/a&gt; suggested at a Wall Street Journal event that a government guarantee could make financing AI chips cheaper. Asked directly whether she meant a taxpayer backstop, she agreed. The uproar was instant, and that same night she walked it back on LinkedIn, writing that OpenAI is not seeking a government backstop. White House AI official David Sacks said there would be no federal bailout, and Sam Altman said taxpayers should not bail out companies that make bad business decisions. Shortly before those comments, though, OpenAI had lobbied the White House to extend chip-manufacturing tax credits to AI data centers, suggesting the government deploy grants, cost-sharing, loans, or loan guarantees. Senator &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://en.wikipedia.org/wiki/Elizabeth_Warren&quot;&gt;Elizabeth Warren&lt;/a&gt; wrote to Altman in January 2026 warning of an old pattern, where companies take on huge debt, make risky bets, and demand a public rescue when the bets sour.&lt;/p&gt;
&lt;p&gt;Then in July 2026, Altman floated giving the US government a 5% stake in OpenAI, worth about $42.6 billion, pitched as letting ordinary Americans share in the upside. Read it the other way. Once the public owns 5% of OpenAI, a collapse stops being a purely private loss and becomes a loss on an asset held in the public’s name. That is pre-selling the downside while keeping the upside private for as long as possible.&lt;/p&gt;
&lt;p&gt;The fourth path is an eventual IPO landing straight in your index fund. OpenAI confidentially submitted a draft S-1 to the SEC on 8 June 2026, and while Altman said it would not list in 2026, a 2027 listing looks likely. Meanwhile the market plumbing has shifted. In 2026 Nasdaq introduced a fast-entry rule letting a newly listed company big enough to rank among the top 40 of the Nasdaq-100 be forced into the index within 15 trading days of its IPO, rather than waiting the usual 3 months. More than $30 trillion in global wealth tracks these indexes, so the moment OpenAI goes public and uses the fast lane, index funds and pension accounts become automatic buyers, whether their holders want the exposure or not. The insiders who got in cheap finally have someone to sell to.&lt;/p&gt;
&lt;h2 id=&quot;the-part-that-complicates-the-doom-story&quot;&gt;The part that complicates the doom story&lt;/h2&gt;
&lt;p&gt;I do not think OpenAI is doomed, and I want to be fair about that. Back in 2019, before ChatGPT existed, Altman joked that OpenAI had no revenue plan, that once it built a truly intelligent system it would just ask the AI how to make money. Seven years later it earns billions and serves close to a billion people a week. If it really does hit something like $350 billion in annual revenue by 2030, today’s losses will look like one of the boldest bets in business history, and everyone holding a piece will be glad they did.&lt;/p&gt;
&lt;p&gt;The size of the bet has changed who is making it. This is no longer a few venture capitalists rolling dice on a startup. It is chipmakers guaranteeing leases, cloud giants booking paper profits, utilities passing costs to households, a government being offered a stake, and index funds waiting to buy in on day 15. Every one of those players has a reason to believe, because each books the upside early. The group that never got asked is the one at the end of the line, the people with a 401(k), an electricity bill, and a tax return. The real question is not whether OpenAI can build the future. It is who pays for it if the future arrives late.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>openai</category><category>chatgpt</category><category>ai</category><category>artificial-intelligence</category><category>ai-agent</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Anthropic Just Posted $16.8</title><link>https://soursen.org/articles/anthropic-just-posted-16-8/</link><guid isPermaLink="true">https://soursen.org/articles/anthropic-just-posted-16-8/</guid><description>The numbers coming out of the AI industry are the biggest in the history of technology, and a growing group of analysts thinks that is the problem. I went through the real figures behind the bubble debate, Anthropic’s revenue, the infrastructure commitments, the debt, the falling cost of intelligence, and who is actually paying, because the case for a correction and the case against it both rest on the same facts.</description><pubDate>Thu, 22 Oct 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;In the middle of October 2026, the Financial Times got hold of a number Anthropic had shared with potential investors at a private meeting, and it set off a week of nerves across the AI industry. The number was not bad. It was enormous. Anthropic told investors its revenue for the third quarter reached &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://thenextweb.com/news/anthropic-q3-revenue-16-8bn-ipo&quot;&gt;$16.8 billion&lt;/a&gt;, a figure Reuters and Bloomberg both matched soon after. For a company this young, that is close to unprecedented. The nervousness is not about the size of the number. It is about its shape, because the quarter before, the growth rate was far higher, and in the AI industry a merely gigantic number that is decelerating reads as a warning.&lt;/p&gt;
&lt;p&gt;I want to walk through the actual figures on both sides of the bubble argument, because this is a case where the bulls and the bears are looking at the same data and drawing opposite conclusions, and the only way to think clearly about it is to see the numbers plainly.&lt;/p&gt;
&lt;h2 id=&quot;the-number-that-started-the-panic&quot;&gt;The number that started the panic&lt;/h2&gt;
&lt;p&gt;Here is the sequence that has people talking. Per Reuters, Anthropic’s revenue went from roughly $4.7 billion in the first quarter of 2026 to about $11.6 billion in the second, then to $16.8 billion in the third. That is growth of around 147% from Q1 to Q2, then about 45% from Q2 to Q3. Both are extraordinary rates most companies would never see. The issue is the drop between them. A business the whole AI narrative leans on going from 147% to 45% quarterly growth, in a single step, is the kind of thing that makes a market built on the promise of endless acceleration start asking questions.&lt;/p&gt;
&lt;p&gt;That is the core of the bear case, and it is a mathematical one. The analyst Alberto Romero put it cleanly in his writing on the subject, it only takes some finite amount of time to turn a proud exponential into a disappointing sigmoid, an S shaped curve that rises fast and then flattens. Every exponential eventually hits a binding constraint and bends over. The question for Anthropic is whether what we just saw is noise in a still steep curve or the first visible bend.&lt;/p&gt;
&lt;p&gt;The counterpoint is just as real. Reuters reported that investors who attended the meeting actually came away upbeat about the growth and the path to profit, and Anthropic told them the quarter was cash flow positive, a milestone most frontier AI labs have not reached. A deceleration from a once in history growth rate is not the same as a collapse. Both things are true at once, and which one matters more is exactly what the market is trying to price.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.BX_8HX6e_1XyEFp.webp&quot; alt=&quot;Figure 1. Anthropic’s quarterly revenue in 2026, with the growth rate falling even as the absolute numbers climb. The revenue roughly tripled in 2 quarters, and the quarter on quarter growth rate dropped from about 147% to about 45%.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1150&quot; height=&quot;636&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 1. Anthropic’s quarterly revenue in 2026, with the growth rate falling even as the absolute numbers climb. The revenue roughly tripled in 2 quarters, and the quarter on quarter growth rate dropped from about 147% to about 45%. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://thenextweb.com/news/anthropic-q3-revenue-16-8bn-ipo&quot;&gt;Author chart based on Reuters and Financial Times reporting&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-valuation-that-has-to-come-true&quot;&gt;The valuation that has to come true&lt;/h2&gt;
&lt;p&gt;The reason the growth rate matters so much is the price tag attached to it. Bloomberg reported that some prospective investors see a fair valuation for Anthropic in the range of $1.8 trillion to $2 trillion ahead of a possible IPO. To grow into a number like that, the company cannot just stay huge, it has to keep compounding at rates that only make sense if AI revenue keeps expanding into enormous new markets. Anthropic recorded an operating loss of more than $8 billion in 2025 on revenue of roughly $4.6 billion, which is normal for a lab spending to reach the frontier, and it is also a lot of ground to make up. A valuation near $2 trillion is a bet that the compounding does not stop.&lt;/p&gt;
&lt;h2 id=&quot;the-money-being-spent-is-the-part-that-cannot-be-undone&quot;&gt;The money being spent is the part that cannot be undone&lt;/h2&gt;
&lt;p&gt;Set the revenue aside and look at the spending, because that is where the risk gets concrete. Building AI means building data centers, and data centers cost real money on fixed timelines whether or not the customers show up. A data center that costs $10 billion still costs $10 billion if the AI payoff arrives a decade late. The construction workers, the energy companies, the landowners, and the lenders all want dollars, not tokens, and they want them on schedule.&lt;/p&gt;
&lt;p&gt;The lenders are where the strain is showing first. Take CoreWeave, a company that rents out GPUs to AI firms after pivoting from crypto mining. It has taken on roughly $52 billion in debt obligations, and in mid October 2026 the &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cryptobriefing.com/coreweave-shares-plunge-bond-yields-15/&quot;&gt;bond market turned on it&lt;/a&gt;. Its 2032 notes hit a 15% yield intraday on 13 October, and the stock fell as much as 16% the next day. A 15% yield is what investors demand to hold debt they consider risky, and this is for a company at the center of the supposedly most promising industry in history. Its interest bill, around $640 million in the second quarter alone, now runs ahead of its adjusted operating income. CoreWeave is an extreme case, but it is not an isolated one.&lt;/p&gt;
&lt;p&gt;The pressure reaches the giants too. Oracle raised $43 billion in debt in a recent fiscal year and ended with negative free cash flow. Even Alphabet, one of the most stable companies on earth, generated about $39 billion in cash from operations in a single quarter and spent about $45 billion on infrastructure, turning its free cash flow negative for the first time since it went public in 2004. When Google has to borrow to keep building, the scale of the commitment is clear.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.yVDyoHvf_Z1wfz58.webp&quot; alt=&quot;Figure 2. The debt stress is showing up first at the companies financing the buildout. CoreWeave’s bond yields climbing toward 15% signal that lenders now treat a central AI infrastructure company as a risky borrower.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1183&quot; height=&quot;606&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 2. The debt stress is showing up first at the companies financing the buildout. CoreWeave’s bond yields climbing toward 15% signal that lenders now treat a central AI infrastructure company as a risky borrower. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cryptobriefing.com/coreweave-shares-plunge-bond-yields-15/&quot;&gt;Author chart based on CoreWeave bond market reporting, October 2026&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-gap-between-what-is-being-spent-and-what-is-coming-back&quot;&gt;The gap between what is being spent and what is coming back&lt;/h2&gt;
&lt;p&gt;The single clearest way to frame the bubble question comes from &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://techcrunch.com/2026/07/09/can-ai-answer-the-3-trillion-question/&quot;&gt;Sequoia’s David Cahn&lt;/a&gt;, who has been tracking it since 2023. He started by asking how much revenue the industry needed to justify its infrastructure spending, and called it AI’s $200 billion question. By 2026 his updated math puts AI infrastructure spending for the year at about $1.5 trillion, and the revenue the industry would need to earn to justify those chips and data centers at around $3 trillion, a figure he thinks is probably an underestimate. Set that against where the industry actually is, and the distance is the whole argument.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cryptobriefing.com/gartner-ai-infrastructure-write-down-risk/&quot;&gt;Gartner&lt;/a&gt; put a number on the downside. It estimated that more than $3.5 trillion in AI infrastructure spending could face impairment by 2030, meaning written down as worth less than what was paid for it, if the current investment pace holds. The firm also projected that data centers will generate trillions of dollars less in revenue by 2030 than they would need to justify their cost and deliver expected returns. These are forecasts, not facts, and forecasts about 2030 are easy to wave away. The bear case is that markets do not wait for 2030. If investors become convinced today that the returns will not arrive on time, they sell today.&lt;/p&gt;
&lt;h2 id=&quot;the-cost-of-intelligence-is-falling-off-a-cliff&quot;&gt;The cost of intelligence is falling off a cliff&lt;/h2&gt;
&lt;p&gt;Here is the figure I find hardest to argue with, and it cuts against every AI company at once. Per &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://marginalrevolution.com/marginalrevolution/2026/09/the-price-of-intelligence-is-falling.html&quot;&gt;Epoch AI&lt;/a&gt;, the cost of achieving a fixed level of AI performance has been falling about 47% every quarter since 2023. That works out to roughly a thirteenfold reduction every year. The capability you spent billions to build this year gets dramatically cheaper for someone else to reproduce next year.&lt;/p&gt;
&lt;p&gt;That is wonderful for anyone using AI and hard for anyone selling it at a premium. Google’s models improve, Meta has caught up from behind, Chinese labs ship capable models at low prices, and open weight alternatives handle the ordinary tasks for free. OpenAI sits toe to toe with Anthropic in the same markets, code, agents, and enterprise. When the thing you sell gets 13 times cheaper to make every year and several well funded competitors sell something similar, holding a premium price is genuinely difficult. The bull reply is Jevons paradox, that cheaper intelligence drives so much more usage that total revenue grows anyway. That can be true, and rising usage still does not guarantee rising profit when competition is this fierce and the product is drifting toward a commodity.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.BkcArp-J_ZgImY8.webp&quot; alt=&quot;Figure 3. The true exponential in AI may be the price collapse. Per Epoch AI, the cost of a fixed level of performance has fallen about 47% per quarter since 2023, roughly 13 times cheaper each year, which squeezes everyone selling frontier models.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1150&quot; height=&quot;641&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 3. The true exponential in AI may be the price collapse. Per Epoch AI, the cost of a fixed level of performance has fallen about 47% per quarter since 2023, roughly 13 times cheaper each year, which squeezes everyone selling frontier models. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://marginalrevolution.com/marginalrevolution/2026/09/the-price-of-intelligence-is-falling.html&quot;&gt;Author chart based on Epoch AI data&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;almost-nobody-is-actually-paying-yet&quot;&gt;Almost nobody is actually paying yet&lt;/h2&gt;
&lt;p&gt;For all the money moving, the customer base is thinner than the valuations imply. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cryptobriefing.com/us-households-pay-ai-services-a16z/&quot;&gt;Andreessen Horowitz&lt;/a&gt;, in its State of Markets report, found that only about 2% of US households were paying for any AI service as of April 2026. After this much progress and this much capability, 2% is a strikingly small share. On the enterprise side the picture is broad but shallow, nearly 30% of S&amp;amp;P 500 companies reported some AI impact, while only about 2% reported a tracked, quantifiable metric. Adoption is wide and usage is light.&lt;/p&gt;
&lt;p&gt;There are early signs of belt tightening too. Reporting on Anthropic’s own customers suggests some of its largest corporate accounts have trimmed their spending as they build their own tools, and a few companies have capped what employees can spend on AI each month. None of this is fatal on its own. It does complicate the story that enterprise demand is an unstoppable escalator.&lt;/p&gt;
&lt;h2 id=&quot;what-a-bubble-actually-means&quot;&gt;What a bubble actually means&lt;/h2&gt;
&lt;p&gt;A bubble is one of the most misunderstood words in this debate, so it is worth being precise about what it means. A bubble does not mean the technology is fake, or that every company goes bust, or that AI fails to change the world. It means the core premise behind the industry’s valuations has been inflated beyond what reality can support in the expected timeframe. The internet was real and world changing, and the dot com bubble still popped, because the prices had run ahead of the business. A bubble is a correction mechanism, not a verdict on the technology.&lt;/p&gt;
&lt;p&gt;That distinction is why smart people land on opposite sides. You can believe AI is the most important technology of the century and also believe its current valuations are too high, those are not contradictory positions. The companies are, by most measures of product and growth, among the best startups ever built. The argument is about price and timing, not about whether the tools work.&lt;/p&gt;
&lt;h2 id=&quot;why-the-whole-market-is-watching-1-company&quot;&gt;Why the whole market is watching 1 company&lt;/h2&gt;
&lt;p&gt;The reason this is not a niche tech story is that the stock market has become unusually concentrated on AI. Reuters noted that the S&amp;amp;P 500 rose about 14% in a recent stretch even though more than 40% of its stocks were down, meaning a handful of large AI linked names carried the whole index. The Kobeissi Letter pointed out that the correlation between the equal weighted S&amp;amp;P 500 and the standard index had fallen to its lowest level on record, lower than during COVID and lower than the dot com peak. A market held up by a few names is a market with a thin margin for disappointment.&lt;/p&gt;
&lt;p&gt;That is what ties Anthropic’s revenue growth to your retirement account. The financial engineering underneath is getting elaborate too. A UBS analysis described deals where investors buy chips from Google, lease them to Anthropic, and Google backstops parts of the arrangement, so several companies’ financial health depends on the same stream of future AI revenue showing up. There is nothing inherently wrong with structuring deals that way, and it does mean a shortfall in one place can ripple through many balance sheets at once.&lt;/p&gt;
&lt;h2 id=&quot;what-would-actually-settle-it&quot;&gt;What would actually settle it&lt;/h2&gt;
&lt;p&gt;Both sides agree on the test, they just disagree on the result. It comes down to whether run rate revenue, the annualized version of a recent strong month or quarter, turns into durable actual revenue, people and companies paying enough, for long enough, to justify the spending. If Anthropic reaccelerates in the fourth quarter, reaches its run rate targets, and enterprise adoption deepens, the bears will have been early and wrong. If growth keeps flattening while the infrastructure bills come due on schedule, the case for a correction gets stronger each quarter.&lt;/p&gt;
&lt;p&gt;I do not know which way it breaks, and anyone who tells you they do with certainty is selling something. What I can say is that the numbers to watch are specific and public. Watch Anthropic’s next quarter for whether growth stabilizes or keeps sliding. Watch the bond yields at companies like CoreWeave for whether lenders calm down or tighten further. Watch the share of households and enterprises actually paying, not just experimenting. Those figures, not the rhetoric, are where this gets decided.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>artificial-intelligence</category><category>ai</category><category>anthropic-claude</category><category>claude</category><category>bubble</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Your GPU Says 100% and Your Model Still Crawls. The Real Bottleneck Is the Kernel You Never See.</title><link>https://soursen.org/articles/your-gpu-says-100-and-your-model-still-crawls-the-real-bottleneck-is/</link><guid isPermaLink="true">https://soursen.org/articles/your-gpu-says-100-and-your-model-still-crawls-the-real-bottleneck-is/</guid><description>nvidia-smi tells you the card is busy. It does not tell you the card is working. I want to show you the gap between those 2 things, because closing it is often one number you type in Python.</description><pubDate>Fri, 16 Oct 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;I have watched a lot of training runs sit at 100% GPU utilization and still take twice as long as they should. The first few times, I did what most people do. I assumed the card was the limit, asked for a bigger one, and moved on. That was the wrong read. The card was barely working. It just looked busy.&lt;/p&gt;
&lt;p&gt;The thing deciding your real speed is the kernel, the small program the GPU actually runs when you call something like torch.matmul. You almost never write one by hand. You pick them without knowing it, every time you set a batch size or a vocabulary size, and those picks can swing training time by 25% or more on the exact same hardware. I want to walk through what a kernel is, why your tensor shapes silently choose a fast or slow one, what the GPU spends its time waiting on, and the handful of changes that actually move the number.&lt;/p&gt;
&lt;h2 id=&quot;why-100-utilization-is-lying-to-you&quot;&gt;Why 100% utilization is lying to you&lt;/h2&gt;
&lt;p&gt;Start with the number everyone trusts. You run &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://developer.nvidia.com/system-management-interface&quot;&gt;nvidia-smi&lt;/a&gt;, the command that reports how busy your GPU is, and it says 100%. The instinct is to read that as the chip being full. It is not what the number means.&lt;/p&gt;
&lt;p&gt;That 100% only says at least 1 kernel was running during the sampling window. NVIDIA’s own documentation defines GPU utilization as the fraction of time over the past sample period that 1 or more kernels was executing, with no account of how much of the chip each kernel used. A single kernel using a few dozen threads out of the tens of thousands a modern GPU can run will still report 100% as long as it never stops. The card looks maxed out while almost all of its math units sit idle.&lt;/p&gt;
&lt;p&gt;So the real question underneath your step time is not whether a kernel is running. It is how much of the chip each kernel keeps busy, and how long each one waits on memory or on the CPU before it can do anything. Those are 3 separate problems, and they have 3 separate fixes.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.7QsGGDoE_1HdP6p.webp&quot; alt=&quot;Figure 1. What nvidia-smi actually measures. A kernel using a sliver of the chip reports the same 100% as one using all of it.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1090&quot; height=&quot;596&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 1. What nvidia-smi actually measures. A kernel using a sliver of the chip reports the same 100% as one using all of it. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://developer.nvidia.com/system-management-interface&quot;&gt;NVIDIA System Management Interface docs&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;what-a-kernel-actually-is&quot;&gt;What a kernel actually is&lt;/h2&gt;
&lt;p&gt;A GPU kernel has nothing to do with the Jupyter kernel you restart when a notebook hangs. That one is the Python process running your cells. A GPU kernel is 1 small function the GPU copies across your data and runs thousands of times at once.&lt;/p&gt;
&lt;p&gt;Each copy runs as a thread, and a GPU thread is far cheaper than a CPU thread. The GPU launches thousands of them together, each working on 1 slice of the result. Threads are grouped into blocks, and every thread reads its own index to figure out which slice it owns. A simple vector add of 2 arrays with 262,144 elements each might launch 1,024 blocks of 256 threads, which is 262,144 copies of the same tiny function, each adding 1 pair of numbers.&lt;/p&gt;
&lt;p&gt;The GPU runs those copies on the small processors that make up the chip, which NVIDIA calls &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.nvidia.com/cuda/cuda-c-programming-guide/&quot;&gt;streaming multiprocessors&lt;/a&gt;, or SMs. Each SM has its own math cores and its own small, fast on chip memory. An &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.nvidia.com/en-us/data-center/a100/&quot;&gt;A100&lt;/a&gt; has 108 of these SMs. An H100 has 132, with 128 math cores each. That SM count matters more than it looks, and in a minute you will see it decide your speed directly.&lt;/p&gt;
&lt;p&gt;Every kernel has 3 moving parts. The code is the set of steps each copy runs, identical for every thread. The launch configuration is how many blocks run and how many threads sit in each block. The data is the tensors the kernel reads out of GPU memory and the result it writes back. In plain PyTorch you touch none of that code. You choose 3 numbers instead, the batch size, the sequence length, and the vocabulary size, and PyTorch hands those to NVIDIA’s math library, which picks the kernel that runs.&lt;/p&gt;
&lt;h2 id=&quot;the-reason-all-of-this-matters-in-one-chart&quot;&gt;The reason all of this matters, in one chart&lt;/h2&gt;
&lt;p&gt;Before the mechanics, here is the pressure behind them. Over the past 20 years the peak math throughput of accelerators grew about 3x every 2 years, roughly 60,000x in total. Memory bandwidth, the rate at which the chip can pull numbers in from its own memory, grew only about 1.6x every 2 years, roughly 100x in total. The interconnect between chips grew slower still, about 1.4x every 2 years. Those numbers come from &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/abs/2403.14123&quot;&gt;AI and Memory Wall&lt;/a&gt;, a 2024 analysis by Gholami and colleagues, and they describe a widening gap. The chip can do math far faster than it can be fed.&lt;/p&gt;
&lt;p&gt;Model size pulled in the opposite direction over the same window. The same paper measures Transformer parameter counts growing about 410x every 2 years and training compute about 750x every 2 years. So models got hungrier at a rate that dwarfs how fast memory learned to feed them. That mismatch is why so much of modern performance work is really memory work, and why a kernel that keeps every math core busy can still spend most of its time idle.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.CQKAGPX8_Z1VXGtW.webp&quot; alt=&quot;Figure 2. Over 20 years, compute grew about 60,000x while memory bandwidth grew about 100x. The gap between the 2 lines is where most slow kernels live.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1091&quot; height=&quot;621&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 2. Over 20 years, compute grew about 60,000x while memory bandwidth grew about 100x. The gap between the 2 lines is where most slow kernels live. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/abs/2403.14123&quot;&gt;AI and Memory Wall, Gholami et al. 2024&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;You can see the same gap inside a single modern GPU. An A100 does about 312 trillion 16 bit operations a second but reads from its memory at about 2 terabytes a second. An H100 pushes compute to roughly 990 trillion operations a second while bandwidth climbs only to 3.35 terabytes a second. Divide the 2 and you get a hard rule of thumb. On an A100 a kernel has to do about 150 math operations for every byte it loads to keep the math cores busy. On an H100 that number rises to about 295. Any operation below its chip’s line is limited by memory, not math, and most of the small operations in a real model fall well below it.&lt;/p&gt;
&lt;h2 id=&quot;how-your-tensor-shapes-secretly-pick-a-slow-kernel&quot;&gt;How your tensor shapes secretly pick a slow kernel&lt;/h2&gt;
&lt;p&gt;Here is the part that surprised me the first time I measured it. The CPU never runs a kernel. It drops the launch into a queue and keeps going, and the GPU doles the kernel’s blocks out to its SMs. Each block runs on 1 SM from start to finish, and the blocks go out in rounds. Once every SM is holding a block, the leftover blocks wait for the next round.&lt;/p&gt;
&lt;p&gt;For a matrix multiply, the number of blocks comes from the shape of the result. The library slices the output into rectangles called tiles and gives each tile to 1 block. NVIDIA’s own &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html&quot;&gt;matrix multiplication guide&lt;/a&gt; shows what the rounds cost on an A100 and its 108 SMs. With the library’s largest tiles, each SM handles 1 block at a time. A result that comes out exactly 12 tiles wide and 9 tiles tall is 108 tiles, which fills all 108 SMs in a single clean round. Now make that result 8 columns wider. Every row suddenly needs a 13th tile, so the launch jumps to 117 blocks. The first 108 run together in round 1, then the last 9 run alone in round 2 while 99 SMs sit completely empty. You added roughly 0.5% of extra work and nearly doubled the time. Nothing changed except the width of 1 matrix.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.NyZL76sd_Z14kVXS.webp&quot; alt=&quot;Figure 3. 117 blocks on 108 SMs. The second round carries 9 blocks and leaves 99 processors idle, so a tiny size change roughly doubles the time. This is called the tile quantization effect.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1090&quot; height=&quot;610&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 3. 117 blocks on 108 SMs. The second round carries 9 blocks and leaves 99 processors idle, so a tiny size change roughly doubles the time. This is called the tile quantization effect. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html&quot;&gt;NVIDIA matrix multiplication performance guide&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;This is why your batch size, sequence length, and vocabulary size quietly decide your speed. PyTorch sends its matrix multiplies to &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.nvidia.com/cuda/cublas/&quot;&gt;cuBLAS&lt;/a&gt;, NVIDIA’s math library, and cuBLAS selects a kernel from the exact dimensions of the 2 matrices being multiplied, which those 3 numbers set. On an A100, the fastest kernels run best when the dimensions divide cleanly by 64, and odd dimensions ran slowest in NVIDIA’s own testing.&lt;/p&gt;
&lt;p&gt;Andrej Karpathy ran straight into this in February 2023 with &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/karpathy/nanoGPT&quot;&gt;nanoGPT&lt;/a&gt;, his small GPT training repo. A language model’s final layer scores every token in its vocabulary, so the vocabulary size is 1 side of that layer’s matrix multiply. nanoGPT’s vocabulary had 50,257 entries, an awkward odd number. Karpathy padded it up to 50,304, which is 64 times 786. The 47 extra slots match no real token, so the model does about 0.09% more arithmetic, and yet &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://simonwillison.net/2023/Feb/4/andrej-karpathy/&quot;&gt;training ran about 25% faster&lt;/a&gt; on the same GPU. In his words the padded size “goes down a different kernel path with much higher occupancy.” Higher occupancy means more of each SM stayed busy. The lesson I took from it is blunt. When you pick a vocabulary, a hidden dimension, or any size that lands inside a matrix multiply, round it up to a multiple of 64. It is free speed.&lt;/p&gt;
&lt;h2 id=&quot;a-full-sm-is-not-the-same-as-a-fast-kernel&quot;&gt;A full SM is not the same as a fast kernel&lt;/h2&gt;
&lt;p&gt;A kernel that fills all 132 SMs can still crawl, because a thread that is not computing is usually waiting on memory. This is the second of the 3 problems, and for small operations it is the dominant one.&lt;/p&gt;
&lt;p&gt;Your tensors live in the GPU’s main memory, the separate stacks of chips next to the processors. Every kernel fetches its inputs from there and writes its results back there, and a single fetch takes hundreds of times longer than a single step of arithmetic. So a kernel that does only a little math per number it loads spends most of its life waiting for the next number to arrive.&lt;/p&gt;
&lt;p&gt;A big matrix multiply is the good case. It reuses every value it loads across thousands of calculations, so the SMs stay fed. The small steps are the bad case, things like activation functions such as ReLU, normalization layers, and softmax. They do 2 or 3 operations per value and then sit idle until the next value shows up. The clearest measurement of this I know of comes from &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/abs/2007.00072&quot;&gt;Data Movement Is All You Need&lt;/a&gt;, a 2020 study of BERT, Google’s language model. The authors found that normalization and elementwise operations were 0.2% of the total arithmetic yet consumed about 37% of the runtime of a BERT layer. Almost all of that time went to shipping data to and from memory, not to the math itself. These are called memory bound operations, and most models are full of them.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-04.CMrV2YwN_Z4lW4O.webp&quot; alt=&quot;Figure 4. In a BERT layer, normalization and elementwise steps are 0.2% of the arithmetic but about 37% of the runtime. The slow part is memory traffic, not math.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1090&quot; height=&quot;591&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 4. In a BERT layer, normalization and elementwise steps are 0.2% of the arithmetic but about 37% of the runtime. The slow part is memory traffic, not math. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/abs/2007.00072&quot;&gt;Data Movement Is All You Need, Ivanov et al. 2020&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;There is a clean way to predict which case you are in before you ever run the model, and it has a name, the roofline. Take any operation and compute its arithmetic intensity, the number of math operations it does per byte it moves to and from memory. Plot that against the chip’s own ratio of peak compute to peak bandwidth, which I gave earlier as about 150 for an A100. If your operation’s intensity sits below that line, memory is your ceiling and faster math cores will not help you. If it sits above, compute is your ceiling. A large matrix multiply with big dimensions can reach hundreds or thousands of operations per byte, which is why it runs near peak. A softmax or a layer norm does a handful of operations per value, lands far below the line, and runs at a small fraction of peak no matter how expensive your GPU is. The roofline is the single most useful mental picture I have for GPU speed, because it tells you in advance whether a bigger card or a smarter kernel is the fix.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-05.D6ZtoAU8_Z1hDEua.webp&quot; alt=&quot;Figure 5. The roofline. An operation below the A100’s line of about 150 operations per byte is limited by memory, so a faster card does not help it. Fusion moves work rightward, toward the compute ceiling.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1091&quot; height=&quot;636&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 5. The roofline. An operation below the A100’s line of about 150 operations per byte is limited by memory, so a faster card does not help it. Fusion moves work rightward, toward the compute ceiling. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.nvidia.com/deeplearning/performance/dl-performance-gpu-background/index.html&quot;&gt;Roofline model, after Williams et al. and NVIDIA guidance&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The fix is fusion, which means merging several small steps into 1 kernel so it touches memory less. Run x.cos().cos() as 2 separate kernels and they hit main memory 4 times. The first reads x, writes the intermediate result, the second reads that back, writes the final answer. Fuse them into 1 kernel and the intermediate value never leaves the chip, so only 2 trips remain and the pair runs roughly twice as fast. Plain PyTorch cannot do this on its own, because it runs 1 operation at a time and never gets to see the next one coming. The fix is a single line, model = torch.compile(model), which reads your whole model up front and fuses the chains it can. Horace He’s &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://horace.io/brrr_intro.html&quot;&gt;Making Deep Learning Go Brrrr From First Principles&lt;/a&gt; is the clearest explanation of why this works, and it is where I would send anyone who wants the full mental model.&lt;/p&gt;
&lt;h2 id=&quot;the-third-wait-is-the-cpu-and-it-hides-in-plain-sight&quot;&gt;The third wait is the CPU, and it hides in plain sight&lt;/h2&gt;
&lt;p&gt;The last thing a kernel waits on is Python itself. Every kernel gets launched from the CPU, 1 at a time, and Python is painfully slow next to the GPU. In the time it takes Python to add 2 numbers, an A100 can run on the order of 9 to 10 million floating point operations. A large kernel hides that lag, because the GPU is still grinding on the current one when the next launch arrives. A tiny kernel does not. It finishes before Python can hand over the next one, and the GPU goes dark in the gap.&lt;/p&gt;
&lt;p&gt;The fix here is a &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://pytorch.org/blog/accelerating-pytorch-with-cuda-graphs/&quot;&gt;CUDA graph&lt;/a&gt;. It records a whole sequence of kernel launches once, then replays all of them with a single launch, which deletes the per kernel Python overhead. In PyTorch you get one by passing a mode to the compiler, torch.compile(model, mode=”reduce-overhead”). Horace He’s piece has a quick test to tell whether this is even your problem. Double your batch size. If the step takes only about 10% longer, the actual work was barely costing anything and you were overhead bound, so cutting launch overhead is where your time is.&lt;/p&gt;
&lt;h2 id=&quot;who-writes-their-own-kernels-and-what-it-buys-them&quot;&gt;Who writes their own kernels, and what it buys them&lt;/h2&gt;
&lt;p&gt;Most of the speed work for most teams is choosing shapes and flipping on torch.compile. Writing kernels by hand is a different sport, and a few teams play it because speed is their product. Their numbers are worth seeing, because they show the ceiling.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://unsloth.ai/&quot;&gt;Unsloth&lt;/a&gt;, an open source library for fine tuning LLMs, rewrote the model code as kernels in &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://openai.com/index/triton/&quot;&gt;Triton&lt;/a&gt;, a Python like language for GPU code. On an A100 it fine tunes popular models around 2x faster than standard Hugging Face training while using roughly 40% to 60% less memory, with no loss in accuracy, per Hugging Face’s own &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://huggingface.co/blog/unsloth-trl&quot;&gt;benchmark writeup&lt;/a&gt;. If you fine tune with it, you are already running handwritten kernels.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/IST-DASLab/marlin&quot;&gt;Marlin&lt;/a&gt; is the kernel &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.vllm.ai/en/latest/&quot;&gt;vLLM&lt;/a&gt; reaches for when a model stores its weights as 4 bit numbers, a quarter of the usual 16 bits. Shrinking the weights means a quarter of the memory traffic, and for a memory bound workload that is almost all of the cost. The &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/abs/2408.11743&quot;&gt;Marlin paper&lt;/a&gt; measures the matrix multiply running about 3.9x faster than the FP16 version at batch sizes of 16 to 32, which is close to the theoretical 4x ceiling for 4 bit weights. The speedup fades as batches grow and the work turns compute bound, down to roughly 1.5x at batch 128, but near the ideal where it counts.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cursor.com/&quot;&gt;Cursor&lt;/a&gt;, the AI code editor, wrote its own &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cursor.com/blog/kernels&quot;&gt;MXFP8 kernels&lt;/a&gt; for training its mixture of experts models. Their expert layers run about 3.5x faster than the BF16 version on both the forward and backward passes, and end to end their training runs about 1.5x faster on NVIDIA’s Blackwell chips versus BF16, and about 2x faster than their old Hopper setup, all with training loss they describe as nearly indistinguishable from full precision over 10,000 steps.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-06.x07YHy-E_Zy4Err.webp&quot; alt=&quot;Figure 6. What handwritten kernels buy 3 teams that sell or serve at scale. Each number is from the team’s own published measurements.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1091&quot; height=&quot;606&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 6. What handwritten kernels buy 3 teams that sell or serve at scale. Each number is from the team’s own published measurements. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://huggingface.co/blog/unsloth-trl&quot;&gt;Hugging Face, Marlin paper, Cursor engineering blog&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;There is also a cautionary case, and it is a good one, because it shows what a broken kernel costs when nobody is watching the output. In its &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues&quot;&gt;postmortem of 3 overlapping bugs&lt;/a&gt;, Anthropic described a kernel problem inside Claude. The model picks each next token from a shortlist of the most likely candidates, which a top k kernel builds. A deployment on August 25, 2025 tripped a latent bug in the XLA compiler for TPUs, Google’s AI chips, and the approximate top k kernel began, for certain batch sizes, dropping the single highest probability token off the shortlist. The symptom was subtle quality degradation that was maddening to reproduce. Anthropic confirmed the damage on Claude Haiku 3.5, rolled that model back on September 4, and switched to an exact top k kernel, trading a little speed to guarantee the right token survived. A kernel that is fast and wrong is worse than a kernel that is slow and right.&lt;/p&gt;
&lt;h2 id=&quot;what-is-oversold-and-what-holds-up&quot;&gt;What is oversold, and what holds up&lt;/h2&gt;
&lt;p&gt;“Write custom kernels” gets oversold for most teams. People repeat it because handwritten kernels are exactly where Unsloth, Marlin, and Cursor found their speed. But PyTorch, torch.compile, and serving engines like vLLM already ship fast kernels for the common model shapes, so for most of us the real speed work is picking good sizes and turning on the compiler. Unsloth and Cursor sell speed, and vLLM serves at enormous scale, which is why writing kernels pays off for them and usually not for you.&lt;/p&gt;
&lt;p&gt;“The compiler will just fuse it” is half true. torch.compile merges chains of simple steps like activation functions very well. What it cannot do is invent a new way to compute something. It fuses the steps you wrote, in the order you wrote them. Attention is the operation where every token looks back at every token before it, scoring all the pairs, and the fast version, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/abs/2205.14135&quot;&gt;FlashAttention&lt;/a&gt;, exists because researchers rewrote that math so the giant intermediate score matrix never fully lands in memory. The payoff was measured, not promised. The 2022 paper reports a 15% end to end speedup on BERT large against the then standard MLPerf record, a 3x speedup training GPT-2, and 2.4x on long sequence benchmarks, all while cutting attention’s memory use from quadratic to linear in the sequence length. A compiler will not derive any of that from a naive attention loop. Someone had to rethink the algorithm so it respected the memory hierarchy, which is the whole point. The fastest kernels are the ones written by people who treated memory traffic as the real cost.&lt;/p&gt;
&lt;h2 id=&quot;how-to-find-your-own-bottleneck-in-10-minutes&quot;&gt;How to find your own bottleneck in 10 minutes&lt;/h2&gt;
&lt;p&gt;You do not have to guess which of the 3 waits is eating your time. Here is the sequence I run, in order, and it almost always lands on the answer.&lt;/p&gt;
&lt;p&gt;Start with the doubling test from Horace He. Double your batch size and time 1 step. If the step gets much slower in proportion, you are compute bound or memory bound and the work is real. If it barely moves, under about 10% more time, you are overhead bound, the GPU is starving between tiny kernel launches, and torch.compile with mode “reduce-overhead” is your first move because it builds the CUDA graph that removes the gaps.&lt;/p&gt;
&lt;p&gt;If the work is real, record a trace with &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html&quot;&gt;PyTorch’s profiler&lt;/a&gt; for a few steps after the first, since the first steps burn time on 1 time setup. Open the trace and read where the time goes. A wall of small kernels named after normalization, activation, or elementwise operations means you are memory bound, and the fix is fusion through torch.compile, or a prebuilt kernel like FlashAttention for attention. A few large matrix multiply kernels dominating the time means you are compute bound, and now the shape rules pay off, so round your dimensions up to multiples of 64 and check that cuBLAS is not falling into a slow tail wave.&lt;/p&gt;
&lt;p&gt;Serving a quantized model moves the lever again. Weights stored at 4 or 8 bits cut the memory traffic that dominates generation, so a kernel like Marlin or a serving engine like vLLM that ships one gives you most of the speed without writing anything. The order matters. Cheap settings first, profiler second, custom kernels last, and most people never need the last step.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-07.CsH3x1WA_Z2uuzNU.webp&quot; alt=&quot;Figure 7. A decision map for the 3 bottlenecks. The doubling test and a profiler trace point you to the one fix that matters, before you spend on a bigger GPU.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1150&quot; height=&quot;820&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 7. A decision map for the 3 bottlenecks. The doubling test and a profiler trace point you to the one fix that matters, before you spend on a bigger GPU. Source: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html&quot;&gt;Author’s synthesis of PyTorch and NVIDIA profiling guidance&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-one-habit-worth-keeping&quot;&gt;The one habit worth keeping&lt;/h2&gt;
&lt;p&gt;You are choosing GPU kernels every time you set a batch size, a sequence length, or a vocabulary size, whether you meant to or not. Those 3 numbers decide which kernel cuBLAS runs and how many of your SMs it fills, which makes the cheapest speedup available to you a number you type in Python. Karpathy’s was a single padded vocabulary size, worth 25% faster training. Before you reach for a bigger GPU, record a trace with &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html&quot;&gt;PyTorch’s profiler&lt;/a&gt;, see whether your time is going to math, to memory, or to Python, and fix the one that is actually eating your step. The card is probably faster than you think. It is just waiting.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>machine-learning</category><category>deep-learning</category><category>llm</category><category>pytorch</category><category>cuda</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>China Just Overtook America on the Only Metric That Predicts Who Builds the Future</title><link>https://soursen.org/articles/china-just-overtook-america-on-the-only-metric-that-predicts-who-builds/</link><guid isPermaLink="true">https://soursen.org/articles/china-just-overtook-america-on-the-only-metric-that-predicts-who-builds/</guid><description>Note: This article discusses ongoing industry developments and government policy. It is for informational purposes only.</description><pubDate>Fri, 17 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;blockquote&gt;
&lt;p&gt;Note: This article discusses ongoing industry developments and government policy. It is for informational purposes only.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;12 months ago, American AI models handled roughly 70% of all traffic on OpenRouter, the largest neutral marketplace where developers route requests across hundreds of AI models from more than 60 providers. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://officechai.com/ai/share-of-us-models-being-used-on-openrouter-has-collapsed-from-70-to-30-over-the-past-year/&quot;&gt;By June 2026, that figure had collapsed to about 30%&lt;/a&gt;. Chinese models, led by DeepSeek, now handle the majority of tokens processed on the platform.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.DeBRpEBJ_2lNmkG.webp&quot; alt=&quot;US model share of OpenRouter token volume collapsed from 70% in June 2025 to roughly 30% by June 2026. Chinese models now handle the majority.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1488&quot; height=&quot;802&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;US model share of OpenRouter token volume collapsed from 70% in June 2025 to roughly 30% by June 2026. Chinese models now handle the majority. Source: Bloomberg via Exponential View data, OfficeChai, July 2026.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The tipping point arrived during the week of February 9 to 15, 2026, when &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cryptobriefing.com/openrouter-us-models-token-share-collapse/&quot;&gt;Chinese models processed 4.12 trillion tokens on OpenRouter against 2.94 trillion for American models&lt;/a&gt;, the first time Chinese systems had outpaced American ones on the platform. It never went back after that. By June, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://officechai.com/ai/share-of-us-models-being-used-on-openrouter-has-collapsed-from-70-to-30-over-the-past-year/&quot;&gt;DeepSeek alone commanded 16.3% of all token volume on OpenRouter, more than any single American provider including Google, Anthropic, or OpenAI&lt;/a&gt;. Combined Chinese providers, DeepSeek, Xiaomi, Tencent, MiniMax, and Qwen, account for roughly 44% of token volume across the platform’s top 10 models.&lt;/p&gt;
&lt;h2 id=&quot;the-story-is-not-abstract-real-companies-made-this-switch&quot;&gt;The Story Is Not Abstract. Real Companies Made This Switch.&lt;/h2&gt;
&lt;p&gt;AI agent startup Lindy moved 100% of its production traffic from Anthropic’s Claude to DeepSeek’s V4. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.kucoin.com/news/flash/u-s-companies-shift-to-chinese-ai-models-token-usage-surges-to-46&quot;&gt;Founder Flo Crivello told CNBC the switch cut inference costs by roughly 90% and called it a matter of survival, since the company’s AI inference bill had grown larger than its payroll&lt;/a&gt;. The switch was not casual. It required 6 to 9 months of testing, and to address the obvious data security concern, Lindy hosts the Chinese model on US soil through a provider called Atlas Cloud, so requests never actually route through Chinese servers. Crivello has said he would switch back if Anthropic lowered its prices, but until then, he has said the company has no other real option.&lt;/p&gt;
&lt;p&gt;Lindy is not an isolated case. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://letsdatascience.com/blog/startup-moved-100-percent-traffic-claude-to-deepseek&quot;&gt;CNBC reported that Uber burned through its entire annual AI budget in just 4 months in 2026, primarily driven by Claude Code usage, and responded by capping engineers at $1,500 a month and routing non-urgent work to cheaper models&lt;/a&gt;. GitHub faced a similar runaway cost problem with Copilot’s agent mode and was forced to shift from a flat monthly subscription to pay-as-you-go pricing.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;On Vercel’s own AI gateway, which now routes more than a trillion tokens a day, CEO Guillermo Rauch described the same shift from his side of the infrastructure.&lt;/em&gt; “We’re seeing a lot of growth of Gemini, even though it’s not on the news as much, because people are optimizing for production now,” &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://letsdatascience.com/blog/startup-moved-100-percent-traffic-claude-to-deepseek&quot;&gt;&lt;em&gt;Rauch told TechCrunch. “You also bring in open models, so DeepSeek and GLM-5.2 are taking off.”&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;After Zhipu AI’s GLM-5.2 was released in June 2026, it achieved the fastest adoption rate on Vercel’s platform that entire year, with daily token usage increasing roughly 27-fold in its first week.&lt;/p&gt;
&lt;h2 id=&quot;why-it-is-this-cheap&quot;&gt;Why It Is This Cheap&lt;/h2&gt;
&lt;p&gt;The headline pricing gap is close to absurd once you see it directly. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.kucoin.com/news/flash/u-s-companies-shift-to-chinese-ai-models-token-usage-surges-to-46&quot;&gt;DeepSeek V4 Flash charges $0.14 per million input tokens and $0.28 per million output tokens&lt;/a&gt;, compared to $5 input and $30 output for OpenAI’s GPT-5.5, and $5 input and $25 output for Anthropic’s Claude Opus 4.8. On the input side alone, that is roughly 36 times cheaper for a comparable unit of work. On the output side the gap widens to more than 100 times.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.CD_gUgCI_QVbOY.webp&quot; alt=&quot;Per-million-token pricing for leading Chinese and American models. DeepSeek V4 Flash costs a fraction of a cent for the same workload that costs dollars on frontier American models.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1488&quot; height=&quot;806&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Per-million-token pricing for leading Chinese and American models. DeepSeek V4 Flash costs a fraction of a cent for the same workload that costs dollars on frontier American models. Source: OpenRouter, MLQ News, KuCoin, July 2026.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The engine behind this is an architecture approach called mixture of experts, or MoE, which avoids activating an entire model’s full parameter count for every single query. DeepSeek’s V3 architecture used 671 billion total parameters but activated only about 37 billion of them per query, an activation rate of roughly 5.5%, which slashes compute requirements by more than 90% compared to a dense model of similar total size. The current V4 generation pushes this further still, with reported total parameter counts in the hundreds of billions but only a small fraction activated per query.&lt;/p&gt;
&lt;p&gt;Justin Summerville, a data analyst at OpenRouter, put the blended pricing comparison plainly: &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.kucoin.com/news/flash/u-s-companies-shift-to-chinese-ai-models-token-usage-surges-to-46&quot;&gt;Chinese open-source models run roughly 60% to 90% cheaper than Anthropic and OpenAI’s top products across the board&lt;/a&gt;. Run a full realistic workload through Artificial Analysis’s Intelligence Index and the spread becomes concrete: Claude comes out to roughly $4,811 for the workload, ChatGPT $3,357, DeepSeek $1,071, Kimi $948, and GLM-5.2 around $544, a roughly 9-fold gap between the most and least expensive way to finish the same job.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;DeepSeek V4 Flash costs $0.14 per million input tokens. Claude Opus 4.8 costs $5. That is not a rounding error in a pricing table. For a startup burning six figures a month on inference, that gap is the difference between having runway and not.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;export-controls-may-have-accelerated-the-very-thing-they-were-meant-to-prevent&quot;&gt;Export Controls May Have Accelerated the Very Thing They Were Meant to Prevent&lt;/h2&gt;
&lt;p&gt;China did not choose extreme compute efficiency because it is an elegant engineering philosophy. It was forced into it. US export controls have cut Chinese labs off from Nvidia’s most advanced chips, pushing companies like DeepSeek onto domestic alternatives, primarily Huawei’s Ascend line. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.csis.org/analysis/deepseek-huawei-export-controls-and-future-us-china-ai-race&quot;&gt;DeepSeek’s own researchers have assessed that Huawei’s best chip, the Ascend 910C, delivers roughly 60% of an Nvidia H100’s performance for AI inference&lt;/a&gt;, a real and meaningful gap, but one that Chinese labs have partially offset through the kind of aggressive architectural efficiency that MoE represents.&lt;/p&gt;
&lt;p&gt;With brute-force scaling largely off the table due to hardware constraints, Chinese labs had little choice but to engineer around the limitation rather than simply outspend it. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://beam.ai/agentic-insights/nvidia-china-market-share-zero-huawei-12-billion&quot;&gt;Nvidia CEO Jensen Huang has said publicly that conceding the Chinese market entirely “probably does not make a lot of strategic sense” and that the decision “has already largely backfired,”&lt;/a&gt; specifically because Chinese developers are now building an entirely separate software ecosystem around Huawei’s Ascend hardware rather than Nvidia’s CUDA, an ecosystem that, once mature, can be exported globally independent of Nvidia’s chips entirely.&lt;/p&gt;
&lt;p&gt;In June 2026, Washington extended export control authority beyond hardware for the first time, restricting access to deployed commercial AI models directly. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.trendingtopics.eu/open-weight-models-from-china-are-capturing-a-growing-share-of-ai-usage/&quot;&gt;Export controls temporarily pulled Anthropic’s newly launched Claude Fable 5 from the market just days after its June debut, and OpenAI’s GPT-5.6 rollout was initially restricted to a small government-vetted list&lt;/a&gt;. For every developer, government, and startup outside that approved list, the practical effect was straightforward: they reached for the free, self-hostable Chinese alternative instead.&lt;/p&gt;
&lt;h2 id=&quot;smart-money-is-paying-billions-for-a-seat-with-no-steering-wheel&quot;&gt;Smart Money Is Paying Billions for a Seat With No Steering Wheel&lt;/h2&gt;
&lt;p&gt;In mid-June 2026, DeepSeek closed its first-ever external funding round, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.forbes.com/sites/anishasircar/2026/06/17/deepseek-just-raised-74-billion-heres-the-catch/&quot;&gt;raising over $7.4 billion at a post-money valuation above $50 billion, making it China’s most valuable AI startup&lt;/a&gt;. Tencent contributed roughly $1.5 billion and battery manufacturer CATL added a further stake, alongside China’s National AI Industry Investment Fund. Founder Liang Wenfeng personally committed approximately $3 billion of his own money to the round.&lt;/p&gt;
&lt;p&gt;The deal structure is the real tell here. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.forbes.com/sites/anishasircar/2026/06/17/deepseek-just-raised-74-billion-heres-the-catch/&quot;&gt;Most external investors did not buy equity in DeepSeek directly. Instead, their capital went into a limited partnership controlled entirely by Liang, carrying a 5-year lock-up and zero voting rights for the investors involved&lt;/a&gt;. Only China’s state-backed AI fund secured direct equity with voting rights and no lock-up. Investors are paying billions of dollars for financial exposure to DeepSeek with no say whatsoever in how the company is run, which tells you how much conviction the smart money currently has in the underlying business regardless of governance terms.&lt;/p&gt;
&lt;p&gt;By mid-July, barely a month after closing that round, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://en.cryptonomist.ch/2026/07/14/deepseek-new-funding/&quot;&gt;DeepSeek was already reportedly seeking a new round at a $71 billion pre-money valuation, a 42% premium over its June post-money figure&lt;/a&gt;. Even at that scale, DeepSeek remains considerably smaller than its American counterparts. Anthropic closed a $65 billion round in late May 2026 at a $965 billion valuation, and OpenAI raised $122 billion at a valuation around $852 billion. DeepSeek is competing on price and volume in a different league from the capital scale of its US rivals.&lt;/p&gt;
&lt;h2 id=&quot;why-volume-is-not-the-whole-story&quot;&gt;Why Volume Is Not the Whole Story&lt;/h2&gt;
&lt;p&gt;Stop here and you would walk away with an incomplete picture. China dominates volume. It does not yet dominate value, and the gap between those 2 words is where the actual competitive picture gets interesting.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.DPfeur_g_LKmUB.webp&quot; alt=&quot;Anthropic’s Claude processes a small fraction of OpenRouter’s token volume but captures a disproportionate share of platform dollar revenue, reflecting its position in high-value, complex reasoning work.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1488&quot; height=&quot;893&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Anthropic’s Claude processes a small fraction of OpenRouter’s token volume but captures a disproportionate share of platform dollar revenue, reflecting its position in high-value, complex reasoning work. Source: OpenRouter platform analysis, 2026.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;By platform-level revenue analysis, Anthropic reportedly captures a substantially outsized share of OpenRouter’s actual dollar revenue relative to its token share, reflecting the American frontier’s continued strength in high-stakes, complex reasoning and agentic error recovery, where cost is rarely the deciding factor for the customer. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://tech-insider.org/au/chinese-ai-models-openrouter-2026/&quot;&gt;Claude Opus 4.7 still holds the number 3 spot on OpenRouter’s overall leaderboard, the highest-ranked closed-source model on the platform&lt;/a&gt;, even as Anthropic’s overall share by pure token volume has fallen from 29.1% to 13.3% over 12 months, with 6 separate Chinese models now ranking above it by volume alone.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Analysts estimate the American frontier still leads on deep reasoning and agentic error recovery by roughly 3 to 8 months, though some, including Perplexity’s leadership, have put that lead closer to a full year. Roughly 96% of enterprise inference teams reportedly still keep an OpenAI or Anthropic account active alongside whatever cheaper models they have adopted for high-volume, lower-sensitivity work.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The honest read is that Chinese models took the volume and the cheap, high-repetition work. They have not yet taken the complex, high-stakes work that compliance officers and enterprise procurement teams still route to American frontier labs.&lt;/p&gt;
&lt;h2 id=&quot;the-data-question-and-the-distillation-fight-that-got-genuinely-ugly&quot;&gt;The Data Question, and the Distillation Fight That Got Genuinely Ugly&lt;/h2&gt;
&lt;p&gt;There is a catch that gets skipped in the enthusiasm around Chinese model pricing, and it is data governance. When a developer calls a Chinese model’s own API directly rather than through a US-hosted intermediary, requests route through servers physically located in China. Under China’s 2017 National Intelligence Law, Chinese companies operating those servers are required to cooperate with state authorities on request. This is precisely why Lindy went to the trouble of hosting DeepSeek’s model on US soil through Atlas Cloud rather than calling DeepSeek’s API directly.&lt;/p&gt;
&lt;p&gt;The competitive fight between Chinese and American labs has also turned genuinely contentious. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.trendingtopics.eu/deepseek-raises-7-4-billion-only-the-chinese-state-gets-voting-rights/&quot;&gt;In February 2026, Anthropic accused DeepSeek, Moonshot AI, and MiniMax of running what it described as an industrial-scale distillation attack against Claude&lt;/a&gt;, using approximately 24,000 fraudulently created accounts to extract more than 16 million interactions from Claude, in violation of both the platform’s terms of service and regional access restrictions, since Claude is not commercially available inside China. Anthropic attributed the largest share of the extraction activity, more than 13 million interactions, to MiniMax specifically, with DeepSeek responsible for a more targeted set of roughly 150,000 exchanges focused on Claude’s reasoning capabilities.&lt;/p&gt;
&lt;p&gt;Distillation, training a smaller model to mimic a larger one’s outputs, is a widely used and legitimate technique in AI research, including by Anthropic itself for its own smaller models. Anthropic’s argument is that it becomes illegitimate specifically when a competitor uses it to extract a rival’s capabilities in a fraction of the original time and cost, and the company has tied the allegation to national security concerns, arguing illicitly distilled models could lack critical safety safeguards. None of the named companies have publicly responded to the specific allegations. It is also worth noting, in fairness, that Anthropic itself faces its own active lawsuits from music labels over training on copyrighted lyrics without permission, which makes the moral high ground in this particular fight considerably more contested than either side’s public statements suggest.&lt;/p&gt;
&lt;h2 id=&quot;compute-access-still-favours-america&quot;&gt;Compute Access Still Favours America&lt;/h2&gt;
&lt;p&gt;Despite the software-level efficiency gains, a genuine hardware ceiling still constrains China’s ceiling for frontier-scale training specifically. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://newsletter.semianalysis.com/p/huawei-ascend-production-ramp&quot;&gt;Analyst estimates put total Chinese advanced AI chip production, primarily Huawei’s Ascend line, at roughly 1 million units for 2026&lt;/a&gt;, with Huawei’s best chip still trailing an Nvidia H100 by a meaningful margin on real-world inference performance. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.cfr.org/articles/chinas-ai-chip-deficit-why-huawei-cant-catch-nvidia-and-u-s-export-controls-should-remain&quot;&gt;The Council on Foreign Relations has estimated that even under aggressive production assumptions, Huawei would still produce only around 4% of the aggregate AI computing power that Nvidia produces globally&lt;/a&gt;, though it is worth noting this estimate comes from a US-based policy organisation generally supportive of maintaining export controls, and other analysts, including those covering Huawei’s rapid production ramp, describe a narrowing gap rather than a fixed one.&lt;/p&gt;
&lt;h2 id=&quot;what-the-split-means-for-anyone-with-money-in-the-market&quot;&gt;What the Split Means for Anyone With Money in the Market&lt;/h2&gt;
&lt;p&gt;The market has effectively split into 2 distinct stacks. A premium American stack selling frontier reasoning at a meaningful per-token cost, and a Chinese industrial stack selling adequate intelligence at a fraction of the price, scaling on volume rather than margin.&lt;/p&gt;
&lt;p&gt;That split matters directly for anyone holding a broad market index fund, because American hyperscalers have made an enormous capital bet on the premium side of that split remaining durable and monetisable. The largest US technology companies are projected to spend well over $700 billion on AI-related capital expenditure in 2026 alone, a substantial increase year over year, with some analyst projections suggesting the industry crosses $1 trillion annually within the next 2 years. Multiple investment analysts have flagged a widening gap between what is being spent on AI infrastructure and what AI is demonstrably earning back in revenue, a gap some have compared in scale and speed to prior historical infrastructure investment cycles.&lt;/p&gt;
&lt;p&gt;If the American frontier’s reasoning lead is measured in months rather than years, and continues narrowing while the rest of the world standardises on free or near-free Chinese infrastructure for the majority of production workloads, the question of who ultimately generates the return to justify that capital spending becomes genuinely important for anyone whose retirement account or index fund holds a slice of that bet, whether they chose that exposure directly or not.&lt;/p&gt;
&lt;p&gt;The people who currently benefit most clearly from this split are already visible. Chinese labs are building the default, low-cost layer that a large share of global AI infrastructure runs on, a form of durable technical influence that export controls have not managed to prevent and may have inadvertently accelerated. American frontier labs still hold the premium, high-margin reasoning work, for now. The people bearing the most direct risk are the shareholders funding a historically large capital buildout against a competitive moat that may be measured in months rather than years, and the ordinary investors exposed to that bet through funds they never actively chose.&lt;/p&gt;
&lt;p&gt;Whether an American frontier lead measured in months constitutes a durable moat worth a trillion dollars in committed capital, or a luxury margin on the verge of being commoditised the way most technical advantages eventually are, is the question genuinely worth sitting with rather than assuming an answer to.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>china</category><category>ai</category><category>artificial-intelligence</category><category>america</category><category>llm</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>The AI Subscription Era Is Ending. Here Is What Replaces It.</title><link>https://soursen.org/articles/the-ai-subscription-era-is-ending-here-is-what-replaces-it/</link><guid isPermaLink="true">https://soursen.org/articles/the-ai-subscription-era-is-ending-here-is-what-replaces-it/</guid><description>There is a specific moment happening in AI pricing right now that is worth understanding clearly, because it is the first real signal of how the economics of this entire industry are likely to shift over the next few years.</description><pubDate>Tue, 14 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;There is a specific moment happening in AI pricing right now that is worth understanding clearly, because it is the first real signal of how the economics of this entire industry are likely to shift over the next few years.&lt;/p&gt;
&lt;p&gt;Claude Fable 5, Anthropic’s most capable publicly released model, launched on June 9, 2026 as the first model in the company’s new Mythos-class tier. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://claudefa.st/blog/guide/development/fable-5-usage-credits&quot;&gt;It was suspended just 3 days later, on June 12, under a US export-control directive&lt;/a&gt;, after researchers found a way to get the model to identify and demonstrate exploit code. The controls were lifted on June 30 and Fable 5 returned to global availability on July 1 with a new, considerably stricter safety classifier.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.BdqLWT2r_2mGSpn.webp&quot; alt=&quot;Claude Fable 5’s first 6 weeks: launch, suspension, restoration, and the shift to metered billing.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1600&quot; height=&quot;670&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Claude Fable 5’s first 6 weeks: launch, suspension, restoration, and the shift to metered billing. Source: Anthropic announcements and reporting from ClaudeFast, DigitalApplied, and TechTimes, July 2026.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;That restoration came with a deadline attached. Fable 5 was included for up to 50% of weekly usage limits on Pro, Max, Team, and select Enterprise plans, but only until a cutoff date. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.digitalapplied.com/blog/claude-fable-5-usage-credits-july-7-pricing-guide-2026&quot;&gt;Anthropic has pushed that deadline back twice&lt;/a&gt;, first from July 7 to July 12, then to July 19. Metered usage credits, billed at API rates, now begin July 20, 2026. Whatever the exact date turns out to be when this settles, the direction is clear: Fable 5 is moving out of the subscription and into pay-per-token billing.&lt;/p&gt;
&lt;h2 id=&quot;the-numbers-that-make-this-significant&quot;&gt;The Numbers That Make This Significant&lt;/h2&gt;
&lt;p&gt;Fable 5 lists at $10 per million input tokens and $50 per million output tokens on Anthropic’s official pricing page. That is exactly double Claude Opus 4.8’s rate of $5 and $25, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://platform.claude.com/docs/en/about-claude/pricing&quot;&gt;making Fable 5 the single most expensive model Anthropic has ever priced for general use&lt;/a&gt;. For comparison, Claude Sonnet 5, which launched June 30 as the new default for Free and Pro plans, runs at an introductory rate of $2 input and $10 output through August 31, 2026, after which it rises to $3 and $15.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.CTcUuas1_29GFsA.webp&quot; alt=&quot;Fable 5’s per-token pricing against the rest of Anthropic’s current model lineup.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1488&quot; height=&quot;802&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Fable 5’s per-token pricing against the rest of Anthropic’s current model lineup. Source: Anthropic API pricing documentation, retrieved July 2026.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;There are cost-reduction levers built into how Fable 5 is billed, and they matter if you are actually managing a budget around this model. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://upstash.com/blog/keep-claude-fable-5-costs-under-control&quot;&gt;The Batch API halves both rates to $5 and $25&lt;/a&gt;, effectively bringing Fable 5 down to Opus 4.8-equivalent pricing for any workload that can tolerate asynchronous processing. Prompt caching drops the input price by 90%, down to roughly $1 per million tokens for repeated content such as system prompts or stable file context. For long agentic sessions in Claude Code, where the same codebase scaffolding gets re-read on every turn, caching is genuinely the single most effective lever available. Usage credits also carry a $2,000 per day spending cap as a safety mechanism.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Fable 5 is exactly double the price of Opus 4.8 and 5 times Sonnet 5’s introductory rate. It is the most expensive model Anthropic has ever priced for general use.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;is-this-a-permanent-shift-or-a-temporary-squeeze&quot;&gt;Is This a Permanent Shift or a Temporary Squeeze?&lt;/h2&gt;
&lt;p&gt;Here is where it gets genuinely more complicated than a simple story about prices only going up. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.androidheadlines.com/2026/07/claude-fable-5-drops-subscriptions-pay-per-use-credits.html&quot;&gt;Thariq Shihipar, an engineer on the Claude Code team, stated publicly on X&lt;/a&gt; that Anthropic intends to restore Fable 5 to standard subscription plans once server capacity allows. Anthropic itself has said the metered-billing shift is intended to be temporary, not a permanent repricing of the model.&lt;/p&gt;
&lt;p&gt;Whether that holds is genuinely uncertain, and it is worth being honest about that uncertainty rather than assuming the worst-case outcome is the only possible one. Running a model with a full 1 million token context window at the scale Fable 5 was designed for is expensive to serve. The suspension and restoration cycle Fable 5 went through in its first weeks likely compressed the timeline Anthropic had for scaling capacity to meet demand. It is entirely plausible that the current metered period reflects a genuine, temporary capacity constraint rather than a deliberate long-term strategy to migrate users off subscriptions.&lt;/p&gt;
&lt;p&gt;It is also true that independent testing has found the new safety classifier introduced alongside Fable 5’s restoration to be, in the words of evaluation platform BridgeMind, incredibly aggressive, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.androidheadlines.com/2026/07/claude-fable-5-drops-subscriptions-pay-per-use-credits.html&quot;&gt;according to reporting from Android Headlines&lt;/a&gt;. Whether that reflects genuine safety necessity following the export-control episode or an overcorrection is a separate and legitimate question from the pricing question, and the 2 should not be conflated even though they arrived at the same time.&lt;/p&gt;
&lt;h2 id=&quot;the-broader-economic-question-underneath-the-pricing&quot;&gt;The Broader Economic Question Underneath the Pricing&lt;/h2&gt;
&lt;p&gt;Whatever happens with Fable 5 specifically, the pricing structure exposes something true about the entire subsidised-AI-subscription model that has defined the past 3 years of this industry. Every flat-rate AI subscription is a bet that the average user costs less in compute than the subscription price charges. That bet becomes harder to sustain as models get more capable and as usage patterns shift toward longer, more agentic sessions that consume dramatically more tokens per interaction than a simple chat message.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;Sam Altman made a version of this argument explicit at BlackRock’s Infrastructure Summit in Washington, DC on March 11, 2026.&lt;/em&gt; “We see a future where intelligence is a utility, like electricity or water, and people buy it from us on a meter, and use it for whatever they want to use it for,” &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.aol.com/news/sam-altman-says-ai-eventually-152157395.html&quot;&gt;&lt;em&gt;Altman said, as reported by Business Insider&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;He tied this directly to compute economics: if AI companies do not build enough capacity to meet demand, prices rise or access gets rationed. The comment drew significant public criticism, generating over 1,700 comments in 48 hours on Reddit’s r/singularity, with many describing the framing as dystopian given OpenAI’s founding mission was explicitly to advance AI unconstrained by the need to generate financial return.&lt;/p&gt;
&lt;h2 id=&quot;what-the-actual-data-says-about-ai-roi-right-now&quot;&gt;What the Actual Data Says About AI ROI Right Now&lt;/h2&gt;
&lt;p&gt;The pricing question only matters if AI usage is actually generating measurable value for the people paying for it, and here the data is genuinely sobering. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/&quot;&gt;MIT’s NANDA initiative published research in 2025&lt;/a&gt; based on 150 executive interviews, a survey of 350 employees, and analysis of 300 public AI deployments. The finding: approximately 5% of generative AI pilots achieve measurable, rapid impact on revenue or profit and loss. The remaining 95% stall, delivering little to no measurable financial impact.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.JqoBBMDf_Z2nz97D.webp&quot; alt=&quot;MIT NANDA’s 2025 finding on enterprise generative AI pilot outcomes. The study’s methodology, based on a relatively small interview sample, has drawn some criticism, but the directional pattern is echoed in separate McKinsey survey data.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;988&quot; height=&quot;990&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;MIT NANDA’s 2025 finding on enterprise generative AI pilot outcomes. The study’s methodology, based on a relatively small interview sample, has drawn some criticism, but the directional pattern is echoed in separate McKinsey survey data.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;It is worth being precise about what this study actually found rather than repeating the headline uncritically, because the number has been widely cited and occasionally misrepresented. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/&quot;&gt;The report’s lead author, Aditya Challapally, told Fortune&lt;/a&gt; that the core issue is not model quality but what the report calls a learning gap, meaning organisations have not integrated these tools into workflows in a way that produces measurable outcomes. Generic tools like ChatGPT succeed for individual productivity precisely because of their flexibility, but that same flexibility is what causes them to stall in enterprise deployments that need consistent, repeatable outcomes.&lt;/p&gt;
&lt;p&gt;Separately, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://bradenkelley.com/2025/09/mckinsey-is-wrong-that-80-companies-fail-to-generate-ai-roi/&quot;&gt;McKinsey’s own survey data, cited widely alongside the MIT figures&lt;/a&gt;, found that nearly 80% of companies report using generative AI, while just as many report no significant bottom-line impact from that usage. The methodology of the MIT study specifically has drawn genuine criticism, including from Marketing AI Institute, which noted the zero-return finding rested on just 52 interviews the report itself describes as directionally accurate rather than based on official company reporting. The critique is fair and worth noting. The broader directional pattern, that most enterprise AI pilots are not yet producing clear financial returns, shows up across multiple independent sources rather than resting on one contested number.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;MIT’s research found roughly 5% of generative AI pilots achieve measurable P&amp;amp;L impact. The other 95% stall. The core problem, according to the study’s lead author, is organisational integration, not model capability.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;What makes this relevant to the Fable 5 pricing story is straightforward. If prices for the most capable models rise while the majority of enterprise deployments are not yet proving out a clear financial return, the gap between AI spending and AI value widens rather than narrows. That is not an argument that AI has no value. Back-office automation, where MIT’s research found the highest actual returns despite receiving less than half the typical AI budget, is a clear counterexample. It is an argument that the easy, exploratory phase of AI adoption, where trying things at low cost made sense because the downside was small, is becoming a smaller window as the most capable models get more expensive to run.&lt;/p&gt;
&lt;h2 id=&quot;where-local-models-actually-stand-right-now&quot;&gt;Where Local Models Actually Stand Right Now&lt;/h2&gt;
&lt;p&gt;One reasonable response to rising frontier model costs is to ask whether local, self-hosted models can fill the gap. The honest answer in mid-2026 is partially, for specific workloads, and not yet for the kind of long, coherent, agentic work that models like Fable 5 are built for.&lt;/p&gt;
&lt;p&gt;Local models running on consumer-grade hardware, even well-specified setups with 64GB or more of RAM, currently handle structured tasks well: parsing data against precise instructions, generating summaries from defined inputs, and similar bounded jobs where the model does not need to maintain coherence across a long, exploratory session. These tasks work particularly well in asynchronous pipelines, where a local model can run for many hours with several instances checking each other’s outputs, trading wall-clock time for the cost savings of not paying per-token API prices.&lt;/p&gt;
&lt;p&gt;What local models are not yet reliably capable of is sustained, agentic coding work across long sessions with the kind of judgement and context retention that the frontier proprietary models currently offer. The gap is closing, and some industry observers have suggested Fable-level capability could run on consumer hardware within roughly 2 years, though that specific timeline is a prediction rather than a confirmed roadmap from any lab, and should be read with appropriate scepticism given how often specific AI capability timelines have proven optimistic in the past.&lt;/p&gt;
&lt;h2 id=&quot;what-this-actually-means-for-how-you-use-these-tools&quot;&gt;What This Actually Means for How You Use These Tools&lt;/h2&gt;
&lt;p&gt;Setting aside the uncertainty about whether Fable 5’s pricing shift is temporary or permanent, there are practical implications worth acting on regardless of which way it resolves.&lt;/p&gt;
&lt;p&gt;Match the model to the task rather than defaulting to the most capable option for everything. Fable 5’s advantage shows up specifically in long-horizon coherence and its full 1 million token context window. For most day-to-day work, including code review, routine content generation, and shorter interactive sessions, Sonnet 5 at its current introductory pricing delivers close to Opus-level performance at a fraction of the cost. Save the most expensive model for the tasks where its specific capabilities are actually load-bearing to the outcome.&lt;/p&gt;
&lt;p&gt;If you are running agentic workflows, invest time in prompt caching and context management rather than treating it as an optional optimisation. The difference between a cached and uncached input token is a 90% cost reduction, and that difference compounds significantly across long sessions where the same context gets re-read on every turn.&lt;/p&gt;
&lt;p&gt;Treat AI spending the way you would treat any other measurable business input; track what specific tasks are producing quantifiable value and which ones are not. Given that the majority of enterprise AI pilots are not yet demonstrating clear returns according to the available research, the discipline of measuring before scaling matters more now than it did when trying things at near-zero marginal cost carried little risk.&lt;/p&gt;
&lt;p&gt;The subscription-subsidised era of frontier AI access may or may not be ending permanently. What is verifiably true right now is that the most capable models are being priced and metered more precisely than before, the evidence for enterprise AI value remains genuinely mixed, and the tools for managing cost, caching, batching, and careful model selection, matter more than they did 6 months ago. Whatever comes next, understanding the actual mechanics rather than reacting to the headline is the more useful position to be in.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>artificial-intelligence</category><category>claude</category><category>llm</category><category>sam-altman</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>I Tried Making AI Image Generation 90x Cheaper. Here Is What I Learned About GPU Quantization.</title><link>https://soursen.org/articles/i-tried-making-ai-image-generation-90x-cheaper-here-is-what-i-learned/</link><guid isPermaLink="true">https://soursen.org/articles/i-tried-making-ai-image-generation-90x-cheaper-here-is-what-i-learned/</guid><description>The bill had a comma in it.</description><pubDate>Mon, 13 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;The bill had a comma in it.&lt;/p&gt;
&lt;p&gt;I was doing freelance work for an e-commerce client who needed product background replacement at scale. Their catalogue had thousands of SKUs and they were processing every image through a paid API at $0.039 per call. 4 cents sounds like nothing until you are generating multiple edited variants for every product in a large batch job. Then it is a monthly number sitting in a dashboard with a comma in it, waiting for someone to ask why.&lt;/p&gt;
&lt;p&gt;The idea was straightforward. Take an open-weight image editing model, run it on a rented GPU, pay for electricity instead of API calls. After testing several options on Replicate, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct&quot;&gt;Qwen-VL-Max from Alibaba’s Qwen series&lt;/a&gt; was the one that kept a shirt looking like the same shirt after editing, same patterns, same cut, just a different background or style treatment. Open weights, good quality, no meter running per call. Done, I thought.&lt;/p&gt;
&lt;p&gt;It was not done. The model took an afternoon to pick. The week went to figuring out what to run it on, and specifically to one fact that nobody in the tutorials warned me about: the architecture name printed on a GPU determines which fast numeric formats that GPU can run natively, and that determines your per-image cost more than any other single variable.&lt;/p&gt;
&lt;h2 id=&quot;why-per-image-cost-is-the-metric-that-actually-matters&quot;&gt;Why Per-Image Cost Is the Metric That Actually Matters&lt;/h2&gt;
&lt;p&gt;The natural instinct when renting cloud GPUs is to sort by price/hour and pick the cheapest. That instinct is wrong for inference workloads, and understanding why requires thinking about what you are actually paying for.&lt;/p&gt;
&lt;p&gt;A pricier GPU that runs a model 3 times faster costs 3 times less per image despite having a higher hourly rate. The metric that matters is hourly rate divided by images per hour. Everything else is noise. The thing that determines how many images per hour you can produce is the numeric format the model runs in, and whether the GPU’s tensor cores can execute that format natively.&lt;/p&gt;
&lt;p&gt;The fast formats come from quantization is that shrinking the model’s weight values from 32-bit or 16-bit floating point down to 8-bit or 4-bit representations. A 20-billion-parameter model stored in full 16-bit precision requires roughly 40 GB of video memory just to load. At 8-bit precision, that same model compresses to around 20 GB. At 4-bit, closer to 10 GB. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.vllm.ai/en/stable/features/quantization/fp8/&quot;&gt;vLLM’s documentation on FP8 quantization&lt;/a&gt; confirms the core trade-off that FP8 allows approximately 2x reduction in model memory requirements and up to 1.6x improvement in throughput with minimal impact on accuracy. The memory saving is real. The throughput saving is only real if the GPU’s tensor cores actually support the format natively.&lt;/p&gt;
&lt;h2 id=&quot;how-to-read-a-gpu-name-as-a-technical-specification&quot;&gt;How to Read a GPU Name as a Technical Specification&lt;/h2&gt;
&lt;p&gt;NVIDIA GPU names are not marketing labels. They encode the hardware architecture, and the architecture tells you everything about which numeric formats get native tensor core acceleration. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://awesomeagents.ai/hardware/nvidia-rtx-4090/&quot;&gt;The Awesome Agents RTX 4090 analysis from April 2026&lt;/a&gt; describes it clearly that consumer cards encode the architecture in the first 2 digits of the model number. The 30-series is Ampere. The 40-series is Ada Lovelace. The 50-series is Blackwell. Datacenter cards use initial letters instead: the A100 is Ampere, the H100 is Hopper, the B200 is Blackwell.&lt;/p&gt;
&lt;p&gt;Underneath the marketing name is a number your actual code cares about is the CUDA compute capability. You can query it directly with one line of Python:&lt;/p&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;import torch&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;print(torch.cuda.get_device_name(0), torch.cuda.get_device_capability(0))&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;# RTX 4090 returns (8, 9), meaning sm_89, meaning Ada Lovelace&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The compute capability maps directly to which numeric formats get hardware acceleration. This is not something that drivers can change or that software updates can retrofit. It is fixed by the silicon.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.vllm.ai/en/stable/features/quantization/fp8/&quot;&gt;According to vLLM’s official FP8 documentation&lt;/a&gt;, FP8 computation is supported on NVIDIA GPUs with compute capability 8.9 or higher, which covers Ada Lovelace and Hopper architectures. Ampere GPUs including the A100 and the RTX 3090 have no native FP8 path at all. FP4, the 4-bit floating point format, is Blackwell-only and is not available even on Hopper. The generation gap is not negotiable.&lt;/p&gt;
&lt;h2 id=&quot;the-3-things-people-call-8-bit-quantization-they-are-not-the-same&quot;&gt;The 3 Things People Call 8-bit Quantization (They Are Not the Same)&lt;/h2&gt;
&lt;p&gt;When I started researching quantization, I assumed 8-bit meant one thing. It means at least 3 different things, and 2 of them have completely opposite performance characteristics despite using the same number of bits.&lt;/p&gt;
&lt;h2 id=&quot;int8-weight-only-quantization&quot;&gt;INT8 weight-only quantization&lt;/h2&gt;
&lt;p&gt;This is the most widely available 8-bit format and the most commonly misunderstood. Picture a library where every book has been printed in a pocket edition, half the normal size. The shelves hold twice as many books. But every time you want to read a page, you photocopy it back to full size before processing it. The math runs at full 16-bit precision even though the storage was 8-bit. You save memory. You do not save compute time. In some benchmarks, INT8 weight-only is actually slower than running the model in full 16-bit, because the dequantisation step adds overhead without providing any tensor core acceleration.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.DMbQ-Pw4_Z1Ic5rt.webp&quot; alt=&quot;INT8 weight-only saves memory but the math still runs in BF16, no tensor core acceleration. FP8 W8A8 both saves memory AND runs natively on Ada/Hopper tensor cores for real throughput gains.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1488&quot; height=&quot;808&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;INT8 weight-only saves memory but the math still runs in BF16, no tensor core acceleration. FP8 W8A8 both saves memory AND runs natively on Ada/Hopper tensor cores for real throughput gains. Source: vLLM docs, torchao benchmarks, NVIDIA architecture&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;fp8-full-quantization&quot;&gt;FP8 full quantization&lt;/h2&gt;
&lt;p&gt;FP8 is a completely different situation. It is a real 8-bit floating point format with 2 variants: E4M3 for higher accuracy with values up to 448, and E5M2 for wider range. More importantly, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.rightnowai.co/guides/gpu-comparison/rtx-4090&quot;&gt;RightNow AI’s RTX 4090 CUDA guide confirms&lt;/a&gt; that the 4th-generation tensor cores in Ada Lovelace hardware execute FP8 matrix multiplications natively. The tensor cores multiply FP8 values directly rather than converting them first. You get the memory saving from 8-bit storage, and you get real compute acceleration from tensor core throughput. Same 8 bits as INT8. Opposite performance outcome because one format lights up the hardware and the other does not.&lt;/p&gt;
&lt;h2 id=&quot;the-rtx-4090-fp8-nuance&quot;&gt;The RTX 4090 FP8 nuance&lt;/h2&gt;
&lt;p&gt;There is a widespread misconception that the RTX 4090 does not support FP8 because some documentation says FP8 requires compute capability 9.0, which is Hopper only. That 9.0 requirement applies specifically to NVIDIA’s Transformer Engine, the high-level API that adds automatic scaling and calibration on top of FP8 computation. The underlying FP8 tensor cores are present in Ada Lovelace at compute capability 8.9. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.vllm.ai/en/stable/features/quantization/fp8/&quot;&gt;The vLLM documentation confirms&lt;/a&gt; that FP8 W8A8 (weights and activations both in FP8) is officially supported on Ada Lovelace. The RTX 4090 has 330 TFLOPS of FP8 compute, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://awesomeagents.ai/hardware/nvidia-rtx-4090/&quot;&gt;confirmed by Awesome Agents’ detailed specifications&lt;/a&gt;. What it lacks compared to an H100 is the Transformer Engine’s automatic scaling hardware. FP8 on a 4090 is real and fast. It is not H100 FP8. That is a meaningful distinction for fine-tuned production inference pipelines, but for practical inference workloads it is not the bottleneck.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;INT8 and FP8 are both 8-bit. But INT8 keeps the compute full-size and FP8 lights up the tensor cores. Same number of bits. Opposite performance outcome.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;the-night-the-model-loaded-and-produced-static&quot;&gt;The Night the Model Loaded and Produced Static&lt;/h2&gt;
&lt;p&gt;Getting FP8 actually working took 3 attempts, and the most educational one was the failure.&lt;/p&gt;
&lt;p&gt;The first attempt used INT8 weight-only quantization with a distilled LoRA applied on top. It worked in the sense that images came out correctly. It did not work in the sense that inference took 7 seconds/step, because torchao’s compiled kernels did not match the installed PyTorch version and quietly fell back to an unoptimised code path. The fix was pinning torchao to match the PyTorch version exactly. Lesson learned: the error does not announce itself. It just runs slowly.&lt;/p&gt;
&lt;p&gt;The second attempt was the educational failure. I pointed the diffusers library at a prebuilt FP8 checkpoint and ran inference. The model loaded without errors. I submitted a generation request. The output folder filled with pure noise, white static across the entire image, every time. I spent longer than I want to admit convinced the problem was my prompt before I admitted it was the loader.&lt;/p&gt;
&lt;p&gt;The checkpoint used a scaling format associated with ComfyUI’s FP8 implementation. The diffusers loader read the weight tensors correctly but did not read the corresponding scale factors, which are required to interpret raw FP8 values as meaningful numbers. Without the scale factors, the model ran the weights as raw bit patterns that meant nothing. The result was correct-looking inference at the software level and complete garbage at the output level, which is among the least helpful failure modes a system can have. The credits ran the whole time.&lt;/p&gt;
&lt;p&gt;What actually worked was LightX2V, an inference engine built specifically for large diffusion models with native FP8 support and CPU offloading. The offloading is the key for fitting a 58 GB model on a 24 GB card: the text encoder stays resident in GPU memory, the transformer layers stream through in phases, and LightX2V manages the scheduling automatically. Generation time landed at approximately 4.5 seconds per image, roughly 6 times faster than the INT8 path. That is the number that ended up in the per-image cost calculation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/modeltc/lightx2v&quot;&gt;LightX2V is available on GitHub&lt;/a&gt; and supports several large diffusion architectures beyond the one I used here.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;At 4.5 seconds/image on a 4090 rented at the market midpoint of $0.50/hr, GPU time works out to roughly $0.00063 per image against $0.039 for the API. That is over 60 times cheaper in GPU time alone.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;the-gpu-comparison-table&quot;&gt;The GPU Comparison Table&lt;/h2&gt;
&lt;p&gt;Here is the full picture of the GPUs worth considering for this workload, with current prices from RunPod community pods as of late June 2026. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.runpod.io/gpu-instance/pricing&quot;&gt;RunPod’s live pricing&lt;/a&gt; fluctuates and the figures below reflect the range I observed, not a guaranteed rate.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.Bi3wyVVS_5LxVm.webp&quot; alt=&quot;Sources: RunPod, Vast.ai, Lambda Labs, JarvisLabs, GetDeploying all July 2026. Prices fluctuate; Vast.ai spot rates change minute to minute.&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;915&quot; height=&quot;564&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Sources: RunPod, Vast.ai, Lambda Labs, JarvisLabs, GetDeploying all July 2026. Prices fluctuate; Vast.ai spot rates change minute to minute.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The FP8 column is where the decision makes itself. The RTX 3090 and A100 are Ampere with no native FP8 path. They strand you on the slow INT8 or full-precision route regardless of configuration. The H100 has genuine Hopper-grade FP8 with the Transformer Engine, but at $1.50 to $3.29 per hour it costs 3 to 5 times as much as a 4090 for a workload that runs 1 model at a time. The 4090 sits at the intersection of cheapest GPU with FP8 tensor cores and sufficient VRAM for models up to roughly 30 to 34 billion parameters with offloading.&lt;/p&gt;
&lt;h2 id=&quot;the-per-image-math&quot;&gt;The Per-Image Math&lt;/h2&gt;
&lt;p&gt;At approximately 4.5 seconds per image on a 4090, here is how the GPU time cost works out across the price range:&lt;/p&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;High end  (RunPod $0.65/hr):  (0.65 / 3600) × 4.5 = ~$0.00081 per image&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Midpoint  (avg ~$0.50/hr):    (0.50 / 3600) × 4.5 = ~$0.00063 per image&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Low end   (Vast.ai $0.35/hr): (0.35 / 3600) × 4.5 = ~$0.00044 per image&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Against the original API cost of $0.039 per image, that is roughly 48 to 90 times cheaper in GPU time alone. There is an honest asterisk: these figures are GPU time only for a card kept fully busy. Idle pod time, storage costs, and the week of engineering to make it work are not included. For context, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://aws.amazon.com/ec2/instance-types/g5/&quot;&gt;AWS’s g5.xlarge instance&lt;/a&gt;, which uses a NVIDIA A10G (an Ampere card with 24 GB and no FP8), runs at roughly $1.00 per hour and delivers slower inference without the FP8 path. It was not a competitive option.&lt;/p&gt;
&lt;h2 id=&quot;the-part-that-takes-most-of-the-time&quot;&gt;The Part That Takes Most of the Time&lt;/h2&gt;
&lt;p&gt;Almost none of the week went to the model. The model decision took an afternoon. The rest of the time went to CUDA version conflicts, mismatched cuDNN libraries, and inference engines that ran correctly on the 4090 but broke on other architectures.&lt;/p&gt;
&lt;p&gt;Ada Lovelace requires CUDA 11.8 or newer. A Blackwell card like the RTX 5090 requires CUDA 12.8 and breaks Python wheels that compiled and ran correctly on the 4090. The error message &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;http://libcudnn.so/&quot;&gt;libcudnn.so&lt;/a&gt;.9 not found is common enough in these setups that it is effectively a rite of passage. The fix is unglamorous: a pre-flight check that validates the environment and fails before you have paid to download 40 GB of weights, combined with a pinned base Docker image that I now refuse to modify without good reason.&lt;/p&gt;
&lt;p&gt;The method that actually holds across future GPU choices is simpler than any of the debugging. Read the GPU name, identify the architecture, look up which numeric formats that architecture supports natively, then do the per-image math before committing to any hardware. That process took an afternoon. Making FP8 actually run correctly took the rest of the week. No GPU comparison table warns you about that second half.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>artificial-intelligence</category><category>llm</category><category>generative-ai-tools</category><category>self-awareness</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Why You Should Stop Using Ollama in 2026 and What to Use Instead</title><link>https://soursen.org/articles/why-you-should-stop-using-ollama-in-2026-and-what-to-use-instead/</link><guid isPermaLink="true">https://soursen.org/articles/why-you-should-stop-using-ollama-in-2026-and-what-to-use-instead/</guid><description>Let me start with a confession. I used Ollama for a long time, and I recommended it to almost everyone who asked me about running AI models locally. It deserved that recommendation when it launched.</description><pubDate>Thu, 09 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Let me start with a confession. I used Ollama for a long time, and I recommended it to almost everyone who asked me about running AI models locally. It deserved that recommendation when it launched. You typed one command, a model loaded, and you had a running LLM on your own hardware with zero configuration. It was genuinely the easiest on-ramp to local inference that existed.&lt;/p&gt;
&lt;p&gt;That recommendation no longer holds, and I want to explain exactly why, with real numbers rather than vague complaints, because the situation has changed enough in 2026 that continuing to point people toward Ollama is actively doing them a disservice.&lt;/p&gt;
&lt;p&gt;The issue is not that Ollama became bad. The issue is that the alternatives got significantly better while Ollama got distracted building a cloud business, a proprietary backend, and a platform identity that quietly abandoned the local-first promise it was built on. The gap between what Ollama offers and what you could have instead is now wide enough to matter for every developer running models seriously.&lt;/p&gt;
&lt;h2 id=&quot;the-performance-gap-you-can-actually-feel&quot;&gt;The Performance Gap You Can Actually Feel&lt;/h2&gt;
&lt;p&gt;Start with the most immediate problem because it is the one you will notice on day one. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://gigagpu.com/ollama-vs-llama-cpp-ease-vs-performance/&quot;&gt;GigaGPU’s April 2026 benchmark&lt;/a&gt; tested Llama 3 8B Q4_K_M on an RTX 5090 and documented 95 tokens/second with the llama.cpp server against 82 tokens/second with Ollama running the same model. The 14% overhead on NVIDIA hardware comes entirely from Ollama’s Go-based API layer, the automatic context management system, and the model loading abstractions that sit between you and the actual inference engine.&lt;/p&gt;
&lt;p&gt;That 14% figure is actually the conservative end of what the benchmarks show. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://singhajit.com/llm-inference-speed-comparison/&quot;&gt;Ajit Singh’s comprehensive benchmark published in January 2026 and updated in April&lt;/a&gt; found that llama.cpp runs consistently 10 to 25% faster than Ollama on NVIDIA GPUs across multiple models and hardware configurations. On an RTX 3060 running a 14B model, llama.cpp produces approximately 23 tokens/second; Ollama produces measurably fewer from the same hardware. At 100 concurrent users on a dedicated GPU server, this gap means needing proportionally more hardware with Ollama to serve the same load.&lt;/p&gt;
&lt;p&gt;The Apple Silicon story is even more stark, because on a Mac the performance difference between Ollama and the fastest available alternative is not 14%, it is multiple times larger.&lt;/p&gt;
&lt;p&gt;The benchmark that makes this clearest is &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://antekapetanovic.com/blog/qwen3.5-apple-silicon-benchmark/&quot;&gt;Ante Kapetanovic’s direct comparison on an M4 Max running Qwen3.5 35B&lt;/a&gt;. MLX via the HTTP server produces 90 to 108 tokens/second. llama.cpp produces roughly 71 tokens per second. Ollama produces approximately 43 tokens/second. On the same machine, with the same model family, the fastest tool is more than 2.5 times faster than Ollama. The gap exists because MLX is built specifically for Apple Silicon, using Apple’s unified memory architecture and Metal GPU access directly rather than routing through a general-purpose abstraction layer.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://yage.ai/share/mlx-apple-silicon-en-20260331.html&quot;&gt;A separate benchmark on a Mac mini M4 Pro with 64GB unified memory running Qwen3-Coder-30B&lt;/a&gt; showed MLX achieving approximately 130 tokens per second compared to Ollama’s 43 tokens per second with its llama.cpp backend. That is a 3x difference on mainstream Apple Silicon hardware that a significant portion of developers are actually using. On an M4 Max with 128GB, MLX reached 130 tokens per second versus Ollama’s 43.5.&lt;/p&gt;
&lt;p&gt;Why does Ollama fall this far behind on Apple Silicon specifically? Because until March 2026, Ollama was using the llama.cpp backend on Apple Silicon rather than MLX, which meant it was routing compute through a general C++ inference engine using Metal shaders rather than a framework designed from the ground up for Apple’s chip architecture. Ollama announced a switch to MLX on Apple Silicon in March 2026, but that switch is still in preview, not the default, and it does not address the prior year of users running suboptimal inference without realising it.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;The creator of llama.cpp, Georgi Gerganov, commented publicly on Ollama’s performance issues:&lt;/em&gt; “LM Studio is using the upstream ggml implementation which is significantly better and well optimized. Looking at Ollama’s modifications in ggml, they have too much branching in their MXFP4 kernels and the attention sinks implementation is really inefficient. Along with other inefficiencies, I expect the performance is going to be quite bad in Ollama.” &lt;em&gt;This is not a competitor complaining. This is the author of the underlying technology describing specific implementation choices in Ollama’s codebase that produce measurable slowdowns.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;the-proprietary-fork-that-trapped-your-models&quot;&gt;The Proprietary Fork That Trapped Your Models&lt;/h2&gt;
&lt;p&gt;The performance problem is annoying. The next problem is a more serious breach of trust.&lt;/p&gt;
&lt;p&gt;During 2024 and into 2025, Ollama forked ggml, the low-level tensor computation library that llama.cpp is built on, and built their own custom inference engine on top of it. They stopped using llama.cpp as their backend entirely. This was not disclosed prominently. It happened gradually, and most users had no idea it was occurring.&lt;/p&gt;
&lt;p&gt;The practical consequence for anyone running Ollama during this period was that model files downloaded through Ollama were stored in a hashed filename format specific to Ollama’s own registry. You could not take those files and use them directly with llama.cpp, LM Studio, or any other inference tool. The models you had downloaded and the disk space you had spent acquiring them were effectively locked inside Ollama’s ecosystem. If you wanted to switch tools, you had to re-download everything from scratch.&lt;/p&gt;
&lt;p&gt;This is vendor lock-in, and it is particularly uncomfortable when the vendor’s entire brand identity is built on open-source credentials. The weights themselves were open. The format Ollama stored them in was not compatible with anything else. The distinction is meaningful and the company chose not to make it visible.&lt;/p&gt;
&lt;p&gt;In May 2026, Ollama released v0.30.0-rc15, which switches back to using llama.cpp directly. GGUF models are first-class again. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/ollama/ollama/releases/tag/v0.30.0-rc15&quot;&gt;The community response&lt;/a&gt; acknowledged this as a correction but not as a vindication. Three things drove the reversal: the custom backend had fallen too far behind on new model architectures including Multi-Token Prediction and hybrid attention, OpenAI’s GPT-OSS release required day-one support that Ollama could not deliver by reimplementing llama.cpp from scratch, and the performance benchmarks had become too embarrassing to keep ignoring in public. Switching back does not erase the period when your models were locked in. It just means they are unlocked again.&lt;/p&gt;
&lt;h2 id=&quot;the-attribution-problem-and-the-desktop-app-launch&quot;&gt;The Attribution Problem and the Desktop App Launch&lt;/h2&gt;
&lt;p&gt;Open-source software runs on attribution. When you build on someone else’s work, you credit it. This is not a legal formality in most cases. It is the social contract that makes contributors willing to keep contributing and that gives users confidence they understand what they are actually running.&lt;/p&gt;
&lt;p&gt;Ollama built its entire original product on llama.cpp. For years, their README did not mention llama.cpp clearly. The attribution was buried, incomplete, or absent depending on where you looked. Their binary distributions were vague about dependencies. The community that built llama.cpp had to watch a product built primarily on their work position itself as an independent platform without proper acknowledgment.&lt;/p&gt;
&lt;p&gt;The desktop application launch made this worse. Ollama released a desktop GUI without including it in the main GitHub repository. The licence was unclear. The source code was not publicly available at launch. For a project whose reputation is built on open-source principles, launching a closed-source application without disclosing that clearly is a meaningful integrity gap. The source code eventually appeared in the repository, but the initial rollout communicated something about the company’s priorities that the correction did not undo.&lt;/p&gt;
&lt;p&gt;The model naming issue compounds this pattern. When DeepSeek released its R1 model family, Ollama listed the smaller distilled variants, specifically models like DeepSeek-R1-Distill-Qwen-32B with 32 billion parameters, simply as DeepSeek-R1 in their library. The full DeepSeek-R1 model has 671 billion parameters and behaves fundamentally differently from the distilled versions. Consumer hardware cannot run the full model. The distilled variants are useful and capable, but they are not the same thing as the flagship model.&lt;/p&gt;
&lt;p&gt;The confusion this created was extensive. Developers and researchers were sharing results claiming they had run the DeepSeek-R1 on their consumer laptops, when they had actually run a model that is one-twentieth the size. Ollama knew the distinction. They named the models the way they did anyway, presumably because the flagship model name drives more downloads than an accurate description of the distilled variant would.&lt;/p&gt;
&lt;h2 id=&quot;the-cloud-service-that-made-everything-worse&quot;&gt;The Cloud Service That Made Everything Worse&lt;/h2&gt;
&lt;p&gt;Ollama began as a local-first tool. Run models on your own machine, keep your data private, no server in the loop. That was the value proposition, and it was a genuine one. The rise of local inference tools matters because it means developers can work with powerful models without sending their prompts to a third-party API.&lt;/p&gt;
&lt;p&gt;Then Ollama took venture capital funding and built a cloud service.&lt;/p&gt;
&lt;p&gt;This is not inherently wrong. Companies need to grow. But the cloud service they built has been documented as deeply unreliable, and the shift in incentives it represents undermines the local-first promise in ways that matter for users who chose Ollama specifically because of that promise.&lt;/p&gt;
&lt;p&gt;The documented failure rates on Ollama Cloud are not minor reliability issues. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/ollama/ollama/issues/14673&quot;&gt;GitHub issue #14673&lt;/a&gt; and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/ollama/ollama/issues/15453&quot;&gt;issue #15453&lt;/a&gt; document a 29.7% failure rate on Qwen3.5 models and a reported 95% failure rate across all models on Ollama Cloud Pro. Users have reported inference timeouts exceeding 60 seconds that break agent workflows which depend on reliable response times. Tool calling returns 500 errors when tools are enabled on cloud models. Users paying $100 per month have been throttled after five days of use, not four days as initially reported to support, but confirmed in community threads to have started as early as day five.&lt;/p&gt;
&lt;p&gt;Support response times during documented outages have been measured in weeks rather than hours, with no incident communication from the company to affected users. The Reddit threads and GitHub issues from paying users describe a service that is not production-ready by any standard definition of that phrase.&lt;/p&gt;
&lt;p&gt;The privacy question that sits underneath the cloud service is worth stating clearly. If you chose Ollama because you wanted local inference with no data leaving your machine, and Ollama is now actively encouraging you to use a cloud service, the company’s incentives around your data have shifted. They have not published detailed privacy documentation explaining exactly what happens to prompts sent to Ollama Cloud. They are a VC-backed company with a cloud business. The original promise and the current business model are in tension.&lt;/p&gt;
&lt;h2 id=&quot;llamacpp-closed-the-convenience-gap&quot;&gt;llama.cpp Closed the Convenience Gap&lt;/h2&gt;
&lt;p&gt;The strongest historical argument for using Ollama despite its performance overhead was convenience. Setting up llama.cpp directly required downloading the source, compiling from scratch, separately obtaining model weights, converting formats, and manually configuring a server. That process took 15 to 30 minutes even for experienced developers, and it was genuinely intimidating for people new to local AI.&lt;/p&gt;
&lt;p&gt;That argument no longer holds in 2026. llama.cpp now ships with a router mode for model switching and a built-in web chat interface. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://gigagpu.com/ollama-vs-llama-cpp-ease-vs-performance/&quot;&gt;GigaGPU documents&lt;/a&gt; that the setup process still takes longer than Ollama’s single-command approach, but the gap has narrowed to the point where the performance benefit justifies it for any serious use case. For developers who still want Ollama’s ease of use with llama.cpp’s performance, LM Studio provides a polished graphical interface on a regularly updated llama.cpp backend with proper GGUF support and model switching built in.&lt;/p&gt;
&lt;p&gt;The feature gap has also widened in llama.cpp’s favour during the period Ollama spent on its custom backend. Multi-Token Prediction support was merged into llama.cpp and remained broken or absent in Ollama’s custom engine. Structured output worked correctly in llama.cpp while being broken in Ollama’s fork. New model architectures including hybrid attention patterns received llama.cpp support within days of release, while Ollama’s custom backend lagged by months. SEED OSS 36B, a significant open model release, was unavailable through Ollama for months because the team declined to update their llama.cpp fork to support it.&lt;/p&gt;
&lt;h2 id=&quot;what-to-use-instead&quot;&gt;What to Use Instead&lt;/h2&gt;
&lt;p&gt;The decision depends on your specific situation, so here is how to think about it rather than a single universal recommendation.&lt;/p&gt;
&lt;p&gt;If you are running on NVIDIA hardware and want maximum single-user inference speed with full control over quantization, context length, and memory allocation, use &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/ggerganov/llama.cpp&quot;&gt;llama.cpp directly&lt;/a&gt;. It is 10 to 25% faster than Ollama on NVIDIA across the benchmarks currently available, gives you access to every inference parameter, and now has a built-in web interface and model switching that removes the main convenience argument for Ollama.&lt;/p&gt;
&lt;p&gt;If you want the graphical experience Ollama offered but with a properly maintained llama.cpp backend, use &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://lmstudio.ai/&quot;&gt;LM Studio&lt;/a&gt;. It provides a polished interface, proper GGUF support, model discovery and switching, and a backend that tracks llama.cpp releases rather than maintaining a divergent fork. The model files are standard GGUF format, so you can use them with any other inference tool without re-downloading.&lt;/p&gt;
&lt;p&gt;If you are on Apple Silicon and running models with long output requirements such as coding agents or long-form generation, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/ml-explore/mlx-examples/tree/main/llms&quot;&gt;MLX via mlx-lm&lt;/a&gt; is the correct choice. On M4-generation hardware, MLX is 2.5 to 3 times faster than Ollama for steady-state token generation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/0ssamaak0/oMLX&quot;&gt;oMLX&lt;/a&gt; is specifically optimised for coding agent scenarios and uses SSD KV cache persistence to reduce time-to-first-token for repeated prefixes from 30 to 90 seconds down to 1 to 3 seconds.&lt;/p&gt;
&lt;p&gt;If you are building a multi-user or production deployment, neither Ollama nor llama.cpp is the right tool. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/vllm-project/vllm&quot;&gt;vLLM&lt;/a&gt; provides proper continuous batching, production-grade concurrency, and industry-standard reliability. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.com/sgl-project/sglang&quot;&gt;SGLang&lt;/a&gt; offers structured generation and agent-oriented features with comparable production maturity. Both are the tools actually used at inference scale. Ollama’s own cloud service failure rates demonstrate what happens when a product designed for single-user local inference is pushed into production workloads without the engineering required to support them.&lt;/p&gt;
&lt;h2 id=&quot;the-honest-summary&quot;&gt;The Honest Summary&lt;/h2&gt;
&lt;p&gt;Ollama earned its reputation. In 2023 and much of 2024, it was the right recommendation for almost everyone who wanted to run models locally without dealing with the complexity of the underlying tools. That was genuinely valuable, and the project deserves credit for making local AI accessible to developers who would not have encountered it otherwise.&lt;/p&gt;
&lt;p&gt;The situation in 2026 is different. Ollama is running 10 to 25% slower than llama.cpp on NVIDIA and 2 to 3 times slower than MLX on Apple Silicon. It spent a year with a proprietary backend that locked model files into a non-portable format. It launched a cloud service with documented failure rates above 25%, support response times measured in weeks, and a privacy posture that contradicts the local-first promise it was built on. It misnamed models in ways that created widespread confusion about what users were actually running. It fell months behind on feature support while its upstream dependency kept shipping.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The alternatives are not harder to use than they used to be. They are easier. The convenience gap that justified the performance penalty has closed. The trust issues that accumulated during 2024 and 2025 have not.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;You do not have to migrate everything at once. Download LM Studio today and point it at the same GGUF files you are already using. If you are on a Mac, install mlx-lm and run a comparison on the same model. The performance difference will be immediately obvious, and the setup time is shorter than you probably expect.&lt;/p&gt;
&lt;p&gt;The open-source local AI ecosystem is healthier and more capable than it has ever been. The tools built on genuine community transparency are faster, more flexible, and more honest about what they are. That is where your time and your data belong.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ollama</category><category>llm</category><category>ai</category><category>artificial-intelligence</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>The Number Nobody Tells You to Check When Buying a GPU for Local AI</title><link>https://soursen.org/articles/the-number-nobody-tells-you-to-check-when-buying-a-gpu-for-local-ai/</link><guid isPermaLink="true">https://soursen.org/articles/the-number-nobody-tells-you-to-check-when-buying-a-gpu-for-local-ai/</guid><description>If you have spent any time researching GPUs for running AI models locally, you have probably heard the same advice repeated everywhere: get as much VRAM as possible. That advice is not wrong.</description><pubDate>Tue, 07 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;If you have spent any time researching GPUs for running AI models locally, you have probably heard the same advice repeated everywhere: get as much VRAM as possible. That advice is not wrong. VRAM capacity determines whether a model loads at all, and if your GPU does not have enough, the model either refuses to run or gets partially offloaded to system RAM at a fraction of the speed. VRAM is the hard gate.&lt;/p&gt;
&lt;p&gt;But here is what most guides skip over entirely. Once your GPU has enough VRAM to hold the model, the number that determines how fast the model actually generates text is not VRAM size. It is memory bandwidth, and the difference between GPUs on this metric is enormous, sometimes three to four times faster on paper-comparable hardware.&lt;/p&gt;
&lt;p&gt;Understanding this distinction will save you from two categories of mistake that are genuinely common right now. The first is buying a GPU with plenty of VRAM but mediocre bandwidth and wondering why inference feels sluggish. The second is spending two to three times more money on a flashy system spec-sheet because it promises large unified memory, only to discover that the bandwidth connecting that memory to the processor makes it feel slower than a used GPU you could have bought for a third of the price.&lt;/p&gt;
&lt;h2 id=&quot;why-memory-bandwidth-controls-inference-speed&quot;&gt;Why Memory Bandwidth Controls Inference Speed&lt;/h2&gt;
&lt;p&gt;Before getting into the numbers, it helps to understand exactly what happens inside a GPU every time an LLM generates a single token.&lt;/p&gt;
&lt;p&gt;The GPU does not load the model once and then keep it in fast registers forever. For every token it generates, it reads the relevant portions of the model weights from VRAM into the compute cores, runs the attention and feed-forward calculations for that token, and then moves on to the next one. This cycle repeats for every single token in the output. The model weights are large, the cycle is continuous, and the speed of the memory connection between VRAM and compute cores is what sets the pace.&lt;/p&gt;
&lt;p&gt;This is why LLM inference is described as memory-bound rather than compute-bound. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://houtini.com/articles/best-gpus-for-running-local-llms/&quot;&gt;Houtini’s July 2026 GPU guide&lt;/a&gt; describes it precisely that LLM inference is memory-bandwidth-bound, meaning the GPU’s compute units are frequently idle, waiting for data to arrive from VRAM rather than being limited by raw compute throughput. This has been confirmed in peer-reviewed research across multiple architectures, with arXiv papers on LLM inference consistently documenting that decoding becomes memory-bound during autoregressive generation, where weight movement from memory dominates the latency profile. Adding more compute cores to a memory-bound workload does almost nothing. Adding more bandwidth changes everything.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.BvXso3e1_148gL1.webp&quot; alt=&quot;Illustration for the section “Why Memory Bandwidth Controls Inference Speed”&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1168&quot; height=&quot;1500&quot;&gt;&lt;/p&gt;
&lt;p&gt;Here is a concrete way to think about the difference. VRAM capacity is like the size of your desk, determining how much material you can have in front of you at once. Memory bandwidth is how fast you can reach into the filing cabinet and pull out whatever you need next. You could have a desk the size of a warehouse floor, but if fetching each document takes fifteen minutes, you will not get much work done regardless of how much space you have.&lt;/p&gt;
&lt;h2 id=&quot;the-math-that-makes-this-real&quot;&gt;The Math That Makes This Real&lt;/h2&gt;
&lt;p&gt;The relationship between memory bandwidth and inference speed is direct enough to calculate approximately. For a model with N parameters at Q4 quantization, which uses roughly 0.5 bytes per parameter, each token generation requires the GPU to read approximately the full model weights from VRAM. A 13B parameter model at Q4 quantization requires reading roughly 6 to 7 GB of data per token generated. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-3090/&quot;&gt;Hardware Corner’s benchmark data&lt;/a&gt; shows this playing out exactly as the theory predicts across real hardware.&lt;/p&gt;
&lt;p&gt;Working through the bandwidth math for a 13B model at Q4 quantization:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;em&gt;At 960 GB/s (RTX 3090, RX 7900 XTX): approximately 19 milliseconds/token, or roughly 50 tokens/second&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;&lt;em&gt;At 504 GB/s (mid-range desktop GPU): approximately 36 milliseconds/token, or roughly 28 tokens/second&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;&lt;em&gt;At 256 GB/s (integrated APU memory): approximately 70 milliseconds/token, or roughly 14 tokens/second&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;That is a 3.5 times difference in speed from hardware that might cost similar amounts of money depending on what you buy. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://knowledgelib.io/computing/components/gpus-for-llm-inference/2026&quot;&gt;The AI Knowledge Library’s June 2026 GPU comparison&lt;/a&gt; puts the RTX 5090’s bandwidth advantage in context: at 1,792 GB/s it is 78% faster than the RTX 4090’s 1,008 GB/s. Hardware Corner’s benchmarks show the RTX 5090 delivering 102.7 tokens per second on Qwen3 14B at Q4 quantization with a 16K context window, making the bandwidth advantage visible in real-world output speed.&lt;/p&gt;
&lt;h2 id=&quot;gpu-comparison-table-ranked-by-memory-bandwidth-july-2026&quot;&gt;GPU Comparison Table: Ranked by Memory Bandwidth (July 2026)&lt;/h2&gt;
&lt;p&gt;This table covers the GPUs worth evaluating for local LLM inference in July 2026, sorted by memory bandwidth. Prices reflect approximate current ranges from Amazon and major retailers. The last two entries are included specifically to illustrate the bandwidth trap described in the section that follows. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-3090/&quot;&gt;Hardware Corner’s LLM benchmark database&lt;/a&gt; and &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://houtini.com/articles/best-gpus-for-running-local-llms/&quot;&gt;Houtini’s July 2026 GPU guide&lt;/a&gt; were the primary sources for confirmed bandwidth figures.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-02.yNw9Zn6Z_ZIHVf1.webp&quot; alt=&quot;Illustration for the section “GPU Comparison Table: Ranked by Memory Bandwidth (July 2026)”&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;761&quot; height=&quot;709&quot;&gt;&lt;/p&gt;
&lt;h2 id=&quot;2-expensive-machines-that-will-disappoint-you&quot;&gt;2 Expensive Machines That Will Disappoint You&lt;/h2&gt;
&lt;p&gt;The bandwidth table above reveals two categories of hardware that get marketed heavily toward the local AI crowd and consistently underdeliver on the metric that actually matters for inference speed. Both deserve a direct warning.&lt;/p&gt;
&lt;h2 id=&quot;the-amd-ryzen-ai-max-mini-pc-trap&quot;&gt;The AMD Ryzen AI Max Mini PC Trap&lt;/h2&gt;
&lt;p&gt;Mini PCs built around the AMD Ryzen AI Max+ 395 processor are being marketed aggressively in mid-2026 at prices ranging from $2,000 to $4,000. The headline specification is unified memory up to 96GB, which sounds compelling when you are used to discrete GPUs topping out at 24 to 32GB for consumer hardware. The problem is that this unified memory is DDR5 system RAM shared between the CPU, the integrated Radeon 8060S GPU, and the rest of the system, with a total memory bandwidth of approximately 256 GB/s. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://singhajit.com/llm-inference-speed-comparison/&quot;&gt;Ajit Singh’s April 2026 benchmark analysis&lt;/a&gt; confirms that on Apple Silicon, MLX achieves 30 to 50% faster throughput than Ollama, and the Apple M-series chips reach their performance partly because their unified memory bandwidth is dramatically higher than DDR5 systems. The Ryzen AI Max+ 395 does not have that advantage.&lt;/p&gt;
&lt;p&gt;At 256 GB/s, the Ryzen AI Max+ 395 will generate tokens on a 13B model at roughly 14 tokens per second in theoretical maximum. A used RTX 3090 with 936 GB/s bandwidth will generate those same tokens at roughly 50 tokens per second, nearly 3.5 times faster, and currently sells for approximately $1,000 to $1,500 on the used market in July 2026 based on live eBay and resale data. The mini PC costs two to four times more and runs AI inference at roughly one-third the speed.&lt;/p&gt;
&lt;p&gt;There are legitimate reasons to buy one of these systems. They are quiet, compact, power-efficient, and capable general-purpose computers. If your workload is mostly CPU-bound tasks, light productivity, and occasional small model inference, they are reasonable machines. If your primary use case is running local LLMs at speed, the bandwidth limitation means you are paying a significant premium for a machine that will feel slow specifically on the task you bought it for.&lt;/p&gt;
&lt;h2 id=&quot;the-nvidia-dgx-spark-trap&quot;&gt;The NVIDIA DGX Spark Trap&lt;/h2&gt;
&lt;p&gt;NVIDIA’s DGX Spark was announced at a $3,000 price point and marketed as a personal AI supercomputer. The specification that gets the most attention is 128GB of unified LPDDR5X memory via the Grace-Blackwell SoC. 128GB is a genuinely useful amount of capacity for running very large models that do not fit on consumer hardware. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://houtini.com/articles/best-gpus-for-running-local-llms/&quot;&gt;Houtini’s July 2026 analysis&lt;/a&gt; notes that the MI300X, AMD’s enterprise datacenter card, delivers 5,300 GB/s of bandwidth on paper but costs over $20,000 and ships in rackmount form factors. The DGX Spark sits at a consumer price point with consumer bandwidth.&lt;/p&gt;
&lt;p&gt;The DGX Spark’s LPDDR5X memory delivers approximately 273 GB/s of bandwidth. An RTX 3090 available used for approximately $1,000 to $1,500 delivers 936 GB/s. For any model that fits in 24GB, the used RTX 3090 will generate tokens roughly 3.4 times faster than the DGX Spark at less than a third of the price.&lt;/p&gt;
&lt;p&gt;The DGX Spark makes genuine sense for one specific situation: you need to load a model larger than 60GB that simply does not fit on any consumer discrete GPU. At that point the bandwidth penalty is unavoidable because no alternative exists. For everything up to approximately 32B parameters at Q4 quantization, which covers the majority of models most developers actually use day-to-day, a much cheaper discrete GPU with higher bandwidth will outperform it on the metric that determines whether the experience feels fast or frustrating.&lt;/p&gt;
&lt;h2 id=&quot;the-value-picks-worth-buying-right-now&quot;&gt;The Value Picks Worth Buying Right Now&lt;/h2&gt;
&lt;h2 id=&quot;rtx-3090-used-around-1000-to-1500&quot;&gt;RTX 3090 (Used, Around $1,000 to $1,500)&lt;/h2&gt;
&lt;p&gt;The RTX 3090 is the best value GPU for local LLM inference in mid-2026 if you do not need more than 24GB of VRAM. According to live July 2026 market data from &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;http://resaleprices.com/&quot;&gt;ResalePrices.com&lt;/a&gt; and GPUPoet, used RTX 3090 units are currently averaging approximately $1,250, with a range of roughly $1,000 to $1,500 depending on condition, seller, and AIB model. The RTX 5080, which costs over $1,200 new, delivers 960 GB/s, a 2.6% bandwidth advantage over the RTX 3090. The RTX 4090 delivers 1,008 GB/s, which is meaningfully faster, but commands $1,800 or more.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://houtini.com/articles/best-gpus-for-running-local-llms/&quot;&gt;Houtini’s analysis&lt;/a&gt; notes one practical consideration that the RTX 3090 draws 350W under load, which is real power consumption requiring a capable power supply and adequate case airflow. Budget 200W of PSU headroom above the GPU’s rated TDP. A system that browns out mid-inference does not fail quietly.&lt;/p&gt;
&lt;h2 id=&quot;rx-7900-xtx-around-800-to-1000&quot;&gt;RX 7900 XTX (Around $800 to $1,000)&lt;/h2&gt;
&lt;p&gt;AMD’s RX 7900 XTX matches the RTX 6000 Ada’s 960 GB/s bandwidth with a full 384-bit memory bus and 24GB of GDDR6, at a fraction of the price of the professional card. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://knowledgelib.io/computing/components/gpus-for-llm-inference/2026&quot;&gt;The AI Knowledge Library’s GPU comparison confirms&lt;/a&gt; that ROCm 7.2, released in March 2026, achieved full software parity with CUDA on Linux for the primary inference frameworks. llama.cpp, vLLM, and LM Studio all work correctly with ROCm in mid-2026 without the driver friction that plagued AMD GPU inference in earlier years. If you are on Windows rather than Linux, the software story is still slightly less mature, but the gap has narrowed considerably from where it was twelve months ago.&lt;/p&gt;
&lt;p&gt;2 RX 7900 XTX cards running in parallel via ROCm gives you 48GB of total VRAM at under $2,000, which is a setup that handles virtually every open model available for local inference at the time of writing.&lt;/p&gt;
&lt;h2 id=&quot;radeon-ai-pro-r9700-around-1350&quot;&gt;Radeon AI PRO R9700 (Around $1,350)&lt;/h2&gt;
&lt;p&gt;AMD’s R9700 is the workstation card for users who want 32GB of VRAM with ECC memory reliability at a price point below the professional NVIDIA alternatives. Bandwidth is more modest at 645 GB/s due to the 256-bit memory bus, which is a real trade-off relative to the RTX 3090 and RX 7900 XTX. But the 32GB capacity is the sweet spot for running 30B to 34B models fully in VRAM without quantization compromises. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://corelab.tech/llmgpu/&quot;&gt;CoreLab’s LLM GPU guide&lt;/a&gt; notes the 32GB buffer as the practical ceiling for most developers in 2026 running coding assistant and reasoning-heavy workloads with extended context.&lt;/p&gt;
&lt;h2 id=&quot;quick-decision-guide&quot;&gt;Quick Decision Guide&lt;/h2&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-03.CtJsshRt_fRoKR.webp&quot; alt=&quot;Illustration for the section “Quick Decision Guide”&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;630&quot; height=&quot;300&quot;&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-one-thing-worth-remembering&quot;&gt;The One Thing Worth Remembering&lt;/h2&gt;
&lt;p&gt;VRAM capacity tells you whether a model can run on your GPU at all. Memory bandwidth tells you whether you will enjoy using it. Both numbers matter, but they answer different questions, and conflating them is what leads people to spend $3,000 on a system with 128GB of unified memory and then discover it generates tokens more slowly than a card they could have bought for $800 used.&lt;/p&gt;
&lt;p&gt;Check the bandwidth number before you buy. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-3090/&quot;&gt;Hardware Corner’s GPU LLM benchmark database&lt;/a&gt; gives you tested tokens-per-second figures across models and context lengths for most cards worth considering. The math from bandwidth to inference speed is direct enough that theoretical limits translate to real-world experience with reasonable accuracy.&lt;/p&gt;
&lt;p&gt;The used GPU market in mid-2026 is unusually good for buyers. New Blackwell cards have pushed prices down on the Ampere and Ada generation hardware that was premium twelve months ago. An RTX 3090 at approximately $1,000 to $1,500 on the used market, with 936 GB/s of bandwidth and 24GB of VRAM, is a legitimately excellent local inference machine. It will outperform systems costing two to three times more on the workload you bought it for.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Your future self running a 32B parameter model at 50 tokens per second will be considerably happier than the version who bought the flashy spec sheet and is watching a loading spinner.&lt;/p&gt;
&lt;/blockquote&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>gpu</category><category>ai</category><category>localai</category><category>artificial-intelligence</category><category>nvidia</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Japan Gave the World Robots, Bullet Trains, and PlayStation. So Why Is It Losing the AI Race?</title><link>https://soursen.org/articles/japan-gave-the-world-robots-bullet-trains-and-playstation-so-why-is-it/</link><guid isPermaLink="true">https://soursen.org/articles/japan-gave-the-world-robots-bullet-trains-and-playstation-so-why-is-it/</guid><description>This is the question that does not quite make sense on the surface.</description><pubDate>Tue, 30 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;This is the question that does not quite make sense on the surface. Japan is the country that gave the world the Shinkansen, precision manufacturing that redefined what quality meant in every industry it touched, consumer robotics that no other nation could match, and entertainment hardware that shaped how an entire generation experienced technology. The country practically invented the idea of engineering perfection as a competitive advantage.&lt;/p&gt;
&lt;p&gt;Now look at the AI landscape. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.bitget.com/amp/news/detail/12560605383253&quot;&gt;Japan has 251 AI companies, including 93 startups&lt;/a&gt;. The United States has approximately 30,000. China has around 5,000. Germany has 1,500. Japan is the world’s fourth-largest economy, home to some of the most technically sophisticated manufacturing on earth, and it has roughly one-eighth the AI company density of Germany.&lt;/p&gt;
&lt;p&gt;Generative AI usage among Japanese people sits at 26.7%, according to &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://medium.com/@email_2235/generative-ai-risks-and-countermeasures-why-japan-lags-behind-what-the-dangers-are-and-how-to-1d97ccff7986&quot;&gt;Japan’s Ministry of Internal Affairs and Communications 2025 White Paper&lt;/a&gt;. In the United States, the figure is 68.8%. In China, 81.2%. Japan’s usage rate is not just lower. It is approximately 40% of America’s and 30% of China’s. Among corporations, only 1.9% of Japanese firms have adopted AI in core business functions, compared to 6.1% in the United States.&lt;/p&gt;
&lt;p&gt;So what happened? The answer is not one thing. It is several things that compounded on each other over 30 years, and understanding them separately is the only way to understand how they combined into something this significant.&lt;/p&gt;
&lt;h2 id=&quot;the-lost-decade-created-a-corporate-culture-that-punishes-risk&quot;&gt;The Lost Decade Created a Corporate Culture That Punishes Risk&lt;/h2&gt;
&lt;p&gt;The starting point for understanding Japan’s technology trajectory in the 21st century is the 1990s, not the 2020s. When Japan’s asset price bubble collapsed at the start of the 1990s, it triggered what became known as the Lost Decade, a period of economic stagnation, corporate restructuring, and profound institutional conservatism that reshaped how Japanese companies thought about innovation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://eastasiaforum.org/2026/06/03/japans-ai-transition-faces-stumbling-blocks/&quot;&gt;East Asia Forum’s June 2026 analysis&lt;/a&gt; documented the direct consequence that since the Lost Decade, the share of loss-making Japanese firms has increased from 40 to 50% to 60 to 70%. Government subsidisation policies keep many of these firms alive but have reinforced productivity stagnation rather than resolving it.&lt;/p&gt;
&lt;p&gt;The cultural consequence of that era is harder to quantify but easier to observe. Japanese corporate culture adopted a deep aversion to failure. The instinct became to wait, to study, to build consensus before moving, to ensure that any new initiative had been de-risked as thoroughly as possible before commitment. In a period of incremental, predictable technological change, this approach was viable. In the era of AI, where the field changes monthly and first-mover advantages compound rapidly, it became a structural disadvantage.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.bitget.com/amp/news/detail/12560605383253&quot;&gt;Sam Ghiotti, founder and CEO of Habitto&lt;/a&gt;, described the cultural gap at Japan Fintech Week 2026 in terms that are worth sitting with: “The biggest shock as a founder in Japan is that Americans need to be first and Japanese need to be right.” That is not a criticism. It is a description of two genuinely different philosophies of innovation. The problem is that in the current AI landscape, being right and being first are not equally weighted. The technology moves fast enough that waiting for certainty means arriving after the market has already formed.&lt;/p&gt;
&lt;h2 id=&quot;the-talent-problem-is-structural&quot;&gt;The Talent Problem Is Structural&lt;/h2&gt;
&lt;p&gt;Japan’s IT talent shortage is not the kind that can be solved by training more people quickly. It is a convergence of demographic decline and technological acceleration that was decades in the making. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://japandev.jp/employer-blog/japan-220000-it-talent-shortage-2026-hiring-crisis&quot;&gt;JapanDev’s April 2026 analysis&lt;/a&gt; put the current IT talent gap at 220,000 positions. The Japanese government’s own Ministry of Economy, Trade and Industry projects a shortage of 790,000 IT workers by 2030. The country is losing 500,000 working-age adults per year while simultaneously trying to digitise industries that have operated on analogue processes for decades.&lt;/p&gt;
&lt;p&gt;The researcher numbers are particularly revealing. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cissokomamady.com/japans-talent-crisis-in-the-age-of-agentic-ai-the-definitive-smoking-gun-special-report/&quot;&gt;Research published by Cissoko and Company in March 2026&lt;/a&gt; found that between 2011 and 2022, the number of researchers in Japan’s technology sector decreased by 27%, falling to approximately 145,662. This happened during the exact period when AI research was accelerating globally. Overall PhD researchers in Japanese enterprises grew by 8% in the same window. The technology sector specifically moved in the opposite direction.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://soursen.org/_astro/img-01.BkMpoIIA_v5pQp.webp&quot; alt=&quot;Illustration for the section “The Talent Problem Is Structural”&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; width=&quot;1177&quot; height=&quot;1000&quot;&gt;&lt;/p&gt;
&lt;p&gt;The OECD’s own labour market study on Japan, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.oecd.org/en/publications/artificial-intelligence-and-the-labour-market-in-japan_b825563e-en/full-report/ai-use-in-the-japanese-workplace_faf178d2.html&quot;&gt;published in November 2025&lt;/a&gt;, found that Japanese companies struggle significantly more than US counterparts with AI-related talent shortages. The most pressing problem is not a lack of PhDs or engineers in the abstract. It is a lack of employees capable of promoting AI adoption using workplace experience and basic AI knowledge, the middle layer of people who translate technical capability into business application.&lt;/p&gt;
&lt;p&gt;The language barrier compounds all of this in a way that is easy to underestimate if you have not worked inside the Japanese tech ecosystem. The vast majority of foundational AI research is published in English. The most important open-source models, the documentation, the research papers, the GitHub repositories, the developer communities: essentially all of it is in English first. Japanese engineers working at the frontier of AI need strong English proficiency in addition to technical ability. That combination narrows the talent pool significantly in a country where workplace English proficiency remains lower than most other developed economies.&lt;/p&gt;
&lt;h2 id=&quot;the-software-gap-that-hardware-excellence-could-not-hide&quot;&gt;The Software Gap That Hardware Excellence Could Not Hide&lt;/h2&gt;
&lt;p&gt;Japan’s technological reputation rests primarily on hardware. The Shinkansen is a feat of precision engineering. PlayStation hardware was genuinely ahead of its time in multiple generations. Japanese robotics represent decades of mechanical and electromechanical excellence. Japanese manufacturing quality systems influenced global industry for fifty years.&lt;/p&gt;
&lt;p&gt;Software is a fundamentally different discipline. It rewards iteration speed, failure tolerance, rapid deployment, and the ability to ship something imperfect and improve it in production. These are not values that the Japanese corporate culture of the Lost Decade era naturally produced. The result was that while Japan dominated global hardware manufacturing, it never built a comparable domestic software industry. The companies that could have pivoted from hardware excellence to software innovation largely did not.&lt;/p&gt;
&lt;p&gt;The gap became visible at scale when the smartphone era arrived. Japan had Sharp, Sony, Fujitsu, and Panasonic all building mobile handsets. It had its own domestic mobile internet ecosystem called i-mode that predated the western mobile web by years. None of it translated into global software platform dominance. Apple and Google captured the smartphone software layer, and Japanese hardware companies watched their mobile businesses shrink. The pattern repeated itself with cloud computing, where Amazon, Microsoft, and Google established the dominant infrastructure globally while Japanese firms remained primarily consumers rather than providers. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.ainvest.com/news/japan-ai-problem-technology-2606/&quot;&gt;Japan’s AI infrastructure market is projected to exceed $5.5 billion in 2026&lt;/a&gt;, 7 times its size three years ago, but the underlying models and platforms are almost entirely American.&lt;/p&gt;
&lt;h2 id=&quot;the-corporate-structure-that-slowed-everything-down&quot;&gt;The Corporate Structure That Slowed Everything Down&lt;/h2&gt;
&lt;p&gt;Japan’s keiretsu system, the interlocking networks of companies, banks, and suppliers that dominated Japanese business for decades, provided extraordinary stability during Japan’s high-growth era. It reduced short-termism, protected workers, and enabled the long-horizon investment that produced world-class manufacturing. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.ainvest.com/news/japan-ai-problem-technology-2606/&quot;&gt;AInvest’s June 2026 analysis&lt;/a&gt; described the same system as a barrier to the speed and risk-taking that AI development requires. Lifetime employment guarantees, cross-shareholding arrangements, and deeply embedded consensus-building processes make rapid pivots extremely difficult. A startup in Silicon Valley can decide to rebuild its entire technical stack in a week. A division of a large Japanese keiretsu company might spend six months building the internal consensus required to initiate a pilot programme.&lt;/p&gt;
&lt;p&gt;This is not abstract. The OECD study found that Japanese firms were slow to adopt AI even for basic tasks including translation, document drafting, and coding. These are not cutting-edge applications. They are entry-level AI use cases that require minimal technical sophistication to implement. The barrier is not capability. It is institutional inertia.&lt;/p&gt;
&lt;p&gt;The venture capital ecosystem that produced OpenAI, Anthropic, and the wave of AI startups in the US simply does not exist at comparable scale in Japan. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.disruptingjapan.com/the-future-of-ai-looks-very-different-in-japan/&quot;&gt;AI startups accounted for over 60% of global VC investment in 2025&lt;/a&gt;. Japan’s venture market, while growing, operates at a fraction of the capital deployment and risk tolerance of Silicon Valley. The result is that Japan produces very few AI-native companies and a large number of traditional companies trying to add AI capabilities to existing operations.&lt;/p&gt;
&lt;h2 id=&quot;what-japan-is-actually-doing-about-it&quot;&gt;What Japan Is Actually Doing About It&lt;/h2&gt;
&lt;p&gt;Having laid out the problems, it is important to be precise about what is changing, because the picture is not static.&lt;/p&gt;
&lt;p&gt;The Japanese government has moved faster than most people outside Japan realise. Japan passed an AI Promotion Act in May 2025, adopted a formal AI Basic Plan in December 2025, created a Prime Minister-led AI Strategic Headquarters, and committed more than 10 trillion yen in public support for AI and semiconductor infrastructure. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.helloworldjapan.com/en/articles/japan-ai-strategy-government-plan-explained&quot;&gt;Hello World Japan’s May 2026 analysis&lt;/a&gt; noted that the official language of Japan’s AI strategy is unusually direct that the government explicitly acknowledges it is behind and frames the strategy as recovery rather than regulation. The Ministry of Economy, Trade and Industry has earmarked 1 trillion yen over five years from fiscal 2026 to develop foundation models at the one-trillion-parameter scale.&lt;/p&gt;
&lt;p&gt;Foreign investment is arriving at scale. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.bitget.com/amp/news/detail/12560605383253&quot;&gt;Microsoft announced a 1.6 trillion yen investment in Japan from 2026 to 2029&lt;/a&gt;. Nvidia has partnered with Fujitsu to build full-stack AI infrastructure. SoftBank launched an AI Boost Program in September 2025 providing free access to high-performance GPU systems and Nvidia engineering support for Japanese startups.&lt;/p&gt;
&lt;p&gt;Adoption is finally accelerating at the consumer level too. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://bworldonline.com/bloomberg/2026/05/26/751938/japans-an-ai-laggard-that-could-be-its-edge/&quot;&gt;Microsoft’s AI Economy Institute Q1 2026 report found&lt;/a&gt; that AI adoption in Japan increased 3.4 percentage points in the first quarter of 2026, more than three times faster than the global average. NTT Docomo data shows usage nearly doubled between February 2025 and February 2026. The country is moving. It is just moving from a position that was significantly further back than most observers had appreciated.&lt;/p&gt;
&lt;p&gt;Japan’s most interesting AI company right now is &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.disruptingjapan.com/the-future-of-ai-looks-very-different-in-japan/&quot;&gt;Sakana AI&lt;/a&gt;, co-founded by David Ha, former head of research at Google Brain. Sakana AI is pursuing a different philosophy from the frontier model race: smaller, more efficient models that draw inspiration from nature-based collective intelligence rather than brute-force scaling. Ha has argued publicly that the frontier model approach may be flawed and expensive in ways that create genuine opportunity for a different path. Whether that argument proves correct will say a great deal about whether Japan’s latecomer position is a permanent disadvantage or a chance to skip directly to what works.&lt;/p&gt;
&lt;h2 id=&quot;the-counterintuitive-case-for-japan&quot;&gt;The Counterintuitive Case for Japan&lt;/h2&gt;
&lt;p&gt;Here is something worth sitting with. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://bworldonline.com/bloomberg/2026/05/26/751938/japans-an-ai-laggard-that-could-be-its-edge/&quot;&gt;BusinessWorld’s May 2026 piece&lt;/a&gt; made an argument that runs against the obvious reading of the data: Japan’s late start may actually be an advantage. The country can observe what worked and what did not in the US and China, skip the expensive failed experiments, and deploy AI directly into problems it genuinely has solutions for.&lt;/p&gt;
&lt;p&gt;Japan has an ageing population and a shrinking labour force. Those are real constraints that AI can genuinely address in ways that produce measurable economic value. There is less of the cultural anxiety about AI displacing workers when the workforce is already shrinking and the pressure is to find workers at all. The adoption case in Japan is not philosophical. It is practical and urgent.&lt;/p&gt;
&lt;p&gt;Japan also has world-class hardware manufacturing, robotics infrastructure, and manufacturing precision that most AI-native countries do not. The integration of AI with physical systems, the embodied AI frontier that most US AI companies are only beginning to approach, is a domain where Japan’s existing capabilities are genuinely relevant. The question is whether the cultural and institutional barriers can be overcome fast enough to deploy those advantages before the window closes.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Japan did not fail at AI because it lacked intelligence or technical capability. It fell behind because the institutional structures and cultural values that made it excellent at hardware-era innovation are precisely the ones that slow you down in software-era innovation. The country built extraordinary tools for a world that was moving slower than the one that arrived.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The honest answer to why Japan is behind in AI is that it is the accumulated consequence of 30 years of corporate conservatism, a structural talent shortage that demographic decline made almost inevitable, a software culture that never matched the hardware excellence, and an institutional speed mismatch with an industry that rewards moving fast and accepting failure as a data point rather than a catastrophe.&lt;/p&gt;
&lt;p&gt;The question that matters now is whether the government investment, the foreign capital, the accelerating adoption curve, and companies like Sakana AI pursuing genuinely different approaches can compound quickly enough to change the trajectory in a meaningful timeframe. The window is real but it is not unlimited.&lt;/p&gt;
&lt;p&gt;Japan has surprised the world before. The question is whether the surprise is still possible in a field where the compounding advantage of being early is this significant.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>japan</category><category>ai</category><category>artificial-intelligence</category><category>llm</category><category>playstation</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>I Quit AI Tools for 30 Days. What It Revealed About My Own Thinking Scared Me.</title><link>https://soursen.org/articles/i-quit-ai-tools-for-30-days-what-it-revealed-about-my-own-thinking/</link><guid isPermaLink="true">https://soursen.org/articles/i-quit-ai-tools-for-30-days-what-it-revealed-about-my-own-thinking/</guid><description>It started with an email.</description><pubDate>Thu, 25 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;It started with an email.&lt;/p&gt;
&lt;p&gt;Not a complicated one. A short, professional message to a colleague I had worked with for years. Something I would have written in 3 minutes without a second thought two years ago. But I sat there, cursor blinking, and felt something I had not felt in a while the slight panic of not knowing where to start without a prompt box in front of me.&lt;/p&gt;
&lt;p&gt;That moment made me run an experiment. 30 days. No AI writing tools. No AI research shortcuts. No using a chatbot to draft, outline, summarise, or structure anything. Just my own brain, the one I had spent years developing and then, gradually, quietly outsourced.&lt;/p&gt;
&lt;p&gt;What I found over those 30 days was not just personal. It maps almost exactly onto what researchers studying AI dependence have been documenting since 2024, and the picture they are drawing is uncomfortable for anyone who has built their workflow around these tools.&lt;/p&gt;
&lt;h2 id=&quot;what-the-research-says-about-brains-that-always-have-an-easy-answer-available&quot;&gt;What the Research Says About Brains That Always Have an Easy Answer Available&lt;/h2&gt;
&lt;p&gt;There is a term researchers now use for what I experienced with that email. It is called cognitive offloading: the transfer of mental processes to an external system. The concept itself is not new. We have always used tools to extend our cognitive capacity. Writing things down rather than memorising them is cognitive offloading. A calculator is cognitive offloading. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.sciencedirect.com/science/article/pii/S2451958826001764&quot;&gt;What a 2025 study of 666 participants by researcher Michael Gerlich found&lt;/a&gt;, however, is that frequent AI usage shows a significant negative relationship with critical thinking capabilities, with cognitive offloading serving as the mediating factor. In other words, the more you use AI to do your thinking, the less capable you become at doing it yourself.&lt;/p&gt;
&lt;p&gt;A separate research framework published in the journal npj Artificial Intelligence in 2026 described it as follows: passive, uncritical reliance on AI may weaken activity-dependent brain plasticity and erode cognition. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.thealgorithmicbridge.com/p/what-the-studies-say-about-how-ai&quot;&gt;Researchers at The Algorithmic Bridge who compiled dozens of these studies&lt;/a&gt; summarised the consensus as this: when AI becomes a cognitive prosthetic, the underlying cognitive muscles weaken from disuse. This is not a metaphor. It is a description of how neural pathways actually function.&lt;/p&gt;
&lt;p&gt;The Wharton School ran three preregistered experiments in early 2026 with 1,372 participants, studying what they called cognitive surrender, the tendency to accept AI-generated answers without evaluation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://publichealth.gmu.edu/news/2026-03/ai-and-rise-cognitive-overload&quot;&gt;A Boston Consulting Group study published in Harvard Business Review&lt;/a&gt; surveyed nearly 1,500 full-time employees and found that a meaningful share reported symptoms they called AI brain fry: mental fog, slower decision-making, and a strange sense that their thinking had become crowded. Workers described spending more time managing AI outputs than they had previously spent simply doing the underlying work.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/pdf/2603.26707&quot;&gt;The Anthropic Economic Index’s January 2026 report&lt;/a&gt; identified something it called deskilling risk: the gradual decline of occupational competence in categories where AI absorbs the most cognitively demanding tasks. The report noted that this effect appears not just in educational settings but at industry scale, affecting workers who consistently outsource complex cognitive tasks to AI before developing sufficient judgment on those tasks themselves.&lt;/p&gt;
&lt;p&gt;This is the thing that the productivity content around AI does not tell you. The tools are genuinely useful for specific tasks. The problem is not the tasks. It is the cumulative effect of never doing the hard part yourself, over months and years, across hundreds of small decisions that together constitute your ability to think.&lt;/p&gt;
&lt;h2 id=&quot;the-attention-problem-is-already-here-ai-is-just-accelerating-it&quot;&gt;The Attention Problem Is Already Here, AI Is Just Accelerating It&lt;/h2&gt;
&lt;p&gt;Before AI tools entered the picture, there was already a significant and well-documented crisis in sustained attention. Layering AI dependency on top of a population that was already struggling to focus is not a neutral act.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.demandsage.com/screen-time-statistics/&quot;&gt;Data compiled by DemandSage in 2026&lt;/a&gt; shows the average American checks their phone 96 times per day. That is once every 10 minutes during waking hours. The average person spends 6 hours and 54 minutes on screens daily, with Americans averaging closer to 7 hours. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://yougov.com/en-us/articles/53735-for-many-americans-their-smartphone-is-the-last-thing-they-see-at-night-and-the-first-thing-they-see-in-the-morning&quot;&gt;A YouGov survey from late 2025&lt;/a&gt; found that 45% of adults under 30 believe their phone has a mostly negative impact on their attention span, while 41% say it has damaged their ability to focus.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sqmagazine.co.uk/social-media-attention-span-statistics/&quot;&gt;A meta-analysis of attention studies cited by SQ Magazine&lt;/a&gt; found that the average social media user’s focus on a single piece of content has dropped from 12.1 seconds in 2015 to 8.25 seconds in 2025. Heavy users spending five or more hours daily on social media are 33% more likely to experience attention fragmentation symptoms. Exposure to short-form video content before studying reduced students’ reading attention span by 31%.&lt;/p&gt;
&lt;p&gt;What this means in practice is that most people are arriving at their work with fragmented attention, then immediately reaching for an AI tool to compensate for that fragmentation rather than rebuilding the focus capacity that the fragmentation has eroded. The AI does the thinking. The scrolling occupies the gap. The muscle of sustained independent thought never gets used, so it does not get stronger. Over time, it weakens.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;We do not think anymore. We fetch. The question worth asking is what happens to the fetching mechanism when the server goes down.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;what-actually-happened-during-the-30-days&quot;&gt;What Actually Happened During the 30 Days&lt;/h2&gt;
&lt;p&gt;The first week was genuinely uncomfortable in a way I had not expected. The discomfort was not the absence of convenience. It was the confrontation with how dependent my cognitive workflow had become on getting a quick first draft, a quick summary, a quick framework, before I was willing to engage with a problem myself.&lt;/p&gt;
&lt;p&gt;I had to write a strategy document on day 4 sitting with a blank page for the first 20 minutes without opening a prompt box felt physically strange, like trying to remember a word that has moved just out of reach. My instinct was not to think harder. It was to look for a shortcut. That instinct, appearing that quickly and that insistently, was the data point that made me take the experiment seriously.&lt;/p&gt;
&lt;p&gt;By the end of week 2, something had shifted. Problems I would previously have handed to an AI for initial framing were starting to resolve themselves in the background, the way a word you have been trying to remember surfaces an hour later when you have stopped actively searching for it. The synthesis of existing knowledge, the connections between things I already understood, started becoming available again in a way they had not been for months.&lt;/p&gt;
&lt;p&gt;Week 3 brought what felt like a qualitative improvement in the texture of my thinking. Solutions that had been feeling complex started revealing simpler structures. I was not getting smarter. I was getting less interrupted, including by myself.&lt;/p&gt;
&lt;p&gt;Week 4 felt like returning to a version of my working brain from several years earlier. Not better overall, but more capable of sustained engagement with a single problem without wanting to hand it off.&lt;/p&gt;
&lt;p&gt;The neuroscience behind this progression is documented. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.psychologytoday.com/us/blog/psych-unseen/202605/your-brain-on-ai-cognitive-offloading-debt-and-atrophy&quot;&gt;Psychology Today’s review of the cognitive offloading research&lt;/a&gt; describes how deep focus causes neural pathways to myelinate, becoming faster and more efficient over time. Skills that begin as effortful become automatic. Flow states become more accessible. Learning compounds. Distraction has the opposite effect: pathways never solidify, everything stays effortful, you never escape the beginner phase of a skill because you never push through the difficulty that produces growth.&lt;/p&gt;
&lt;h2 id=&quot;the-resolution-failure-rate-and-what-it-actually-measures&quot;&gt;The Resolution Failure Rate and What It Actually Measures&lt;/h2&gt;
&lt;p&gt;June is the most useful time to think about this because it is the moment when millions of people simultaneously attempt to change their habits try hard and then discover they cannot. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.sciencedirect.com/science/article/pii/S2451958826001764&quot;&gt;Research cited across multiple studies identifies January 10 as what some researchers call Quitter’s Day&lt;/a&gt;, the point at which 91% of New Year’s resolutions have already been abandoned. By February, only about 36% of people are still pursuing whatever they committed to on January 1. By the following December, fewer than 9% have completed what they set out to do.&lt;/p&gt;
&lt;p&gt;This failure rate is usually framed as a motivation problem or a goal-setting problem. That framing is convenient because it locates the failure in personal character rather than in the environment that produced the character. The more accurate framing is an attention problem. Most people cannot sustain effort toward a single goal for an extended period because the nervous system they are operating has been trained, through years of short-cycle reward loops, to require constant novelty and stimulation. Sustained effort toward a single goal is the opposite of what that training produces.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.polytechnique-insights.com/en/columns/neuroscience/generative-ai-the-risk-of-cognitive-atrophy/&quot;&gt;A 2025 CHI Conference study on generative AI and critical thinking&lt;/a&gt; surveyed knowledge workers and found self-reported reductions in cognitive effort specifically in contexts where AI was available. Workers were not choosing to think less hard because they were lazy. They were responding to an environment that offered an easier path, and the brain, which is energy-efficient by design, took it. The same mechanism drives resolution failure: the environment offers the easier path constantly, and sustained effort requires actively choosing the harder one every single time.&lt;/p&gt;
&lt;p&gt;The reason a 30-day AI detox produced a meaningful cognitive shift is the same reason resolution failure is so consistent. Thirty days is long enough for the environment to change. When the easier path is not available, the harder one becomes the only path, and you either take it or nothing happens at all. Most people discover, within a few days of having the easier path removed, that they are more capable than the easier path had been giving them credit for.&lt;/p&gt;
&lt;h2 id=&quot;the-uncomfortable-truth-about-what-wins-over-time&quot;&gt;The Uncomfortable Truth About What Wins Over Time&lt;/h2&gt;
&lt;p&gt;The most counterintuitive finding from both the research and from personal experience is how low the bar has become for sustained, focused effort.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sqmagazine.co.uk/social-media-attention-span-statistics/&quot;&gt;A 2025 study on attention fragmentation&lt;/a&gt; found that individuals who use social media during work or study sessions are 2.2x more likely to make attention-related errors. Prolonged exposure to rapid content streams decreases working memory efficiency by 11%. 95% of people, according to attention researchers, cannot sustain ninety minutes of genuinely uninterrupted focus on a single complex problem.&lt;/p&gt;
&lt;p&gt;That 95% figure is the relevant competitive context. You are not competing with the most focused people in your industry. You are competing with everyone in your industry, most of whom are operating with significantly degraded attention capacity and compensating for it with AI tools that are introducing their own reliability problems. The gap between what most people can do and what a person with reclaimed sustained focus can do has widened, not narrowed, as AI tools have proliferated.&lt;/p&gt;
&lt;p&gt;The reason this is counterintuitive is that AI tools feel like they are levelling the playing field. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.sciencedirect.com/science/article/pii/S2451958826001764&quot;&gt;The ScienceDirect review of AI cognitive effects&lt;/a&gt; calls this the levelling effect: performance equalisation across individuals with different baseline competencies, because AI compensates for skill gaps. In the short term, someone with weaker writing ability can produce a stronger email using AI. In the medium term, the gap between that person and the person who has continued to develop their writing ability independently widens, because the person using AI is getting no practice while the person not using AI is getting all of it.&lt;/p&gt;
&lt;p&gt;One year of genuinely focused effort on a single skill or domain, in the environment that currently exists, produces outcomes that would have been considered exceptional a decade ago simply because so few people are doing it. Not because the bar for exceptional has changed. Because the percentage of people capable of clearing it has dropped.&lt;/p&gt;
&lt;h2 id=&quot;what-to-actually-do-with-this&quot;&gt;What to Actually Do With This&lt;/h2&gt;
&lt;p&gt;The conclusion here is not that AI tools are bad and should be abandoned. That is not what 30 days showed me. What 30 days showed me is that the pattern of use matters far more than the tool itself.&lt;/p&gt;
&lt;p&gt;The distinction that &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.thealgorithmicbridge.com/p/what-the-studies-say-about-how-ai&quot;&gt;the npj Artificial Intelligence research identified&lt;/a&gt; is between passive reliance and active co-creation. Passive reliance means handing a problem to the AI before you have engaged with it yourself. Active co-creation means doing the thinking first, forming your own view, developing your own draft or analysis, and then using AI to pressure-test, extend, or improve what you have already produced. The first pattern atrophies the skill. The second develops it.&lt;/p&gt;
&lt;p&gt;The practical version of this is simpler than it sounds. Before opening a prompt box, sit with the problem for twenty minutes. Write a rough version yourself, even if it is imperfect. Form a view before asking for one. Then use the tool. The cognitive work the first twenty minutes require is the work that builds the capacity. The AI assistance that follows is genuinely useful rather than a substitute for developing the judgment to evaluate it.&lt;/p&gt;
&lt;p&gt;The attention piece requires a separate intervention. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.psychologytoday.com/us/blog/psych-unseen/202605/your-brain-on-ai-cognitive-offloading-debt-and-atrophy&quot;&gt;Researchers who study focus restoration&lt;/a&gt; consistently find that sustained improvement in attention capacity requires periods of genuine disconnection: no notifications, no tab switching, no phone within reach. The University of Texas study found that the mere presence of a smartphone on a desk reduces cognitive capacity even when it is face down and silent. Attention is not just about what you are looking at. It is about what your brain knows is available.&lt;/p&gt;
&lt;p&gt;90 minutes of genuinely uninterrupted focus per day, sustained over a year, produces compounding results that are difficult to model in advance because so few people have done it in the current environment. The people who have done it tend to describe the experience the same way: the work started feeling different somewhere around the three-month mark, easier in a way that did not feel like it was getting simpler, more like moving through familiar territory with better navigation.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The year that feels like discipline at the start becomes the decade that feels like freedom at the end. The compounding is real. So is the delay before you can see it.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The 30 days I ran were not about proving that AI is bad. They were about recovering something I had not noticed losing. The experiment worked. The capacity came back. What I know now that I did not know before starting is that it can come back, faster than you would expect, once the environment stops making the easier path available every six seconds.&lt;/p&gt;
&lt;p&gt;The question is not whether you are capable of focused, independent thought. You are. The question is whether the environment you have built around yourself makes that capacity easy to access or systematically undermines it.&lt;/p&gt;
&lt;p&gt;That question is worth answering honestly before deciding how to use the tools available to you.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>artificial-intelligence</category><category>ai</category><category>chatgpt</category><category>anthropic-claude</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>The AI Industry Built a $5 Trillion Debt Pile on One Bet. That Bet Is Not Paying Off.</title><link>https://soursen.org/articles/the-ai-industry-built-a-5-trillion-debt-pile-on-one-bet-that-bet-is-not/</link><guid isPermaLink="true">https://soursen.org/articles/the-ai-industry-built-a-5-trillion-debt-pile-on-one-bet-that-bet-is-not/</guid><description>Something shifted in the AI narrative over the past few months, and it is worth paying attention to. The CEOs who spent years telling anyone who would listen that AI would eliminate entire categories of human work have gone quiet on that point.</description><pubDate>Tue, 23 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Something shifted in the AI narrative over the past few months, and it is worth paying attention to. The CEOs who spent years telling anyone who would listen that AI would eliminate entire categories of human work have gone quiet on that point. The companies charging flat monthly subscriptions for AI tools have abruptly switched to usage-based pricing. The most valuable AI companies in history are rushing toward IPOs while their own financial documents show losses that do not narrow as they grow.&lt;/p&gt;
&lt;p&gt;These are not random coincidences. They are connected events, all flowing from the same underlying problem. The AI industry built itself on a specific financial bet, poured trillions of dollars into that bet, and is now confronting the fact that the bet is not working out the way it needed to.&lt;/p&gt;
&lt;p&gt;This piece is an attempt to trace that logic clearly, from the debt to the failed premise to the cost crisis to the IPO scramble, because understanding the connection between these things matters more than tracking any single headline in isolation.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Note: This article reflects my analysis of publicly available data and should not be taken as financial advice.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;the-debt-and-the-bet-it-created&quot;&gt;The Debt and the Bet It Created&lt;/h2&gt;
&lt;p&gt;Start with the money, because the scale of it is not widely appreciated. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.bloomberg.com/news/articles/2025-10-07/jpmorgan-says-1-2-trillion-debt-tied-to-ai-tops-bank-high-grade&quot;&gt;JPMorgan analysts reported in late 2025&lt;/a&gt; that AI-related debt had reached $1.2 trillion in investment-grade markets alone, making it the single largest segment in that market, surpassing even US banks. That was just one slice. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.kucoin.com/news/flash/jpmorgan-upgrades-ai-infrastructure-investment-forecast-to-5-5-trillion-tech-giants-shift-to-debt-financing&quot;&gt;JPMorgan subsequently raised its total AI infrastructure investment forecast to $5.5 trillion by 2030&lt;/a&gt;, noting that approximately $4.1 trillion of that total is expected to come from debt financing, with loans covering an average of 85 percent of project costs.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.mellon.com/insights/insights-articles/record-breaking-ai-related-debt-issuance-in-2025.html&quot;&gt;Morgan Stanley and JPMorgan jointly project&lt;/a&gt; that the technology sector will need to issue as much as $1.5 trillion in new debt over the next few years to finance AI and data centre infrastructure alone. UBS analysts forecast $900 billion in new debt from global companies in 2026 specifically. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cryptobriefing.com/jpmorgan-ai-capex-data-center-debt/&quot;&gt;Since late 2025, global bond issuance related to AI and data centres has already exceeded $300 billion&lt;/a&gt;. Nvidia completed a $25 billion bond offering that drew $850 billion in demand, its first bond issuance in five years.&lt;/p&gt;
&lt;p&gt;This debt does not sit on a spreadsheet in silence. It has to be serviced. To service even a fraction of the outstanding AI-related debt at typical market rates over a ten-year period requires generating hundreds of billions of dollars in profit annually. That is not a modest ask. It is larger than the annual profit of almost every company in the world.&lt;/p&gt;
&lt;p&gt;To meet that requirement, AI cannot be a useful productivity tool or a consumer subscription product. It has to replace human labour at a scale that fundamentally restructures the economy. The math does not work any other way. This is the bet buried inside the debt: that AI will generate a displacement of human workers so large and so fast that the revenue replacing those wages becomes sufficient to service what has been borrowed.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Geoffrey Hinton, the researcher widely credited as one of the founding figures of modern AI, has publicly stated that the debt the industry has accumulated on the premise of mass labour replacement poses a genuine systemic risk if the technology fails to deliver on that premise at sufficient scale.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That is the context for everything that follows. Not hype. Not consumer enthusiasm. A specific financial bet, made at a specific scale, that requires a specific outcome to avoid a very large problem.&lt;/p&gt;
&lt;h2 id=&quot;the-jobs-are-not-going-anywhere-and-here-is-the-evidence&quot;&gt;The Jobs Are Not Going Anywhere, and Here Is the Evidence&lt;/h2&gt;
&lt;p&gt;The honest read of the available data on AI and employment is not what the narrative of the past three years has suggested. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2026/01/07/ai-layoffs-convenient-corporate-fiction-true-false-oxford-economics-productivity/&quot;&gt;Oxford Economics published a detailed analysis in January 2026&lt;/a&gt; concluding that companies do not appear to be replacing workers with AI on any significant scale. The firm’s test for this was direct: if AI were genuinely replacing labour, output per remaining worker should be surging. It is not. Economy-wide productivity growth has actually decelerated, tracking cyclical economic patterns rather than any technological disruption.&lt;/p&gt;
&lt;p&gt;Oxford highlighted data from Challenger, Gray and Christmas showing that while AI was cited as the reason for approximately 55,000 US job cuts in the first eleven months of 2025, that figure represented just 4.5% of total reported job losses. Standard market and economic conditions accounted for four times as many. The report concluded that AI-related job losses remain statistically limited. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://laweconcenter.org/resources/ai-productivity-and-labor-markets-a-review-of-the-empirical-evidence/&quot;&gt;The International Center for Law and Economics reviewed the wider academic literature&lt;/a&gt; in early 2026 and reached the same conclusion, finding essentially zero measurable effects on earnings or employment hours in multiple large datasets through 2024 and 2025.&lt;/p&gt;
&lt;p&gt;On the corporate deployment side, a survey of 6,000 C-suite executives across multiple countries found that nearly 90% reported AI had no measurable impact on employment or productivity over the past three years. Companies including Starbucks and Pizza Hut attempted to deploy AI systems to replace customer-facing workers and were forced to reverse those decisions when performance issues made the systems unusable in production. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://workplaceinsight.net/evidence-of-ai-driven-job-losses-remains-limited-says-oxford-economics-report/&quot;&gt;A study by Sinch surveying 2,500 AI decision-makers&lt;/a&gt; found a 74% rollback or shutdown rate for AI customer communications agents, a category widely considered the most immediately replaceable by current AI systems.&lt;/p&gt;
&lt;p&gt;Oxford Economics posed the question directly: if jobs are being replaced, where is the productivity surge? It has not appeared. That absence is not a minor data point. It is the central empirical challenge to the premise on which multiple trillions of dollars of debt has been issued.&lt;/p&gt;
&lt;h2 id=&quot;why-ai-keeps-failing-at-the-tasks-that-would-justify-the-bet&quot;&gt;Why AI Keeps Failing at the Tasks That Would Justify the Bet&lt;/h2&gt;
&lt;p&gt;There is a specific reason the productivity surge is not appearing, and it is worth being precise about it because the popular explanation, that the technology just needs more time or better deployment, does not match what the research actually shows.&lt;/p&gt;
&lt;p&gt;The core problem is reliability. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai&quot;&gt;Stanford HAI’s 2026 AI Index Report&lt;/a&gt; documented that hallucination rates across 26 leading AI models range from 22% to 94% depending on the task type. GPT-4o’s accuracy on certain tasks dropped from 98.2% to 64.4% under more realistic testing conditions. The AI Incident Database recorded 362 documented failures in production deployments in 2025, a 55% increase from 233 in 2024.&lt;/p&gt;
&lt;p&gt;In software development, the domain where AI investment has been most concentrated, the reliability picture is particularly clear. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/pdf/2507.09089&quot;&gt;Research from METR published in 2026&lt;/a&gt; studied experienced open-source developers using AI coding tools and found that AI assistance actually increased task completion time by 19%. Developers estimated the tools would save them 20% of their time. The measured outcome ran in the opposite direction. The explanation was consistent across participants: more time was spent debugging and correcting AI-generated errors than was saved in initial generation.&lt;/p&gt;
&lt;p&gt;Research from Veracode found that 45% of AI-generated code contains security vulnerabilities. A separate study found AI-generated code carries 70% more bugs than human-written code. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026&quot;&gt;OpenAI’s own internal research found&lt;/a&gt; that simply increasing computing power or providing more training data cannot reduce AI hallucinations below current levels. The 2026 International AI Safety Report, written by over 100 domain experts, independently confirmed this finding: AI systems remain prone to basic errors of fact and logic in ways that do not resolve with more capability investment.&lt;/p&gt;
&lt;p&gt;This matters enormously for the financial argument, because it rules out the most obvious escape route. If the reason AI is not replacing workers is deployment problems or insufficient capability, more investment could theoretically fix it. If the reason is that the systems have an inherent reliability ceiling that cannot be engineered away within current architectural approaches, the premise of the bet cannot be rescued by spending more money on it.&lt;/p&gt;
&lt;h2 id=&quot;the-pricing-crisis-that-exposed-the-subsidy&quot;&gt;The Pricing Crisis That Exposed the Subsidy&lt;/h2&gt;
&lt;p&gt;The most recent development in this story is the one that has brought the underlying economics into sharpest focus. Over the past several months, AI companies have moved away from flat subscription pricing toward usage-based token pricing. The consequences have been immediate and severe.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://techcrunch.com/2026/05/30/what-a-joke-github-copilots-new-token-based-billing-spurs-consternation-among-devs/&quot;&gt;GitHub Copilot switched to token-based billing on June 1, 2026&lt;/a&gt;, replacing its flat subscription model with a credit system where each AI request draws down a finite monthly allowance. Heavy users of agentic and repository-wide features immediately reported bills surging from $39 per month to over $800. Some documented cases showed increases of more than twenty times the previous cost. This was not a price increase in the conventional sense. It was the removal of a subsidy that had been making the tool appear cheaper than its actual operating cost.&lt;/p&gt;
&lt;p&gt;Uber’s CTO told The Information that the company had burned through its entire 2026 AI budget in four months, primarily from AI coding tools including Claude Code. He described going back to the drawing board on AI spending entirely. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2026/04/28/nvidia-executive-cost-of-ai-is-greater-than-cost-of-employees/&quot;&gt;Fortune reported that a Goldman Sachs research note&lt;/a&gt; concluded that companies are overrunning their initial AI inference budgets by orders of magnitude. An Nvidia vice president stated publicly that the cost of compute for his team now exceeds the cost of the employees that compute is supposed to augment.&lt;/p&gt;
&lt;p&gt;Senior leaders at Uber made the implication explicit: AI is now more expensive than the human workers it was supposed to replace, and unlike human workers, AI usage does not correlate to delivering new features. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://usagebox.com/articles/github-copilot-usage-based-billing-2026&quot;&gt;Microsoft cancelled its internal Claude Code licences&lt;/a&gt; for the same reason, citing cost as prohibitive. Reports surfaced of a single company accumulating a $500 million bill with Anthropic in a single month.&lt;/p&gt;
&lt;p&gt;This pricing shift reveals something important. The flat subscription prices that most users had been paying were not prices at all. They were marketing investments, designed to embed AI tools deeply into workflows before the real cost of operating them was disclosed. Now that the subsidised phase is ending, the actual economics of AI at scale are visible for the first time, and they do not support the case for AI replacing human labour.&lt;/p&gt;
&lt;p&gt;If AI costs more than human workers and does not reliably produce better output, the industrial logic of the replacement narrative collapses entirely. This is almost certainly why the executives who built careers on that narrative have gone unusually quiet about it in recent months.&lt;/p&gt;
&lt;h2 id=&quot;the-financial-condition-of-the-companies-making-the-bet&quot;&gt;The Financial Condition of the Companies Making the Bet&lt;/h2&gt;
&lt;p&gt;Against this backdrop, the financial condition of the leading AI companies becomes more significant. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.wheresyoured.at/news-openai-had-a-negative-122-operating-margin-in-q1-2026-and-chatgpt-growth-has-stalled/&quot;&gt;OpenAI’s Q1 2026 financials, reported by The Information&lt;/a&gt;, showed revenue of $5.7 billion alongside an operating margin of negative 122%, meaning the company lost $1.22 for every dollar of revenue it generated. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sacra.com/c/openai/&quot;&gt;Sacra’s analysis of OpenAI’s projected cash burn&lt;/a&gt; puts the 2026 figure at approximately $27 billion, rising to approximately $63 billion in 2027. The company does not reach cash-flow positive until 2030 on its own projections.&lt;/p&gt;
&lt;p&gt;OpenAI’s total net loss for 2025 was approximately $39 billion, up from $5 billion in 2024. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.tradingkey.com/analysis/stocks/us-stocks/261972340-openai-cost-revenue-q1-2026-tradingkey&quot;&gt;The company has committed to $665 billion in long-term computing procurement&lt;/a&gt;, nearly twenty times its total 2025 expenditures. These commitments are not yet on the balance sheet but represent a very real future obligation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://sacra.com/c/openai/&quot;&gt;OpenAI filed confidentially with the SEC for an IPO on June 8, 2026&lt;/a&gt;, targeting a valuation exceeding $1 trillion, having raised $122 billion at an $852 billion post-money valuation in March 2026.&lt;/p&gt;
&lt;p&gt;Anthropic’s position is presented as more favourable, with claims of approaching non-GAAP operating profitability in Q2 2026. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.wheresyoured.at/news-openai-had-a-negative-122-operating-margin-in-q1-2026-and-chatgpt-growth-has-stalled/&quot;&gt;Ed Zitron’s analysis of those figures&lt;/a&gt; argues that the profitability claim depends on accounting adjustments and a specific two-month compute cost reduction rather than underlying operational improvement, and that both Anthropic and OpenAI benefit from heavily discounted compute supplied by their major backers, meaning their stated losses already understate the true cost of operations.&lt;/p&gt;
&lt;p&gt;The model these companies have been running is now clearly visible. Raise equity from major tech companies and venture capital. Offer services at a loss to build user base and dependency. Switch to debt markets when equity sources slow down. Now that debt markets are also under strain as AI debt has saturated the corporate bond market and borrowing costs have risen, the remaining option is to access public market capital through IPOs before the financial picture deteriorates further.&lt;/p&gt;
&lt;h2 id=&quot;the-ipos-and-what-they-tell-you&quot;&gt;The IPOs and What They Tell You&lt;/h2&gt;
&lt;p&gt;OpenAI’s confidential S-1 filing and Anthropic’s parallel preparation for public markets are best understood in this context. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://futuresearch.ai/openai-financial-forecast/&quot;&gt;Financial analysis from FutureSearch&lt;/a&gt; estimates OpenAI’s 2026 GAAP losses at $25 to $26 billion, approximately 80% higher than the widely cited non-GAAP figure. The projected market cap of roughly $860 billion at IPO is almost exactly the current private valuation, suggesting public markets are being asked to validate a price that private markets already set rather than providing genuine price discovery.&lt;/p&gt;
&lt;p&gt;The cash that a successful IPO generates is meaningful but finite. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://marketwise.com/investing/openai-losses-surge-to-21-billion-as-ai-bubble-grows-bigger/&quot;&gt;MarketWise’s analysis&lt;/a&gt; notes that OpenAI’s Q1 2026 cash burn consumed a large fraction of its then-available cash in a single quarter. At current burn rates, IPO proceeds provide runway measured in years rather than a path to sustainability. Both companies are also reportedly considering significant token price cuts to compete for enterprise market share, a move that would widen already substantial losses further.&lt;/p&gt;
&lt;p&gt;This is the classic pattern of a company that needs a liquidity event before its financial trajectory becomes publicly visible. The IPO is not evidence of success. It is a mechanism for distributing the risk of the current financial position across a broader base of investors before the question of whether the underlying bet pays off is resolved.&lt;/p&gt;
&lt;p&gt;Whether these IPOs succeed or fail matters for reasons beyond the companies themselves. AI debt now represents the largest single segment in the investment-grade bond market, exceeding US banks. The financial systems that price and hold that debt are exposed to the same underlying question: does AI generate enough economic value at sufficient scale to service the obligations written against it?&lt;/p&gt;
&lt;h2 id=&quot;what-this-means-in-practice&quot;&gt;What This Means in Practice&lt;/h2&gt;
&lt;p&gt;The picture that emerges from laying these pieces together is not one of a technology that failed. It is one of an industry that borrowed against a specific future and is now learning that the future is arriving more slowly and more expensively than the borrowing assumed.&lt;/p&gt;
&lt;p&gt;AI is genuinely useful in many contexts. It changes how specific tasks get done and accelerates certain kinds of work in certain conditions. The research supporting meaningful productivity gains in narrow, well-defined tasks is real. None of that requires disputing.&lt;/p&gt;
&lt;p&gt;What the evidence does not support is the premise that funded the debt that AI would replace human labour at the scale and speed required to generate the returns needed to service it. Oxford Economics found no displacement at scale. METR found AI slows down experienced developers on complex tasks. The companies that bet hardest on near-term replacement, from Starbucks to various enterprise software vendors, have quietly rehired the workers they let go. Meanwhile the actual cost of operating AI at scale, now becoming visible as the subsidy era ends, is higher than the human labour it was supposed to replace.&lt;/p&gt;
&lt;p&gt;The U-turns on the jobs narrative, the token-based pricing shift, the IPO timing, the desperate search for government infrastructure contracts and backing: these are not separate stories. They are responses to the same underlying problem.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The question that matters now is not whether AI is impressive. It is whether the industry structured around it can generate enough real economic value to service the debt written against a premise that the evidence has not validated. That question does not yet have a clear answer, and the next eighteen months will likely define it.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The most honest position is uncertainty. AI may find the use cases that justify the investment. The technology may improve in ways that change the reliability picture. The companies may find paths to profitability that current projections do not capture. All of those remain possible.&lt;/p&gt;
&lt;p&gt;What is no longer possible is the claim that the bet is paying off on the original timeline. It is not, and the people who made it are now telling you so through their actions, even as their words continue to describe a different story.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>artificial-intelligence</category><category>llm</category><category>ipo</category><category>anthropic-claude</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>A Tech CEO Just Banned All AI Across His Entire Company. Here Is Why He Is Not Entirely Wrong.</title><link>https://soursen.org/articles/a-tech-ceo-just-banned-all-ai-across-his-entire-company-here-is-why-he/</link><guid isPermaLink="true">https://soursen.org/articles/a-tech-ceo-just-banned-all-ai-across-his-entire-company-here-is-why-he/</guid><description>Let me tell you about a conversation I did not expect to have.</description><pubDate>Fri, 19 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Let me tell you about a conversation I did not expect to have.&lt;/p&gt;
&lt;p&gt;A CEO of a mid-sized tech company reached out recently with a policy he had just put in place. No AI. Not limited to certain departments. Not restricted to a specific category of tools. No exceptions for engineering, no carve-outs for sales, no trial period. A complete and immediate ban on every large language model across every function in the company.&lt;/p&gt;
&lt;p&gt;My first instinct was to write it off as an overcorrection. My second instinct was to actually read what he sent me. What followed was one of the more honest accounts of what happens when AI adoption runs faster than the judgment required to use it well, and it is worth sitting with.&lt;/p&gt;
&lt;p&gt;This is not a story about AI being bad. It is a story about what happens when an organisation treats a powerful tool as a substitute for thinking rather than a support for it.&lt;/p&gt;
&lt;h2 id=&quot;why-a-total-ban-is-even-conceivable-right-now&quot;&gt;Why a Total Ban Is Even Conceivable Right Now?&lt;/h2&gt;
&lt;p&gt;To understand why a CEO would go this far, you need to understand the moment we are in. The past 2 years have been defined by a gap between the story companies tell about AI and the reality they are living with. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://hbr.org/2026/01/companies-are-laying-off-workers-because-of-ais-potential-not-its-performance&quot;&gt;Harvard Business Review published research in January 2026&lt;/a&gt; showing that companies are making layoff decisions based on AI’s potential rather than its demonstrated performance. The headline was blunt: firms are cutting people based on what they hope AI will do, not what it is currently doing.&lt;/p&gt;
&lt;p&gt;The cost picture is getting harder to ignore. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2026/04/28/nvidia-executive-cost-of-ai-is-greater-than-cost-of-employees/&quot;&gt;Fortune reported in April 2026&lt;/a&gt; that an Nvidia executive stated plainly that the cost of compute for his team was far beyond the cost of its employees. Uber’s CTO told The Information the same month that his company had burned through its entire 2026 AI budget in four months, describing the experience as going back to the drawing board. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techbrew.com/stories/2026/05/01/ai-makes-human-labor-look-cheap-again&quot;&gt;A Goldman Sachs research note&lt;/a&gt; put it directly: companies are overrunning their initial budgets for AI inference by orders of magnitude. Some software companies are already spending roughly 10% of their total engineering labour costs on AI tools alone.&lt;/p&gt;
&lt;p&gt;The productivity picture is equally complicated. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2026/02/17/ai-productivity-paradox-ceo-study-robert-solow-information-technology-age/&quot;&gt;A Fortune report from February 2026&lt;/a&gt; surveyed thousands of CEOs and found that while about two thirds reported using AI, actual usage averaged only around 1.5 hours per week per executive. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2026/02/17/ai-productivity-paradox-ceo-study-robert-solow-information-technology-age/&quot;&gt;ManpowerGroup’s 2026 Global Talent Barometer&lt;/a&gt;, which surveyed nearly 14,000 workers across 19 countries, found that while regular AI use increased 13% in 2025, confidence in the technology’s actual utility fell 18% over the same period.&lt;/p&gt;
&lt;p&gt;The reliability data is not flattering either. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai&quot;&gt;Stanford HAI’s 2026 AI Index&lt;/a&gt; found that hallucination rates across 26 leading models range from 22% to 94% depending on the task. The AI Incident Database recorded 362 documented incidents in 2025, up 55% from 233 in 2024. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://suprmind.ai/hub/insights/ai-hallucination-statistics-research-report-2026/&quot;&gt;A separate analysis by Suprmind&lt;/a&gt; found that 51% of organisations using AI have experienced at least one negative consequence, with nearly a third specifically attributing problems to AI inaccuracy.&lt;/p&gt;
&lt;p&gt;This is the environment in which the CEO I spoke with was operating. Not an outlier. Not a technophobe. A person running a real business who had watched the gap between the AI promise and the AI reality widen until it started costing him customers, credibility, and months of engineering time.&lt;/p&gt;
&lt;h2 id=&quot;4-specific-failures-that-led-to-the-ban&quot;&gt;4 Specific Failures That Led to the Ban&lt;/h2&gt;
&lt;p&gt;The ban did not come from ideology. It came from 4 specific, documented failures inside the business. Each one on its own might have been manageable. Together they created a pattern the CEO decided he needed to break completely.&lt;/p&gt;
&lt;h2 id=&quot;one-the-email-that-should-never-have-left-the-building&quot;&gt;One: The Email That Should Never Have Left the Building&lt;/h2&gt;
&lt;p&gt;The first crack appeared when a customer support agent forwarded a complaint to the CEO about an email the customer had received. The email was supposedly a personal message from a member of the company’s customer-facing team. It was not. It was AI-generated, unedited, factually imprecise, and written in the kind of language that makes people feel they are being processed rather than helped.&lt;/p&gt;
&lt;p&gt;What made this worse was not the single email. It was what the CEO found when he started looking. He asked his team for more examples of AI-generated content that had reached customers without meaningful human review. What came back was extensive. Emails with errors. Marketing copy so laden with AI-generated filler phrases it communicated nothing. A pattern of output leaving the company that no one had been accountable for reviewing.&lt;/p&gt;
&lt;p&gt;This is not an isolated phenomenon. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://markets.financialcontent.com/lightport.lightport1/article/bizwire-2025-6-17-ai-underdelivers-at-work-62-of-employees-say-ai-is-significantly-overhyped-but-86-admit-they-arent-using-ai-to-its-full-potential-according-to-new-research-from-goto&quot;&gt;A GoTo survey of 2,500 employees conducted in 2025&lt;/a&gt; found that 76% of workers said AI tools frequently produce outputs that require significant correction before use. The same survey found that 86% of employees are not using AI to its full potential, partly because they do not trust its accuracy. The result is a specific failure mode where people use the tool anyway, trust it more than they should, and stop applying the editorial judgment that would catch the errors.&lt;/p&gt;
&lt;h2 id=&quot;two-the-prospects-who-did-not-exist&quot;&gt;Two: The Prospects Who Did Not Exist&lt;/h2&gt;
&lt;p&gt;The CEO had scheduled a review with the sales team to understand why their close rate had been falling through 2026. When they pulled the data, one number stood out immediately. The rate of disqualified prospects had nearly doubled compared to the prior year.&lt;/p&gt;
&lt;p&gt;The cause took some digging. The sales team had been using AI to assist with research and prospect identification. The AI had cast a significantly wider net, identifying far more potential targets than the team could have found manually. The problem was that many of those targets were not qualified leads. Some were companies that had gone out of business. In at least one case, the company the AI had identified did not exist at all. The model had effectively inferred a company into existence based on patterns in the data it had seen.&lt;/p&gt;
&lt;p&gt;This is the hallucination problem in a sales context. It does not announce itself. The output looks like a normal research document with company names, descriptions, and contact details. There is no error message. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://suprmind.ai/hub/insights/ai-hallucination-statistics-research-report-2026/&quot;&gt;Columbia Journalism Review found&lt;/a&gt; that 8 leading generative search tools gave incorrect answers on more than 60% of tested queries where sources could be verified. When that failure mode reaches your sales pipeline, you waste the time of everyone who touches those leads, and you never quite know which part of your pipeline is built on solid ground.&lt;/p&gt;
&lt;h2 id=&quot;three-the-agent-that-invented-its-own-data&quot;&gt;Three: The Agent That Invented Its Own Data&lt;/h2&gt;
&lt;p&gt;One of the company’s developers had built an AI agent to automate a recurring reporting task. The agent would scrape data from a specific source, structure it, and generate a report that fed into a business decision process. Nobody had mandated the agent. Nobody had formally reviewed it. It had just appeared in the workflow and people had started relying on it.&lt;/p&gt;
&lt;p&gt;Then the data source the agent depended on changed its structure. The agent could no longer scrape what it needed rather than failing loudly and stopping, the agent continued generating the report using whatever it could access, treating the missing data as if an entire business line had collapsed to zero. The output looked like a legitimate report showing a catastrophic decline in a part of the business.&lt;/p&gt;
&lt;p&gt;What followed was a week of panic and manual re-analysis before someone connected the dots. The developer who had built the agent was not the person responsible for the report it generated. He was quietly re-running months of reports by hand while trying to understand what had gone wrong, and not telling anyone why.&lt;/p&gt;
&lt;p&gt;This failure pattern is well documented. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026&quot;&gt;The International AI Safety Report 2026&lt;/a&gt;, authored by more than 100 domain experts, identifies AI systems being derailed by simple errors during multi-step tasks as a core reliability challenge in current production deployments. The specific problem of agents that continue operating confidently after losing access to the data they need is a known failure mode, not an edge case.&lt;/p&gt;
&lt;h2 id=&quot;four-two-versions-of-the-same-module-two-databases-no-one-in-charge&quot;&gt;Four: Two Versions of the Same Module, Two Databases, No One in Charge&lt;/h2&gt;
&lt;p&gt;This was the one that ended the conversation about whether to ban AI. The CEO discovered that two developers, working independently and both using AI-assisted coding tools, had built two different versions of the same module. Each version connected to a different database. Each database contained a different set of data stored in incompatible schemas. Neither developer knew the other version existed.&lt;/p&gt;
&lt;p&gt;The AI coding tools had made both developers faster and more confident. They had also removed the kind of friction that forces people to check what already exists before building something new. The result was a codebase with an invisible fork in it that nobody had explicitly created and nobody had signed off on.&lt;/p&gt;
&lt;p&gt;This is the vibe coding problem at its most concrete. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.techbrew.com/stories/2026/05/01/ai-makes-human-labor-look-cheap-again&quot;&gt;Tech Brew reported in May 2026&lt;/a&gt; that around 70% of code committed at Uber now originates with AI assistance. That same report noted Microsoft had to cap individual engineer AI tool usage after discovering some engineers were spending between $500 and $2,000 per month. The speed AI enables is real. So is the technical debt that accumulates when speed replaces the review processes that normally prevent this kind of duplication.&lt;/p&gt;
&lt;h2 id=&quot;why-the-ban-makes-a-certain-kind-of-sense&quot;&gt;Why the Ban Makes a Certain Kind of Sense&lt;/h2&gt;
&lt;p&gt;Reading those 4 failures in sequence, the ban becomes easier to understand. This was not a CEO who decided AI was bad in principle. This was a CEO who had watched AI do something specific and damaging inside his organisation, which was not the tool itself, but what the tool did to his people’s judgment.&lt;/p&gt;
&lt;p&gt;Without AI, a sales researcher who cannot find a prospect moves on or asks a colleague. With AI, they have a plausible-looking result in seconds, and the plausibility of the format suppresses the instinct to question the content. Without AI, a developer who needs a module searches the codebase first. With AI, they can generate a module faster than they can search for one, so they generate it.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;AI did not make his team incompetent. It made their existing lapses in judgment faster, more confident, and harder to catch. That is a specific problem with a specific character, and it is not solved by better prompting.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The CEO said something in our exchange that I have been thinking about since. He told me the ban was partly permanent because he needed to expose the people who had been using AI to hide. The people whose work had looked competent because they were good at prompting AI, but whose underlying judgment had quietly atrophied. He wanted to see who could still do the work without the tool. That is not an unreasonable thing to want to know. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.thomsonreuters.com/en/insights/articles/the-human-side-of-ai-the-growing-risks-of-ubiquitous-use-of-ai-on-talent&quot;&gt;Thomson Reuters research published in February 2026&lt;/a&gt; raised exactly this concern, citing what experts describe as agency decay: the gradual erosion of independent problem-solving capacity when people consistently offload cognitive work to AI before developing their own judgment on a task.&lt;/p&gt;
&lt;h2 id=&quot;why-the-ban-will-not-work-permanently&quot;&gt;Why the Ban Will Not Work Permanently&lt;/h2&gt;
&lt;p&gt;Being honest about the logic behind the ban also means being honest about its limits.&lt;/p&gt;
&lt;p&gt;The failures described above were not failures of AI. They were failures of process, oversight, and accountability that AI made more visible and more expensive. A bad sales researcher producing made-up prospect lists with AI is a worse version of a bad sales researcher producing made-up prospect lists without it. Removing the AI does not produce a good sales researcher. It produces a bad one who works more slowly.&lt;/p&gt;
&lt;p&gt;The competitive math is also not going away. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.ynetnews.com/tech-and-digital/article/syglv32twl&quot;&gt;Gartner projects global IT spending will reach $6.31 trillion in 2026&lt;/a&gt;, a 13.5% increase driven primarily by AI infrastructure. Companies that figure out how to use AI with proper oversight will outpace those that do not at some point, even if that point is further out than the hype suggests. A ban that is genuinely permanent puts a company on the wrong side of that gap eventually.&lt;/p&gt;
&lt;p&gt;The more interesting finding from &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2026/05/11/ai-automation-layoffs-gartner-study-roi/&quot;&gt;the Gartner study on AI-driven layoffs&lt;/a&gt; is that 80% of companies surveyed reported workforce reductions tied to AI, yet there was no correlation between those reductions and higher ROI. Companies cut people and invested in AI. The ROI did not materialise at the rate expected. What that data suggests is not that AI does not work. It suggests that deploying AI without building the human judgment infrastructure around it does not work.&lt;/p&gt;
&lt;p&gt;The CEO I spoke with understands this. He said he does not know if the ban is permanent. He knows he needs to say it is permanent for now, both to flush out the accountability problems and to buy himself the time to figure out what a more disciplined approach actually looks like. That is not a long-term strategy. It is a pause with a specific purpose.&lt;/p&gt;
&lt;h2 id=&quot;what-the-right-version-of-this-looks-like&quot;&gt;What the Right Version of This Looks Like&lt;/h2&gt;
&lt;p&gt;The problem this CEO ran into is not unique to his company. It is the predictable result of treating AI adoption as a goal rather than treating good output as the goal. The tool gets deployed. The speed increases. The review processes that catch errors get skipped because the tool was supposed to eliminate the need for them. The errors accumulate until they cost something real.&lt;/p&gt;
&lt;p&gt;The companies getting this right are treating AI the same way they would treat any powerful new capability: with explicit boundaries, clear ownership, and review processes that survive the temptation to skip them when things are moving fast.&lt;/p&gt;
&lt;p&gt;That means defining which tasks benefit from AI assistance and which ones require the kind of judgment that AI does not reliably provide. It means keeping humans accountable for AI output rather than treating AI output as something that exists outside the normal accountability structure. It means building verification into the workflow rather than adding it at the end when a problem has already reached a customer.&lt;/p&gt;
&lt;p&gt;It also means accepting that some of the friction AI removes is friction that was doing useful work. The developer who searches the codebase before writing new code is doing something valuable even when it is slower. The sales researcher who cannot find a prospect and picks up the phone is building something AI cannot replicate. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.thomsonreuters.com/en/insights/articles/the-human-side-of-ai-the-growing-risks-of-ubiquitous-use-of-ai-on-talent&quot;&gt;The EY 2025 Work Reimagined Survey&lt;/a&gt;, which gathered data from 15,000 employees and 1,500 employers across 29 countries, found that 64% of employees report a perceived increase in workloads over the past year despite AI adoption, with only 5% describing themselves as maximising AI to genuinely transform their work. The tool is everywhere. The judgment about how to use it is not.&lt;/p&gt;
&lt;p&gt;The CEO who banned AI entirely got one thing exactly right. He identified that the problem was not the tool. It was what the tool did to the people using it when there were no real consequences for bad output. The ban is his way of restoring those consequences. Whether it is the right method is a reasonable question. That the underlying problem is real is not.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>artificial-intelligence</category><category>ceo</category><category>llm</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>You Are Using Claude Wrong. Here Is How the Best Product Managers Actually Use It.</title><link>https://soursen.org/articles/you-are-using-claude-wrong-here-is-how-the-best-product-managers/</link><guid isPermaLink="true">https://soursen.org/articles/you-are-using-claude-wrong-here-is-how-the-best-product-managers/</guid><description>There is a gap between how most product managers use AI and how the best ones use it. The majority treat it like a faster search engine. They open a chat window, type a question, get a generic answer, and wonder why the output never quite fits.</description><pubDate>Tue, 16 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;There is a gap between how most product managers use AI and how the best ones use it. The majority treat it like a faster search engine. They open a chat window, type a question, get a generic answer, and wonder why the output never quite fits. The ones getting genuinely useful results are doing something different. They are giving the AI a standing brief, a set of instructions that travel with every session so they never have to explain context again.&lt;/p&gt;
&lt;p&gt;That mechanism is called a Claude Skill, and specifically, a file called SKILL.md. It is not a plugin, not a paid feature, and not something that requires technical knowledge to set up. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.anthropic.com/en/docs/claude-code/skills&quot;&gt;Anthropic’s documentation on Claude Skills&lt;/a&gt; describes it as a folder-based instruction system that Claude reads before responding. In practice, it is the difference between an AI assistant that produces generic output and one that already knows your team’s format, your product’s context, and your definition of good work.&lt;/p&gt;
&lt;p&gt;This piece covers what Skills are, why they matter more for product managers than almost any other role, and how to build one from scratch that actually improves your output.&lt;/p&gt;
&lt;h2 id=&quot;why-product-management-is-the-perfect-use-case-for-ai-skills&quot;&gt;Why Product Management Is the Perfect Use Case for AI Skills&lt;/h2&gt;
&lt;p&gt;Think about what a typical week looks like as a PM. You write PRDs in a specific format your team has agreed on. You score feature requests using the same prioritisation framework every time. You synthesise user interview notes into structured insights using the same analytical approach. You draft release notes in your product’s voice. You send stakeholder updates that follow a consistent structure and level of detail.&lt;/p&gt;
&lt;p&gt;Almost all of it is pattern application at scale. You take a large surface area of different problems and run a small set of established formats and frameworks against them. That is precisely what AI is well suited for, applying known patterns consistently across varied inputs.&lt;/p&gt;
&lt;p&gt;The problem with how most PMs currently use AI is that every session starts at zero. You open Claude, begin typing, and spend the first several exchanges re-establishing context. Here is our PRD format. Here is how we score features. Here is the voice we use in release notes. The AI produces something reasonable, but it required significant setup, and the next session you do it all over again.&lt;/p&gt;
&lt;p&gt;The data on structured AI usage is worth understanding clearly. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://survey.stackoverflow.co/2025/ai/&quot;&gt;The Stack Overflow 2025 Developer Survey&lt;/a&gt;, covering over 49,000 respondents across 177 countries, found that 84% of developers now use or plan to use AI tools, but only 52% report that AI has had a positive effect on their productivity. The gap between those two numbers is largely explained by how people use the tools, not whether they use them. Developers who use AI daily report significantly higher satisfaction and productivity gains than those using it infrequently, because they have learned how to structure their interactions.&lt;/p&gt;
&lt;p&gt;Product management work is at least as repetitive and structured as code review and documentation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.getpanto.ai/blog/ai-coding-assistant-statistics&quot;&gt;DX’s analysis of over 135,000 developers&lt;/a&gt; found that AI tools save an average of 3.6 hr/week when used consistently within structured workflows. Daily users saw even larger gains and significantly higher throughput on their core deliverables. The pattern holds across knowledge work generally: the productivity benefit comes from systematic integration, not occasional use.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A prompt is a sticky note you write and throw away. A Skill is a trained colleague who already knows how your team works, shows up the moment you need them, and never has to be onboarded again.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The broader picture is consistent. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://openai.com/business/guides-and-resources/the-state-of-enterprise-ai-2025-report/&quot;&gt;OpenAI’s 2025 State of Enterprise AI report&lt;/a&gt;, drawing on usage data from over one million business customers and surveys across 9,000 workers, found that 75% of workers who use AI at work report improvements in the speed or quality of their output. The same report found that 75% of users say they can now complete tasks they previously could not perform at all. The ceiling on what structured AI use enables is still being discovered.&lt;/p&gt;
&lt;h2 id=&quot;what-a-claude-skill-actually-is&quot;&gt;What a Claude Skill Actually Is&lt;/h2&gt;
&lt;p&gt;A Skill is a folder. Inside that folder lives a file called SKILL.md. Claude reads this file when you invoke the skill and follows the instructions inside it for that session. That is the entire mechanism. No app to install, no platform to configure, no code to write. The &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.anthropic.com/en/docs/claude-code/skills&quot;&gt;official Claude Skills documentation&lt;/a&gt; walks through the technical setup, which takes roughly ten minutes the first time you do it.&lt;/p&gt;
&lt;p&gt;The SKILL.md file contains two things. A short description that tells Claude when to activate the skill, written in plain language that specifies the trigger conditions and the type of task. A longer set of instructions that Claude follows once activated, written like you are briefing a capable new team member who needs to understand your standards before starting work.&lt;/p&gt;
&lt;p&gt;The folder can also contain supporting files. A template document Claude references when generating output. Example outputs that show Claude the quality standard to match. A context file with your product’s north star metric, target users, and voice guidelines. These supporting files are what take a basic skill from useful to genuinely powerful, because you are giving Claude institutional knowledge rather than just instructions.&lt;/p&gt;
&lt;p&gt;The cleanest mental model is this. Right now, every time you open Claude, you are working with a capable generalist who knows nothing about your specific context. A Skill turns that generalist into a specialist who already knows your team’s format, your product’s constraints, and your quality bar before you type your first word.&lt;/p&gt;
&lt;h2 id=&quot;the-5-pm-tasks-that-map-almost-perfectly-to-skills&quot;&gt;The 5 PM Tasks That Map Almost Perfectly to Skills&lt;/h2&gt;
&lt;h2 id=&quot;prd-generation&quot;&gt;PRD Generation&lt;/h2&gt;
&lt;p&gt;Writing a product requirements document is the most obvious and highest-value use case. The format is fixed. The sections are established. The quality bar is consistent. The only thing that changes is the input, the problem statement or feature idea you are speccing out.&lt;/p&gt;
&lt;p&gt;A PRD skill tells Claude exactly which sections to include, how long each should be, what language register to use, when to ask clarifying questions before starting, and what assumptions to flag for your review. You paste in a problem statement. Claude produces a structured draft in your team’s format. You spend your time on the thinking inside the document rather than the structure around it.&lt;/p&gt;
&lt;h2 id=&quot;user-research-synthesis&quot;&gt;User Research Synthesis&lt;/h2&gt;
&lt;p&gt;Qualitative research synthesis is time-consuming and pattern-heavy, which makes it well suited for Skills. Most of what takes time in synthesis is structural: organising quotes into themes, tagging observations by user type, writing up jobs-to-be-done statements in consistent language. A research synthesis skill handles all of that structurally, freeing the PM to focus on interpretation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/superagency-in-the-workplace-empowering-people-to-unlock-ais-full-potential-at-work&quot;&gt;McKinsey’s Superagency in the Workplace report&lt;/a&gt; identified document drafting and research synthesis as two of the knowledge work tasks where AI delivers the clearest and most consistent time savings.&lt;/p&gt;
&lt;h2 id=&quot;feature-prioritisation-scoring&quot;&gt;Feature Prioritisation Scoring&lt;/h2&gt;
&lt;p&gt;Whether your team uses RICE, ICE, Kano, or a custom framework, applying it consistently across a large backlog is tedious and prone to the kind of subtle inconsistency that makes prioritisation debates harder than they need to be. A prioritisation skill applies your exact scoring criteria to each feature request with the same weighting and the same interpretation of each factor. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://survey.stackoverflow.co/2025/ai/&quot;&gt;The Stack Overflow 2025 survey&lt;/a&gt; found that 69% of developers using AI agents report an increase in personal productivity on specific tasks. Repeatable analytical work like backlog scoring is exactly the category where that productivity gain shows up most reliably.&lt;/p&gt;
&lt;h2 id=&quot;release-notes&quot;&gt;Release Notes&lt;/h2&gt;
&lt;p&gt;Release notes require a specific voice, a consistent level of technical detail calibrated to your audience, and the discipline to focus on user benefit rather than engineering implementation. A release note skill takes commit messages or Jira ticket summaries as input and produces user-facing notes in your product’s established voice. The output is consistent across every release regardless of which PM is writing it.&lt;/p&gt;
&lt;h2 id=&quot;stakeholder-updates&quot;&gt;Stakeholder Updates&lt;/h2&gt;
&lt;p&gt;Weekly status updates follow a predictable format: what shipped, what is in progress, what is blocked, what the metrics look like. A stakeholder update skill takes your current sprint data and produces a formatted update in whatever structure your organisation expects. The cognitive load of drafting is eliminated. The time goes to verifying accuracy rather than writing from scratch.&lt;/p&gt;
&lt;h2 id=&quot;how-to-write-a-skillmd-file-the-full-anatomy&quot;&gt;How to Write a SKILL.md File: The Full Anatomy&lt;/h2&gt;
&lt;h2 id=&quot;part-1-the-description-block&quot;&gt;Part 1: The Description Block&lt;/h2&gt;
&lt;p&gt;Every SKILL.md file starts with a YAML front matter block at the top. Two fields are required: a name and a description. The description is the most important part of the entire file because it is what Claude reads to decide whether to activate your skill for a given request.&lt;/p&gt;
&lt;p&gt;Write the description as if you are training a junior PM on exactly which situations call for this skill. Be specific about the trigger phrases, the types of requests it handles, and the output it produces. Vague descriptions produce missed activations. The more precisely you describe when the skill applies, the more reliably Claude will activate it at the right moment.&lt;/p&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;name: prd-generator&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;description: &amp;gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Drafts a product requirements document in our team&amp;#39;s format. Use this skill when the user asks to write a PRD, draft requirements, spec out a feature, or turn a problem statement into a structured document. Also activates when the user shares a customer complaint or research finding and asks what to build.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;part-2-when-to-use-this-skill&quot;&gt;Part 2: When to Use This Skill&lt;/h2&gt;
&lt;p&gt;After the front matter, open with a brief section that gives Claude the complete picture of when this skill applies and what input it needs to get started. Include any clarification it should request before proceeding if the input is ambiguous.&lt;/p&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;## When to use this skill &lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;The user has a problem statement, feature idea, customer complaint, or rough draft and wants a structured PRD that matches the team&amp;#39;s format and quality bar.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;If the problem statement is vague, ask for: the target user, the scope boundary, and the primary success metric before starting. Do not begin drafting until these are clear.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;part-3-the-instructions&quot;&gt;Part 3: The Instructions&lt;/h2&gt;
&lt;p&gt;This is the core of your skill. Write numbered instructions as if you are onboarding a capable new team member who needs to understand your process before starting. Step by step ordering matters because it tells Claude the sequence in which to approach the task, not just the components to include.&lt;/p&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;## Instructions &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;1. Read the input carefully. Identify the core problem, the affected user, and the proposed direction if one exists.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;2. If any of the following are missing, ask before drafting:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;   - Which user segment experiences this problem&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;   - What success looks like in measurable terms&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;   - What is explicitly out of scope &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;3. Draft the PRD using the format specified below. &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;4. Use plain language. Avoid PM jargon unless the user has used it first. Write for an engineer who needs to build this, not an executive who needs to approve it.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;5. Mark every assumption you make with [ASSUMPTION] so the user can verify or correct it before sharing the doc. &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;6. End with a list of open questions that need answering before the team begins work.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;part-4-supporting-files&quot;&gt;Part 4: Supporting Files&lt;/h2&gt;
&lt;p&gt;A Skill folder can contain more than just the SKILL.md file. This is where the real leverage comes from, because you are giving Claude your institutional knowledge rather than just a set of instructions.&lt;/p&gt;
&lt;p&gt;A template file like prd-template.md gives Claude the exact structure to populate rather than inferring it from instructions. An examples folder with two or three of your best previous PRDs gives Claude pattern-matching data for the quality standard you expect. A context file called business-context.md can contain your product’s north star metric, current quarterly goals, target user personas, and brand voice guidelines. Referencing this file from your skill instructions with a line like read business-context.md before generating output and align all content to our product principles ensures Claude is working within your actual strategic context.&lt;/p&gt;
&lt;p&gt;The compounding effect of structured context is significant. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://github.blog/news-insights/research/survey-ai-wave-grows/&quot;&gt;GitHub’s research on AI tool adoption&lt;/a&gt;, conducted across 2,000 enterprise developers in four countries, found that developers who had learned to work with structured AI workflows, rather than ad-hoc prompting, reported markedly higher productivity gains and satisfaction scores. The same pattern applies to any knowledge worker who invests in building that structure upfront. The context file is the mechanism for providing that structure systematically rather than rebuilding it every session.&lt;/p&gt;
&lt;h2 id=&quot;a-complete-working-example-the-prd-generator-skill&quot;&gt;A Complete Working Example: The PRD Generator Skill&lt;/h2&gt;
&lt;p&gt;Here is what a production-ready PRD generator SKILL.md looks like in full. This is the kind of file you build once and refine over several weeks as you see how the output compares to your actual standards.&lt;/p&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;---&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;name: prd-generator&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;description: &amp;gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Drafts a product requirements document in our team&amp;#39;s format. Use this skill when the user asks to write a PRD, draft requirements, spec out a feature, or turn a problem statement into a structured doc. Also use when a user shares a customer pain point and wants to know what to build.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;---&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;# PRD Generator &lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;## When to use this skill&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;The user has a problem statement, feature idea, or rough draft and wants a structured PRD that matches our team&amp;#39;s format. &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;## Before you start&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Read business-context.md to understand our product, users, and current strategic priorities. All PRD content should align to the north star metric defined there. &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;## Instructions&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;1. Identify the core problem and the user it affects.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;2. Ask if missing: target user segment, success metric, scope boundary, or launch timeline.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;3. Draft the PRD using the format below.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;4. Keep language concrete. No passive voice.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;5. Flag every assumption with [ASSUMPTION].&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;6. Close with open questions to resolve before build starts.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;## PRD Format &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Problem Statement: One to two sentences. What is broken or missing, for whom, and at what cost to the user or the business. &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Target User: Who experiences this problem. How often. What they currently do instead, and why that solution falls short. &lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Proposed Solution: What we are building. What we are explicitly not building. Why this solution and not an alternative.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Success Primary metric: the single number that moves if this works.&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Guardrail metrics: what we are not allowed to break.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light github-dark&quot; style=&quot;--shiki-light:#24292e;--shiki-dark:#e1e4e8;--shiki-light-bg:#fff;--shiki-dark-bg:#24292e;overflow-x:auto;white-space:pre-wrap;word-wrap:break-word&quot; tabindex=&quot;0&quot; data-language=&quot;text&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span&gt;Open Questions: Unresolved dependencies, unknowns, and decisions that need owner sign-off before the team begins work.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;skills-vs-mcp-understanding-the-difference&quot;&gt;Skills vs MCP: Understanding the Difference&lt;/h2&gt;
&lt;p&gt;If you have been following the AI tooling space, you have likely come across &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.anthropic.com/news/model-context-protocol&quot;&gt;Model Context Protocol&lt;/a&gt;, known as MCP. Both Skills and MCP extend what Claude can do, but they address fundamentally different problems and are not interchangeable.&lt;/p&gt;
&lt;p&gt;A Skill is an internal playbook. It tells Claude how to approach a task, what format to use, what quality bar to meet, and what context to apply. It operates entirely within the conversation. It cannot reach outside the session to fetch live data, call an API, or connect to an external service. Skills are instructions, not integrations.&lt;/p&gt;
&lt;p&gt;MCP is the connection layer between Claude and the outside world. An MCP server lets Claude read your actual &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://linear.app/&quot;&gt;Linear&lt;/a&gt; tickets in real time, query your &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://notion.so/&quot;&gt;Notion&lt;/a&gt; workspace, pull data from your analytics platform, or interact with any external service that has an MCP integration. MCP is about live data access. Skills are about consistent behaviour.&lt;/p&gt;
&lt;p&gt;The two work together rather than replacing each other. A skill tells Claude how to write a PRD in your team’s format. An MCP connection lets Claude pull the relevant customer feedback from your research repository as it writes. The skill governs the output structure and quality. The MCP integration provides the live context the skill operates on.&lt;/p&gt;
&lt;p&gt;The &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.anthropic.com/en/docs/claude-code/mcp&quot;&gt;Anthropic MCP documentation&lt;/a&gt; covers the available integrations and how to connect them. For most PMs getting started, the right sequence is to build your core skills first, get the output quality to the standard you want, and then layer in MCP connections to feed those skills live data from your tools.&lt;/p&gt;
&lt;h2 id=&quot;how-to-get-started-without-overwhelming-yourself&quot;&gt;How to Get Started Without Overwhelming Yourself&lt;/h2&gt;
&lt;p&gt;The mistake most people make when they first encounter Skills is trying to build a comprehensive library immediately. They map out fifteen skills they want, start building all of them simultaneously, and end up with fifteen half-finished files that none of them use consistently.&lt;/p&gt;
&lt;p&gt;Start with one skill. Pick the PM task you repeat most frequently, which for most people is PRD writing, and build only that. Do not move on until you have used it in at least ten real sessions and refined it based on what the output gets wrong.&lt;/p&gt;
&lt;p&gt;Use Claude to help you write the skill itself. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview&quot;&gt;Anthropic’s prompting guide&lt;/a&gt; covers how to structure instruction documents effectively. In practice, the fastest way to get a starting point is to paste two or three of your best previous PRDs into Claude and ask it to generate a SKILL.md file that would produce outputs at that quality standard. The result will not be perfect, but it gives you a draft to refine rather than a blank page.&lt;/p&gt;
&lt;p&gt;Treat the description field as your primary quality lever. If the skill is not activating when you expect it to, the description is almost always the cause. Rewrite it with more specific trigger phrases and more precise task language. Add examples of the kinds of requests that should activate it.&lt;/p&gt;
&lt;p&gt;Build the context file early. A business-context.md document that contains your product’s north star metric, your current quarterly priorities, your target user definitions, and your brand voice guidelines is worth building even before you have a second skill. Every skill you build after can reference it, which means the entire library benefits from that context document automatically.&lt;/p&gt;
&lt;p&gt;The compounding benefit is the point. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://survey.stackoverflow.co/2025/ai/&quot;&gt;The Stack Overflow 2025 Developer Survey&lt;/a&gt; found a clear correlation between usage frequency and productivity gains. Developers who used AI tools daily reported significantly higher satisfaction and output quality than those using them occasionally, specifically because they had developed structured workflows that applied the tools to their most repetitive tasks. The skill library is how you build that structure for product management work.&lt;/p&gt;
&lt;h2 id=&quot;the-shift-that-is-already-happening&quot;&gt;The Shift That Is Already Happening&lt;/h2&gt;
&lt;p&gt;Every major PM tool is going AI-native in 2026. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://linear.app/blog/ai&quot;&gt;Linear’s AI features&lt;/a&gt; now generate issue descriptions and suggest priorities. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.figma.com/ai/&quot;&gt;Figma’s AI capabilities&lt;/a&gt; assist with design iteration and content generation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.notion.so/product/ai&quot;&gt;Notion AI&lt;/a&gt; drafts documents and synthesises notes across your workspace. These are tool-specific implementations, which means the AI behaves differently depending on which tool you are in, and none of them carry your specific context from one to another.&lt;/p&gt;
&lt;p&gt;A Claude Skills library sits above all of that. It is a workflow layer you own, that carries your institutional context across every tool and every session, that compounds in value as you refine it, and that no tool migration can take away from you. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/superagency-in-the-workplace-empowering-people-to-unlock-ais-full-potential-at-work&quot;&gt;McKinsey’s research on AI in the workplace&lt;/a&gt; found that organisations where employees use AI daily and with structured workflows are far ahead of those where usage is fragmented across departments without coordination. The individual equivalent of that is building your own workflow layer rather than depending on whatever each tool happens to offer.&lt;/p&gt;
&lt;p&gt;The adoption numbers are clear on the direction. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://survey.stackoverflow.co/2025/ai/&quot;&gt;84% of developers in 2025 use or plan to use AI tools&lt;/a&gt;, up from 76% in 2024 and 70% in 2023. The trend is consistent and accelerating. What is not accelerating at the same rate is structured, intentional use that actually compounds. Most people are using AI generically. The PMs who build a skills library now are building a compounding advantage over those who continue relying on tool-native AI implementations that reset context with every session.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>artificial-intelligence</category><category>claude</category><category>product-management</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Prompt Engineering Is Dead. Here Is What Replaced It.</title><link>https://soursen.org/articles/prompt-engineering-is-dead-here-is-what-replaced-it/</link><guid isPermaLink="true">https://soursen.org/articles/prompt-engineering-is-dead-here-is-what-replaced-it/</guid><description>I want to share something that took me an embarrassingly long time to figure out, even though the evidence was sitting right in front of me the whole time.</description><pubDate>Tue, 09 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;I want to share something that took me an embarrassingly long time to figure out, even though the evidence was sitting right in front of me the whole time.&lt;/p&gt;
&lt;p&gt;We spent months debating model choices. GPT-4o versus Claude 3.5 Sonnet versus Gemini. Every switch was a 2 to 3 week engineering project. Re-running evaluations. Recalibrating prompts. Fixing regressions that appeared out of nowhere. Each switch felt significant because it was expensive.&lt;/p&gt;
&lt;p&gt;And then we made a change that had nothing to do with the model at all. We reorganised how user profile information was structured before it entered the context window. Same model. Same task. The intent classification accuracy moved more than the last two model upgrades combined.&lt;/p&gt;
&lt;p&gt;That was the moment I realised we had been optimising the wrong thing.&lt;/p&gt;
&lt;p&gt;The thing that actually separates AI products that work from AI products that disappoint is not the model underneath them. It is what those products put into the model’s context window at each call, in what form, in what order, retrieved through what logic. That discipline has a name now: context engineering. And I think it is the most underrated and most important concept in AI product development right now.&lt;/p&gt;
&lt;p&gt;Let me show you exactly what I mean, and then, because understanding the concept is only half the value, I want to give you a practical guide to finding and landing the roles that are being created around this discipline.&lt;/p&gt;
&lt;h2 id=&quot;why-everyone-is-still-arguing-about-the-wrong-thing&quot;&gt;Why Everyone Is Still Arguing About the Wrong Thing&lt;/h2&gt;
&lt;p&gt;Here is the conversation happening in almost every AI team right now. Should we use GPT-5 or Claude Sonnet? Is Gemini better for reasoning tasks? What about the new open-source models from DeepSeek? The debate is about models, because models are visible. They have names, version numbers, benchmark scores, and press releases.&lt;/p&gt;
&lt;p&gt;The problem with this conversation is that it is increasingly a debate about a commodity layer.&lt;/p&gt;
&lt;p&gt;Andrej Karpathy, one of the most respected voices in AI engineering, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://x.com/karpathy/status/1816919122807238740&quot;&gt;has been arguing for several years&lt;/a&gt; that the model itself is the least differentiated part of a production AI system. His framing, what you call a prompt is only one slice of what you are actually handing the model. The rest of it, the retrieved documents, the conversation history, the user profile, the tool definitions, the few-shot examples, the structured constraints, is what actually determines whether the system behaves intelligently or not.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://intuitionlabs.ai/articles/context-engineering-vs-prompt-engineering-ai&quot;&gt;Gartner formalised this in mid-2025&lt;/a&gt; with a declaration that made a lot of AI teams uncomfortable context engineering is in, and prompt engineering is out. Their recommendation to enterprise AI leaders was to stop focusing on prompt phrasing and start building context-aware architectures with dynamic data pipelines.&lt;/p&gt;
&lt;p&gt;The numbers from actual deployments back this up. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://datahub.com/blog/context-engineering-vs-prompt-engineering/&quot;&gt;DataHub’s 2026 State of Context Management Report&lt;/a&gt; surveyed IT and data leaders across the industry and found that 82% agreed that prompt engineering alone is no longer sufficient to power AI at scale. The same report found that 95% of data teams plan to invest in context engineering capabilities during 2026. That is not a niche opinion. That is the whole industry arriving at the same conclusion from different directions.&lt;/p&gt;
&lt;p&gt;So what exactly is context engineering? Let me give you the clearest definition I can manage.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;CONTEXT ENGINEERING: A WORKING DEFINITION&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Context engineering is the systematic design of what information enters the model at each call: what is included, what is excluded, in what form, in what order, retrieved through what mechanism, and at what computational cost. Prompt engineering is craft at the level of words. Context engineering is craft at the level of architecture.&lt;/p&gt;
&lt;p&gt;The prompt is one component. Context engineering covers prompt design, retrieval design, memory design, tool design, structured output design, and cache design. It is where all the consequential technical decisions in AI product development converge.&lt;/p&gt;
&lt;h2 id=&quot;what-actually-goes-into-a-single-model-call&quot;&gt;What Actually Goes Into a Single Model Call&lt;/h2&gt;
&lt;p&gt;To understand why context engineering matters so much, you need to see what is actually being sent to the model when a user types something. Most people assume it is just the prompt. It is not even close.&lt;/p&gt;
&lt;p&gt;Take a customer support AI handling a message like ‘my payment did not go through.’ That sounds simple. But a production system handling that message might be assembling something like this before it touches the model:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The system prompt: agent persona, behavioural constraints, formatting rules, escalation guidelines&lt;/li&gt;
&lt;li&gt;Tool definitions: schemas and descriptions for every callable function, such as checking payment status, issuing refunds, escalating to a human&lt;/li&gt;
&lt;li&gt;Few-shot examples: two or three carefully chosen prior conversations that teach the model how to handle edge cases in payment disputes&lt;/li&gt;
&lt;li&gt;User profile: pulled from a database at runtime, account age, subscription tier, past complaint history, preferred language&lt;/li&gt;
&lt;li&gt;Conversation history: the current session’s prior turns, compressed and formatted&lt;/li&gt;
&lt;li&gt;Retrieved knowledge: relevant sections of payment policy documents pulled from a vector store based on the semantic similarity to the user’s message&lt;/li&gt;
&lt;li&gt;The actual user message: the thing the user typed&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;7 components. Assembled in a specific order. Formatted in a specific way. All of that is what the model actually processes. The prompt is one of those seven. The other six are where the product lives.&lt;/p&gt;
&lt;p&gt;Change the order of those components and your outputs change. Change which few-shot examples you include and your outputs change. Change how you compress the conversation history and your outputs change. None of those changes require switching models. All of them require context engineering.&lt;/p&gt;
&lt;p&gt;The teams that have figured this out are the ones whose products feel qualitatively different even when running on the same underlying model as a competitor. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://aispaces.substack.com/p/the-end-of-prompt-engineering-why&quot;&gt;Enterprise AI implementations at Salesforce, ServiceNow, and Microsoft&lt;/a&gt; that actually work are almost entirely context systems, CRM history, customer data, policy documents, past tickets, all injected at runtime. The model is the reasoning engine. The context system is the product.&lt;/p&gt;
&lt;h2 id=&quot;the-five-disciplines-that-make-up-context-engineering&quot;&gt;The Five Disciplines That Make Up Context Engineering&lt;/h2&gt;
&lt;p&gt;When I try to make context engineering practical for the teams I work with, I break it into five specific disciplines. None of these are brand new ideas in isolation. But thinking of them as a coherent set, each one building on the others, is what makes the difference between tinkering and engineering.&lt;/p&gt;
&lt;h2 id=&quot;discipline-one-selection&quot;&gt;Discipline One: Selection&lt;/h2&gt;
&lt;p&gt;The first question in context engineering, and the most counterintuitive one, is not what to include. It is what to leave out.&lt;/p&gt;
&lt;p&gt;The instinct when you have access to a large context window is to fill it. You have a knowledge base, so why not retrieve everything potentially relevant? You have a long conversation history, so why not include all of it? You have a detailed user profile, so why not pass the whole schema?&lt;/p&gt;
&lt;p&gt;The reason not to is grounded in research that every AI team should know.&lt;/p&gt;
&lt;p&gt;A landmark study from Stanford and UC Berkeley, commonly called ‘Lost in the Middle,’ &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://arxiv.org/abs/2307.03172&quot;&gt;found that LLMs do not attend uniformly across their context window.&lt;/a&gt; They attend strongly to the beginning and end and systematically underweight the middle. The performance drop when critical information sits in the middle rather than at the edges exceeds 30% on multi-document question answering tasks.&lt;/p&gt;
&lt;p&gt;Chroma Research extended this in&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.firecrawl.dev/blog/context-engineering&quot;&gt;2025 by testing 18 production LLMs&lt;/a&gt;, including Claude 4 Sonnet, GPT-4.1, and Gemini 2.5 Pro, on tasks across contexts ranging from 10,000 to 500,000 tokens. Every single model showed monotonically decreasing accuracy as context grew. Not some models. All of them. The steepest degradation occurred in the 100,000 to 500,000 token range, which is exactly the range where teams often assume they can safely dump everything.&lt;/p&gt;
&lt;p&gt;A separate analysis found that current LLMs effectively use only 10 to 20% of their context window for retrieval tasks, with accuracy varying significantly based on where in the window the information appears. Stuffing a 200,000 token context window is not just wasteful. It actively dilutes the attention the model pays to the things that matter.&lt;/p&gt;
&lt;p&gt;Selection is an active design problem. For a payment support use case, when a user asks about a failed transaction, you do not retrieve their entire account history. You retrieve the last three transactions and the specific policy section covering payment failures. You pass the top two or three retrieved documents ranked by query specificity, not the top ten.&lt;/p&gt;
&lt;p&gt;Useful information is the smallest set that lets the model do the task correctly. Anything beyond that is noise that costs you tokens, attention, and latency.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;Before adding anything to your context, ask: does removing this change the model output on the task I care about? If the answer is no, remove it. The goal is the minimum sufficient context, not the maximum available context.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;discipline-two-compression&quot;&gt;Discipline Two: Compression&lt;/h2&gt;
&lt;p&gt;Once you accept that you cannot put everything in context, the second discipline is what to do with the things you do not put in. Especially conversation history.&lt;/p&gt;
&lt;p&gt;A fifty-turn conversation cannot live in the prompt verbatim without becoming expensive and cognitively overwhelming for the model. But the information in those turns matters. So how do you preserve it without dumping it?&lt;/p&gt;
&lt;p&gt;3 patterns that actually work in production:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;When a conversation has been running long, replace older turns with a structured summary that preserves the actionable state: ‘Customer is on the Pro plan, has reported two failed payments this week, previous agent offered a courtesy credit that was not applied.’ The model loses verbatim detail but keeps the information it needs to act correctly.&lt;/li&gt;
&lt;li&gt;Do not keep state in the prompt at all when you can avoid it. Persist it to a database and retrieve specific fields on demand via tool calls. This is both cheaper and more reliable than asking the model to extract meaning from a long conversation history it is already underattending to.&lt;/li&gt;
&lt;li&gt;This is a technique highlighted in the&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://manus.im/blog/context-engineering-for-agents&quot;&gt;Manus context engineering post:&lt;/a&gt; periodically rewriting the current task state into the end of the context so the most recent attention window sees the most important information. For long agent tasks, this is what separates an agent that stays on plan from one that slowly drifts.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The key insight is that raw history and necessary history are not the same thing. Most of what was said in a conversation is no longer relevant by the time you are three turns in. The work of compression is identifying the small fraction that is.&lt;/p&gt;
&lt;h2 id=&quot;discipline-three-structure&quot;&gt;Discipline Three: Structure&lt;/h2&gt;
&lt;p&gt;Same information. Different organisation. Noticeably different model behaviour. This is the discipline that took me longest to take seriously and that turned out to deliver some of the largest gains.&lt;/p&gt;
&lt;p&gt;A few specific things that are documented and reproducible:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Structured tags outperform flowing paragraphs. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.anthropic.com/engineering/building-effective-agents&quot;&gt;Claude in particular performs&lt;/a&gt; measurably better when context is wrapped in XML-style tags than when the same content is written as natural prose. The same user profile, restructured with clear semantic boundaries, produces more accurate references to that profile in the model output. Anthropic documents this in their own engineering guidance.&lt;/li&gt;
&lt;li&gt;Models attend more strongly to instructions at the very start and the very end of context, because of the architectural biases we covered in the selection discipline. Constraints you want the model to consistently respect: safety boundaries, output format requirements, persona definitions, belong at the edges. Not buried in the middle where they will be underweighted.&lt;/li&gt;
&lt;li&gt;Every major model now has a structured tool-calling API. Using it is more reliable than describing tools in the system prompt and parsing string outputs. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.anthropic.com/engineering/building-effective-agents&quot;&gt;Anthropic has published data showing that rewriting tool descriptions alone&lt;/a&gt; can lift task success 5 to 20% on the same model without any other changes. That improvement is entirely at the context layer, not the model layer.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Treat context like any other designed interface. It should have explicit structure, predictable ordering, and clear semantic boundaries between components. The model is not a human reader who can infer context from narrative flow. It responds to architecture.&lt;/p&gt;
&lt;h2 id=&quot;discipline-four-cache-discipline&quot;&gt;Discipline Four: Cache Discipline&lt;/h2&gt;
&lt;p&gt;This is the most technical of the five disciplines, and the one that product people are most tempted to leave entirely to engineers. That is a mistake, because product decisions made without understanding caching can silently multiply serving costs by three to five times.&lt;/p&gt;
&lt;p&gt;Here is the mechanism in plain language. Modern LLM serving systems reuse computation across calls when the beginning of the prompt matches between calls. This is called KV-cache. A high cache hit rate dramatically reduces both latency and cost. A low one dramatically increases both.&lt;/p&gt;
&lt;p&gt;What destroys cache hit rates? Changes to the beginning of your prompt. And the things that cause those changes often look completely innocent from a product perspective:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Adding the current date to the system prompt causes a cache miss every day&lt;/li&gt;
&lt;li&gt;Personalising the system prompt with the user’s name causes cache misses across users&lt;/li&gt;
&lt;li&gt;Dynamically reordering tool lists based on user context causes cache misses on every reorder&lt;/li&gt;
&lt;li&gt;Making any change to your prompt structure to fix an edge case causes cache misses across the entire deployment&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each of these can look like a small, sensible product improvement and quietly make your serving costs three to five times higher. The fix is not to never personalise. It is to push personalisation to the end of context rather than the beginning, preserving the stable prefix that the cache can reuse.&lt;/p&gt;
&lt;p&gt;Treat your prompt structure like a database schema. It can be changed, but changes need to be deliberate, tested, and deployed with awareness of the downstream cost impact. Monitoring cache hit rate as a first-class metric, not an afterthought, is what makes this discipline real rather than theoretical.&lt;/p&gt;
&lt;h2 id=&quot;discipline-five-evaluation&quot;&gt;Discipline Five: Evaluation&lt;/h2&gt;
&lt;p&gt;The 5th discipline is the one without which the other four are guesswork.&lt;/p&gt;
&lt;p&gt;Every change you make to your context, adding a few-shot example, restructuring the prompt, swapping retrieval logic, modifying tool descriptions, should pass through an evaluation gate before it ships. Without that gate, you are operating on vibes. You might be improving things. You might be regressing them. You have no way to know.&lt;/p&gt;
&lt;p&gt;What makes context engineering evaluation harder than general model evaluation is that the changes are often subtle. A new few-shot example might improve one failure mode and quietly degrade another. A restructured user profile might increase accuracy on profile-dependent queries while slowing down every call because the prompt is now longer. The gains and regressions live in single-digit percentages, and you cannot see them without measurement.&lt;/p&gt;
&lt;p&gt;This is the compounding mechanism that makes context engineering a genuine moat. If you have been running an evaluation pipeline for a year, you have accumulated evidence about which changes help and which hurt across hundreds of edge cases. A competitor starting today cannot replicate that accumulated knowledge. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://karozieminski.substack.com/p/context-engineering-product-builders-guide-2026&quot;&gt;Stanford, SambaNova, and UC Berkeley published research in October 2025&lt;/a&gt; on Agentic Context Engineering, describing systems where agents update their own context playbooks based on what works across tasks. The direction the whole field is moving is toward context that learns and improves over time. Measurement is the precondition for that.&lt;/p&gt;
&lt;h2 id=&quot;why-this-is-the-moat-and-why-most-teams-have-not-found-it-yet&quot;&gt;Why This Is the Moat and Why Most Teams Have Not Found It Yet&lt;/h2&gt;
&lt;p&gt;I want to make a specific argument here and back it up, because ‘this is the moat’ is easy to say and meaningless without the reasoning.&lt;/p&gt;
&lt;h2 id=&quot;the-model-layer-is-commoditising-fast&quot;&gt;The Model Layer Is Commoditising Fast&lt;/h2&gt;
&lt;p&gt;The gap between the top closed-source models and the best open-weight alternatives has narrowed every release cycle for the past two years. DeepSeek, Mistral, and Llama variants are landing within months of frontier closed-source quality on most benchmarks. The marginal advantage of using the best available model is shrinking, and the rate of shrinkage is itself accelerating.&lt;/p&gt;
&lt;p&gt;Any product whose differentiation reduces to ‘we use the latest model’ has, by definition, a temporary and rapidly expiring advantage. Six months at most before a competitor is on the same model or one that performs equivalently.&lt;/p&gt;
&lt;h2 id=&quot;the-context-layer-does-not-commoditise&quot;&gt;The Context Layer Does Not Commoditise&lt;/h2&gt;
&lt;p&gt;Your context-layer assets are not in any model’s training data. They cannot be replicated by a competitor switching to the same API. They include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Your accumulated correction database: Every time a user edited an AI output in your product, that is a labelled example of what right looks like for that specific failure mode in your specific domain. No competitor has access to it.&lt;/li&gt;
&lt;li&gt;Your calibrated few-shot pool: Examples handpicked from your evaluation set, tested against your specific quality bar, refined over months of iteration on real user behaviour. Each one took judgment to select. The whole pool took time to build.&lt;/li&gt;
&lt;li&gt;Your tool descriptions: Rewriting these alone can lift task success by 5 to 20% on the same model. That improvement is now embedded in your product and is not in anyone else’s.&lt;/li&gt;
&lt;li&gt;Your prompt structure: The specific layout that maximises your cache hit rate, your retrieval relevance, your structured output reliability. The result of dozens of experiments you ran and your competitor has not.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each of these is individually small. Cumulatively they are enormous. And unlike model upgrades, which are a one-time step function, these assets compound. The correction database grows every day. The few-shot pool gets better with every evaluation cycle. The prompt structure improves with every A/B test. A competitor starting today cannot catch up because they are not starting at zero, they are starting behind a moving target.&lt;/p&gt;
&lt;h2 id=&quot;why-most-teams-have-not-found-it-yet&quot;&gt;Why Most Teams Have Not Found It Yet&lt;/h2&gt;
&lt;p&gt;If context engineering is this important, why is not everyone doing it? Four honest reasons.&lt;/p&gt;
&lt;p&gt;First, the work is invisible. A team can spend a quarter improving context engineering and have nothing to demo. No new features. No visible UI changes. Just better outputs in a hundred small ways. That is genuinely hard to justify in a product review against teams shipping flashier work.&lt;/p&gt;
&lt;p&gt;Second, it sits in an organisational gap. Context engineering is partly product, what information matters, in what form, and partly engineering, how to retrieve it and cache it efficiently. Product managers assume it is an engineering detail. Engineers assume it is a product decision. Without an explicit owner, it gets done badly or not at all.&lt;/p&gt;
&lt;p&gt;Third, the discipline is not taught anywhere. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://neo4j.com/blog/agentic-ai/context-engineering-vs-prompt-engineering/&quot;&gt;Neo4j’s analysis&lt;/a&gt; notes that context engineering only emerged as a formalised concept in mid-2025. There is no university course. No PM training programme. Anyone learning this is learning it on the job from first-party engineering blogs that move faster than any curriculum.&lt;/p&gt;
&lt;p&gt;Fourth, the compounding takes time to become visible. The first month of context engineering investment looks unimpressive. The sixth month starts to feel meaningful. The twelfth month is when the gap becomes obvious. Teams that need quarterly wins struggle to justify it. The teams willing to invest anyway are the ones building the actual moat.&lt;/p&gt;
&lt;h2 id=&quot;what-this-means-for-your-career-and-where-the-roles-are&quot;&gt;What This Means for Your Career and Where the Roles Are&lt;/h2&gt;
&lt;p&gt;Here is the part most articles on context engineering leave out entirely. Understanding this concept is not just useful for building better AI products. It is one of the most valuable career differentiators in the technical job market right now.&lt;/p&gt;
&lt;p&gt;The roles being created around context engineering are real, they are well-compensated, and they are being filled through channels that most people are not watching. Let me get specific.&lt;/p&gt;
&lt;h2 id=&quot;the-roles-that-are-actually-being-created&quot;&gt;The Roles That Are Actually Being Created&lt;/h2&gt;
&lt;p&gt;Based on the patterns visible across AI teams that are doing this well, the roles with genuine demand in 2026 look like this:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Context architects: people who own the design of what enters the model’s context at each call, how it is structured, and how it is evaluated. Neither a traditional PM nor a traditional engineer. A new kind of role that sits at the intersection.&lt;/li&gt;
&lt;li&gt;Retrieval engineers: people who build and maintain the pipelines that decide what gets pulled from knowledge bases at runtime, with what ranking logic, formatted how. The quality of retrieval is often the single largest driver of output quality in production systems.&lt;/li&gt;
&lt;li&gt;Evaluation engineers: people who build and maintain the eval pipelines that let context changes be measured before they ship. Without this role, context engineering cannot compound. With it, every week of iteration is permanently additive.&lt;/li&gt;
&lt;li&gt;AI product managers with context fluency: PMs who understand KV-cache well enough to know when a product decision will triple serving costs, who own the few-shot pool, who treat the prompt structure as a product asset not an implementation detail.&lt;/li&gt;
&lt;li&gt;Technical writers for AI systems: people who write the tool descriptions, the system prompt language, the few-shot examples. The quality of these artefacts directly determines model behaviour. Most AI teams are genuinely bad at this and know it.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;What all of these roles share? They require understanding the problem at a level most people do not have yet. That gap is exactly where the opportunity is.&lt;/p&gt;
&lt;h2 id=&quot;the-thing-that-will-look-obvious-in-hindsight&quot;&gt;The Thing That Will Look Obvious in Hindsight&lt;/h2&gt;
&lt;p&gt;Two years from now I think there will be a role on most serious AI product org charts that does not have a clear name yet. It sits between product management and machine learning engineering. It owns the context layer explicitly. It treats the prompt structure as a product asset, the few-shot pool as a competitive advantage, the evaluation pipeline as an infrastructure investment.&lt;/p&gt;
&lt;p&gt;The teams that figure this out first will be the ones whose products feel qualitatively different from everything else in their category, even when they are running on the same underlying models as everyone else. The difference will be invisible from the outside and enormous from the inside.&lt;/p&gt;
&lt;p&gt;Right now, there is a window where understanding context engineering clearly is a genuine differentiator because most people and most teams do not understand it yet. That window will close as the concept becomes mainstream and the textbooks catch up.&lt;/p&gt;
&lt;p&gt;The model is the visible part of the AI product. It is the part that gets the press releases and the benchmark announcements and the heated debates on social media. Context is the part that actually decides whether the product works, whether it compounds, and whether it is possible to build something a competitor cannot replicate.&lt;/p&gt;
&lt;p&gt;Start treating it that way. The compounding starts from the first week you do.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;If this reframed how you think about AI products, share it with someone who is still arguing about which model to use.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;&lt;strong&gt;Disclaimer: This article reflects personal analysis and commentary based on publicly available research and engineering documentation. All statistics are sourced from cited research.&lt;/strong&gt;&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>artificial-intelligence</category><category>writing-prompts</category><category>prompt-engineering</category><category>chatgpt</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>Your AI Writing Tool Is Running on Borrowed Time and Borrowed Money</title><link>https://soursen.org/articles/your-ai-writing-tool-is-running-on-borrowed-time-and-borrowed-money/</link><guid isPermaLink="true">https://soursen.org/articles/your-ai-writing-tool-is-running-on-borrowed-time-and-borrowed-money/</guid><description>Let me start by asking you something honest. If every AI writing tool disappeared tomorrow, could you still do your job?</description><pubDate>Fri, 05 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Let me start by asking you something honest. If every AI writing tool disappeared tomorrow, could you still do your job?&lt;/p&gt;
&lt;p&gt;If the answer is yes, great. If there is a long pause before you answer, then this piece is for you. And I want to have a straight conversation with you about something most people writing about AI are not willing to say clearly. AI might not be around in the form you know it forever. And if you have built your writing practice on top of it as a crutch rather than a teacher, that is a problem you need to start solving today.&lt;/p&gt;
&lt;p&gt;I know that sounds alarming. So let me show you exactly what I mean, and then I will show you what to do about it. Not in a vague, motivational way. In a concrete, this is the actual thing to try right now way.&lt;/p&gt;
&lt;h2 id=&quot;the-money-behind-ai-is-not-what-you-think-it-is&quot;&gt;The Money Behind AI Is Not What You Think It Is&lt;/h2&gt;
&lt;p&gt;Here is something most people using AI tools every day have no idea about. The companies building those tools are not profitable. Not even close.&lt;/p&gt;
&lt;p&gt;According to &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.fortune.com/2025/11/12/openai-cash-burn-rate-annual-losses-2028-profitable-2030-financial-documents&quot;&gt;internal financial documents obtained by the Wall Street Journal&lt;/a&gt;, OpenAI is projected to post roughly $74 billion in operating losses in 2028 alone. The company does not expect to turn a profit until 2030 at the earliest. Over the entire period from 2023 through 2028, OpenAI is &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://finance.yahoo.com/news/openais-own-forecast-predicts-14-150445813.html&quot;&gt;forecasting $44 billion&lt;/a&gt; in cumulative losses before it sees a single year in the black.&lt;/p&gt;
&lt;p&gt;To give you a sense of the scale, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://medium.com/@ZombieCodeKill/is-openai-in-trouble-b9f8cd2a9130&quot;&gt;Microsoft’s SEC filings&lt;/a&gt; revealed that OpenAI lost $12 billion in a single quarter, the three months from July to September 2025. That is not a typo. Twelve billion dollars. In 3 months. From one company and their revenue growth, while real and impressive, is still not keeping pace with how fast the costs are growing.&lt;/p&gt;
&lt;p&gt;So where is all the money coming from? Investors. Trillions of dollars of investment capital flowing into AI companies based on a promise of future returns that the companies themselves are not yet able to deliver.&lt;/p&gt;
&lt;p&gt;Harvard economist Jason Furman has estimated that AI investment accounted for roughly &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.oliverwyman.com/our-expertise/insights/2026/jan/impact-ai-bubble-burst-on-global-financial-markets.html&quot;&gt;92% of US GDP growth&lt;/a&gt; in the first half of 2025. Nearly every dollar of economic growth in the world’s largest economy is now connected, in some way, to the assumption that AI will deliver on its promises. That is a staggering level of concentration.&lt;/p&gt;
&lt;p&gt;This is not a secret. Financial analysts are talking about it openly. The question they are debating is not whether the math is sustainable. It is whether there is a graceful landing or an ugly one.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;THE NUMBERS IN ONE PLACE&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;OpenAI projected losses: $14B in 2026, $74B operating losses in 2028, total $44B from 2023 to 2028&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;OpenAI in Q3 2025: lost $12 billion in a single quarter (Microsoft SEC filings)&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;OpenAI current spend: approximately $1.69 for every $1.00 of revenue it generates&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;AI investment share of US GDP growth H1 2025: approximately 92% (Harvard economist Jason Furman)&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;OpenAI expected break-even: 2029 to 2030, assuming everything goes to plan&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;but-wait-is-it-not-working-in-business-either&quot;&gt;But Wait. Is It Not Working in Business Either?&lt;/h2&gt;
&lt;p&gt;Here is the part that gets even more interesting. The pitch to investors has been that corporations will adopt AI at scale, reduce their labour costs, and pay handsomely for the tools that make it possible. That was the whole business case. That was what was supposed to justify the trillions.&lt;/p&gt;
&lt;p&gt;It is not working out that way.&lt;/p&gt;
&lt;p&gt;A study from MIT found that &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://builtin.com/articles/ai-bubble-pop-risks&quot;&gt;95% of generative AI pilots failed&lt;/a&gt; to produce meaningful business impact. Not 10%. Not 30%. Ninety-five percent. Boston Consulting Group found that &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://medium.com/@iram.ahmed1997/the-ai-reckoning-why-the-bubble-is-bursting-in-2026-and-what-we-can-still-build-from-the-ruins-2388aa8bbc4f&quot;&gt;74% of companies are struggling&lt;/a&gt; to generate real business value from AI at any meaningful scale.&lt;/p&gt;
&lt;p&gt;Gartner found that 40% of agentic AI projects get cancelled before they even launch and the US Census Bureau, which tracks AI adoption across the economy, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.aol.com/finance/ai-bubble-burst-mounting-data-173859691.html&quot;&gt;recently reported&lt;/a&gt; something notable: for the first time since 2023, AI adoption among large companies is dipping, not growing. The six-week rolling average for companies with more than 250 employees dropped from a high of 13.5% to around 12%. That is a small number but it is the first reversal of a previously unbroken upward trend.&lt;/p&gt;
&lt;p&gt;Man Group, one of the world’s largest hedge funds, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.man.com/insights/the-ai-bubble&quot;&gt;published an analysis at the end of 2025&lt;/a&gt; stating plainly that &lt;em&gt;“it is increasingly a question of when, not if, the AI bubble bursts.”&lt;/em&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.brookings.edu/articles/is-there-an-ai-bubble/&quot;&gt;Brookings Institution&lt;/a&gt; has published similar analysis. These are not fringe voices. These are mainstream financial institutions doing the math.&lt;/p&gt;
&lt;p&gt;None of this means AI disappears tomorrow. Some version of it will absolutely survive. AI tools for translation, image editing, data processing, code completion. Those are genuinely useful, the costs are manageable, and they work. Some will run locally on your laptop or phone without needing a data centre the size of a small city.&lt;/p&gt;
&lt;p&gt;What might not survive, or might change dramatically in cost and availability, is the version you rely on right now. The always-on, generate-anything-on-demand model running on someone else’s servers, subsidised by investor capital that is still waiting for a return.&lt;/p&gt;
&lt;p&gt;That is the specific version of AI you should not be dependent on.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;WHAT THE CORPORATE DATA SAYS&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;95% of generative AI pilots fail to produce meaningful business impact (MIT)&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;74% of companies struggle to scale AI value at all (Boston Consulting Group)&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;40% of agentic AI projects are cancelled before launch (Gartner, 2026)&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;Large company AI adoption declined for the first time since 2023 (US Census Bureau, 2026)&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;Built In estimates the AI bubble may be 17 times the size of the dot-com crisis&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;how-ai-writing-actually-works-and-why-that-matters&quot;&gt;How AI Writing Actually Works, and Why That Matters&lt;/h2&gt;
&lt;p&gt;Before we talk about what to do, I want to make sure you understand what AI is actually doing when it writes for you. Because once you understand the mechanism, the right strategy becomes obvious.&lt;/p&gt;
&lt;p&gt;Think about how you learned to write. You read thousands of books, articles, letters, essays, terrible blog posts, brilliant ones. Over time your brain started to pattern-match. You developed a sense of rhythm. You learned which sentences land and which ones clunk. You do not consciously calculate this anymore. You just feel when a sentence is right.&lt;/p&gt;
&lt;p&gt;AI does something structurally similar, but mechanically. The companies behind these tools scraped an enormous amount of professionally written text and broke it down into what are called tokens. Every word is a token. Common phrases are tokens too. The AI learned the statistical probability of which tokens follow which other tokens, across billions of examples.&lt;/p&gt;
&lt;p&gt;So when you type a prompt, the model runs a chain of probability calculations and assembles an output that is, statistically, very likely to resemble good writing. It knows that ‘once upon a time’ is a phrase. It knows ‘once upon a fish’ is not. It knows which sentence structures tend to appear in persuasive essays and which tend to appear in instruction manuals.&lt;/p&gt;
&lt;p&gt;And it is genuinely good at certain things. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.allaboutai.com/resources/ai-statistics/ai-writing/&quot;&gt;Research published by Grammarly in 2024&lt;/a&gt; found that AI scores 32% better than humans on grammar accuracy. It is consistent. It does not have bad days. It will not accidentally use passive voice in paragraph four after being told not to. It is a remarkable pattern-matching machine.&lt;/p&gt;
&lt;p&gt;But here is the fundamental ceiling it cannot break through.&lt;/p&gt;
&lt;p&gt;AI can only assemble combinations of tokens that have a reasonable probability of appearing together based on what it has already seen. It cannot say something that has never been said before. It cannot make a genuine observation drawn from lived experience. It cannot have a point of view that is not, in some sense, a synthesis of points of view that already existed in its training data.&lt;/p&gt;
&lt;p&gt;That is not a temporary limitation waiting for the next model release. It is structural. It is what the technology is. A very sophisticated echo of everything that has already been written.&lt;/p&gt;
&lt;p&gt;And readers can feel it. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://cdh.princeton.edu/news/2025/11/11/everyone-prefers-human-writers-even-ai/&quot;&gt;Princeton’s Center for Digital Humanities published research in November 2025&lt;/a&gt; finding that both humans and AI systems show a strong preference for human-authored writing over AI-authored writing, even when the content quality is equivalent. We are wired to respond to genuine human voice. We can tell the difference, often without being able to articulate why.&lt;/p&gt;
&lt;p&gt;The engagement data reflects this too. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://wordcountai.com/blog/ai-vs-human-writing&quot;&gt;WordCount AI’s 2026 analysis&lt;/a&gt; of thousands of content pieces found that human-written content gets 40 to 60% longer average time on page than AI-written content and social sharing rates favour human content by 3 to 4 times on average. AI-only content has bounce rates 20 to 30% higher. Pure AI output scores 47% worse on originality than human writing.&lt;/p&gt;
&lt;p&gt;This is the thing most people using AI to write have not sat with long enough. The tool that feels like it is helping you write better might actually be making the output less worth reading.&lt;/p&gt;
&lt;h2 id=&quot;the-only-smart-way-to-use-ai-if-you-are-a-writer&quot;&gt;The Only Smart Way to Use AI If You Are a Writer&lt;/h2&gt;
&lt;p&gt;Okay. So the business model behind AI is under real strain. The corporate adoption story is not playing out as promised. And the writing AI produces, while technically competent, consistently underperforms human writing on the metrics that actually matter for building an audience.&lt;/p&gt;
&lt;p&gt;So what do you do? You stop using AI as the writer. You start using it as the teacher.&lt;/p&gt;
&lt;p&gt;This is not a semantic distinction. It is a completely different relationship with the tool that produces completely different outcomes, and it is the only approach that makes sense given everything we just covered.&lt;/p&gt;
&lt;p&gt;Here is what most writers currently do. They type a prompt. They get a draft. They edit it. They change some words. They adjust the tone. Essentially, the AI pulls the words and the human supervises. The writer becomes an editor of someone else’s work, except the someone else is a probability engine that has no actual understanding of what they are trying to say.&lt;/p&gt;
&lt;p&gt;The problem with that approach is not just that the output is weaker. The problem is that you are not getting better. You are outsourcing the part of writing that builds the skill. And when the tool changes or disappears or gets expensive or gets paywalled, you are left having made no progress.&lt;/p&gt;
&lt;h2 id=&quot;the-method-that-actually-works&quot;&gt;The Method That Actually Works&lt;/h2&gt;
&lt;p&gt;Start with a blank page. Write your own draft. Yes, it will be rough. That is entirely fine. Every good writer in history has written terrible first drafts. Hemingway said all first drafts are garbage. He was right and he was one of the greatest writers of the twentieth century.&lt;/p&gt;
&lt;p&gt;Write the whole thing yourself. Your thoughts. Your structure. Your argument. Your examples. Your voice. The mess of it is where the actual thinking happens.&lt;/p&gt;
&lt;p&gt;Then bring AI in. But not to rewrite. To critique.&lt;/p&gt;
&lt;p&gt;Ask it specific questions about your draft. Which paragraph is the strongest? Why? What is the weakest sentence in the piece? Where does the argument lose its thread? Where did I use passive voice when active would have been sharper? Is the opening doing enough work, or is it just warming up?&lt;/p&gt;
&lt;p&gt;Here is a critical instruction when you do this: tell it not to praise you. Tell it you want critique, not encouragement. AI is a sycophant by default. It is designed to please the person it is talking to. If you do not explicitly instruct it to be critical, it will tell you everything is great and offer only minor suggestions. That is useless. You want it uncomfortable. You want it to find the weak spots.&lt;/p&gt;
&lt;p&gt;Once you have the critique, go back to your draft and make changes yourself. Do not ask AI to rewrite the weak paragraph. Rewrite it. Use what it told you as the brief. Now you are learning. Now the feedback is building your skill rather than bypassing it.&lt;/p&gt;
&lt;p&gt;Feed the revised draft back in. Run another round of critique. Focus this time on sentence level. Which sentences are doing nothing? Where is the pacing slow? Which transitions are lazy?&lt;/p&gt;
&lt;p&gt;Do this as many rounds as it takes. Each round, you are not just improving this piece. You are building the internal editor that will make your next piece better without needing the external feedback loop at all.&lt;/p&gt;
&lt;p&gt;THE SIMPLE WORKFLOW&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Step 1: Write your own first draft from scratch. All of it. No prompts.&lt;/li&gt;
&lt;li&gt;Step 2: Ask AI to critique it. Not rewrite it. Critique.&lt;/li&gt;
&lt;li&gt;Step 3: Tell it explicitly not to praise you. Tell it to find the problems.&lt;/li&gt;
&lt;li&gt;Step 4: Take the critique and rewrite the weak parts yourself.&lt;/li&gt;
&lt;li&gt;Step 5: Feed the revised draft back in. Repeat at sentence level.&lt;/li&gt;
&lt;li&gt;Step 6: Keep going until you have genuinely improved the piece, not just changed words.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;em&gt;What you are building is not better output. You are building a better writer.&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-specific-questions-to-ask-ai-about-your-draft&quot;&gt;The Specific Questions to Ask AI About Your Draft&lt;/h2&gt;
&lt;p&gt;The quality of your critique session depends entirely on how you ask. Here are questions that actually produce useful feedback:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;“What is the single strongest paragraph in this draft and why?” Move it to the opening. Your best material should come first.&lt;/li&gt;
&lt;li&gt;“What is the weakest sentence and what makes it weak?” This trains your ear over time.&lt;/li&gt;
&lt;li&gt;“Where does this piece lose momentum?” Pacing problems are hard to self-diagnose.&lt;/li&gt;
&lt;li&gt;“Is there anything in here that does not belong? Any section that drifts from the main point?” Ruthless editing is a learnable skill.&lt;/li&gt;
&lt;li&gt;“Is my ending strong or does it just stop?” Endings are where most writers go wrong.&lt;/li&gt;
&lt;li&gt;“Am I saying anything here that a thousand other articles have already said, or is there a genuine original observation?” This is the most important question.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;That last question is where AI earns its keep as a teacher. Because it has read everything. It knows when you are being original and when you are just echoing what already exists. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.sciencedirect.com/science/article/pii/S2215039025000219&quot;&gt;ScienceDirect research from 2025&lt;/a&gt; found that AI-generated text consistently uses fewer interactional markers and shows weaker authorial presence than human writing. AI can spot the absence of those things in your draft and tell you where you sound generic. That is genuinely useful feedback.&lt;/p&gt;
&lt;h2 id=&quot;what-this-means-for-your-career-and-where-the-real-opportunities-are&quot;&gt;What This Means for Your Career and Where the Real Opportunities Are&lt;/h2&gt;
&lt;p&gt;Understanding AI clearly, its real capabilities and its structural limits, is not just useful for improving your writing. It is one of the most valuable professional assets you can have right now.&lt;/p&gt;
&lt;p&gt;Because here is what the market actually needs and cannot find enough of: people who can write with genuine voice, understand what AI can and cannot do, and use that combination intelligently. Not people who are anti-AI. Not people who are fully dependent on AI. People who can do both, strategically.&lt;/p&gt;
&lt;p&gt;The data backs this up. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;http://parse.ly/&quot;&gt;Parse.ly&lt;/a&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.allaboutai.com/resources/ai-statistics/ai-writing/&quot;&gt;’s 2025 analysis&lt;/a&gt; found that human-edited AI content sees 16% higher engagement than even fully human-written content. The winning combination is a human who understands AI using it as a tool, not a replacement. 78% of top-ranking content now follows a hybrid model where human expertise shapes AI-assisted output. That is the standard.&lt;/p&gt;
&lt;p&gt;The writers getting hired in 2026 are not the ones who produce the most volume. They are the ones who produce the most signal in an environment drowning in noise. That requires a real voice, formed through practice, not borrowed from a language model.&lt;/p&gt;
&lt;h2 id=&quot;the-thing-nobody-wants-to-hear-but-you-need-to&quot;&gt;The Thing Nobody Wants to Hear, But You Need to&lt;/h2&gt;
&lt;p&gt;Here is the uncomfortable truth at the centre of all of this.&lt;/p&gt;
&lt;p&gt;The writers who are going to thrive in whatever version of AI-assisted media comes next are not the ones who learned to use AI most fluently. They are the ones who used AI as an accelerant on top of a real underlying skill. The ones who will struggle are the ones who used AI as a shortcut around the hard part. The ones who never really wrote because they did not have to. The ones who got a lot done but did not get better.&lt;/p&gt;
&lt;p&gt;If that describes you, today is the day to change the relationship. Not because AI is going away necessarily. But because your skill should not be contingent on whether it stays.&lt;/p&gt;
&lt;p&gt;Think about a person who spent years working a job they disliked to support their family, and now finally has the time to write. That person does not need ten years to develop a writing practice. They need the right feedback loop applied consistently over a manageable period of time. AI as teacher can compress that timeline significantly. AI as ghostwriter cannot help them at all.&lt;/p&gt;
&lt;p&gt;The tool is genuinely useful. It is just most useful when you point it at your weakness rather than your blank page. Start with the draft. Let the AI find the holes. Patch them yourself. Repeat until the piece is good. Then do it again on the next one.&lt;/p&gt;
&lt;p&gt;That is the whole method. And if the bubble bursts tomorrow, you will still have everything you built.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;If this changed how you think about using AI, share it with a writer who might need to hear it.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Disclaimer: This article reflects personal analysis and commentary. All financial figures and statistics are sourced from cited research and publicly available reporting. Opinions on the AI industry trajectory are the author’s own.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>openai</category><category>artificial-intelligence</category><category>llm</category><category>ai-tools</category><author>hello@soursen.org (Shivashish Yadav)</author></item><item><title>The AI CEOs Said Humans Were Replaceable. The Winning Hackathon Products Disagree</title><link>https://soursen.org/articles/the-ai-ceos-said-humans-were-replaceable-the-winning-hackathon-products/</link><guid isPermaLink="true">https://soursen.org/articles/the-ai-ceos-said-humans-were-replaceable-the-winning-hackathon-products/</guid><description>I want to tell you something that might change how you read every AI press release from this point forward.</description><pubDate>Thu, 04 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;blockquote&gt;
&lt;p&gt;What 3 prize-winning products from a major Google hackathon actually reveal about where AI is right now and what it means for the jobs everyone said were disappearing.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;I want to tell you something that might change how you read every AI press release from this point forward.&lt;/p&gt;
&lt;p&gt;Not because AI is a scam or a bubble, it isn’t. But because there’s a specific and repeatable gap between the pitch and the product, and once you learn to see it, you can’t unsee it.&lt;/p&gt;
&lt;p&gt;Here’s where I want to start. Every AI lab CEO has, at some point in the last two years, told an audience some version of the same story, the models are getting smarter, the context windows are getting longer, the reasoning chains are getting more reliable, and soon any month now the AI will handle the work. Your workflows. Your decisions. Your contracts and by implication, a meaningful chunk of your headcount. Then Google DeepMind ran the &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://gemini3.devpost.com/&quot;&gt;Gemini 3 Hackathon&lt;/a&gt; a global competition open from December 2025 to February 2026, with a $50,000 grand prize and thousands of entries from developers around the world. The judges looked at everything. And the three products they rewarded most weren’t the ones that pushed AI farthest into autonomy. They were the ones that built the most thoughtful systems for managing what AI can’t yet do reliably.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;1. Grand prize: a team that kept the human making the final call. &lt;br/&gt;
2. Second place: a team that forced AI to narrate its reasoning so humans could check it. &lt;br/&gt;
3. Third: a team that built a workaround for the fact that AI wakes up with no memory every single session.&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Read that again slowly. The winning designs are a map of AI’s current limitations, dressed up as features.&lt;/p&gt;
&lt;p&gt;I want to walk you through each one what the product does, what limitation it’s actually compensating for, and what that tells you about the real shape of AI work in 2026 and then, because understanding the landscape is only half the value, I want to give you something practical a concrete playbook for finding and landing the jobs this AI transition is actually creating, using the platforms and tactics that are working right now.&lt;/p&gt;
&lt;p&gt;Let’s get into it.&lt;/p&gt;
&lt;h2 id=&quot;ai-still-cant-be-trusted-to-decide-alone&quot;&gt;AI Still Can’t Be Trusted to Decide Alone&lt;/h2&gt;
&lt;h2 id=&quot;what-the-winning-product-did&quot;&gt;What the Winning Product Did?&lt;/h2&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://devpost.com/software/globot-341w9q?_gl=1*1y0icxl*_gcl_au*NjcyODQ3NjU2LjE3ODA1MzcwMTk.*_ga*MTk1NDU1MDAwMC4xNzgwNTM3MDE5*_ga_0YHJK3Y10M*czE3ODA1MzcwMTkkbzEkZzEkdDE3ODA1MzcwNDYkajMzJGwwJGgw&quot;&gt;The grand prize went to a product called Globot&lt;/a&gt;. The scenario it’s designed for is genuinely high-stakes, a logistics or supply chain manager needs to make a million-dollar rerouting decision while a geopolitical crisis or natural disaster is breaking in real time. Every minute costs money. Every wrong call costs more.&lt;/p&gt;
&lt;p&gt;The architecture uses 5 AI agents. The first 4 each own a specific slice of the problem. One pulls geopolitical signals from Reuters and Bloomberg. One calculates the financial impact of different routing options. One redraws the shipping routes. One checks whether the reroute would affect the cargo’s insurance status.&lt;/p&gt;
&lt;p&gt;Then a fifth agent runs and its only job is to go back through what the other 4 just produced and actively look for errors, contradictions, and gaps in the reasoning.&lt;/p&gt;
&lt;p&gt;After all 5 agents have finished, the system hands the consolidated output to a human being for final sign-off. That human approval step isn’t a safety layer added on top of an otherwise autonomous system. It’s a core architectural component. Without it, the product doesn’t work as intended.&lt;/p&gt;
&lt;h2 id=&quot;what-that-actually-tells-you&quot;&gt;What That Actually Tells You&lt;/h2&gt;
&lt;p&gt;You’ve probably heard of this design pattern under different names. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback&quot;&gt;Anthropic baked a version of it into Constitutional AI&lt;/a&gt;, where one model generates an output and a second model evaluates it against a written set of principles. Different companies call these agents ‘critics,’ ‘judges,’ ‘reviewers,’ or ‘validators.’ The name changes. The underlying design doesn’t, you add a checking layer because you don’t fully trust the layer it’s checking.&lt;/p&gt;
&lt;p&gt;And here’s the thing the pitch decks skip. Even after the critic runs, you still can’t trust the system to make a high-stakes decision autonomously. The critic can hallucinate too. The reasoning can look internally consistent and still be built on a false premise. The logic can pass a surface-level check and fail in the field.&lt;/p&gt;
&lt;p&gt;This isn’t speculation. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://mlq.ai/news/carnegie-mellon-study-finds-ai-agents-fail-at-office-tasks-nearly-70-of-the-time/&quot;&gt;Carnegie Mellon research — in collaboration with Salesforce&lt;/a&gt; found that leading AI agents fail &lt;strong&gt;approximately 70% of standard office tasks when operating autonomously.&lt;/strong&gt; These weren’t edge cases or adversarial prompts. These were routine tasks, data analysis, writing performance reviews, basic problem-solving. Tasks that any competent human employee handles without much thought.&lt;/p&gt;
&lt;p&gt;So ‘humans are needed’ isn’t a thoughtful gesture toward human dignity in the workplace. It’s an engineering necessity. The phrase is load-bearing euphemism for: ‘the AI alone isn’t reliable enough yet.’&lt;/p&gt;
&lt;p&gt;And here’s what that creates downstream something none of the lab CEOs seemed to factor into their headcount projections:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Someone has to build the agents and keep them running&lt;/li&gt;
&lt;li&gt;Domain experts have to validate the outputs at regular intervals&lt;/li&gt;
&lt;li&gt;A human sits at the end of every high-stakes pipeline before anything goes live&lt;/li&gt;
&lt;li&gt;That human needs enough expertise to catch what the critic missed — which means you can’t fill that seat cheaply&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You aren’t saving the cost of human labour. You’re restructuring it. The roles changed shape. In many cases, they became more expensive and harder to fill. The jobs didn’t disappear they evolved into something the original pitch didn’t anticipate and definitely didn’t budget for.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;THE QUESTION TO ASK ANY “AUTONOMOUS AI” PITCH &lt;br/&gt;&lt;/strong&gt;&lt;/em&gt;
&lt;em&gt;What happens if you remove the human from the loop? If the honest answer is “it stops being reliable for anything that matters,” then you’re not buying an autonomous system. You’re buying a sophisticated first-draft machine with a human still doing the final check. That’s still valuable but it’s a very different product claim.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;the-black-box-is-still-a-black-box-but-now-it-talks&quot;&gt;The Black Box Is Still a Black Box, But Now It Talks&lt;/h2&gt;
&lt;h2 id=&quot;what-the-winning-product-did-1&quot;&gt;What the Winning Product Did&lt;/h2&gt;
&lt;p&gt;2nd place went to a product called &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://devpost.com/software/aegis-autonomous-multi-agent-crisis-command?_gl=1*1ux30xj*_gcl_au*NjcyODQ3NjU2LjE3ODA1MzcwMTk.*_ga*MTk1NDU1MDAwMC4xNzgwNTM3MDE5*_ga_0YHJK3Y10M*czE3ODA1MzcwMTkkbzEkZzEkdDE3ODA1MzcwODkkajYwJGwwJGgw&quot;&gt;Aegis&lt;/a&gt;. The problem it solves is real and important, when a major natural disaster hits, a regional emergency response system can receive tens of thousands of distress signals within minutes. Calls. Texts. Social media posts. Drone feeds. All arriving simultaneously, all requiring triage decisions that determine who gets help first.&lt;/p&gt;
&lt;p&gt;No human team can process that volume in real time. The architecture is, again, multi-agent coordinator routes incoming signals, a triage agent assigns priority scores, a surveillance agent analyses drone imagery, a logistics agent calculates the fastest rescue routes, and a reporter summarises the mission after the fact.&lt;/p&gt;
&lt;p&gt;The problem the team ran into during early testing, users couldn’t act on the priority scores. Not because the scores were wrong but because the AI gave them a ranked list with no visible reasoning. One casualty ranked above another, but there was no explanation of why. Black box output in a life-or-death context. Nobody would trust it.&lt;/p&gt;
&lt;p&gt;The fix was straightforward in concept and genuinely useful in practice, force every agent to display its reasoning next to its recommendation, visible on the main interface in real time. You see the priority score and the logic behind it, side by side.&lt;/p&gt;
&lt;p&gt;The industry calls this ‘Explainable AI.’ It’s been a buzzword in enterprise AI for years and it’s a real improvement. But it’s worth being precise about what it actually does.&lt;/p&gt;
&lt;h2 id=&quot;showing-the-reasoning-isnt-the-same-as-opening-the-box&quot;&gt;Showing the Reasoning Isn’t the Same as Opening the Box&lt;/h2&gt;
&lt;p&gt;Explainable AI gives the black box a narrator. That’s useful. But it’s not the same as making the black box transparent.&lt;/p&gt;
&lt;p&gt;The model is still doing what it does. The explanation you see is the model’s own account of its reasoning which can itself be wrong, incomplete, or post-hoc rationalisation of a decision it made through a process that isn’t actually inspectable. You’re not seeing inside the model. You’re seeing a readable summary it generated about itself.&lt;/p&gt;
&lt;p&gt;And here’s the dangerous second-order effect that doesn’t make it into the product demo. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.deloitte.com/uk/en/services/consulting/research/automation-bias.html&quot;&gt;Deloitte UK, in research prepared for the UK Ministry of Defence, documented something called&lt;/a&gt; &lt;strong&gt;automation bias&lt;/strong&gt;: the tendency for people to trust automated outputs over their own judgment, even when the evidence points the other way. Their report describes it like this imagine you’re driving and you see a 30mph sign on the road. Your car’s dashboard shows 60. Do you trust the sign or the instrument panel? Studies show a meaningful proportion of people accelerate. The machine has replaced their judgment instead of supporting it.&lt;/p&gt;
&lt;p&gt;The &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.deloitte.com/uk/en/Industries/defense-security-justice/perspectives/defence-ai-a-human-centred-perspective.html&quot;&gt;Deloitte report’s conclusion&lt;/a&gt; about AI in defence contexts is unambiguous: &lt;em&gt;“In this situation, the use of AI has unwittingly impaired the operator’s judgment, rather than improving it.”&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;This isn’t just a military problem. &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://academic.oup.com/isq/article/68/2/sqae020/7638566&quot;&gt;A 2023 study by researchers from the University of Pennsylvania and the Council on Foreign Relations&lt;/a&gt; surveyed 9,000 people across nine countries and found strong evidence for automation bias in high-stakes decision-making with a twist that should give you pause, &lt;strong&gt;people with moderate AI familiarity showed the highest automation bias.&lt;/strong&gt; Not beginners, not experts the people in the middle. The ones who know enough to feel confident, but not enough to catch the subtle failures.&lt;/p&gt;
&lt;p&gt;When you attach a confident, well-structured explanation to an automated decision, the human reviewer is statistically more likely to approve it including when it’s wrong. The explainability feature that was added to increase trust has also, as a side effect, increased the likelihood that errors slip through unchallenged. Add to this the financial reality. Every additional agent in the pipeline isn’t free.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://beam.ai/agentic-insights/anthropics-new-billing-split-reveals-what-ai-agents-actually-cost&quot;&gt;Gartner’s 2026 analysis&lt;/a&gt; found that agentic workloads consume &lt;strong&gt;5 to 30 times more compute than standard single-model interactions&lt;/strong&gt; for equivalent business outcomes. In a multi-agent setup where each agent re-ingests prior outputs as context, a single task can run 10–15× the token count of a one-model call. Anthropic now bills agentic tool calls as a separate line item.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://fortune.com/2026/05/22/microsoft-ai-cost-problem-tokens-agents/&quot;&gt;Fortune reported in May 2026&lt;/a&gt; that inference costs now represent 85% of enterprise AI spend up from 40% in 2023. A five-agent system with a critic is not a cheap solution. It can run to four or five figures in US dollars monthly for a single modest project. That’s in the neighbourhood of a full-time engineer salary. Not instead of in addition to.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;TWO TRUE THINGS THAT COEXIST &lt;br/&gt;&lt;/strong&gt;&lt;/em&gt;
&lt;em&gt;Explainable AI is better than unexplainable AI. Showing reasoning is more useful than hiding it. Both of these are genuinely true. But a confident, readable explanation of a wrong answer is arguably more dangerous than a cryptic one because it’s much harder to push back on something that looks rigorous. The explanation is not proof of correctness.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;llms-start-from-scratch-every-single-time&quot;&gt;LLMs Start From Scratch Every Single Time&lt;/h2&gt;
&lt;h2 id=&quot;what-the-winning-product-did-2&quot;&gt;What the Winning Product Did&lt;/h2&gt;
&lt;p&gt;3rd place went to a product called &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://devpost.com/software/netra-empowering-the-visually-impaired?_gl=1*1eu6y3r*_gcl_au*NjcyODQ3NjU2LjE3ODA1MzcwMTk.*_ga*MTk1NDU1MDAwMC4xNzgwNTM3MDE5*_ga_0YHJK3Y10M*czE3ODA1MzcwMTkkbzEkZzEkdDE3ODA1MzczNTIkajYwJGwwJGgw&quot;&gt;Netra&lt;/a&gt;. It’s an accessibility tool for people with limited vision. The demo shows a user asking the system to help locate their water bottle. The AI scans the room, identifies where the bottle is relative to surrounding objects, and narrates the location precisely ‘on the desk, to the left of your silver laptop, approximately a metre away.’&lt;/p&gt;
&lt;p&gt;Genuinely useful. The underlying architecture is what the team called a ‘Memory Palace’ a persistent file that builds a spatial map of the user’s environment over time. The idea is that the AI accumulates context about a person’s space across sessions, so it doesn’t have to rediscover everything from zero each time the app opens.&lt;/p&gt;
&lt;p&gt;It’s a thoughtful solution to a real problem. And it illustrates with uncomfortable precision one of the most structurally significant limitations of current large language models.&lt;/p&gt;
&lt;h2 id=&quot;now-move-the-water-bottle&quot;&gt;Now Move the Water Bottle&lt;/h2&gt;
&lt;p&gt;Seriously. Move the water bottle to the kitchen. Don’t tell the AI. Come back the next day. The AI will still tell you the bottle is on the desk, to the left of the laptop. It rdwill say so with the same calm, measured confidence as yesterday. Because from the model’s perspective, the file says it’s there, the file is treated as ground truth, and the model has no mechanism to distinguish between data that’s current and data that’s six weeks stale.&lt;/p&gt;
&lt;p&gt;This is &lt;strong&gt;statelessness&lt;/strong&gt; one of the defining structural characteristics of current LLMs. Every session, the model starts fresh. It has no continuous thread of experience. No awareness of time passing. No internal sense that the world might have changed since its last update. The Memory Palace workaround solves the ‘no memory’ problem and simultaneously creates a new one: a model that treats stale data with the same confidence as fresh data, because it has no mechanism to tell the difference.&lt;/p&gt;
&lt;p&gt;At demo scale, this is manageable. At production scale, it becomes a data retention liability, a contamination risk, a scope creep issue, and in certain contexts medical, financial, legal a potential regulatory problem.&lt;/p&gt;
&lt;h2 id=&quot;klarna-found-out-the-hard-way&quot;&gt;Klarna Found Out the Hard Way&lt;/h2&gt;
&lt;p&gt;You want to see what statelessness looks like in a production deployment that didn’t work? Look at Klarna. In February 2024, the Swedish fintech company announced with considerable fanfare that its AI assistant had handled 2.3 million customer service conversations in its first month of global operation. Equivalent, they claimed, to the work of &lt;strong&gt;700 full-time agents&lt;/strong&gt;, operating 24/7 across 35 languages.&lt;/p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.maginative.com/article/klarna-dials-back-its-ai-customer-service-strategy-now-its-hiring-humans-again/&quot;&gt;The CEO publicly described wanting Klarna to be OpenAI’s ‘favourite guinea pig.’&lt;/a&gt; By mid-2025, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://lasoft.org/blog/klarna-walks-back-ai-overhaul-rehires-staff-after-customer-service-backlash/&quot;&gt;Klarna was quietly rebuilding&lt;/a&gt; its human customer service capacity. The statelessness problem was central, &lt;strong&gt;if a customer switched from chat to phone, or if their connection dropped mid-session, the bot forgot them completely.&lt;/strong&gt; Every channel restart was a blank slate. No history. No context. No continuity. Customers had to explain their situation from the beginning every single time.&lt;/p&gt;
&lt;p&gt;CEO Sebastian Siemiatkowski’s &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://vibegraveyard.ai/story/klarna-ai-assistant-customer-service-shift/&quot;&gt;own words to Bloomberg&lt;/a&gt; were remarkably candid: &lt;em&gt;“We focused too much on efficiency and cost. The result was lower quality, and that’s not sustainable.”&lt;/em&gt; Customer satisfaction had dropped 22%. The company re-hired human agents. The ‘AI replaced 700 workers’ headline became the ‘700 workers were a premature announcement’ correction.&lt;/p&gt;
&lt;p&gt;As of 2026, &lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://www.digitalapplied.com/blog/klarna-reverses-ai-layoffs-replacing-700-workers-backfired&quot;&gt;Digital Applied describes&lt;/a&gt; Klarna as ‘the canonical enterprise cautionary tale’ the concrete data point that investors and boards now require executives to explicitly address when proposing AI-driven headcount reductions.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;THE PATTERN TO SPOT &lt;br/&gt;&lt;/strong&gt;&lt;/em&gt;
&lt;em&gt;Every time you see “persistent memory,” “context retention,” or “long-term learning” in an AI product pitch, ask how fresh is the data? How does the system know when its memory is outdated? What happens when the memory is wrong and the model doesn’t know it’s wrong? Right now, in most implementations, the honest answer is, it doesn’t know. And it’ll tell you confidently anyway.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;how-to-read-any-ai-pitch-backward&quot;&gt;How to Read Any AI Pitch Backward&lt;/h2&gt;
&lt;p&gt;Let me give you a framework that’ll save you a lot of money and confusion. Every solution generates a new set of problems. That’s not a cynical statement it’s just how technology works. The railroad created rail engineers, signal operators, and track maintenance crews. Nobody predicted those specific jobs from the original pitch. But they arrived, reliably, because the technology needed them.&lt;/p&gt;
&lt;p&gt;AI is following the same pattern. The jobs aren’t disappearing. They’re changing shape, and in many cases becoming more expensive. The pitch says headcount goes down. The invoice says otherwise.&lt;/p&gt;
&lt;p&gt;Here’s how to read it backward.&lt;/p&gt;
&lt;h2 id=&quot;the-harness-engineer-example&quot;&gt;The ‘Harness Engineer’ Example&lt;/h2&gt;
&lt;p&gt;Forward reading (the pitch) -&lt;em&gt;”We’re hiring AI infrastructure engineers to orchestrate sophisticated multi-agent pipelines.”&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Backward reading — a single model drifts and loses coherence on anything longer than a certain context window. So you split the task across multiple narrow agents, add a critic agent to check their work, and hire a human the ‘harness engineer’ whose job is to make sure the whole crew stays on task and doesn’t spend the compute budget without converging.&lt;/p&gt;
&lt;p&gt;That harness engineer is a specialist hire. In 2026, you’re looking at $180,000–$350,000. And you don’t get just one. Underneath them sits a cost structure that doesn’t make it into the initial pitch conversation:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;A multi-agent pipeline with a critic re-ingests each agent’s output as context for the next, so a single task can consume 10–15× the tokens of a one-model call.&lt;/li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://beam.ai/agentic-insights/anthropics-new-billing-split-reveals-what-ai-agents-actually-cost&quot;&gt;Gartner puts the agentic multiplier at 5–30×&lt;/a&gt; versus standard interactions.&lt;/li&gt;
&lt;li&gt;Databases for retrieval, orchestration services to route tasks between agents, memory stores to preserve context across sessions, monitoring dashboards to catch when the pipeline starts hallucinating&lt;/li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer&quot; target=&quot;_blank&quot; href=&quot;https://edrm.net/2025/11/the-new-stanford-carnegie-study-hybrid-ai-teams-beat-fully-autonomous-agents-by-68-7/&quot;&gt;The Stanford–Carnegie Mellon study&lt;/a&gt; found something genuinely worth holding onto, &lt;strong&gt;when humans and AI agents work together with proper collaboration design, task completion improves by 68.7% over fully autonomous AI.&lt;/strong&gt; That’s not an argument for removing the human. That’s an argument for designing the collaboration properly from day one.&lt;/li&gt;
&lt;li&gt;The guardrails tell the story too. Iteration caps, cost ceilings, and timeout limits aren’t safety polish they’re admissions that left unconstrained, an agent will keep running and keep spending without necessarily reaching a useful conclusion&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;And finally the thing that should give every executive pause. You are building a new operating model that most people in the organisation have never worked with before, can’t audit in the traditional sense, and have almost no playbook for managing when something goes wrong.&lt;/p&gt;
&lt;p&gt;None of that was in the pitch. All of it is a tax on the model’s current limitations, repackaged with a new job title.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;&lt;strong&gt;THE BACKWARD READING CHECKLIST&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;1. What limitation is the workaround actually compensating for? That’s the real AI gap.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;2. What new problem does the workaround introduce? There’s always one.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;3. What is the full cost? compute, infrastructure, human oversight combined? Not just the licence.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;4. What happens when the AI is wrong? Is the failure visible and recoverable, or silent and compounding?&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;5. What new roles does this create? Budget for them now, or get surprised by them on the invoice.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;what-this-means-for-you-and-where-the-real-jobs-are&quot;&gt;What This Means for You and Where the Real Jobs Are&lt;/h2&gt;
&lt;p&gt;Here’s the part that gets missed in most AI discourse. The conversation is almost entirely about which jobs AI will eliminate. Much less attention goes to the specific, well-compensated roles that the AI transition is actively creating and the fact that most people are still looking for those roles in the wrong places.&lt;/p&gt;
&lt;p&gt;Let me be direct about something if you understand what you just read if you understand human should be in the middle architecture, automation bias, statelessness, and the real cost structure of agentic AI you are already more useful to most organisations than the majority of people applying for AI-adjacent roles right now.&lt;/p&gt;
&lt;p&gt;The question is how to get in front of the people doing the hiring. And that’s where it gets interesting, because the answer is almost never ‘apply on LinkedIn.’&lt;/p&gt;
&lt;h2 id=&quot;the-roles-the-ai-transition-is-actually-creating&quot;&gt;The Roles the AI Transition Is Actually Creating&lt;/h2&gt;
&lt;p&gt;Before we get into how to find these roles, let’s be specific about what they are. Based on the patterns visible in the hackathon winners and the broader industry trajectory, the roles with real demand in 2026 are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;AI workflow designers:&lt;/strong&gt; people who map out which parts of a task should go to an agent and which need a human, and can design the handoff between them clearly&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Agent oversight specialists:&lt;/strong&gt; the humans sitting at the end of high-stakes pipelines who understand the domain well enough to catch what the critic missed&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Prompt and system architects:&lt;/strong&gt; people who write the instructions that tell each agent what it’s responsible for, and can debug the pipeline when it drifts&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;AI output auditors:&lt;/strong&gt; roles in legal, finance, healthcare, and logistics where the cost of an AI error is high enough that dedicated human review is structurally required&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Infrastructure engineers for agentic stacks:&lt;/strong&gt; the people who build and maintain the databases, orchestration layers, memory stores, and monitoring tools that multi-agent systems run on&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Technical writers and documentation specialists:&lt;/strong&gt; because every one of those agents needs clear instructions, every workflow needs a runbook, and most AI teams are terrible at this&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Notice something about that list? None of those roles is ‘AI replaces this person.’ All of them exist because AI has created new problems that need humans to manage. The original pitch is still wrong. The jobs are still here.&lt;/p&gt;
&lt;h2 id=&quot;what-the-hackathon-actually-proved&quot;&gt;What the Hackathon Actually Proved&lt;/h2&gt;
&lt;p&gt;The three products that won the Gemini 3 Hackathon are genuinely impressive pieces of work. The teams behind them solved real problems with real sophistication. I’m not trying to diminish that.&lt;/p&gt;
&lt;p&gt;But the reason they won is the most interesting thing about them. They didn’t win because they pushed AI farthest into autonomy. They won because they were the most honest about where autonomy currently breaks and the most thoughtful about designing systems that work within those constraints.&lt;/p&gt;
&lt;p&gt;The grand prize os keep the human in the loop. Second place: show the reasoning so humans can verify it. Third: build a memory system because AI has none. Every one of those design decisions is a response to a real, current, documented limitation of large language models.&lt;/p&gt;
&lt;p&gt;The pitch says the AI will handle it. The hackathon winners say here’s what it can do if you design carefully around what it can’t. Those are not the same statement. Learning to tell them apart in a job interview, in a vendor pitch, in a board presentation is one of the most valuable things you can do in 2026.&lt;/p&gt;
&lt;p&gt;And the good news is the people who can do that clearly are exactly who the industry is hiring. The roles are real. The compensation is real. The work is genuinely interesting.&lt;/p&gt;
&lt;p&gt;You just have to know where to look for it and how to show up in the conversation before the formal listing goes live.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;If this changed how you read an AI pitch, share it with someone who’s about to sit through one. It’ll save them some money.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Disclaimer: This article reflects personal analysis and commentary based on publicly available information. All statistics are sourced from cited research. Product descriptions are based on publicly available hackathon submissions and press coverage.&lt;/p&gt;</content:encoded><dc:creator>Shivashish Yadav</dc:creator><category>AI &amp; GPU compute</category><category>ai</category><category>jobs</category><category>ceo</category><category>llm</category><category>artificial-intelligence</category><author>hello@soursen.org (Shivashish Yadav)</author></item></channel></rss>