{"id":57554,"date":"2025-11-29T16:54:29","date_gmt":"2025-11-29T11:24:29","guid":{"rendered":"https:\/\/officechai.com\/?p=57554"},"modified":"2025-11-29T16:54:32","modified_gmt":"2025-11-29T11:24:32","slug":"openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis","status":"publish","type":"post","link":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/","title":{"rendered":"OpenAI Hasn’t Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis"},"content":{"rendered":"\n

OpenAI appears to have been leapfrogged, at least on benchmarks, by rivals including Google <\/a>and Anthropic <\/a>in the AI race, and it may have to do with one of the first steps in creating a modern LLM — pretraining.<\/p>\n\n\n\n

Semiconductor industry publication SemiAnalysis has claimed that OpenAI hasn’t completed a successful pretraining run since May 2024. SemiAnalysis mentioned this in a piece <\/a>in which it compared the relative strengths of Google’s TPU chips compared to NVIDIA’s GPUs. TPUs have seen plenty of interest in recent days after Google’s Gemini 3 model — trained entirely on TPUs — topped benchmarks, and Google struck a deal<\/a> with Anthropic to supply 1 million TPUs.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

“Pre-training a frontier model remains the hardest and most resource-intensive challenge in AI hardware,” SemiAnalysis wrote in its post. “The TPU platform has passed that test decisively. This stands in sharp contrast to rivals: OpenAI\u2019s leading researchers have not completed a successful full-scale pre-training run that was broadly for a new frontier model since GPT-4o in May 2024, highlighting the significant technical hurdle that Google\u2019s TPU fleet has managed to overcome,” it added.<\/p>\n\n\n\n

Pretraining is\u00a0the initial, foundational phase of training a large language model (LLM) on a massive, diverse dataset of text and code. Its goal is to teach the model a general understanding of language, including its grammar, semantics, and factual knowledge, by having it predict the next word in a sequence. This results in a versatile, but “raw,” base model that can then be customized for specific tasks through a process called\u00a0fine-tuning.\u00a0<\/p>\n\n\n\n

SemiAnalysis’ claims would imply that OpenAI’s pre-training runs for newer models likely failed or were aborted, and models like GPT-5 were trained over pre-training runs of older models. Interestingly, Google has now come out and said that it’s still seeing improvements in pre-training. “The secret behind Gemini 3? Simple: Improving pre-training & post-training,\u201d Google DeepMind’s Oriol Vinyals had said<\/a>. \u201cPre-training: Contra the popular belief that scaling is over\u2014which we discussed in our NeurIPS \u201925 talk with Ilya Sutskever and Quoc Le\u2014the team delivered a drastic jump. The delta between 2.5 and 3.0 is as big as we\u2019ve ever seen. No walls in sight!\u201d he had added.<\/p>\n\n\n\n

GPT-5 had also largely failed to meet its (admittedly high) expectations, and even though it had topped benchmarks, was soon overshadowed by newer releases from Google and Anthropic. If SemiAnalysis’s information is correct, it’s possible that GPT-5 was unable to benefit from the pre-training advances that Google saw, which caused the model to be relatively weak in comparison. While it’s impossible to tell if the SemiAnalysis report is accurate — information such as this is carefully kept under wraps by top labs — it could be a plausible explanation for OpenAI losing the AI benchmark crown for the first time since the release of ChatGPT in late 2022.<\/p>\n","protected":false},"excerpt":{"rendered":"

OpenAI appears to have been leapfrogged, at least on benchmarks, by rivals including Google and Anthropic in the AI race, and it may…<\/p>\n","protected":false},"author":1,"featured_media":50667,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1029],"tags":[],"class_list":["post-57554","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"yoast_head":"\nOpenAI Hasn't Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"OpenAI Hasn't Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis\" \/>\n<meta property=\"og:description\" content=\"OpenAI appears to have been leapfrogged, at least on benchmarks, by rivals including Google and Anthropic in the AI race, and it may...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/\" \/>\n<meta property=\"og:site_name\" content=\"OfficeChai\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/OfficeChai\/\" \/>\n<meta property=\"article:published_time\" content=\"2025-11-29T11:24:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-29T11:24:32+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"OfficeChai Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@OfficeChai\" \/>\n<meta name=\"twitter:site\" content=\"@OfficeChai\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"OfficeChai Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/\",\"url\":\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/\",\"name\":\"OpenAI Hasn't Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis\",\"isPartOf\":{\"@id\":\"https:\/\/officechai.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg?fit=1200%2C630&ssl=1\",\"datePublished\":\"2025-11-29T11:24:29+00:00\",\"dateModified\":\"2025-11-29T11:24:32+00:00\",\"author\":{\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2\"},\"breadcrumb\":{\"@id\":\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg?fit=1200%2C630&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg?fit=1200%2C630&ssl=1\",\"width\":1200,\"height\":630},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/officechai.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"OpenAI Hasn’t Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/officechai.com\/#website\",\"url\":\"https:\/\/officechai.com\/\",\"name\":\"OfficeChai\",\"description\":\"Startups, Businesses And Careers\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/officechai.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2\",\"name\":\"OfficeChai Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g\",\"caption\":\"OfficeChai Team\"},\"description\":\"Dotting the i's, crossing the t's.\",\"url\":\"https:\/\/officechai.com\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"OpenAI Hasn't Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/","og_locale":"en_US","og_type":"article","og_title":"OpenAI Hasn't Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis","og_description":"OpenAI appears to have been leapfrogged, at least on benchmarks, by rivals including Google and Anthropic in the AI race, and it may...","og_url":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/","og_site_name":"OfficeChai","article_publisher":"https:\/\/www.facebook.com\/OfficeChai\/","article_published_time":"2025-11-29T11:24:29+00:00","article_modified_time":"2025-11-29T11:24:32+00:00","og_image":[{"width":1200,"height":630,"url":"http:\/\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg","type":"image\/jpeg"}],"author":"OfficeChai Team","twitter_card":"summary_large_image","twitter_creator":"@OfficeChai","twitter_site":"@OfficeChai","twitter_misc":{"Written by":"OfficeChai Team","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/","url":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/","name":"OpenAI Hasn't Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis","isPartOf":{"@id":"https:\/\/officechai.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#primaryimage"},"image":{"@id":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg?fit=1200%2C630&ssl=1","datePublished":"2025-11-29T11:24:29+00:00","dateModified":"2025-11-29T11:24:32+00:00","author":{"@id":"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2"},"breadcrumb":{"@id":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#primaryimage","url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg?fit=1200%2C630&ssl=1","contentUrl":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg?fit=1200%2C630&ssl=1","width":1200,"height":630},{"@type":"BreadcrumbList","@id":"https:\/\/officechai.com\/ai\/openai-hasnt-completed-a-successful-full-scale-pretraining-run-since-gpt-4o-in-may-2024-says-semianalysis\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/officechai.com\/"},{"@type":"ListItem","position":2,"name":"OpenAI Hasn’t Completed A Successful Full-Scale Pretraining Run Since GPT-4o In May 2024, Says SemiAnalysis"}]},{"@type":"WebSite","@id":"https:\/\/officechai.com\/#website","url":"https:\/\/officechai.com\/","name":"OfficeChai","description":"Startups, Businesses And Careers","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/officechai.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2","name":"OfficeChai Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/officechai.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g","caption":"OfficeChai Team"},"description":"Dotting the i's, crossing the t's.","url":"https:\/\/officechai.com\/author\/admin\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/02\/MixCollage-01-Feb-2025-04-32-AM-8995.jpg?fit=1200%2C630&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p685C6-eYi","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/57554","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/comments?post=57554"}],"version-history":[{"count":1,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/57554\/revisions"}],"predecessor-version":[{"id":57555,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/57554\/revisions\/57555"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/media\/50667"}],"wp:attachment":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/media?parent=57554"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/categories?post=57554"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/tags?post=57554"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}