{"id":50126,"date":"2024-12-14T10:56:03","date_gmt":"2024-12-14T05:26:03","guid":{"rendered":"http:\/\/officechai.com\/?p=50126"},"modified":"2025-01-19T03:08:49","modified_gmt":"2025-01-18T21:38:49","slug":"data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever","status":"publish","type":"post","link":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/","title":{"rendered":"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever"},"content":{"rendered":"\n<p>AI has been progressing at breakneck pace over the last couple of years, but one of the chief resources behind this rapid progress might be rapidly depleting.<\/p>\n\n\n\n<p>Former OpenAI Chief Scientist and founder of SSI Ilya Sutskever has said that data is the &#8220;fossil fuel&#8221; of AI, and will inevitably soon be exhausted. He said that even as computing resources such as chips and algorithms were still improving, the gains from these advances could be stymied by the lack of fresh data to train models on.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" fetchpriority=\"high\" decoding=\"async\" width=\"640\" height=\"336\" src=\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?resize=640%2C336\" alt=\"\" class=\"wp-image-49869\" srcset=\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?resize=1024%2C538&amp;ssl=1 1024w, https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?resize=300%2C158&amp;ssl=1 300w, https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?resize=768%2C403&amp;ssl=1 768w, https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?w=1200&amp;ssl=1 1200w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/figure>\n\n\n\n<p>&#8220;Pre-training as we know it will unquestionably end,&#8221; he <a href=\"https:\/\/www.youtube.com\/watch?v=1yvBqasHLZs&amp;ab_channel=seremot\">said <\/a>at a talk at NeurIPS. &#8220;Pre training will end. Why will it end? Because while compute is growing through better hardware, better algorithms, and larger clusters &#8212; all those things keep increasing your compute &#8212; the data is not growing because we have but one internet,&#8221; he added.<\/p>\n\n\n\n<p>&#8220;We have but one intranet,&#8221; Sutskever went on. &#8220;We have but one intranet. You could even say, you can even go as far as to say that data is the fossil fuel of AI. It was like created somehow, and now we use it, and we&#8217;ve achieved peak data, and there&#8217;ll be no more. We have to deal with the data that we have. I would still still let us go quite far, but this is, there&#8217;s only one internet,&#8221; he said.<\/p>\n\n\n\n<p>Pre-training is one of the initial steps in training an AI model in which the model is fed large amounts of data. This enables it to understand language patters, facts and other concepts. Thus far, researchers had been using data scraped from the internet to train these models. But researchers have already used up most of the data that was available, which means that there&#8217;s will soon be no more data available to make even more sophisticated AI models.<\/p>\n\n\n\n<p>But Sutskever says that this doesn&#8217;t mean that progress in AI is about to end.  He said there are new interfaces, such as agents, being created, which could help AI solve newer problems. Researchers could also create &#8220;synthetic data&#8221;, which would likely be data created by existing AI models to train future models. And there could be newer approaches such as inference-time compute used in OpenAI&#8217;s o-1 model, which cause the model to think deeply before it responds, which has led to some better outcomes. And Sutskever said that in spite of our rapidly-depleting data stores, these approaches, along with others, could eventually take humanity to superintelligence.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>AI has been progressing at breakneck pace over the last couple of years, but one of the chief resources behind this rapid progress&#8230;<\/p>\n","protected":false},"author":1,"featured_media":49869,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1029,46],"tags":[],"class_list":["post-50126","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-stories"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever\" \/>\n<meta property=\"og:description\" content=\"AI has been progressing at breakneck pace over the last couple of years, but one of the chief resources behind this rapid progress...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/\" \/>\n<meta property=\"og:site_name\" content=\"OfficeChai\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/OfficeChai\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-12-14T05:26:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-01-18T21:38:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"OfficeChai Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@OfficeChai\" \/>\n<meta name=\"twitter:site\" content=\"@OfficeChai\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"OfficeChai Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/\",\"url\":\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/\",\"name\":\"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever\",\"isPartOf\":{\"@id\":\"https:\/\/officechai.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1\",\"datePublished\":\"2024-12-14T05:26:03+00:00\",\"dateModified\":\"2025-01-18T21:38:49+00:00\",\"author\":{\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2\"},\"breadcrumb\":{\"@id\":\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1\",\"width\":1200,\"height\":630},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/officechai.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/officechai.com\/#website\",\"url\":\"https:\/\/officechai.com\/\",\"name\":\"OfficeChai\",\"description\":\"Startups, Businesses And Careers\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/officechai.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2\",\"name\":\"OfficeChai Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g\",\"caption\":\"OfficeChai Team\"},\"description\":\"Dotting the i's, crossing the t's.\",\"url\":\"https:\/\/officechai.com\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/","og_locale":"en_US","og_type":"article","og_title":"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever","og_description":"AI has been progressing at breakneck pace over the last couple of years, but one of the chief resources behind this rapid progress...","og_url":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/","og_site_name":"OfficeChai","article_publisher":"https:\/\/www.facebook.com\/OfficeChai\/","article_published_time":"2024-12-14T05:26:03+00:00","article_modified_time":"2025-01-18T21:38:49+00:00","og_image":[{"width":1200,"height":630,"url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1","type":"image\/jpeg"}],"author":"OfficeChai Team","twitter_card":"summary_large_image","twitter_creator":"@OfficeChai","twitter_site":"@OfficeChai","twitter_misc":{"Written by":"OfficeChai Team","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/","url":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/","name":"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever","isPartOf":{"@id":"https:\/\/officechai.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#primaryimage"},"image":{"@id":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1","datePublished":"2024-12-14T05:26:03+00:00","dateModified":"2025-01-18T21:38:49+00:00","author":{"@id":"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2"},"breadcrumb":{"@id":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#primaryimage","url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1","contentUrl":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1","width":1200,"height":630},{"@type":"BreadcrumbList","@id":"https:\/\/officechai.com\/stories\/data-is-the-fossil-fuel-of-ai-it-will-get-exhausted-ilya-sutskever\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/officechai.com\/"},{"@type":"ListItem","position":2,"name":"Data Is The Fossil Fuel Of AI, It Will Get Exhausted: Ilya Sutskever"}]},{"@type":"WebSite","@id":"https:\/\/officechai.com\/#website","url":"https:\/\/officechai.com\/","name":"OfficeChai","description":"Startups, Businesses And Careers","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/officechai.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2","name":"OfficeChai Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/officechai.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g","caption":"OfficeChai Team"},"description":"Dotting the i's, crossing the t's.","url":"https:\/\/officechai.com\/author\/admin\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2024\/11\/MixCollage-21-Nov-2024-11-10-PM-6589.jpg?fit=1200%2C630&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p685C6-d2u","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/50126","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/comments?post=50126"}],"version-history":[{"count":1,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/50126\/revisions"}],"predecessor-version":[{"id":50128,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/50126\/revisions\/50128"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/media\/49869"}],"wp:attachment":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/media?parent=50126"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/categories?post=50126"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/tags?post=50126"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}