AI supercomputers have become increasingly expensive. Since 2019, the cost of the computing hardware for leading supercomputers has increased at a rate of 1.9x per year. In June 2022, the most expensive cluster was Oak Ridge National Laboratory Frontier, with a reported cost of $200M. Three years later, as of June 2025, the most expensive supercomputer is xAI’s Colossus, estimated to use over $7B of hardware.
| Name | Status | Certainty | Single cluster? | Max OP/s (log) | H100 equivalents | Chip type (primary) | Chip quantity (primary) | Country | Owner | First Operational Date | Note | Sector | Power Capacity (MW) | Hardware Cost | Location | Users | Hardware note | Quote | First Operational Date Note | Certainty Note | Energy Efficiency (log) | Builds Upon | Superseded by | Possible Duplicate | Possible Duplicate Of | Chip type (secondary) | Chip quantity (secondary) | Total number of AI chips | GPU Supplier (primary) | GPU supplier (secondary) | Include in Standard Analysis | Exclude | Rank when first operational | 16-bit OP/s (log) | Max OP/s | 8-bit OP/s | 16-bit OP/s | 32-bit OP/s | Calculated Power Capacity (MW) | Reported Power Capacity (MW) | Energy Efficiency | Calculated Cost | Reported Cost | Reported Cost (Inflation adjusted) | Cost Quote | Noteworthy | Decommissioned Date (if applicable) | Largest existing cluster when first operational | % of largest cluster when first operational | Source 1 | Source 2 | Source 3 | Source 4 | Source 5 | Latitude | Longitude |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Abu Dhabi UAE/USA 5GW Campus Phase 2 | Planned | Likely | Unclear | 22.60 | 2.0e+7 | United Arab Emirates | FLOP/s is a very rough estimate based on stated power capacity and the completion year | Public/Private | 5,000.00 | Abu Dhabi, United Arab Emirates | US Companies,G42 | Uncertain | "Breaking ground on 1GW AI Datacenter, part of a planned 5GW UAE-US artificial intelligence campus; largest such deployment outside of the US"
"The campus, which will span 10 square miles within the Emirate, will be built by G42 and operated in partnership with several US companies." | Planned 2030 | 12.60 | Stargate UAE Phase 2 | Unknown | 22.30 | 4.0e+22 | 4.0e+22 | 2.0e+22 | 5,000.00 | 4.0e+12 | True | https://web.archive.org/web/20250515200336/https://www.commerce.gov/news/press-releases/2025/05/uae-and-us-presidents-attend-unveiling-phase-1-new-5gw-ai-campus-abu | 24.45 | 54.38 | |||||||||||||||||||||||||||||
| Meta $200B Campus Rumor | Planned | Unlikely | Yes | 22.60 | 2.0e+7 | United States of America | Meta AI | FLOP/s is a very rough estimate based on stated power capacity (~5-7GW) and the completion year. Plans for this cluster are very much still rumors/vauge, and it's highly likely that they're changed.
2030 is a rough estiamted date for when this might be completed | Private | 5,000.00 | 2.0e+11 | Meta | Uncertain | "could cost over $200 billion" | Planned 2030 | 12.60 | Unknown | 22.30 | 4.0e+22 | 4.0e+22 | 2.0e+22 | 5,000.00 | 4.0e+12 | 2.0e+11 | 2.0e+11 | $200B | https://web.archive.org/web/20250226194100/https://finance.yahoo.com/news/meta-weighs-200b-ai-data-134016347.html | 39.38 | -97.92 | |||||||||||||||||||||||||||
| DataVolt Neom 1.5 GW Phase 2 | Planned | Likely | Unclear | 22.00 | 5.1e+6 | Saudi Arabia | DataVolt | Planned year is a very rough guess | Private | 1,500.00 | 27°33'21.6"N 35°32'18.4"E | Uncertain | "At the LEAP 2025 event this week DataVolt and Neom signed a $5bn agreement to develop a 1.5GW data center campus in Neom’s Oxagon industrial area.
The first 300MW phase of the site will reportedly go live in 2028" | Planned 2031 | 12.52 | DataVolt Neom 1.5 GW Phase 1 | Unknown | 21.70 | 1.0e+22 | 1.0e+22 | 5.0e+21 | 1,500.00 | 3.3e+12 | 5.0e+9 | http://web.archive.org/web/20250404231606/https://www.datacenterdynamics.com/en/news/datavolt-plans-15gw-data-center-campus-in-neoms-oxagon/ | 24.68 | 46.63 | |||||||||||||||||||||||||||||
| Meta Louisiana Datacenter | Planned | Likely | Unclear | 22.00 | 5.1e+6 | United States of America | Meta AI | FLOP/s is a very rough estimate based on stated power capacity and the completion year | Private | 2,260.00 | Holly Ridge Louisiana 71269 | Meta | Uncertain | "Mark Zuckerberg said the site would total more than 2GW at full build-out and would be used to train its Llama AI models."
"This will include the construction of three combined-cycle combustion turbines with a combined capacity of 2,260MW" | Planned 2030 | 12.35 | Unknown | 21.70 | 1.0e+22 | 1.0e+22 | 5.0e+21 | 2,260.00 | 2.2e+12 | https://archive.ph/rS3ef | 32.48 | -91.76 | ||||||||||||||||||||||||||||||
OpenAI Stargate Abilene Oracle OCI Supercluster Phase 3 | Planned | Likely | Yes | 22.00 | 5.1e+6 | United States of America | Oracle | "That cluster is expected to be leased to Oracle, which in turn will lease it to Microsoft, who will rent it to OpenAI"
Crusoe is involved in this
FLOP/s figure is a very rough estimate based off some rumored specifications about chip performance | Private | 2,200.00 | 5502 Spinks Rd, Abilene, TX 79601 | Cloud,Microsoft,OpenAI | VR200,GB300,GB200 | "The initial plan here was a 1GW facility; SA now thinks it’s going to have a total of 2.2GW of power supplied and a critical IT capacity of 1.8GW."
"SA estimates the campus will house a previously-announced 100k GB200 cluster, as well as a 200k GB300 cluster and a 400k VR200 cluster" | Planned 2027 | Low confidence since plans are still somewhat up in the air | 12.36 | OpenAI Stargate Abilene Oracle OCI Supercluster Phase 2 | Unknown | 21.70 | 1.0e+22 | 1.0e+22 | 5.0e+21 | 2,200.00 | 2.3e+12 | True | https://archive.ph/B3HJm#selection-1045.0-1045.83 | https://archive.ph/UeNZs | https://web.archive.org/web/20250124161138/https://www.transformernews.ai/p/unravelling-the-stargate-spin | 32.49 | -99.79 | |||||||||||||||||||||||||
| South Korea Planned 3GW Cluster | Planned | Likely | Unclear | 22.00 | 5.1e+6 | Korea (Republic of) | FLOP/s is a very rough estimate based on stated power capacity and the completion year | Private | 3,000.00 | southwest South Korea | Uncertain | If completed as envisioned by its backers, the data center will cost as much as $35 billion and pack up to 3 gigawatts of power. | Planned 2028 | 12.22 | Unknown | 21.70 | 1.0e+22 | 1.0e+22 | 5.0e+21 | 3,000.00 | 1.7e+12 | https://www.wsj.com/tech/ai/ai-data-center-with-up-to-3-gigawatts-of-power-is-envisioned-for-south-korea-5141bd77 | 36.54 | 128.17 | ||||||||||||||||||||||||||||||||
| Stargate UAE Phase 2 | Planned | Likely | Unclear | 21.70 | 2.5e+6 | United Arab Emirates | Stargate (OpenAI) | Part of the broader planned 5GW UAE/USA cluster
Planned date is a rough guess
FLOP/s is a very very rough guess based on power capacity and expected completion date | Public/Private | 1,000.00 | Abu Dhabi, United Arab Emirates | OpenAI | Uncertain | "Breaking ground on 1GW AI Datacenter, part of a planned 5GW UAE-US artificial intelligence campus; largest such deployment outside of the US"
"The campus, which will span 10 square miles within the Emirate, will be built by G42 and operated in partnership with several US companies."
"Stargate UAE is a next-generation AI infrastructure cluster that will run in the newly established 5-gigawatt UAE–U.S. AI Campus in Abu Dhabi. Stargate UAE, a 1-gigawatt compute cluster" | Planned 2029 | 12.40 | Stargate UAE Phase 1 | Unknown | 21.40 | 5.0e+21 | 5.0e+21 | 2.5e+21 | 1,000.00 | 2.5e+12 | https://web.archive.org/web/20250515200336/https://www.commerce.gov/news/press-releases/2025/05/uae-and-us-presidents-attend-unveiling-phase-1-new-5gw-ai-campus-abu | https://archive.ph/hQWdM | https://archive.ph/PfVwI | 24.45 | 54.38 | |||||||||||||||||||||||||||
| xAI Colossus 2 Memphis Phase 2 | Planned | Likely | Yes | 21.44 | 1.4e+6 | NVIDIA GB200 NVL2 (per GPU) | 330,000.00 | United States of America | xAI | 550k GB200/GB300 planned. Unclear what the proportion of GB200 vs GB300 will be. The first 110k will be GB200s. I will naively assume that the remaining will be split 50/50 GB200/GB300, so
330k GB200s
220k GB300s
- Elon has claimed that this will be complete in 6-9 months, so presumably early-mid 2026 | Private | 1,563.56 | 5420 Tulane Rd, Memphis, TN 38109 | xAI | 550k GB200/GB300s,Blackwell,GB200,GB300 | At Colossus 2, the first batch of 550k GB200s & GB300s, also for training, start going online in a few weeks. | Planned 2026 | 11.94 | xAI Colossus 2 Memphis Phase 1 | NVIDIA GB300 | 220,000 | 550,000.00 | NVIDIA | NVIDIA | 21.14 | 2.8e+21 | 2.8e+21 | 1.4e+21 | 6.9e+20 | 1,563.56 | 8.8e+11 | True | https://archive.ph/oHQSA#selection-585.0-597.43 | https://www.cnbc.com/2025/05/20/elon-musk-says-ai-could-run-into-power-capacity-issues-by-middle-of-next-year.html | https://web.archive.org/web/20241205161745/https://memphischamber.com/blog/general/xai-memphis-announces-expansion-of-supercomputer-with-addition-of-tech-companies-in-digital-delta/ | https://archive.ph/WYQTZ | 35.00 | -90.04 | ||||||||||||||||||
"OpenAI/Microsoft Mt Pleasant, Wisconsin Phase 2" | Planned | Likely | Yes | 21.54 | 1.8e+6 | NVIDIA GB200 NVL2 (per GPU) | 700,000.00 | United States of America | OpenAI,Microsoft | Rough estimate of chip count from stated power capacity of datacenter.
Estimate from Semianalysis that Microsoft plans to build four of the 300MW buildings
What type of GPUs is unclear, but I'm assuming they're B200s | Private | 1,500.00 | M39X+XG, Mt Pleasant, WI 53177 | OpenAI,Microsoft | GB200,Uncertain | "1.5GW within 2.5 years"
"Microsoft plans to build four 300MW buildings... 1.5GW is like a million GPUs" | Planned 2026 | They announced Jan 2025 that they are currently pausing some planned construction
Estimate from Semianalysis that Microsoft plans to build four of the 300MW buildings, which translates to ~1 million GPUs. Type of GPU uncertain | 12.07 | OpenAI/Microsoft Mt Pleasant, Wisconsin Phase 1 | 700,000.00 | NVIDIA | 21.24 | 3.5e+21 | 3.5e+21 | 1.8e+21 | 8.8e+20 | 1,681.68 | 1,500.00 | 1.2e+12 | https://web.archive.org/web/20240717135010/https://www.tomshardware.com/tech-industry/artificial-intelligence/openai-and-microsoft-reportedly-planning-dollar100-billion-datacenter-project-for-an-ai-supercomputer | https://web.archive.org/web/20240915193816/https://www.datacenterdynamics.com/en/news/microsoft-openai-consider-100bn-5gw-stargate-ai-data-center-report/ | https://www.youtube.com/watch?v=hobvps-H38o | 42.70 | -87.89 | |||||||||||||||||||||
| HUMAIN Saudi Arabia Phase 2 | Planned | Likely | Unclear | 21.48 | 1.5e+6 | Saudi Arabia | Humain | Seems to be partnering with AMD and NVIDIA
FLOP/s is a very rough estimate based on stated power capacity and the completion year | Public/Private | 500.00 | Uncertain | "HUMAIN is making a major investment to build AI factories in the Kingdom of Saudi Arabia with a projected capacity of up to 500 megawatts powered by several hundred thousand of NVIDIA’s most advanced GPUs over the next five years. The first phase of deployment will be an 18,000 NVIDIA GB300 Grace Blackwell AI supercomputer with NVIDIA InfiniBand networking." | Planned 2030 | 12.48 | HUMAIN Saudi Arabia/NVIDIA Phase 1 | Unknown | 21.18 | 3.0e+21 | 3.0e+21 | 1.5e+21 | 500.00 | 3.0e+12 | https://archive.ph/vvAxQ | https://archive.ph/CFcnD | http://web.archive.org/web/20250517061228/https://ir.amd.com/news-events/press-releases/detail/1250/amd-and-humain-form-strategic-10b-collaboration-to-advance-global-ai | 23.38 | 44.65 | |||||||||||||||||||||||||||||
| Fluidstack France Gigawatt Campus | Planned | Likely | Yes | 21.40 | 1.3e+6 | NVIDIA GB200 NVL2 (per GPU) | 500,000.00 | France | Unclear if the 500k chips refers to GPUs or entire chips (which have 2 GPUs) | Public/Private | 1,000.00 | France | Mistral | GB200,Uncertain | "France and UAE to invest billions into 1GW European AI data center"
"The facility’s Phase 1 will ultimately host close to 500,000 next-generation AI chips." | Planned 2028 | 12.10 | 500,000.00 | NVIDIA | 21.10 | 2.5e+21 | 2.5e+21 | 1.3e+21 | 6.3e+20 | 1,201.20 | 1,000.00 | 1.3e+12 | https://web.archive.org/web/20250211011508/https://www.datacenterdynamics.com/en/news/france-and-uae-to-invest-billions-into-1gw-european-ai-data-center/ | https://web.archive.org/web/20250210220905/https://www.hpcwire.com/off-the-wire/fluidstack-to-build-1gw-ai-supercomputer-in-france/ | 47.82 | 2.62 | |||||||||||||||||||||||||
| Reliance Industries Supercomputer | Planned | Likely | Yes | 21.35 | 1.1e+6 | NVIDIA GB200 NVL2 (per GPU) | 450,000.00 | India | Reliance Industries | Chip count estimated from power capacity. Chip type uncertain | Private | 1,000.00 | Jamnagar, India | Reliance Jio,Cloud | GB200 | will supply its Blackwell AI processors for a one-gigawatt data centre Reliance is building in the western state of Gujarat | Planned 2027 | 12.05 | 450,000.00 | NVIDIA | 21.05 | 2.3e+21 | 2.3e+21 | 1.1e+21 | 5.6e+20 | 1,081.08 | 1,000.00 | 1.1e+12 | https://archive.ph/tTvHo | 22.47 | 70.07 | |||||||||||||||||||||||||
OpenAI Stargate Abilene Oracle OCI Supercluster Phase 2 | Planned | Likely | Yes | 21.18 | 7.6e+5 | NVIDIA GB300 | 200,001.00 | United States of America | Oracle | Unclear exact FLOPs for GB300, but sources indicate that it is 1.5x a GB200.
"That cluster is expected to be leased to Oracle, which in turn will lease it to Microsoft, who will rent it to OpenAI"
Crusoe is involved in this | Private | 940.94 | 5502 Spinks Rd, Abilene, TX 79601 | Cloud,Microsoft,OpenAI | GB200,GB300 | "SA estimates the campus will house a previously-announced 100k GB200 cluster, as well as a 200k GB300 cluster and a 400k VR200 cluster" | Planned 2026 | Low confidence since plans are still somewhat up in the air | 11.90 | OpenAI Stargate Abilene Oracle OCI Supercluster Phase 1 | NVIDIA GB200 NVL2 (per GPU) | 100,000 | 300,001.00 | NVIDIA | NVIDIA | 20.88 | 1.5e+21 | 1.5e+21 | 7.5e+20 | 3.8e+20 | 940.94 | 8.0e+11 | https://archive.ph/B3HJm#selection-1045.0-1045.83 | https://archive.ph/UeNZs | https://web.archive.org/web/20250124161138/https://www.transformernews.ai/p/unravelling-the-stargate-spin | 32.49 | -99.79 | |||||||||||||||||||
| OpenAI/Microsoft Atlanta | Planned | Likely | Yes | 21.13 | 6.8e+5 | NVIDIA B200 | 300,000.00 | United States of America | OpenAI,Microsoft | Rough estimate of chip count from stated power capacity of datacenter.
Assuming chips are B200s | Private | 700.00 | 1.3e+10 | 1435 Hwy 54 W, Fayetteville, GA 30214 | OpenAI,Microsoft | B200,Uncertain | "700MW within 1.5 years" | Planned 2026 | Rought estimate from power capacity | 11.98 | 300,000.00 | NVIDIA | 20.83 | 1.4e+21 | 1.4e+21 | 6.8e+20 | 3.4e+20 | 600.60 | 700.00 | 9.6e+11 | 1.3e+10 | https://www.youtube.com/watch?v=hobvps-H38o (this cluster mentioned at 15:01) | 33.45 | -84.52 | ||||||||||||||||||||||
| DataVolt Neom 1.5 GW Phase 1 | Planned | Likely | Unclear | 21.00 | 5.1e+5 | Saudi Arabia | DataVolt | FLOP/s is a very rough estimate based on stated power capacity and the completion year | Private | 300.00 | 27°33'21.6"N 35°32'18.4"E | Uncertain | "At the LEAP 2025 event this week DataVolt and Neom signed a $5bn agreement to develop a 1.5GW data center campus in Neom’s Oxagon industrial area.
The first 300MW phase of the site will reportedly go live in 2028" | Planned 2028 | 12.22 | Unknown | 20.70 | 1.0e+21 | 1.0e+21 | 5.0e+20 | 300.00 | 1.7e+12 | http://web.archive.org/web/20250404231606/https://www.datacenterdynamics.com/en/news/datavolt-plans-15gw-data-center-campus-in-neoms-oxagon/ | 24.68 | 46.63 | |||||||||||||||||||||||||||||||
| Applied Digital Ellendale Possible Phase 3 | Planned | Likely | Yes | 20.95 | 4.5e+5 | NVIDIA GB200 NVL2 (per GPU) | 180,000.00 | United States of America | Applied Digital | It's unclear whether all the GPUs will be intereconnected. The CEO says that it's designed in a way that they can be all interconnected, though it's unclear if they will be. It is also somewhat unclear what type of GPUs they will be using, though I am assuming Blackwell GPUs here
Will be rented to CoreWeave
Number of GPUs estimated from power capacity
CoreWeave holds the option for this building | Private | 400.00 | 3.3e+9 | 9685 87th Ave SE, Ellendale, ND 58436 | Cloud,CoreWeave | Uncertain | Applied Digital expects the first 100 MW data center for CoreWeave to be ready for service in the fourth quarter of calendar 2025. The second building, which is expected to house a 150 MW data center, is currently under construction and is expected to be ready for service in the middle of 2026. Additionally, CoreWeave holds an option for the third 150 MW building, which is currently in the planning stages, anticipated to be ready for service in 2027. | Planned 2027 | 12.05 | Applied Digital CoreWeave Ellendale Phase 2 | 180,000.00 | NVIDIA | 20.65 | 9.0e+20 | 9.0e+20 | 4.5e+20 | 2.3e+20 | 432.43 | 400.00 | 1.1e+12 | 3.3e+9 | https://web.archive.org/web/20240622003111/https://marketscale.com/industries/software-and-technology/applied-digital-and-nvidia-are-solving-ai-applications-efficiency-crisis-with-next-gen-data-centers/ | https://web.archive.org/web/20250603030527/https://ir.applieddigital.com/news-events/press-releases/detail/123/applied-digital-announces-250mw-ai-data-center-lease-with | 46.01 | -98.57 | |||||||||||||||||||||
| Nebius New Jersey | Planned | Likely | Yes | 20.88 | 3.8e+5 | NVIDIA GB200 NVL2 (per GPU) | 150,000.00 | United States of America | Nebius AI | Rough estimate of chip count from stated power capacity of datacenter. | Private | 300.00 | New Jersey, USA | Cloud | GB200,Blackwell,Uncertain | "AI infrastructure company Nebius has announced plans to build a 300MW data center in New Jersey."
"Nebius today confirmed that its recently announced data center in New Jersey will be dedicated solely to NVIDIA Blackwell-architecture GPUs" | Planned 2026 | 12.10 | 150,000.00 | NVIDIA | 20.57 | 7.5e+20 | 7.5e+20 | 3.8e+20 | 1.9e+20 | 360.36 | 300.00 | 1.3e+12 | https://archive.ph/QP6zP#selection-1349.0-1349.96 | https://archive.ph/N4MzN | 40.15 | -74.39 | ||||||||||||||||||||||||
"OpenAI/Microsoft Mt Pleasant, Wisconsin Phase 1" | Planned | Likely | Yes | 20.88 | 3.8e+5 | NVIDIA GB200 NVL2 (per GPU) | 150,000.00 | United States of America | OpenAI,Microsoft | Rough estimate of chip count from stated power capacity of datacenter.
Estimate from Semianalysis that that this is a 300MW building
What type of GPUs is unclear, but I'm assuming they're GB200s | Private | 300.00 | M39X+XG, Mt Pleasant, WI 53177 | OpenAI,Microsoft | GB200,Uncertain | Ahead of Stargate, the companies plan to build a Phase 4 supercomputer in Mount Pleasant, Wisconsin, with operations starting in 2026. | Planned 2026 | Rough estiamate of number of GPUs from power consupmtion estimate. Type of GPU uncertain | 12.10 | 150,000.00 | NVIDIA | 20.57 | 7.5e+20 | 7.5e+20 | 3.8e+20 | 1.9e+20 | 360.36 | 300.00 | 1.3e+12 | https://web.archive.org/web/20240717135010/https://www.tomshardware.com/tech-industry/artificial-intelligence/openai-and-microsoft-reportedly-planning-dollar100-billion-datacenter-project-for-an-ai-supercomputer | https://web.archive.org/web/20240915193816/https://www.datacenterdynamics.com/en/news/microsoft-openai-consider-100bn-5gw-stargate-ai-data-center-report/ | https://www.youtube.com/watch?v=hobvps-H38o | 42.70 | -87.89 | ||||||||||||||||||||||
| Sesterce Grand Est France A | Planned | Likely | Yes | 20.88 | 3.8e+5 | NVIDIA GB200 NVL2 (per GPU) | 150,000.00 | France | Sesterce | Unclear what type of GPUs it will use
It seems very likely that they're using Blackwell. However, given that, the power capacity they cite does not line up with the chip counts they cite. I will assume that the MW figure they cite is accurate, and extrapolate chip counts from there | Private | 300.00 | Grand Est, Fance | Cloud | Uncertain | Sesterce is aiming to add 600MW of capacity across two data centers in Grand Est with 500,000 GPUs by 2028, and 1.2 GW with more than one million GPUs by 2030. | Planned 2028 | 12.10 | 150,000.00 | NVIDIA | 20.57 | 7.5e+20 | 7.5e+20 | 3.8e+20 | 1.9e+20 | 360.36 | 300.00 | 1.3e+12 | https://archive.ph/33xgc | 48.58 | 7.75 | |||||||||||||||||||||||||
| Sesterce Grand Est France B | Planned | Likely | Yes | 20.88 | 3.8e+5 | NVIDIA GB200 NVL2 (per GPU) | 150,000.00 | France | Sesterce | Unclear what type of GPUs it will use
It seems very likely that they're using Blackwell. However, given that, the power capacity they cite does not line up with the chip counts they cite. I will assume that the MW figure they cite is accurate, and extrapolate chip counts from there | Private | 300.00 | Grand Est, Fance | Cloud | Uncertain | Sesterce is aiming to add 600MW of capacity across two data centers in Grand Est with 500,000 GPUs by 2028, and 1.2 GW with more than one million GPUs by 2030. | Planned 2028 | 12.10 | 150,000.00 | NVIDIA | 20.57 | 7.5e+20 | 7.5e+20 | 3.8e+20 | 1.9e+20 | 360.36 | 300.00 | 1.3e+12 | https://archive.ph/33xgc | 48.58 | 7.75 | |||||||||||||||||||||||||
| Sesterce Southern France 250MW | Planned | Likely | Yes | 20.78 | 3.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 120,000.00 | France | Sesterce | Unclear what type of GPUs it will use
It seems very likely that they're using Blackwell. However, given that, the power capacity they cite does not line up with the chip counts they cite. I will assume that the MW figure they cite is accurate, and extrapolate chip counts from there | Private | 250.00 | Southern France | Cloud | Uncertain | the company has said it will establish a 250MW supercomputer with 200,000 GPUs in southern France | Planned 2028 | 12.08 | 120,000.00 | NVIDIA | 20.48 | 6.0e+20 | 6.0e+20 | 3.0e+20 | 1.5e+20 | 288.29 | 250.00 | 1.2e+12 | https://archive.ph/33xgc | 43.70 | 7.26 | |||||||||||||||||||||||||
| Oracle OCI Supercluster B200s | Planned | Likely | Yes | 20.77 | 3.0e+5 | NVIDIA B200 | 131,072.00 | United States of America | Oracle | Uncertain if this will actually be built | Private | 262.41 | 6.6e+9 | Cloud | B200 | OCI Supercluster with up to 131,072 NVIDIA B200 Tensor Core GPUs | Planned 2025 | 12.05 | 131,072.00 | NVIDIA | 20.47 | 5.9e+20 | 5.9e+20 | 2.9e+20 | 1.5e+20 | 262.41 | 1.1e+12 | 6.6e+9 | https://archive.ph/B3HJm#selection-1045.0-1045.83 | 39.38 | -97.92 | |||||||||||||||||||||||||
| Applied Digital CoreWeave Ellendale Phase 2 | Planned | Likely | Yes | 20.74 | 2.8e+5 | NVIDIA GB200 NVL2 (per GPU) | 110,000.00 | United States of America | Applied Digital | It's unclear whether all the GPUs will be intereconnected. The CEO says that it's designed in a way that they can be all interconnected, though it's unclear if they will be. It is also somewhat unclear what type of GPUs they will be using, though I am assuming Blackwell GPUs here
Will be rented to CoreWeave
Number of GPUs estimated from power capacity | Private | 250.00 | 3.3e+9 | 9685 87th Ave SE, Ellendale, ND 58436 | Cloud,CoreWeave | Uncertain | Applied Digital expects the first 100 MW data center for CoreWeave to be ready for service in the fourth quarter of calendar 2025. The second building, which is expected to house a 150 MW data center, is currently under construction and is expected to be ready for service in the middle of 2026. Additionally, CoreWeave holds an option for the third 150 MW building, which is currently in the planning stages, anticipated to be ready for service in 2027. | Planned 2026 | 12.04 | Applied Digital CoreWeave Ellendale Phase 1 | 110,000.00 | NVIDIA | 20.44 | 5.5e+20 | 5.5e+20 | 2.8e+20 | 1.4e+20 | 264.26 | 250.00 | 1.1e+12 | 3.3e+9 | https://web.archive.org/web/20240622003111/https://marketscale.com/industries/software-and-technology/applied-digital-and-nvidia-are-solving-ai-applications-efficiency-crisis-with-next-gen-data-centers/ | https://web.archive.org/web/20250603030527/https://ir.applieddigital.com/news-events/press-releases/detail/123/applied-digital-announces-250mw-ai-data-center-lease-with | 46.01 | -98.57 | |||||||||||||||||||||
| CoreWeave Denton GB200s OpenAI/Microsoft | Planned | Likely | Yes | 20.70 | 2.5e+5 | NVIDIA GB200 NVL2 (per GPU) | 100,000.00 | United States of America | CoreWeave | Rough estimate of chip count from stated power capacity of datacenter.
Core Scientific is also involved.
Said to be "one of the largest GPU supercomputers in North America"
Rumored to be using GB200s and rented to Microsoft for OpenAI | Private | 260.00 | 8161 Jim Christal Rd, Denton, TX 76207 | OpenAI,Microsoft | GB200 | The new agreement brings an additional $1.2 billion in contracted revenue across Core Scientific’s Denton TX location | Planned 2027 | 11.98 | 100,000.00 | NVIDIA | 20.40 | 5.0e+20 | 5.0e+20 | 2.5e+20 | 1.3e+20 | 240.24 | 260.00 | 9.6e+11 | https://web.archive.org/web/20250228154647/https://investors.corescientific.com/news-events/press-releases/detail/110/core-scientific-and-coreweave-announce-1-2-billion-expansion-at-denton-tx-site | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 33.21 | -97.21 | ||||||||||||||||||||||||
| Middle East 100k Cluster | Planned | Unlikely | Yes | 20.70 | 2.5e+5 | NVIDIA GB200 NVL2 (per GPU) | 100,000.00 | We think this is likely the same cluster being administered/operated by Microsoft for G42 | Private | 240.24 | GB200 | There's a 100k GB200 cluster going up in the Middle East | Planned | Alluded to by Semianalysis | 12.02 | True | 100,000.00 | NVIDIA | 20.40 | 5.0e+20 | 5.0e+20 | 2.5e+20 | 1.3e+20 | 240.24 | 1.0e+12 | https://web.archive.org/web/20241006181731/https://www.dwarkeshpatel.com/p/dylan-jon | ||||||||||||||||||||||||||||||
OpenAI Stargate Abilene Oracle OCI Supercluster Phase 1 | Planned | Likely | Yes | 20.70 | 2.5e+5 | NVIDIA GB200 NVL2 (per GPU) | 100,000.00 | United States of America | Oracle | States that it will have over 100k GPUs, implying it will have over 50k GB200s
"That cluster is expected to be leased to Oracle, which in turn will lease it to Microsoft, who will rent it to OpenAI"
Crusoe is involved in this | Private | 240.24 | 5502 Spinks Rd, Abilene, TX 79601 | Cloud,Microsoft,OpenAI | GB200 | OCI Supercluster with over 100,000+ GPUs in NVIDIA GB200 Grace Blackwell Superchips | Planned Q2 2025 | The first phase is under construction already and is very likely to be completed | 12.02 | 100,000.00 | NVIDIA | 20.40 | 5.0e+20 | 5.0e+20 | 2.5e+20 | 1.3e+20 | 240.24 | 1.0e+12 | https://archive.ph/B3HJm#selection-1045.0-1045.83 | https://archive.ph/UeNZs | 32.49 | -99.79 | ||||||||||||||||||||||||
| Stargate UAE Phase 1 | Planned | Likely | Unclear | 20.70 | 2.5e+5 | NVIDIA GB300 | 100,000.00 | United Arab Emirates | Stargate (OpenAI) | Chip amount roughly estimated based on power capacity. Assuming for now that they're using GB300s, but this hasn't been officially confirmed
Part of the broader planned 5GW UAE/USA cluster | Public/Private | 200.00 | Abu Dhabi, United Arab Emirates | OpenAI | Uncertain | "A 1GW Stargate UAE cluster in Abu Dhabi with 200MW expected to go live in 2026"
"Breaking ground on 1GW AI Datacenter, part of a planned 5GW UAE-US artificial intelligence campus; largest such deployment outside of the US"
"The campus, which will span 10 square miles within the Emirate, will be built by G42 and operated in partnership with several US companies."
"Stargate UAE is a next-generation AI infrastructure cluster that will run in the newly established 5-gigawatt UAE–U.S. AI Campus in Abu Dhabi. Stargate UAE, a 1-gigawatt compute cluster" | Planned 2026 | 12.10 | 100,000.00 | NVIDIA | 20.40 | 5.0e+20 | 5.0e+20 | 2.5e+20 | 1.3e+20 | 350.35 | 200.00 | 1.3e+12 | https://web.archive.org/web/20250515200336/https://www.commerce.gov/news/press-releases/2025/05/uae-and-us-presidents-attend-unveiling-phase-1-new-5gw-ai-campus-abu | https://archive.ph/hQWdM | https://archive.ph/PfVwI | 24.45 | 54.38 | |||||||||||||||||||||||
| xAI Colossus Memphis Phase 2 | Existing | Likely | Yes | 20.60 | 2.0e+5 | NVIDIA H100 SXM5 80GB | 150,000.00 | United States of America | xAI | 2025-02-18 | GPU types is from a tweet. Unclear exactly when it was first operational. It was confirmed to be operational Feb 17, 2025, but some evidence suggests that it was operational by December 4th, 2024 | Private | 280.28 | 7.1e+9 | 3231 Riverport Rd, Memphis, TN 38109 | xAI | H100,H200 | This weekend, the
@xAI
team brought our Colossus 100k H100 training cluster online.... Moreover, it will double in size to 200k (50k H200s) in a few months. | 2025-02-18 | 11.85 | xAI Colossus Memphis Phase 1 | xAI Colossus Memphis Phase 3 | NVIDIA H200 SXM | 50,000 | 200,000.00 | NVIDIA | NVIDIA | True | 1 | 20.30 | 4.0e+20 | 4.0e+20 | 2.0e+20 | 9.9e+19 | 280.28 | 7.1e+11 | 7.1e+9 | xAI Colossus Memphis Phase 2 | 1.00 | https://archive.ph/z39uN | https://web.archive.org/web/20241217184541/https://memphischamber.com/economic-development/xai/ | https://web.archive.org/web/20241217184541/https://memphischamber.com/economic-development/xai/ | https://www.youtube.com/watch?v=AUAJ82H12qs&t=260s | 35.07 | -90.12 | |||||||||||
| Project Rainier | Planned | Likely | Yes | 20.43 | 1.3e+5 | Amazon Trainium2 | 400,000.00 | United States of America | Amazon | Private | 350.00 | 55253 Elderberry Rd, New Carlisle, IN 46552 | Anthropic | Trainium2 | a cluster with 400k Trainium2 chips for Anthropic | Planned 2025 | 11.88 | 400,000.00 | Amazon AWS | 20.43 | 2.7e+20 | 2.7e+20 | 400.40 | 350.00 | 7.6e+11 | https://web.archive.org/web/20241204130049/https://semianalysis.com/2024/12/03/amazons-ai-self-sufficiency-trainium2-architecture-networking/ | 41.69 | -86.47 | ||||||||||||||||||||||||||||
| Applied Digital CoreWeave Ellendale Phase 1 | Planned | Likely | Yes | 20.35 | 1.1e+5 | NVIDIA GB200 NVL2 (per GPU) | 45,000.00 | United States of America | Applied Digital | It's unclear whether all the GPUs will be intereconnected. The CEO says that it's designed in a way that they can be all interconnected, though it's unclear if they will be. It is also somewhat unclear what type of GPUs they will be using, though I am assuming Blackwell GPUs here
Will be rented to CoreWeave
Number of GPUs estimated from power capacity | Private | 100.00 | 3.3e+9 | 9685 87th Ave SE, Ellendale, ND 58436 | Cloud,CoreWeave | Uncertain | Applied Digital expects the first 100 MW data center for CoreWeave to be ready for service in the fourth quarter of calendar 2025. The second building, which is expected to house a 150 MW data center, is currently under construction and is expected to be ready for service in the middle of 2026. Additionally, CoreWeave holds an option for the third 150 MW building, which is currently in the planning stages, anticipated to be ready for service in 2027. | Planned Q4 2025 | 12.05 | 45,000.00 | NVIDIA | 20.05 | 2.3e+20 | 2.3e+20 | 1.1e+20 | 5.6e+19 | 108.11 | 100.00 | 1.1e+12 | 3.3e+9 | https://web.archive.org/web/20240622003111/https://marketscale.com/industries/software-and-technology/applied-digital-and-nvidia-are-solving-ai-applications-efficiency-crisis-with-next-gen-data-centers/ | https://web.archive.org/web/20250603030527/https://ir.applieddigital.com/news-events/press-releases/detail/123/applied-digital-announces-250mw-ai-data-center-lease-with | 46.01 | -98.57 | ||||||||||||||||||||||
| Nscale Loughton | Planned | Likely | Yes | 20.35 | 1.1e+5 | NVIDIA GB200 NVL2 (per GPU) | 45,000.00 | United Kingdom of Great Britain and Northern Ireland | Slightly unclear if they refer to GB200 GPUs or entire chips (2 GPUs) | Private | 90.00 | Loughton, England | Cloud | GB200 | The site is equipped to support 50MW of AI and HPC capacity, with the ability to scale the power allocation up to 90MW, all utilising advanced liquid cooling to support generative AI GPU deployments. The site is scheduled to be live in Q4 2026. This UK facility can house up to 45,000 of the latest Nvidia GB200 GPUs. | Planned Q4 2026 | 12.10 | 45,000.00 | NVIDIA | 20.05 | 2.3e+20 | 2.3e+20 | 1.1e+20 | 5.6e+19 | 108.11 | 90.00 | 1.3e+12 | https://web.archive.org/web/20250116140650/https://www.globenewswire.com/news-release/2025/01/13/3008191/0/en/AI-hyperscaler-Nscale-to-invest-2-5-2-billion-in-the-UK-data-centre-industry-over-the-next-three-years.html | 51.65 | 0.06 | ||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Unlikely | Unclear | 20.30 | 1.0e+5 | China | Public/Private | China | Anonymized | Anonymized | 20.30 | 2.0e+20 | 2.0e+20 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| CoreWeave Muskogee | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 40,000.00 | United States of America | CoreWeave | Rough estimate of chip count from stated power capacity of datacenter.
Core Scientific is also involved
NVIDIA GPUs, so presumably GB200s | Private | 100.00 | Muskogee, Oklahoma 35.693709, -95.386957 | CoreWeave,Undisclosed Client | GB200,Uncertain | ground had been broken on a 100MW facility in the Port of Muskogee in West Oklahoma.
Set to go live in 2026, the facility will host CoreWeave’s Nvidia GPUs for an undisclosed client | Planned 2026 | 12.00 | 40,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 1.0e+20 | 5.0e+19 | 96.10 | 100.00 | 1.0e+12 | https://web.archive.org/web/20250115175242/https://www.datacenterdynamics.com/en/news/coreweave-and-core-scientific-break-ground-on-100mw-data-center-in-muskogee-oklahoma/ | 35.75 | -95.37 | |||||||||||||||||||||||||
| EU AI Gigafactory #1 | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 40,000.00 | Still in the soliciting proposal phase, so details are not fully clear, yet. They say they should be at least 100k Hopper-equivalent AI chips, which would be ~40k current frontier chips (Blackwells)
Will be located in Europe, sites still uncertain
They are planning on building five | Public/Private | 96.10 | Europe | Uncertain | They will be integrating massive computing power (e.g., beyond 100,000 advanced AI chips.... Reference here is NVIDIA’s H100 GPUs or equivalent | Planned H2 2026 | 12.02 | 40,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 1.0e+20 | 5.0e+19 | 96.10 | 1.0e+12 | https://web.archive.org/web/20250409164352/https://eurohpc-ju.europa.eu/document/download/47492db7-592e-4ad8-b672-9c822f94afa0_en?filename=Call%20for%20expression%20of%20interest%20in%20AI%20gigafactories%20-%202025%2004%2009%20-%20FINAL.pdf | |||||||||||||||||||||||||||||||
| EU AI Gigafactory #2 | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 40,000.00 | Still in the soliciting proposal phase, so details are not fully clear, yet. They say they should be at least 100k Hopper-equivalent AI chips, which would be ~40k current frontier chips (Blackwells)
Will be located in Europe, sites still uncertain
They are planning on building five | Public/Private | 96.10 | Europe | Uncertain | They will be integrating massive computing power (e.g., beyond 100,000 advanced AI chips.... Reference here is NVIDIA’s H100 GPUs or equivalent | Planned H2 2026 | 12.02 | 40,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 1.0e+20 | 5.0e+19 | 96.10 | 1.0e+12 | https://web.archive.org/web/20250409164352/https://eurohpc-ju.europa.eu/document/download/47492db7-592e-4ad8-b672-9c822f94afa0_en?filename=Call%20for%20expression%20of%20interest%20in%20AI%20gigafactories%20-%202025%2004%2009%20-%20FINAL.pdf | |||||||||||||||||||||||||||||||
| EU AI Gigafactory #3 | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 40,000.00 | Still in the soliciting proposal phase, so details are not fully clear, yet. They say they should be at least 100k Hopper-equivalent AI chips, which would be ~40k current frontier chips (Blackwells)
Will be located in Europe, sites still uncertain
They are planning on building five | Public/Private | 96.10 | Europe | Uncertain | They will be integrating massive computing power (e.g., beyond 100,000 advanced AI chips.... Reference here is NVIDIA’s H100 GPUs or equivalent | Planned H2 2026 | 12.02 | 40,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 1.0e+20 | 5.0e+19 | 96.10 | 1.0e+12 | https://web.archive.org/web/20250409164352/https://eurohpc-ju.europa.eu/document/download/47492db7-592e-4ad8-b672-9c822f94afa0_en?filename=Call%20for%20expression%20of%20interest%20in%20AI%20gigafactories%20-%202025%2004%2009%20-%20FINAL.pdf | |||||||||||||||||||||||||||||||
| EU AI Gigafactory #4 | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 40,000.00 | Still in the soliciting proposal phase, so details are not fully clear, yet. They say they should be at least 100k Hopper-equivalent AI chips, which would be ~40k current frontier chips (Blackwells)
Will be located in Europe, sites still uncertain
They are planning on building five | Public/Private | 96.10 | Europe | Uncertain | They will be integrating massive computing power (e.g., beyond 100,000 advanced AI chips.... Reference here is NVIDIA’s H100 GPUs or equivalent | Planned H2 2026 | 12.02 | 40,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 1.0e+20 | 5.0e+19 | 96.10 | 1.0e+12 | https://web.archive.org/web/20250409164352/https://eurohpc-ju.europa.eu/document/download/47492db7-592e-4ad8-b672-9c822f94afa0_en?filename=Call%20for%20expression%20of%20interest%20in%20AI%20gigafactories%20-%202025%2004%2009%20-%20FINAL.pdf | |||||||||||||||||||||||||||||||
| EU AI Gigafactory #5 | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 40,000.00 | Still in the soliciting proposal phase, so details are not fully clear, yet. They say they should be at least 100k Hopper-equivalent AI chips, which would be ~40k current frontier chips (Blackwells)
Will be located in Europe, sites still uncertain
They are planning on building five | Public/Private | 96.10 | Europe | Uncertain | They will be integrating massive computing power (e.g., beyond 100,000 advanced AI chips.... Reference here is NVIDIA’s H100 GPUs or equivalent | Planned H2 2026 | 12.02 | 40,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 1.0e+20 | 5.0e+19 | 96.10 | 1.0e+12 | https://web.archive.org/web/20250409164352/https://eurohpc-ju.europa.eu/document/download/47492db7-592e-4ad8-b672-9c822f94afa0_en?filename=Call%20for%20expression%20of%20interest%20in%20AI%20gigafactories%20-%202025%2004%2009%20-%20FINAL.pdf | |||||||||||||||||||||||||||||||
| Sesterce Valence | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA GB200 NVL2 (per GPU) | 40,000.00 | France | Sesterce | Unclear what type of GPUs they'll use, but I'm assuming it'll be Blackwell GPUs | Private | 96.10 | Valence, France | Cloud | GB200 | The data center is set to be located in the Rovaltain business park in Valence Romans Agglo, and will house 40,000 GPUs for the training and deployment of AI. | Planned 2026 | 12.02 | 40,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 1.0e+20 | 5.0e+19 | 96.10 | 1.0e+12 | https://archive.ph/33xgc | 44.93 | 4.89 | ||||||||||||||||||||||||||
| Meta 100k | Existing | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA H100 SXM5 80GB | 100,000.00 | United States of America | Meta AI | 2024-10-30 | Reported as operational in October 2024 earnings call | Private | 142.69 | 3.8e+9 | United States of America | Meta | H100 | Meta Platforms is putting the final touches on one such cluster, which will be a little bigger than 100,000 H100s | 2024-10-30 | Reported by The Information | 11.84 | 100,000.00 | NVIDIA | True | 1 | 20.00 | 2.0e+20 | 2.0e+20 | 9.9e+19 | 4.9e+19 | 142.69 | 6.9e+11 | 3.8e+9 | True | xAI Colossus Memphis Phase 1 | 1.00 | https://archive.ph/85KB1 | https://web.archive.org/web/20240901171617/https://www.semianalysis.com/p/100000-h100-clusters-power-network | https://web.archive.org/web/2/https://www.tomshardware.com/tech-industry/artificial-intelligence/meta-is-using-more-than-100-000-nvidia-h100-ai-gpus-to-train-llama-4-mark-zuckerberg-says-that-llama-4-is-being-trained-on-a-cluster-bigger-than-anything-that-ive-seen | https://web.archive.org/web/20241228090751/https://www.tomshardware.com/tech-industry/artificial-intelligence/meta-is-using-more-than-100-000-nvidia-h100-ai-gpus-to-train-llama-4-mark-zuckerberg-says-that-llama-4-is-being-trained-on-a-cluster-bigger-than-anything-that-ive-seen | 39.38 | -97.92 | ||||||||||||||
| OpenAI/Microsoft Goodyear Arizona | Existing | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA H100 SXM5 80GB | 100,000.00 | United States of America | Microsoft,OpenAI | 2024-10-02 | It seems likely that this became operational in phases. It seems like 75k were likely available sometime mid 2024, and ~160k will be available in the future.
There is very little official public data about this cluster, it's almost all infrences/rumors
Microsoft appears to own it and rent it to OpenAI
The Semianalysis lecture estimates that there are 32k H100s (48MW) per building. Satellite data as of Dec 2024 seems to show that 5 building are now complete (though this doesn't imply that the clusters inside have been built or are operational). This would imply that the cluster is growing to ~160k GPUs. It's also unclear if the newer buildings would still have Hoppers or would have Blackwell chips
I personally suspect that this cluster was one of the primary clusters for training GPT4.5 | Private | 142.69 | 3.8e+9 | 33°24'28.7"N 112°21'54.3"W
Goodyear, Arizona, USA | OpenAI | H100 | "You already have 100K clusters. OpenAI in Arizona"
"Our data shows one of OpenAI’s next training supercomputers in Arizona was going to have more than 75,000+ GPUs in a singular site by the middle of next year" | 2024-10-02 | This estimate is from the SemiAnalysis number of GPUs/building, and satellite imagery showing that they've built 5 buildings | 11.84 | 100,000.00 | NVIDIA | True | 1 | 20.00 | 2.0e+20 | 2.0e+20 | 9.9e+19 | 4.9e+19 | 142.69 | 6.9e+11 | 3.8e+9 | xAI Colossus Memphis Phase 1 | 1.00 | https://web.archive.org/web/20241006181731/https://www.dwarkeshpatel.com/p/dylan-jon | https://web.archive.org/web/20240609051243/https://www.semianalysis.com/p/microsoft-swallows-openais-core-team | https://www.youtube.com/watch?v=hobvps-H38o | 33.19 | -111.97 | ||||||||||||||||
| Tesla Cortex Phase 3 | Planned | Likely | Yes | 20.30 | 1.0e+5 | NVIDIA H100 SXM5 80GB | 100,000.00 | United States of America | Tesla | Private | 140.14 | 4.7e+9 | 1 Tesla Rd, Austin, TX 78725 | Tesla | H100 | This will be ~100k H100/H200 with massive storage for video training of FSD & Optimus. | Planned 2026 | From a Musk Tweet | 11.85 | Tesla Cortex Phase 2 | 20,000 | 120,000.00 | NVIDIA | 20.00 | 2.0e+20 | 2.0e+20 | 9.9e+19 | 4.9e+19 | 140.14 | 7.1e+11 | 4.7e+9 | https://web.archive.org/web/20240904184237/https://www.tomshardware.com/desktops/servers/elon-musk-shows-off-cortex-ai-supercluster-first-look-at-teslas-50000-nvidia-h100s | https://web.archive.org/web/20240822222555/https://www.fool.com/earnings/call-transcripts/2024/07/24/tesla-tsla-q2-2024-earnings-call-transcript/ | https://archive.ph/0jHdq#selection-479.86-479.172 | 30.22 | -97.62 | ||||||||||||||||||||
| xAI Colossus Memphis Phase 1 | Existing | Confirmed | Yes | 20.30 | 1.0e+5 | NVIDIA H100 SXM5 80GB | 100,000.00 | United States of America | xAI | 2024-09-02 | Private | 150.00 | 3.8e+9 | 3231 Riverport Rd, Memphis, TN 38109 | xAI | H100 | "With 100k liquid-cooled H100s on a single RDMA fabric"
"This weekend, the @xAI team brought our Colossus 100k H100 training cluster online.... Moreover, it will double in size to 200k (50k H200s) in a few months" | 2024-09-02 | Reported to be online by Musk | 11.82 | xAI Colossus Memphis Phase 2 | 100,000.00 | NVIDIA | True | 1 | 20.00 | 2.0e+20 | 2.0e+20 | 9.9e+19 | 4.9e+19 | 142.69 | 150.00 | 6.6e+11 | 3.8e+9 | xAI Colossus Memphis Phase 1 | 1.00 | https://archive.ph/Bh9Tq | https://web.archive.org/web/20240727065656/https://www.commercialappeal.com/story/news/government/city/2024/07/09/elon-musk-xai-memphis-electricity-water-mlgw/74331544007/ | https://archive.ph/z39uN | 35.07 | -90.12 | |||||||||||||||
| together.ai 36k GB200s | Planned | Likely | Yes | 20.26 | 9.1e+4 | NVIDIA GB200 NVL2 (per GPU) | 36,000.00 | United States of America | Together | Private | 86.49 | Cloud | GB200 | This infrastructure backbone that will power the next generation of AI innovations enables the following customer benefits:
36,000 NVIDIA Blackwell GPUs in GB200 NVL72 Cluster starting Q1 2025 | Planned Q1 2025 | 12.02 | 36,000.00 | NVIDIA | 19.95 | 1.8e+20 | 1.8e+20 | 9.0e+19 | 4.5e+19 | 86.49 | 1.0e+12 | https://web.archive.org/web/20241120213534/https://www.together.ai/blog/nvidia-gb200-together-gpu-cluster-36k | 39.38 | -97.92 | ||||||||||||||||||||||||||||
| HUMAIN Saudi Arabia/NVIDIA Phase 1 | Planned | Confirmed | Yes | 19.95 | 4.5e+4 | NVIDIA GB300 | 18,000.00 | Saudi Arabia | Humain | Public/Private | 63.06 | GB300 | "HUMAIN is making a major investment to build AI factories in the Kingdom of Saudi Arabia with a projected capacity of up to 500 megawatts powered by several hundred thousand of NVIDIA’s most advanced GPUs over the next five years. The first phase of deployment will be an 18,000 NVIDIA GB300 Grace Blackwell AI supercomputer with NVIDIA InfiniBand networking." | Planned 2026 | 11.85 | 18,000.00 | NVIDIA | 19.65 | 9.0e+19 | 9.0e+19 | 4.5e+19 | 2.3e+19 | 63.06 | 7.1e+11 | https://archive.ph/vvAxQ | https://archive.ph/CFcnD | 23.38 | 44.65 | ||||||||||||||||||||||||||||
| Oracle OCI Supercluster H200s | Existing | Likely | Yes | 20.11 | 6.6e+4 | NVIDIA H200 SXM | 65,536.00 | United States of America | Oracle | 2024-11-21 | Unclear exactly when this first became operational, if it was ever built. The announcement November 2024 states that the largest supercomputer in the cloud is now generally available, implying that this cluster has been built and is operational. However, it's possible that they're just advertising their capacity to scale H200 clusters to be the largest supercomputer in the cloud.
It seems very likely that this cluster has been built, but it's not fully certain. I estimate the first operational date to be Q1 2025 to be slightly conservative on the timing | Private | 93.51 | 2.3e+9 | Cloud | H200 | We’re excited to announce the general availability of Oracle Cloud Infrastructure (OCI) Supercluster with NVIDIA H200 Tensor Core GPUs. The largest AI supercomputer available in the cloud*, our latest Supercluster scales up to an industry-leading 65,536 GPUs. | 2024-11-21 | 11.84 | 65,536.00 | NVIDIA | True | 4 | 19.81 | 1.3e+20 | 1.3e+20 | 6.5e+19 | 3.2e+19 | 93.51 | 6.9e+11 | 2.3e+9 | xAI Colossus Memphis Phase 1 | 0.66 | https://archive.ph/B3HJm#selection-1045.0-1045.83 | https://web.archive.org/web/20241214043143/https://www.datacenterdynamics.com/en/news/oracles-65000-gpu-supercluster-now-generally-available/ | 39.38 | -97.92 | |||||||||||||||||||
| Nebius Finland Phase 2 | Planned | Likely | Yes | 20.07 | 6.0e+4 | NVIDIA H100 SXM5 80GB | 60,000.00 | Finland | Nebius AI | Using H100s and H200s, and possibly Blackwell chips | Private | 84.08 | 2.8e+9 | Moreenikatu 6, 04600 Mäntsälä, Finland | Cloud | H100,H200 | "The expansion will see Nebius place upwards of 60,000 GPUs at the facility, growing its capacity from 25MW to 75MW."
"The expansion in Finland will include deployment of NVIDIA H200 Tensor Core GPUs, which will be available to customers from November, in addition to already installed NVIDIA H100 Tensor Core GPUs that form the backbone of Nebius’s fleet of NVIDIA GPUs" | Planned H2 2025 | 11.85 | Nebius 8k Finland Phase 1 | 60,000.00 | NVIDIA | 19.77 | 1.2e+20 | 1.2e+20 | 5.9e+19 | 3.0e+19 | 84.08 | 7.1e+11 | 2.8e+9 | https://web.archive.org/web/20241009115430/https://www.datacenterdynamics.com/en/news/nebius-to-triple-capacity-of-mantsala-data-center-in-finland/ | https://group.nebius.com/newsroom/nebius-to-triple-capacity-at-finland-data-center-to-75-mw | 60.63 | 25.26 | ||||||||||||||||||||||
| G42 Microsoft 100Mw UAE Cluster | Planned | Likely | Yes | 20.04 | 5.5e+4 | NVIDIA H100 SXM5 80GB | 55,000.00 | United Arab Emirates | G42,Microsoft | Estimating "50,000–60,000" as 55k
Seems likely to be the Ajman cluster, but uncertain here
Assuming they're using H100s here | Public/Private | 100.00 | 2.6e+9 | Ajman, United Arab Emirates | Microsoft,Azure | H100 | Private conversations with Microsoft and G42 executives revealed that G42 and Khazna are building three new data centers in the UAE for Microsoft. The centers will be 30, 30, and 100 megawatts each, the equivalent of nearly 100,000 H100 chips (approximately 15,000, 15,000, and 50,000–60,000 respectively). | Planned Q3 2025 | 11.74 | 55,000.00 | NVIDIA | 19.74 | 1.1e+20 | 1.1e+20 | 5.4e+19 | 2.7e+19 | 77.08 | 100.00 | 5.4e+11 | 2.6e+9 | https://web.archive.org/web/20250221170709/https://www.csis.org/analysis/united-arab-emirates-ai-ambitions | https://web.archive.org/web/20241006181731/https://www.dwarkeshpatel.com/p/dylan-jon | 25.41 | 55.44 | ||||||||||||||||||||||
| Project Ceiba Phase 2 | Planned | Confirmed | Yes | 20.02 | 5.2e+4 | NVIDIA GB200 NVL2 (per GPU) | 20,736.00 | United States of America | Amazon,NVIDIA | Private | 49.82 | NVIDIA | GB200 | Alongside the 20,736 B200 GPUs, Ceiba will boast 10,368 Grace CPUs.
Ceiba will be built using the new liquid-cooled GB200 NVL72 platform with fifth generation NVLink | Planned 2025 | Plans were officially reported by NVIDIA | 12.02 | Project Ceiba Phase 1 | 20,736.00 | NVIDIA | 19.71 | 1.0e+20 | 1.0e+20 | 5.2e+19 | 2.6e+19 | 49.82 | 1.0e+12 | https://web.archive.org/web/20240522175821/https://www.datacenterdynamics.com/en/news/aws-upgrades-project-ceiba-to-feature-20736-nvidia-blackwell-gpus-boosting-power-6x-to-414-ai-exaflops/ | https://web.archive.org/web/20240806110553/https://nvidianews.nvidia.com/news/aws-nvidia-generative-ai-innovation | 39.38 | -97.92 | |||||||||||||||||||||||||
| Tesla Cortex Phase 1 | Existing | Confirmed | Yes | 20.00 | 5.0e+4 | NVIDIA H100 SXM5 80GB | 50,000.00 | United States of America | Tesla | 2024-11-15 | Private | 71.34 | 1.9e+9 | 1 Tesla Rd, Austin, TX 78725 | Tesla | H100 | Tesla completed the deployment of its 50,000 Nvidia H100 GPU training cluster during Q4 2024. | Q4 2024 | 11.84 | 50,000.00 | NVIDIA | True | 4 | 19.69 | 9.9e+19 | 9.9e+19 | 4.9e+19 | 2.5e+19 | 71.34 | 6.9e+11 | 1.9e+9 | xAI Colossus Memphis Phase 1 | 0.50 | https://web.archive.org/web/20250217061801/https://www.datacenterdynamics.com/en/news/teslas-50000-gpu-cortex-supercomputer-went-live-in-q4-2024/ | https://web.archive.org/web/20240822222555/https://www.fool.com/earnings/call-transcripts/2024/07/24/tesla-tsla-q2-2024-earnings-call-transcript/ | https://web.archive.org/web/20240904184237/https://www.tomshardware.com/desktops/servers/elon-musk-shows-off-cortex-ai-supercluster-first-look-at-teslas-50000-nvidia-h100s | 30.22 | -97.62 | ||||||||||||||||||
| Tesla Cortex Phase 2 | Planned | Likely | Yes | 20.00 | 5.0e+4 | NVIDIA H100 SXM5 80GB | 50,000.00 | United States of America | Tesla | Private | 70.07 | 2.4e+9 | 1 Tesla Rd, Austin, TX 78725 | Tesla | H100, 50k H100, additional 20k of Tesla's own hardware is also expected | So it will be an incremental for 50,000 H100s plus 20,000 of our hardware for AI5 at Tesla AI computer | Planned 2025 | Musk gave details about this in an earnings call
Some sources mention that the 50k H100s will come online before Tesla's own chips will | 11.85 | Tesla Cortex Phase 1 | 20,000 | 70,000.00 | NVIDIA | 19.69 | 9.9e+19 | 9.9e+19 | 4.9e+19 | 2.5e+19 | 70.07 | 7.1e+11 | 2.4e+9 | https://web.archive.org/web/20240904184237/https://www.tomshardware.com/desktops/servers/elon-musk-shows-off-cortex-ai-supercluster-first-look-at-teslas-50000-nvidia-h100s | https://web.archive.org/web/20240822222555/https://www.fool.com/earnings/call-transcripts/2024/07/24/tesla-tsla-q2-2024-earnings-call-transcript/ | https://web.archive.org/web/20250217061801/https://www.datacenterdynamics.com/en/news/teslas-50000-gpu-cortex-supercomputer-went-live-in-q4-2024/ | 30.22 | -97.62 | ||||||||||||||||||||
| Lawrence Livermore NL El Capitan Phase 2 | Existing | Confirmed | Yes | 19.94 | 4.4e+4 | AMD Instinct MI300A | 44,544.00 | United States of America | US Department of Energy | 2024-11-18 | Public | 35.00 | 6.1e+8 | Lawrence Livermore National Laboratory
7000 East Ave, Livermore, CA 94550 | US Government,Lawrence Livermore NL,Scientific Research | AMD MI300A | the system has 11,136 nodes packed with 44,544 of AMD's MI300A APUs | 2024-11-18 | 12.10 | Lawrence Livermore NL El Capitan Phase 1 | 44,544.00 | AMD | True | 5 | 19.64 | 8.7e+19 | 8.7e+19 | 4.4e+19 | 2.2e+19 | 69.01 | 35.00 | 1.2e+12 | 1.8e+9 | 6.0e+8 | 6.1e+8 | $600M was quoted as the max cost | True | xAI Colossus Memphis Phase 1 | 0.44 | https://web.archive.org/web/20241122140255/https://www.tomshardware.com/pc-components/cpus/amd-powered-el-capitan-is-now-the-worlds-fastest-supercomputer-with-1-7-exaflops-of-performance-fastest-intel-machine-falls-to-third-place-on-top500-list | 37.68 | -121.77 | ||||||||||||||
| CoreWeave H200s | Existing | Likely | Yes | 19.92 | 4.2e+4 | NVIDIA H200 SXM | 42,000.00 | United States of America | CoreWeave | 2024-08-28 | Private | 59.93 | 1.5e+9 | CoreWeave | H200 | These instances are deployed in clusters with up to 42,000 GPUs | 2024-08-28 | 11.84 | 42,000.00 | NVIDIA | True | 1 | 19.62 | 8.3e+19 | 8.3e+19 | 4.2e+19 | 2.1e+19 | 59.93 | 6.9e+11 | 1.5e+9 | CoreWeave H200s | 1.00 | https://web.archive.org/web/20250107015942/https://www.coreweave.com/blog/coreweave-is-the-first-cloud-provider-to-deploy-nvidia-h200-tensor-core-gpus | 39.38 | -97.92 | |||||||||||||||||||||
| YTL AI Johor | Planned | Likely | Yes | 19.89 | 3.9e+4 | NVIDIA GB200 NVL2 (per GPU) | 15,428.00 | Malaysia | YTL Power | It states the cluster will be over 300E18 FLOP/s, and that a single GB200 NVL72 has 1.4E18 FLOP/s, implying they are referring to FP4. (300/1.4)*36=7,714 GB200s=15,428 GPUs (B200s) | Private | 37.06 | 81000 Kulai, Johor, Malaysia
1.6230215444990799, 103.52989382701362 | Cloud | GB200 | YTL is among the first companies to adopt NVIDIA GB200 NVL72, which is a multi-node, liquid-cooled, rack-scale system with fifth-generation NVLink. The supercomputer will be interconnected by NVIDIA Quantum InfiniBand networking platform. The platform acts as a single GPU with 1.4 exaflops of AI performance and 30TB of fast memory and is designed for the most compute-intensive workloads. The YTL AI Supercomputer will surpass more than 300 exaflops of AI compute, making it one of the fastest supercomputers in the world. | Planned Q3 2025 | Plans to use Blackwell chips | 12.02 | 15,428.00 | NVIDIA | 19.59 | 7.7e+19 | 7.7e+19 | 3.9e+19 | 1.9e+19 | 37.06 | 1.0e+12 | https://web.archive.org/web/20240717202025/https://www.ytlaicloud.com/press-releases/ytl-creates-one-of-the-worlds-most-advanced-supercomputers-powered-by-nvidia-grace-blackwell-based-dgx-cloud/ | 1.66 | 103.61 | |||||||||||||||||||||||||
| Nebius Kansas City Phase 2 | Planned | Likely | Yes | 19.86 | 3.6e+4 | NVIDIA B200 | 8,000.00 | United States of America | Nebius AI | Additional capacity was planned to go online Q2 2025
Is using Blackwell GPUs and H200s, unclear what proportion of each. For this, I am assuming have of the MW go to H200s vs B200s | Private | 40.00 | 1.3e+9 | Kansas City, USA | Cloud | H200,Blackwell | will house thousands of state-of-the-art NVIDIA GPUs, primarily NVIDIA Hopper GPUs in the initial phase, with the energy-efficient NVIDIA Blackwell platform expected to arrive in 2025. The colocation can be expanded from an initial 5 MW up to 40 MW, or about 35 thousand GPUs, at full potential capacity | Planned Q2 2025 | 11.95 | Nebius Kansas City Phase 1 | NVIDIA H200 SXM | 18,000 | 26,000.00 | NVIDIA | NVIDIA | 19.55 | 7.2e+19 | 7.2e+19 | 3.6e+19 | 1.8e+19 | 41.24 | 40.00 | 9.0e+11 | 1.3e+9 | https://web.archive.org/web/20250109005023/https://group.nebius.com/newsroom/nebius-expands-in-us-with-first-gpu-cluster-in-kansas-city-offices-in-san-francisco-dallas-and-new-york | 39.10 | -94.58 | |||||||||||||||||||
| Tesla Earnings Call Claim | Existing | Unlikely | Unclear | 19.84 | 3.5e+4 | NVIDIA H100 SXM5 80GB | 35,000.00 | United States of America | Tesla | 2024-03-31 | Unclear if these are all in the same cluster. I personally doublt that it is. Musk diverted 12k H100s intended for Tesla to XAi, and it's unclear how that relates to this cluster. Also, NVIDIA stated that some of Tesla's statements "did not align with what they had sold Tesla" | Private | 49.94 | 1.2e+9 | Tesla | H100 | We've installed and commissioned, meaning they're actually working 35,000 H100 computers or GPUs | 2024-03-31 | Likely not a single superocmputer | 11.84 | 35,000.00 | NVIDIA | 19.54 | 6.9e+19 | 6.9e+19 | 3.5e+19 | 1.7e+19 | 49.94 | 6.9e+11 | 1.2e+9 | https://web.archive.org/web/20241203032810/https://www.fool.com/earnings/call-transcripts/2024/04/23/tesla-tsla-q1-2024-earnings-call-transcript/ | https://web.archive.org/web/20241111165856/https://www.datacenterdynamics.com/en/news/elon-musk-told-nvidia-to-ship-thousands-of-gpus-reserved-for-tesla-to-x-and-xai/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Lambda Labs H100/H200 | Existing | Likely | No | 19.80 | 3.2e+4 | NVIDIA H100 SXM5 80GB | 32,000.00 | United States of America | Lambda Labs | 2023-12-05 | Likely in the US since majority of operations happen there https://lambdalabs.com/service/gpu-cloud/faqs
Unclear if the 32k refers to one single supercomputer, the aggregation of many supercomputers, or the potential that they could hypotheticlaly scale up to | Private | 46.48 | 1.1e+9 | Cloud | H100,H200 | Reserved cloud clusters from 64 to 32,000 H100 or H200 SXM GPUs | 2023-12-05 | Very little information on this. Unclear if this is one cluster or several, and how many are in the largest cluster | 11.83 | NVIDIA H200 SXM | 32,000.00 | NVIDIA | NVIDIA | 19.50 | 6.3e+19 | 6.3e+19 | 3.2e+19 | 1.6e+19 | 46.48 | 6.8e+11 | 1.1e+9 | https://web.archive.org/web/20231205014622/https://lambdalabs.com/nvidia-h100-nvidia-h200-gpus | https://archive.ph/Zzfyw | https://web.archive.org/web/20240413041051/https://www.futuriom.com/articles/news/lambda-scores-320-million-for-ai-infrastructure-cloud/2024/02 | 39.38 | -97.92 | |||||||||||||||||||
| NVIDIA CoreWeave Eos-DFW Rumored Phase 2 | Planned | Likely | Yes | 19.80 | 3.2e+4 | NVIDIA H100 SXM5 80GB | 32,000.00 | United States of America | NVIDIA,CoreWeave | Private | 44.84 | 1.5e+9 | 1000 Coit Rd, Plano, TX 75075 | NVIDIA,CoreWeave | H100 | NVIDIA is doing the same thing in Plano, Texas for a 32,000 GPU cluster that they're building | Planned 2025 | Alluded to in Dylan Patel interview | 11.85 | NVIDIA CoreWeave Eos-DFW Phase 1 | 32,000.00 | NVIDIA | 19.50 | 6.3e+19 | 6.3e+19 | 3.2e+19 | 1.6e+19 | 44.84 | 7.1e+11 | 1.5e+9 | https://web.archive.org/web/20241006181731/https://www.dwarkeshpatel.com/p/dylan-jon | https://web.archive.org/web/20240901052405/https://www.nextplatform.com/2024/03/06/a-tale-of-two-nvidia-eos-supercomputers/ | 33.01 | -96.76 | ||||||||||||||||||||||
| Foxconn Big Innovation Cloud AI factory | Planned | Likely | Yes | 19.70 | 2.5e+4 | NVIDIA GB300 | 5,000.00 | Taiwan | Foxconn | Unclear what proportion of the Blackwell GPUs will be GB300s vs GB200s. I assume it will be 50% each | Private | 29.53 | Taiwan National Science and Technology Council,TSMC,Foxconn | Blackwell | the supercomputer will consist of 10,000 Nvidia Blackwell Ultra GPUs... will include the GB300 NVL72 | Planned 2026 | 11.93 | NVIDIA GB200 NVL2 (per GPU) | 5,000 | 10,000.00 | NVIDIA | NVIDIA | 19.40 | 5.0e+19 | 5.0e+19 | 2.5e+19 | 1.3e+19 | 29.53 | 8.5e+11 | https://web.archive.org/web/20250519123959/https://www.datacenterdynamics.com/en/news/nvidia-and-foxconn-plan-taiwanese-ai-supercomputer/ | 23.78 | 120.93 | ||||||||||||||||||||||||
| Sesterce Pegasus | Planned | Likely | Yes | 19.72 | 2.7e+4 | NVIDIA GB200 NVL2 (per GPU) | 10,496.00 | Sesterce | Private | 25.22 | Cloud | GB200 | "10496 GPUs" | Planned | 12.02 | 10,496.00 | NVIDIA | 19.42 | 5.2e+19 | 5.2e+19 | 2.6e+19 | 1.3e+19 | 25.22 | 1.0e+12 | https://web.archive.org/web/20240823055517/https://www.sesterce.com/reserved-cloud | |||||||||||||||||||||||||||||||
| Google A3 VMs | Planned | Likely | Yes | 19.71 | 2.6e+4 | NVIDIA H100 SXM5 80GB | 26,000.00 | United States of America | The A3 supercomputer seems to refer to an amalgamation of different clusters available as a virtual machine via the cloud and is operational.
26k is the size of the largest individual cluster they say they can build as part of this. It is unclear if a cluster of this size has been constructed. If it has, it could have been in 2024 | Private | 36.44 | 1.2e+9 | Cloud | H100 | "“For our largest customers, we can build A3 supercomputers up to 26,000 GPUs in a single cluster and are working to build multiple clusters in our largest regions,” a Google spokeswoman said in an email" | Planned 2025 | This refers to the largest clusters Google says it can build for its customers | 11.85 | 26,000.00 | NVIDIA | 19.41 | 5.1e+19 | 5.1e+19 | 2.6e+19 | 1.3e+19 | 36.44 | 7.1e+11 | 1.2e+9 | https://web.archive.org/web/20240716093004/https://cloud.google.com/blog/products/compute/introducing-a3-supercomputers-with-nvidia-h100-gpus | https://web.archive.org/web/20240829141902/https://www.hpcwire.com/2023/05/10/googles-new-ai-focused-a3-supercomputer-has-26000-gpus/ | 39.38 | -97.92 | ||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | No | 19.70 | 3.0e+4 | China | 2024-12-15 | Public/Private | China | Anonymized | Anonymized | 19.70 | 5.0e+19 | 5.0e+19 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| ParTec ELBJUWEL | Planned | Confirmed | Yes | 19.70 | 2.5e+4 | Germany | Helmholtz-Zentrum Dresden-Rossendorf (HZDR) | Public | Bautzner Landstraße 400, 01328 Dresden, Germany | Cloud,Academia,Industry | with capabilities extending to about 50 exaflops for 8-bit floating-point operations | Planned | Unknown | 19.40 | 5.0e+19 | 5.0e+19 | 2.5e+19 | https://web.archive.org/web/20241117210443/https://thequantuminsider.com/2024/10/22/partec-ag-and-hzdr-to-build-ai-supercomputer-supporting-research-in-ai-quantum-computing-and-hpc/ | 51.06 | 13.95 | ||||||||||||||||||||||||||||||||||||
| Meta GenAI 2024a | Existing | Confirmed | Yes | 19.69 | 2.5e+4 | NVIDIA H100 SXM5 80GB | 24,576.00 | United States of America | Meta AI | 2024-03-12 | At least one of these was used to train Llama 3 | Private | 35.07 | 8.8e+8 | Meta | H100 | Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta | 2024-03-12 | From official Meta announcement. At least one of these was used to train Llama 3.1 | 11.84 | 24,576.00 | NVIDIA | True | 1 | 19.39 | 4.9e+19 | 4.9e+19 | 2.4e+19 | 1.2e+19 | 35.07 | 6.9e+11 | 8.8e+8 | True | Meta GenAI 2024a | 1.00 | http://web.archive.org/web/20240828225612/https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/ | https://web.archive.org/web/20240827223115/https://www.semianalysis.com/p/gpt-4-architecture-infrastructure | 39.38 | -97.92 | |||||||||||||||||
| Meta GenAI 2024b | Existing | Confirmed | Yes | 19.69 | 2.5e+4 | NVIDIA H100 SXM5 80GB | 24,576.00 | United States of America | Meta AI | 2024-03-12 | At least one of these was used to train Llama 3 | Private | 35.07 | 8.8e+8 | Meta | H100 | Today, we’re sharing details on two versions of our 24,576-GPU data center scale cluster at Meta | 2024-03-12 | From official Meta announcement. At least one of these was used to train Llama 3.1 | 11.84 | 24,576.00 | NVIDIA | True | 1 | 19.39 | 4.9e+19 | 4.9e+19 | 2.4e+19 | 1.2e+19 | 35.07 | 6.9e+11 | 8.8e+8 | Meta GenAI 2024a | 1.00 | http://web.archive.org/web/20240828225612/https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/ | https://web.archive.org/web/20240827223115/https://www.semianalysis.com/p/gpt-4-architecture-infrastructure | 39.38 | -97.92 | ||||||||||||||||||
| "Jupiter, Jülich" | Existing | Confirmed | Yes | 19.67 | 2.4e+4 | NVIDIA GH200 | 23,536.00 | Germany | EuroHPC JU,Julich Supercomputing Center | 2025-06-10 | Public | 18.00 | 2.9e+8 | Jülich Supercomputing Centre (JSC)
52428 Jülich, Germany | GH200 | "JUPITER marks the debut of a quad NVIDIA GH200 Grace Hopper Superchip node configuration, based on Eviden’s BullSequana XH3000 liquid-cooled architecture, with a booster module comprising close to 24,000 NVIDIA GH200 Superchips interconnected with the NVIDIA Quantum-2 InfiniBand networking platform."
"If you look at the Top500 spreadsheet, it tells you the total number of cores used in the machine, plus the number of cores used by accelerators. (These cores are synonymous with a streaming multiprocessor in a GPU architecture, not a count of CUDA cores or tensor cores.) The Jupiter GPU Booster used for the HPL test that have it the number four ranking had 4,801,344 cores and of these, there are 3,106,752 cores allocated to the GPUs, which leaves 1,694,592 cores in the CPU hosts. At 72 cores per Grace, that is 23,536 Grace CPUs and therefore 23,536 Grace-Hopper units and therefore 23,536 Hopper H200 GPUs." | 2025-06-10 | 12.11 | 23,536.00 | NVIDIA | True | 19.37 | 4.7e+19 | 4.7e+19 | 2.3e+19 | 1.2e+19 | 32.98 | 18.00 | 1.3e+12 | 1.1e+9 | 2.9e+8 | 2.9e+8 | EuroHPC says in a statement that the cost of building, delivering, installing, and maintaining the Jupiter machine is €273 million ($287.3 million), and presumably the remaining part of that €500 million is to build or retrofit a datacenter for Jupiter and pay for power and cooling for the machine. | https://web.archive.org/web/20240825120701/https://nvidianews.nvidia.com/news/nvidia-grace-hopper-superchip-powers-jupiter-defining-a-new-class-of-supercomputers-to-propel-ai-for-scientific-discovery | https://archive.is/oyvz9 | https://archive.ph/wip/HgCqo | https://www.anandtech.com/show/21136/nvidia-at-sc23-h200-accelerator-with-hbm3e-and-jupiter-supercomputer-for-2024 | https://top500.org/system/180357/ | 50.92 | 6.36 | ||||||||||||||||
| Inflection AI Cluster | Planned | Likely | Unclear | 19.64 | 2.2e+4 | NVIDIA H100 SXM5 80GB | 22,000.00 | United States of America | Inflection AI | This was in the process of being built, but it is unclear if it ever got built | Private | 31.00 | 1.0e+9 | Inflection | H100 | Along with its partners CoreWeave and NVIDIA, Inflection AI is building the largest AI cluster in the world comprising 22,000 NVIDIA H100 Tensor Core GPUs | Planned | Unclear if this is still happening after the shakeup at Inflection A. They stated they're building this on their website in June 2023 | 11.85 | 22,000.00 | NVIDIA | 19.34 | 4.4e+19 | 4.4e+19 | 2.2e+19 | 1.1e+19 | 30.83 | 31.00 | 7.0e+11 | 1.0e+9 | https://archive.ph/xKdtC | https://web.archive.org/web/20230708074006/https://wccftech.com/inflection-ai-develops-supercomputer-equipped-with-22000-nvidia-h100-ai-gpus/amp/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Oracle OCI MI300x | Existing | Likely | Yes | 19.63 | 2.2e+4 | AMD Instinct MI300X | 16,384.00 | United States of America | Oracle | 2024-11-15 | Unclear exactly when this first became operational, if it was ever built. The announcement September 2024 states that MI300x clusters are now available, and can scale up to 16k GPUs. It's unclear whether this implies that they have already built a 16k MI300x cluster, or if they are just accepting orders to. It seems very likely that this cluster has been built, but I estimate the first operational date as Q4 2024 to be more conservative | Private | 25.05 | 4.5e+8 | Cloud | MI300X | OCI Supercluster with AMD Instinct MI300X accelerators provides high-throughput, ultra-low latency RDMA cluster network architecture for up to 16,384 MI300X GPUs | Q4 2024 | 11.93 | 16,384.00 | AMD | True | 8 | 19.33 | 4.3e+19 | 4.3e+19 | 2.1e+19 | 1.1e+19 | 25.05 | 8.6e+11 | 4.5e+8 | xAI Colossus Memphis Phase 1 | 0.22 | https://archive.ph/gdAb4 | 39.38 | -97.92 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | No | 19.60 | 2.0e+4 | China | 2024-03-15 | Private | China | Anonymized | Anonymized | 19.60 | 4.0e+19 | 4.0e+19 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 19.60 | 2.0e+4 | China | Public/Private | China | Anonymized | Anonymized | 19.60 | 4.0e+19 | 4.0e+19 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Andreessen Horowitz Oxygen | Existing | Likely | Unclear | 19.60 | 2.0e+4 | NVIDIA H100 SXM5 80GB | 20,000.00 | United States of America | Andreessen Horowitz | 2024-10-23 | They have already secured thousands of chips, but plan to expand it to over 20k.
Unclear if this is one cluster or many, if all the chips are H100s, and if they own these chips or are just renting them | Private | 28.54 | 7.6e+8 | Startups via cloud | H100 | Earlier this year, took its focus on the generative technology further as it began accumulating A.I. chips—including the sought-after Nvidia (NVDA) H100 GPUs—with plans to expand its stockpile to more than 20,000, the Information reported this week | 2024-10-23 | Unclear if this is one cluster or many, and if they own these chips or are just renting them | 11.84 | 20,000.00 | NVIDIA | 19.30 | 4.0e+19 | 4.0e+19 | 2.0e+19 | 9.9e+18 | 28.54 | 6.9e+11 | 7.6e+8 | https://web.archive.org/web/20240724120811/https://observer.com/2024/07/andreessen-horowitz-stocking-ai-chips-win-deals-gpu-shortage/ | https://web.archive.org/web/20241110000626/https://techcrunch.com/2024/10/23/andreessen-horowitz-helps-founders-meet-compute-needs-with-oxygen-private-gpu-cluster/ | 39.38 | -97.92 | ||||||||||||||||||||||
| AWS EC2 P5 UltraClusters | Existing | Likely | Unclear | 19.60 | 2.0e+4 | NVIDIA H100 SXM5 80GB | 20,000.00 | United States of America | Amazon | 2023-07-26 | Likely several such clusters in the same availability zone | Private | 29.05 | 7.2e+8 | United States of America | Cloud | H100 | To address customer needs for large-scale and low latency, P5 instances are deployed in the second-generation EC2 UltraClusters, which now provide customers with lower latency across up to 20,000+ NVIDIA H100 Tensor Core GPUs. | 2023-07-26 | Seems likely these are several clusters | 11.83 | 20,000.00 | NVIDIA | 19.30 | 4.0e+19 | 4.0e+19 | 2.0e+19 | 9.9e+18 | 29.05 | 6.8e+11 | 7.2e+8 | https://web.archive.org/web/20240823004421/https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances-powered-by-nvidia-h100-tensor-core-gpus-for-accelerating-generative-ai-and-hpc-applications/ | https://web.archive.org/web/20240826022116/https://aws.amazon.com/ec2/ultraclusters/ | 39.38 | -97.92 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Unclear | 19.60 | 2.0e+4 | 10,000.00 | China | 2024-11-15 | Private | 30.00 | 7.0e+8 | China | 11.82 | 10,000 | 20,000.00 | Anonymized | Anonymized | 19.30 | 4.0e+19 | 4.0e+19 | 2.0e+19 | 9.0e+18 | 30.00 | 6.7e+11 | 7.0e+8 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | No | 19.60 | 2.0e+4 | China | 2024-07-15 | China | True | Anonymized | Anonymized | 19.60 | 4.0e+19 | 4.0e+19 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
NexGen Cloud Hyperstack AQ Compute Supercomputer | Existing | Likely | Yes | 19.51 | 1.6e+4 | NVIDIA H100 SXM5 80GB | 16,384.00 | Norway | NexGen Cloud | 2024-04-19 | They seem to be stating that all 16k GPUs are in the same physical location, but I'm not fully confident in this | Private | 23.38 | 5.8e+8 | Follummoveien 94, 3516 Hønefoss, Norway | Cloud | H100 | "With up to 16,384 H100 80G cards operating in a single cluster"
"by June the supercloud will be comprised of 20,000 Nvidia H100 Tensor Core GPUs" | 2024-04-19 | 11.84 | 16,384.00 | NVIDIA | True | 3 | 19.21 | 3.2e+19 | 3.2e+19 | 1.6e+19 | 8.1e+18 | 23.38 | 6.9e+11 | 5.8e+8 | Meta GenAI 2024a | 0.67 | https://web.archive.org/web/20240911111117/https://www.datacenterdynamics.com/en/news/nexgen-cloud-to-host-ai-supercloud-in-norwegian-data-center/ | https://archive.ph/XcQQY | https://web.archive.org/web/20240524063011/https://www.hyperstack.cloud/nvidia-h100-sxm | 60.21 | 10.24 | |||||||||||||||||
| Oracle OCI Supercluster H100s | Existing | Likely | Yes | 19.51 | 1.6e+4 | NVIDIA H100 SXM5 80GB | 16,384.00 | United States of America | Oracle | 2024-04-15 | Unclear when this was first operational. In November 2023, they state that OCI superclusters "can scale up to" 16k H100s. However, it's unclear whether they had actually built a 16k cluster yet, or were just accepting orders. Musk's xAI was said to be renting 16k of these GPUs in May 2024, signaling that at least one Oracle cluster of this size was operational by then. I mark the first operational date as April 2024 to be slightly conservative.
Also, it's very likely that Oracle has several of these 16k H100 clusters | Private | 23.38 | 5.8e+8 | Cloud,Cohere,xAI | H100 | "With high-performance local storage, high-performance computing (HPC) storage, cluster networking and memory, bare metal instances can be part of an OCI Supercluster that can scale to tens of thousands of NVIDIA H100 GPUs."
"An OCI Supercluster with H100 GPUs can scale up to 16,384 GPUs" | 2024-04 | 11.84 | 16,384.00 | NVIDIA | True | 3 | 19.21 | 3.2e+19 | 3.2e+19 | 1.6e+19 | 8.1e+18 | 23.38 | 6.9e+11 | 5.8e+8 | Meta GenAI 2024a | 0.67 | https://web.archive.org/web/20231023100410/https://blogs.oracle.com/cloud-infrastructure/post/general-availability-oci-compute-nvidia-h100 | https://archive.ph/B3HJm | https://archive.ph/iLAnE | https://archive.ph/cgqbk | 39.38 | -97.92 | |||||||||||||||||
| Paper on Llama 3.1 | Existing | Confirmed | Yes | 19.51 | 1.6e+4 | NVIDIA H100 SXM5 80GB | 16,384.00 | Meta AI | 2024-07-23 | Private | 23.38 | 5.8e+8 | Meta | H100 | 2024-07-23 | 11.84 | True | Meta GenAI 2024a,Meta GenAI 2024b | 16,384.00 | NVIDIA | 3 | 19.21 | 3.2e+19 | 3.2e+19 | 1.6e+19 | 8.1e+18 | 23.38 | 6.9e+11 | 5.8e+8 | Meta GenAI 2024a | 0.67 | https://web.archive.org/web/20250213040824/https://scontent-sjc3-1.xx.fbcdn.net/v/t39.2365-6/468347782_9231729823505907_4580471254289036098_n.pdf?_nc_sid=3c67a6&_nc_zt=14&oe=67B094C0&_nc_gid=Amq2wPDHtohnYaTcmwzmEM_&_nc_cat=110&_nc_ohc=qCynUx1yT0sQ7kNvgGyJpa7&ccb=1-7&oh=00_AYCdVmbLn7nRo_-1yU_lxmEbGI1Bby-xGC_1TZe4QDWaPA&_nc_ht=scontent-sjc3-1.xx | ||||||||||||||||||||||||
| Project Ceiba Phase 1 | Planned | Unlikely | Yes | 19.51 | 1.6e+4 | NVIDIA GH200 | 16,384.00 | United States of America | Amazon,NVIDIA | This plan has been updated to use Blackwell chips. Unclear if this supercomputer was ever completed | Private | 22.96 | 7.5e+8 | Cloud,NVIDIA | GH200 | The system will feature 16,384 GH200 NVL32 Grace Hopper Superchips and is expected to be the world’s largest cloud AI supercomputer. | Planned | This was officially planned, but it appears they have since updated their plan (see Project Ceiba Phase 2) | 11.85 | 16,384.00 | NVIDIA | 19.21 | 3.2e+19 | 3.2e+19 | 1.6e+19 | 8.1e+18 | 22.96 | 7.1e+11 | 7.5e+8 | https://web.archive.org/web/20240228165625/https://www.datacenterdynamics.com/en/news/project-ceiba-aws-and-nvidia-plan-to-build-worlds-largest-cloud-ai-supercomputer/ | 39.38 | -97.92 | ||||||||||||||||||||||||
| Yotta Shakti Cloud D1 | Planned | Likely | Yes | 19.51 | 1.6e+4 | NVIDIA H100 SXM5 80GB | 16,384.00 | India | Yotta Data Services | Private | 22.96 | 7.7e+8 | Yotta Data Center Park, Greater Noida
Plot No 7, Tusiana Village, Knowledge Park V, Greater Noida, Tusyana, Uttar Pradesh 201301, India | Cloud | H100 | Yotta will deploy the first cluster of 16,384 GPUs at NM1... Next, Yotta will deploy a similar-sized cluster at D1 | Planned 2025 | Size is officially stated to be "similar" as NM1 cluster | 11.85 | 16,384.00 | NVIDIA | 19.21 | 3.2e+19 | 3.2e+19 | 1.6e+19 | 8.1e+18 | 22.96 | 7.1e+11 | 7.7e+8 | https://web.archive.org/web/20240521205830/https://yotta.com/media/press-release-yotta-data-services-collaborates-with-nvidia-to-catalyze-indias-ai-transformation/ | ||||||||||||||||||||||||||
| Yotta Shakti Cloud NM1 Phase 2 | Planned | Confirmed | Yes | 19.51 | 1.6e+4 | NVIDIA H100 SXM5 80GB | 16,384.00 | India | Yotta Data Services | Private | 22.96 | 7.7e+8 | Yotta Datacenter Park - Panvel Hiranandani Fortune City. Survey No. 30, MH SH 76, Panvel, Navi Mumbai, Maharashtra 410206, India | Cloud | H100 | Yotta will deploy the first cluster of 16,384 GPUs at NM1 | Planned 2025 | 11.85 | Yotta Shakti Cloud NM1 Phase 1 | 16,384.00 | NVIDIA | 19.21 | 3.2e+19 | 3.2e+19 | 1.6e+19 | 8.1e+18 | 22.96 | 7.1e+11 | 7.7e+8 | https://web.archive.org/web/20240527112515/https://shakticloud.ai/ | https://web.archive.org/web/20240423043934/https://yotta.com/media/press-release-yotta-receives-indias-first-ever-consignment-of-nvidia-h100-the-fastest-gpus-in-the-world-at-its-nm1-data-center-park/ | https://web.archive.org/web/20240521205830/https://yotta.com/media/press-release-yotta-data-services-collaborates-with-nvidia-to-catalyze-indias-ai-transformation/ | 19.03 | 73.02 | ||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 19.48 | 2.0e+4 | China | Public | China | Anonymized | Anonymized | 19.48 | 3.0e+19 | 3.0e+19 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Sakura's B200s Phase 2 | Planned | Confirmed | Yes | 19.48 | 1.5e+4 | NVIDIA B200 | 4,000.00 | Japan | Sakura Internet | This builds on their previous 2k H100s
"Japan's Sakura Internet has secured around 20 billion yen worth ($130 million) of Nvidia's next-generation B200 chips" ~= 4k B200s
10000 GPUs
at least 2k of these are H100s
Unclear amount of B200s
I estimate 4k B200s, and 6k H100s, based on a statement that they acquired $130 million worth of B200s
| Public/Private | 16.42 | 4.8e+8 | Ishikari City, Hokkaido Japan | Cloud | B200,H100 | The company plans to expand the number of GPUs deployed in “Koukaryoku” to fivefold from the initially planned quantity, aiming to incorporate around 10,000 GPUs, including NVIDIA’s latest “NVIDIA HGX B200 system” introduced in March | Planned Q1 2028 | Unclear how many are H100s vs B200s | 11.96 | Sakura's H100s Phase 1 | NVIDIA H100 SXM5 80GB | 6,000 | 10,000.00 | NVIDIA | NVIDIA | 19.17 | 3.0e+19 | 3.0e+19 | 1.5e+19 | 7.5e+18 | 16.42 | 9.1e+11 | 4.8e+8 | https://web.archive.org/web/20240603205526/https://www.trendforce.com/news/2024/04/22/news-japans-sakura-ai-gpu-procurement-reportedly-increases-fivefold-including-purchase-of-nvidia-b200/ | https://web.archive.org/web/20240913153045/https://www.smbom.com/news/13481 | 43.17 | 141.32 | ||||||||||||||||||
| G42 Microsoft 30Mw UAE Cluster A | Planned | Likely | Yes | 19.47 | 1.5e+4 | NVIDIA H100 SXM5 80GB | 15,000.00 | United Arab Emirates | G42,Microsoft | At least one of the 30MW data centers is in Abu Dhabi | Public/Private | 30.00 | 7.1e+8 | Abu Dhabi, United Arab Emirates | Microsoft,Azure | H100 | Private conversations with Microsoft and G42 executives revealed that G42 and Khazna are building three new data centers in the UAE for Microsoft. The centers will be 30, 30, and 100 megawatts each, the equivalent of nearly 100,000 H100 chips (approximately 15,000, 15,000, and 50,000–60,000 respectively). | Planned | 11.69 | 15,000.00 | NVIDIA | 19.17 | 3.0e+19 | 3.0e+19 | 1.5e+19 | 7.4e+18 | 21.02 | 30.00 | 4.9e+11 | 7.1e+8 | https://web.archive.org/web/20250221170709/https://www.csis.org/analysis/united-arab-emirates-ai-ambitions | https://web.archive.org/web/20241006181731/https://www.dwarkeshpatel.com/p/dylan-jon | 24.45 | 54.38 | ||||||||||||||||||||||
| G42 Microsoft 30Mw UAE Cluster B | Planned | Likely | Yes | 19.47 | 1.5e+4 | NVIDIA H100 SXM5 80GB | 15,000.00 | United Arab Emirates | G42,Microsoft | Public/Private | 30.00 | 7.1e+8 | Microsoft,Azure | H100 | Private conversations with Microsoft and G42 executives revealed that G42 and Khazna are building three new data centers in the UAE for Microsoft. The centers will be 30, 30, and 100 megawatts each, the equivalent of nearly 100,000 H100 chips (approximately 15,000, 15,000, and 50,000–60,000 respectively). | Planned | 11.69 | 15,000.00 | NVIDIA | 19.17 | 3.0e+19 | 3.0e+19 | 1.5e+19 | 7.4e+18 | 21.02 | 30.00 | 4.9e+11 | 7.1e+8 | https://web.archive.org/web/20250221170709/https://www.csis.org/analysis/united-arab-emirates-ai-ambitions | https://web.archive.org/web/20241006181731/https://www.dwarkeshpatel.com/p/dylan-jon | 23.85 | 54.26 | ||||||||||||||||||||||||
| Microsoft NVIDIA partnership | Planned | Unlikely | Unclear | 19.47 | 1.5e+4 | NVIDIA H100 SXM5 80GB | 15,000.00 | United States of America | Microsoft | 2023-05-15 | The plans in this article are very vague. This superocmputer probably ended up getting built, but we don't know when, and we likely capture it as another independent entry | Private | 21.79 | 7.2e+8 | H100,A100 | It is the first public cloud to incorporate NVIDIA’s advanced AI stack, adding tens of thousands of NVIDIA A100 and H100 GPUs, NVIDIA Quantum-2 400Gb/s InfiniBand networking and the NVIDIA AI Enterprise software suite to its platform. | 2023-05-15 | Possible duplicate, number of GPUs says "thousands" so this is a very rough estimate | 11.83 | 15,000.00 | NVIDIA | 19.17 | 3.0e+19 | 3.0e+19 | 1.5e+19 | 7.4e+18 | 21.79 | 6.8e+11 | 7.2e+8 | https://web.archive.org/web/20240819113910/https://nvidianews.nvidia.com/news/nvidia-microsoft-accelerate-cloud-enterprise-ai | 39.38 | -97.92 | ||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 19.48 | 1.0e+4 | China | 2024-10-15 | Public/Private | China | 20,000.00 | Anonymized | Anonymized | True | 9 | 19.48 | 3.0e+19 | 3.0e+19 | xAI Colossus Memphis Phase 1 | 0.10 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||
| iGenius Colosseum | Planned | Likely | Yes | 19.46 | 1.5e+4 | NVIDIA GB200 NVL2 (per GPU) | 5,760.00 | Italy | iGenius | 80 servers x 72 GPUs/server = 5760 GPUs | Private | 13.84 | Southern Italy | iGenius | GB200 | iGenius aims to develop colossal AI models using 80 GB200 NVL72 servers | Planned 2025 | 12.02 | 5,760.00 | NVIDIA | 19.16 | 2.9e+19 | 2.9e+19 | 1.4e+19 | 7.2e+18 | 13.84 | 1.0e+12 | https://web.archive.org/web/20250306213958/https://opentools.ai/news/igenius-and-nvidia-to-transform-ai-landscape-with-colosseum-project | 42.50 | 12.65 | ||||||||||||||||||||||||||
| Microsoft Azure Eagle | Existing | Confirmed | Yes | 19.45 | 1.4e+4 | NVIDIA H100 SXM5 80GB | 14,400.00 | United States of America | Microsoft | 2023-11-15 | Said to be part of a larger system | Private | 20.91 | 4.8e+8 | Cloud | H100 | "With 14,400 NVIDIA H100 GPUs (14400/8 = 1800 nodes?), InfiniBand, and Intel Xeon Sapphire Rapids CPUs, the new" | 2023-11-15 | 11.83 | 14,400.00 | NVIDIA | True | 1 | 19.15 | 2.8e+19 | 2.8e+19 | 1.4e+19 | 7.1e+18 | 20.91 | 6.8e+11 | 4.8e+8 | Microsoft Azure Eagle | 1.00 | https://web.archive.org/web/20240901103636/https://www.servethehome.com/microsoft-azure-eagle-is-a-paradigm-shifting-cloud-supercomputer-nvidia-intel/ | 39.38 | -97.92 | ||||||||||||||||||||
| TensorWave MI300X Cluster 1 Phase 2 | Planned | Likely | Yes | 19.42 | 1.3e+4 | AMD Instinct MI300X | 10,000.00 | United States of America | TensorWave | Unclear how many would be in each datacenter. Assuming half in each | Private | 15.02 | 2.7e+8 | Cloud | MI300X | "By the end of 2024, TensorWave aims to have 20,000 MI300X accelerators deployed across two facilities" | Planned H1 2025 | 11.94 | TensorWave MI300X Cluster 1 Phase 1 | 10,000.00 | AMD | 19.12 | 2.6e+19 | 2.6e+19 | 1.3e+19 | 6.5e+18 | 15.02 | 8.7e+11 | 2.7e+8 | https://web.archive.org/web/20241009165502/https://www.theregister.com/2024/04/16/amd_tensorwave_mi300x/ | 39.38 | -97.92 | ||||||||||||||||||||||||
| TensorWave MI300X Cluster 2 | Planned | Likely | Yes | 19.42 | 1.3e+4 | AMD Instinct MI300X | 10,000.00 | United States of America | TensorWave | Unclear how many would be in each datacenter. Assuming half in each | Private | 15.02 | 2.7e+8 | Cloud | MI300X | By the end of 2024, TensorWave aims to have 20,000 MI300X accelerators deployed across two facilities | Planned H1 2025 | 11.94 | 10,000.00 | AMD | 19.12 | 2.6e+19 | 2.6e+19 | 1.3e+19 | 6.5e+18 | 15.02 | 8.7e+11 | 2.7e+8 | https://web.archive.org/web/20241009165502/https://www.theregister.com/2024/04/16/amd_tensorwave_mi300x/ | 39.38 | -97.92 | |||||||||||||||||||||||||
Saudi Data & AI Authority Sovereign AI factory | Planned | Likely | Yes | 19.40 | 1.3e+4 | NVIDIA GB200 NVL2 (per GPU) | 5,000.00 | Saudi Arabia | Saudi Data and Artificial Intelligence Authority | Public | 12.01 | Blackwell,Uncertain | NVIDIA and the Saudi Data & AI Authority (SDAIA) will deploy up to 5,000 Blackwell GPUs for a sovereign AI factory and enable smart city solutions | Planned | 12.02 | 5,000.00 | NVIDIA | 19.10 | 2.5e+19 | 2.5e+19 | 1.3e+19 | 6.3e+18 | 12.01 | 1.0e+12 | https://archive.ph/vvAxQ | 23.38 | 44.65 | |||||||||||||||||||||||||||||
| xAI Fulton Georgia | Planned | Likely | Yes | 19.38 | 1.2e+4 | NVIDIA H100 SXM5 80GB | 12,112.00 | United States of America | xAI | The article indicates that the 12,448 GPUs are 97% Hoppers, 3% Amperes
Location isn't confirmed, but an article from a year earlier reports that xAI received a "10-year tax break on a $700 million project to deploy IT equipment at QTS’ 1025 Jefferson Street site", which matches the description that this site is in Fulton, GA | Private | 20.00 | 1025 Jefferson St NW, Atlanta, GA 30318 | xAI,X | H100,A100 | "The Georgia facility will house an estimated 12,448 Nvidia GPUs. The vast majority of these are Hopper generation H100 GPUs"
"Roughly 3% of the chips are Nvidia's less-powerful A100 GPUs" | Planned 2025 | 11.78 | NVIDIA A100 | 336 | 12,448.00 | NVIDIA | NVIDIA | 19.08 | 2.4e+19 | 2.4e+19 | 1.2e+19 | 6.0e+18 | 17.24 | 20.00 | 6.0e+11 | 7.0e+8 | Of the $700 million in accelerated computing hardware going into the facility, $442 million is allocated to X, and $258 million is allocated to xAI | https://archive.ph/Oeulg | https://web.archive.org/web/20250125054811/https://www.datacenterdynamics.com/en/news/elon-musks-twitterx-granted-10-million-tax-break-for-ai-hardware-in-georgia/ | 33.78 | -84.42 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 19.30 | 1.0e+4 | China | 2025-02-15 | Private | China | 10,000.00 | Anonymized | Anonymized | True | 17 | 2.0e+19 | 2.0e+19 | xAI Colossus Memphis Phase 2 | 0.06 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Foxconn Hon Hai Kaohsiung Supercomputer | Planned | Confirmed | Yes | 19.36 | 1.2e+4 | NVIDIA GB200 NVL2 (per GPU) | 4,608.00 | Taiwan | Foxconn | There was a later announcement for a 10k Blackwell Foxconn supercomputer, so slightly unclear if this just got upgraded or if they are seperate supercomputers | Private | 11.07 | 806, Taiwan, Kaohsiung City, Cianjhen District, Chenggong 2nd Rd, 25號2 樓 | Foxconn | GB200 | will be built around NVIDIA’s groundbreaking Blackwell architecture and feature the GB200 NVL72 platform, which includes a total of 64 racks and 4,608 Tensor Core GPUs | Planned 2026 | 12.02 | 4,608.00 | NVIDIA | 19.06 | 2.3e+19 | 2.3e+19 | 1.2e+19 | 5.8e+18 | 11.07 | 1.0e+12 | https://web.archive.org/web/20241009234027/https://blogs.nvidia.com/blog/foxconn-taiwan-blackwell/ | https://web.archive.org/web/20241008143947/https://www.datacenterdynamics.com/en/news/nvidia-and-foxconn-to-build-taiwans-fastest-ai-supercomputer/ | 22.60 | 120.30 | |||||||||||||||||||||||||
| NVIDIA MLPerf v4.0 Submission 2024 | Existing | Confirmed | Yes | 19.36 | 1.2e+4 | NVIDIA H100 SXM5 80GB | 11,616.00 | NVIDIA | 2024-06-12 | It seems likely that this could be an expansion on the NVIDIA Eos-DFW supercomputer | Private | 16.57 | 4.1e+8 | NVIDIA | H100 | In this round of MLPerf Training, NVIDIA has more than tripled its submission scale to 11,616 H100 GPUs and more than tripled performance to 3.4 minutes to train, | 2024-06-12 | 11.84 | True | NVIDIA CoreWeave Eos-DFW Phase 1 | 11,616.00 | NVIDIA | 6 | 19.06 | 2.3e+19 | 2.3e+19 | 1.1e+19 | 5.7e+18 | 16.57 | 6.9e+11 | 4.1e+8 | Meta GenAI 2024a | 0.47 | https://web.archive.org/web/20240915170526/https://developer.nvidia.com/blog/nvidia-sets-new-generative-ai-performance-and-scale-records-in-mlperf-training-v4-0/ | ||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | No | 19.30 | 1.0e+4 | China | 2024-06-15 | Public/Private | China | Anonymized | Anonymized | 19.30 | 2.0e+19 | 2.0e+19 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Alps Supercomputer Phase 2 | Existing | Confirmed | Yes | 19.33 | 1.1e+4 | NVIDIA GH200 | 10,752.00 | Switzerland | ETH Domain | 2024-09-17 | Total number of sockets = 10,752
There are other portions of this supercomputer, but it is unclear which are in which location, as they are distributed. The 10k Grace Hopper GPUs is by far the largest and most significant portion, though | Public | 11.91 | 1.2e+8 | Swiss National Supercomputing Centre
Via Trevano 131, 6900 Lugano, Switzerland | Swiss Universities | GH200 | has 10,752 of the world’s coveted NVIDIA Grace Hopper superchips | 2024-09-17 | Specs confirmed by the organization building this | 11.95 | Alps Supercomputer Phase 1 | 10,752.00 | NVIDIA | True | 10 | 19.03 | 2.1e+19 | 2.1e+19 | 1.1e+19 | 5.3e+18 | 15.34 | 11.91 | 8.9e+11 | 4.9e+8 | 1.2e+8 | 1.2e+8 | "the CHF100 million ($118 million) supercomputer" | xAI Colossus Memphis Phase 1 | 0.11 | https://web.archive.org/web/20240519135849/https://www.cscs.ch/computers/alps | https://web.archive.org/web/20240415160748/https://www.hpcwire.com/2023/04/05/into-the-alps-what-exactly-is-the-new-swiss-supercomputer-infrastructure/ | https://web.archive.org/web/20241004224121/https://www.hpcwire.com/off-the-wire/cscs-debuts-alps-supercomputer-enhancing-switzerlands-research-infrastructure/ | https://web.archive.org/web/20250215163401/https://www.top500.org/system/180259/ | https://www.swissinfo.ch/eng/science/how-switzerlands-alps-supercomputer-aims-to-advance-ai/87659724 | 46.02 | 8.96 | |||||||||
| Microsoft Azure MLPerf 3.1 Submission | Existing | Confirmed | Yes | 19.33 | 1.1e+4 | NVIDIA H100 SXM5 80GB | 10,752.00 | United States of America | Microsoft | 2023-11-08 | Likely is CoreWeave Eos-DFW Phase 1 | Private | 15.62 | 3.6e+8 | Microsoft | H100 | The GPT-3 LLM model and its 175 billion parameters were trained to completion in four minutes on 1,344 ND H100 v5 virtual machines (VMs), which represent 10,752 NVIDIA H100 Tensor Core GPUs | 2023-11-08 | 11.83 | True | Microsoft Azure Eagle | 10,752.00 | NVIDIA | 1 | 19.03 | 2.1e+19 | 2.1e+19 | 1.1e+19 | 5.3e+18 | 15.62 | 6.8e+11 | 3.6e+8 | NVIDIA CoreWeave Eos-DFW Phase 1 | 1.00 | https://web.archive.org/web/20240415030521/https://azure.microsoft.com/en-us/blog/azure-sets-a-scale-record-in-large-language-model-training/ | 39.38 | -97.92 | |||||||||||||||||||
| NVIDIA CoreWeave Eos-DFW Phase 1 | Existing | Confirmed | Yes | 19.33 | 1.1e+4 | NVIDIA H100 SXM5 80GB | 10,752.00 | United States of America | NVIDIA,CoreWeave | 2023-11-08 | There's also a 4.6k cluster with the same name | Private | 15.62 | 3.6e+8 | 1000 Coit Rd, Plano, TX 75075 | NVIDIA,CoreWeave | H100 | NVIDIA’s AI platform raised the bar for AI training and high performance computing in the latest MLPerf industry benchmarks. Among many new records and milestones, one in generative AI stands out: NVIDIA Eos — an AI supercomputer powered by a whopping 10,752 NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking — completed a training benchmark based on a GPT-3 model with 175 billion parameters trained on one billion tokens in just 3.9 minutes. | 2023-11-08 | Stated in NVIDIA blog post | 11.83 | 10,752.00 | NVIDIA | True | 1 | 19.03 | 2.1e+19 | 2.1e+19 | 1.1e+19 | 5.3e+18 | 15.62 | 6.8e+11 | 3.6e+8 | NVIDIA CoreWeave Eos-DFW Phase 1 | 1.00 | https://web.archive.org/web/20240430123932/https://blogs.nvidia.com/blog/scaling-ai-training-mlperf/ | https://web.archive.org/web/20240901052405/https://www.nextplatform.com/2024/03/06/a-tale-of-two-nvidia-eos-supercomputers/ | 33.01 | -96.76 | |||||||||||||||||
| Oracle OCI Supercluster A100s | Existing | Likely | Yes | 19.31 | 1.0e+4 | NVIDIA A100 | 32,768.00 | United States of America | Oracle | 2023-11-15 | Unclear exactly when this first became operational, if it was ever built. The announcement March 2023 states Oracle OCI can now scale to 32k A100s, but it's unclear whether this implies that a 32k A100 cluster has already been built, or just that they have the capacity to build one. It seems likely that a cluster of this size was built, but there's little direct evidence for this.
I estimate Q4 2023 to be slightly conservative on the timing for when it was first operational. | Private | 27.19 | 6.5e+8 | Cloud | A100 | OCI Supercluster networking can now scale up to 4,096 OCI Compute Bare Metal instances with 32,768 A100 GPUs | Q4 2023 | 11.58 | 32,768.00 | NVIDIA | True | 4 | 19.01 | 2.0e+19 | 2.0e+19 | 1.0e+19 | 5.1e+18 | 27.19 | 3.8e+11 | 6.5e+8 | Microsoft Azure Eagle | 0.72 | https://web.archive.org/web/20240910231627/https://www.oracle.com/news/announcement/nvidia-chooses-oracle-cloud-infrastructure-for-ai-services-2023-03-21/ | https://archive.ph/B3HJm | 39.38 | -97.92 | |||||||||||||||||||
| Google TPUv5e | Existing | Confirmed | Unclear | 19.30 | 1.0e+4 | Google TPU v5e | 50,944.00 | 2023-11-08 | Private | 14.80 | 1.6e+8 | TPUv5e | we used Multislice Training to run what we believe to be the world’s largest publicly disclosed LLM distributed training job (in terms of the number of chips used for training) on a compute cluster of 50,944 Cloud TPU v5e chips (spanning 199 Cloud TPU v5e pods) that is capable of achieving 10 exa-FLOPs (16-bit), or 20 exa-OPs (8-bit), of total peak performance | 2023-11-08 | Unclear whether it is a single cluster | 11.83 | 50,944.00 | 19.00 | 2.0e+19 | 2.0e+19 | 1.0e+19 | 14.80 | 6.8e+11 | 1.6e+8 | https://web.archive.org/web/20240822230040/https://cloud.google.com/blog/products/compute/the-worlds-largest-distributed-llm-training-job-on-tpu-v5e | |||||||||||||||||||||||||||||||
| Imbue 10k Cluster | Existing | Likely | Yes | 19.30 | 1.0e+4 | NVIDIA H100 SXM5 80GB | 10,000.00 | United States of America | Imbue | 2023-09-07 | Private | 14.52 | 3.3e+8 | Imbue | H100 | Dell PowerEdge server cluster with nearly 10,000 NVIDIA H100 Tensor Core GPUs | 2023-09-07 | 11.83 | 10,000.00 | NVIDIA | True | 1 | 19.00 | 2.0e+19 | 2.0e+19 | 9.9e+18 | 4.9e+18 | 14.52 | 6.8e+11 | 3.3e+8 | Tesla 10k H100 Cluster | 1.00 | https://web.archive.org/web/20240423234157/https://investors.delltechnologies.com/news-releases/news-release-details/imbue-develop-next-generation-ai-models-150-million-dell-high | https://web.archive.org/web/20240830093115/https://imbue.com/company/introducing-imbue/ | 39.38 | -97.92 | ||||||||||||||||||||
| Poolside 10k Cluster | Planned | Likely | Yes | 19.30 | 1.0e+4 | NVIDIA H100 SXM5 80GB | 10,000.00 | Poolside | Unclear if they own this or if they're just renting it. | Private | 14.01 | 4.7e+8 | Poolside | H100 | With this new funding round we have been able to scale our training cluster to 10,000 GPUs and start the work of scaling up both RLCEF and our model training. | Planned | Unclear if they own this or if they're just renting it.
Type of GPU unspecified, I'm assuming they're H100s | 11.85 | 10,000.00 | NVIDIA | 19.00 | 2.0e+19 | 2.0e+19 | 9.9e+18 | 4.9e+18 | 14.01 | 7.1e+11 | 4.7e+8 | https://web.archive.org/web/20241007114224/https://poolside.ai/checkpoint/announcing-our-500-million-fundraise-to-make-progress-towards-agi | |||||||||||||||||||||||||||
| Tesla 10k H100 Cluster | Existing | Confirmed | Yes | 19.30 | 1.0e+4 | NVIDIA H100 SXM5 80GB | 10,000.00 | United States of America | Tesla | 2023-08-28 | There are claims of the cluster costing $300 million, however, that was not an official anouncement and might just be a BOTEC "30k per H100 times 10,000 | Private | 14.52 | 3.6e+8 | "on prem" for Tesla | Tesla | H100 | Tomorrow, @Tesla will turn on a massive and very expensive 10,000 unit NVIDIA H100 GPU cluster to help it train FSD | 2023-08-28 | Said it was "going live tomorrow" in Aug 2023 | 11.83 | 10,000.00 | NVIDIA | True | 1 | 19.00 | 2.0e+19 | 2.0e+19 | 9.9e+18 | 4.9e+18 | 14.52 | 6.8e+11 | 3.6e+8 | Tesla 10k H100 Cluster | 1.00 | https://web.archive.org/web/20240802004227/https://www.theregister.com/2023/08/30/tesla_nvidia_supercomputer/ | https://archive.ph/PwU9T | https://web.archive.org/web/20240524114346/https://www.tomshardware.com/news/teslas-dollar300-million-ai-cluster-is-going-live-today | 42.41 | -82.98 | ||||||||||||||||
| SoftBank Planned B200 Superpod | Planned | Likely | Yes | 19.26 | 9.1e+3 | NVIDIA B200 | 4,000.00 | Japan | Softbank | They had 6k GPUs before (2k A100s, 4k H100s), so bringing the total to 10k GPUs would be adding 4k Blackwell GPUs | Private | 8.01 | 2.0e+8 | Japan | Softbank | B200 | The company has further plans to expand this with an Nvidia DGX SuperPOD featuring GGX B200 systems within the fiscal year (FY) 2025, ultimately growing the number of GPUs installed to around 10,000 with a compute capability of 25.7 exaflops. | Planned | 12.05 | 4,000.00 | NVIDIA | 18.95 | 1.8e+19 | 1.8e+19 | 9.0e+18 | 4.5e+18 | 8.01 | 1.1e+12 | 2.0e+8 | https://web.archive.org/web/20241116120654/https://nvidianews.nvidia.com/news/nvidia-and-softbank-accelerate-japans-journey-to-global-ai-powerhouse | https://web.archive.org/web/20241221053240/https://www.datacenterdynamics.com/en/news/softbank-installs-4000-nvidia-h100s-for-japanese-ai-computing-platform/ | 36.39 | 138.59 | |||||||||||||||||||||||
| S. Korea 6th national supercomputer | Planned | Likely | Yes | 19.24 | 8.8e+3 | NVIDIA H100 SXM5 80GB | 8,800.00 | Korea (Republic of) | Ministry of Science and ICT | Public | 12.33 | 4.1e+8 | Korea (Republic of) | Researchers | H100 | A total of 8,800 GPUs will be used for No. 6 | Planned 2026 | Uncertain what type of GPUs will be used | 11.85 | 8,800.00 | NVIDIA | 18.94 | 1.7e+19 | 1.7e+19 | 8.7e+18 | 4.4e+18 | 12.33 | 7.1e+11 | 4.1e+8 | https://web.archive.org/web/20250110174004/https://biz.chosun.com/en/en-science/2025/01/10/A4AEH4EX2JEA7CGFXUOADNCGD4/ | ||||||||||||||||||||||||||
| Sesterce Nordics | Existing | Likely | Yes | 19.21 | 8.2e+3 | NVIDIA H100 SXM5 80GB | 8,192.00 | Sesterce | 2024-06-29 | This cluster was originally planned for Spain, but was later moved to the Nordics | Private | 11.69 | 2.9e+8 | The Nordics | Cloud | H100 | 8192 H100s available from 06/29/2024 to 06/29/2026 (Spain)
Sesterce 1024 Servers Cluster available in Spain Europe End of June. | 2024-06-29 | 11.84 | 8,192.00 | NVIDIA | True | 12 | 18.91 | 1.6e+19 | 1.6e+19 | 8.1e+18 | 4.1e+18 | 11.69 | 6.9e+11 | 2.9e+8 | Meta GenAI 2024a | 0.33 | https://web.archive.org/web/20240523122734/https://gpulist.ai/detail/f6be42a | ||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 19.30 | 8.0e+3 | 30,000.00 | China | Public | 40.00 | China | 11.70 | 30,000.00 | Anonymized | Anonymized | 19.30 | 2.0e+19 | 2.0e+19 | 40.00 | 5.0e+11 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 19.30 | 8.0e+3 | China | 2024-03-15 | Public/Private | China | Anonymized | Anonymized | True | 7 | 19.30 | 2.0e+19 | 2.0e+19 | Microsoft Azure Eagle | 0.60 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | No | 19.30 | 8.0e+3 | China | 2023-04-15 | China | Anonymized | Anonymized | 2.0e+19 | 2.0e+19 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 19.30 | 8.0e+3 | 8,000.00 | China | 2024-08-15 | Private | 10.00 | 3.0e+8 | China | 11.90 | 8,000.00 | Anonymized | Anonymized | True | 15 | 18.90 | 2.0e+19 | 2.0e+19 | 8.0e+18 | 4.0e+18 | 10.00 | 8.0e+11 | 3.0e+8 | Meta GenAI 2024a | 0.30 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Magic G4 Google Cloud Rental | Existing | Confirmed | Yes | 19.20 | 8.0e+3 | NVIDIA H100 SXM5 80GB | 8,000.00 | United States of America | Rented from Google Cloud. Not marked "potential duplicate" because we don't have other operational Google cloud clusters of this size | Private | 11.21 | 3.8e+8 | Magic | H100 | Magic-G4 supercomputer, which will be based on Nvidia H100 Tensor Core graphics processor unit (GPU) chips. Magic says it has 8,000 of the hugely powerful chips | Planned | 11.85 | 8,000.00 | NVIDIA | True | 18.90 | 1.6e+19 | 1.6e+19 | 7.9e+18 | 4.0e+18 | 11.21 | 7.1e+11 | 3.8e+8 | https://archive.ph/DLF9r | https://archive.ph/PZl4u | 39.38 | -97.92 | ||||||||||||||||||||||||
| Nebius 8k Finland Phase 1 | Existing | Likely | Yes | 19.20 | 8.0e+3 | NVIDIA H100 SXM5 80GB | 8,000.00 | Finland | Nebius AI | 2024-05-10 | Private | 11.42 | 2.8e+8 | Moreenikatu 6, 04600 Mäntsälä, Finland | Cloud | H100 | we also constructed an 8,000-GPU supercluster | 2024-05-10 | 11.84 | 8,000.00 | NVIDIA | True | 12 | 18.90 | 1.6e+19 | 1.6e+19 | 7.9e+18 | 4.0e+18 | 11.42 | 6.9e+11 | 2.8e+8 | Meta GenAI 2024a | 0.33 | https://web.archive.org/web/20241008144121/https://www.datacenterdynamics.com/en/news/nebius-deploys-ai-cluster-at-equinix-data-center-in-paris/ | https://archive.ph/x0bXS | 60.63 | 25.26 | |||||||||||||||||||
| Tesla Dojo 1 Planned Phase 2 | Planned | Likely | Yes | 19.20 | 8.0e+3 | Tesla D1 Dojo | United States of America | Tesla | "roughly 8k H100-equivalent" I interpreted this as | Private | San Jose, California | Tesla | Dojo | And Dojo 1 will have roughly 8k H100-equivalent of training online by end of year | Planned | Source is one Musk Tweet. Uncertain on the exact number of chips, because Musk only specifies H100 equivalents | Tesla Dojo 1 Phase 1 | Tesla | 18.90 | 1.6e+19 | 1.6e+19 | 8.0e+18 | https://archive.ph/pM3CE | https://web.archive.org/web/20240903213050/https://techcrunch.com/2024/08/10/teslas-dojo-a-timeline/ | 37.34 | -121.89 | ||||||||||||||||||||||||||||||
| Microsoft GPT-4 cluster | Existing | Likely | Yes | 19.19 | 7.9e+3 | NVIDIA A100 | 25,000.00 | United States of America | Microsoft,OpenAI | 2022-04-30 | Training finished August 2022. Training lasted ~3 months, meaning that it was operational late May at the latest | Private | 21.29 | 5.6e+8 | Des Moines, Iowa | OpenAI | A100 | OpenAI’s training FLOPS for GPT-4 is ~2.15e25, on ~25,000 A100s for 90 to 100 days at about 32% to 36% MFU.... | 2022-04-30 | Uncertain, but seems likely located in Des Moines, Iowa | 11.56 | 25,000.00 | NVIDIA | True | 1 | 18.89 | 1.6e+19 | 1.6e+19 | 7.8e+18 | 3.9e+18 | 21.29 | 3.7e+11 | 5.6e+8 | True | Microsoft GPT-4 cluster | 1.00 | https://archive.ph/qTvzU | https://archive.ph/vqdRc#selection-4109.11-4109.239 | https://web.archive.org/web/20241212034522/https://www.glennklockwood.com/garden/microsoft-supercomputers | https://web.archive.org/web/20250201064042/https://semianalysis.com/2023/07/10/gpt-4-architecture-infrastructure/ | 41.58 | -93.62 | ||||||||||||||
| Aramco Groq Inference Cluster | Existing | Likely | Yes | 19.17 | 7.5e+3 | GroqChip LPU v1 | 19,725.00 | Saudi Arabia | Saudi Aramco | 2025-01-15 | This cluster is designed for only inference, not training | Private | 8.49 | 6.7e+8 | Dammam, Saudi Arabia | Saudi Aramco,Cloud | GroqChip | "19,725 LPUs deployed Dammam, January 2025" | 2025-01 | 11.64 | 19,725.00 | Unknown | True | 28 | 18.57 | 1.5e+19 | 1.5e+19 | 3.7e+18 | 8.49 | 4.4e+11 | 6.7e+8 | xAI Colossus Memphis Phase 1 | 0.07 | https://web.archive.org/web/20250212161800/https://www.linkedin.com/posts/sundeepm_groq-aramco-digital-build-the-activity-7294312309013848064-T1fY/ | 26.43 | 50.10 | ||||||||||||||||||||
| Oak Ridge NL Frontier | Existing | Confirmed | Yes | 19.16 | 7.3e+3 | AMD Radeon Instinct MI250X | 37,632.00 | United States of America | US Department of Energy | 2022-05-30 | (9408 nodes)x(4 GPUs/node) = 37,632 GPUs | Public | 40.00 | 6.2e+8 | Oak Ridge National Laboratory
5200, 1 Bethel Valley Rd, Oak Ridge, TN 37830 | US Government,Academia,Industry | AMD MI250X | The system has 74 Olympus rack HPE cabinets, each with 128 AMD compute nodes, and a total of 9,408 AMD compute nodes....
Each Frontier compute node consists of [1x] 64-core AMD “Optimized 3rd Gen EPYC” CPU (with 2 hardware threads per physical core) with access to 512 GB of DDR4 memory. Each node also contains [4x] AMD MI250X | 2022-05-30 | Details released on official government website | 11.56 | 37,632.00 | AMD | True | 2 | 19.16 | 1.4e+19 | 1.4e+19 | 1.4e+19 | 3.6e+18 | 40.07 | 40.00 | 3.6e+11 | 9.7e+8 | 6.0e+8 | 6.2e+8 | $600M | True | Microsoft GPT-4 cluster | 0.92 | https://web.archive.org/web/20240720224959/https://docs.olcf.ornl.gov/systems/frontier_user_guide.html | https://web.archive.org/web/20240823003004/https://www.ornl.gov/news/frontier-supercomputer-debuts-worlds-fastest-breaking-exascale-barrier | 36.01 | -84.27 | ||||||||||||
| Microsoft Sweden Gävle | Planned | Likely | Yes | 19.12 | 6.7e+3 | 6,666.00 | Sweden | Microsoft | The "Reported Manual Flop/s" column is the number of INT8 FLOPs that this cluster could do assuming it was using (20k/3) GPUs, and they were all H100s. I assume that the true FLOP/s number will be higher as I assume they will likely use some Blackwell chips. Unclear how many GPUs will be in each of the three stated data centers, so this (20k/3) is a lower bound on how large the largest cluster must be. Also, it is unclear what type of GPUs will be used (though presumable some H100s and Blackwells). | Private | 805 91 Gävle, Sweden | Cloud,Microsoft | Microsoft said it will place more than 20,000 graphic processing units (GPUs) at its data centers in Sandviken, Gävle, and Staffanstorp | Planned 2027 | Unclear how many GPUs will be in which location, and which type. FLOP/s count is lower bound on FLOP/s assuming that they're all H100s (and there are 20k/3 GPUs in each cluster) | 6,666.00 | Unknown | 18.82 | 1.3e+19 | 1.3e+19 | 6.6e+18 | https://web.archive.org/web/20240915193858/https://www.datacenterdynamics.com/en/news/microsoft-to-deploy-20000-gpus-in-sweden/ | 60.62 | 17.11 | ||||||||||||||||||||||||||||||||
| Microsoft Sweden Sandviken | Planned | Likely | Yes | 19.12 | 6.7e+3 | 6,666.00 | Sweden | Microsoft | The "Reported Manual Flop/s" column is the number of INT8 FLOPs that this cluster could do assuming it was using (20k/3) GPUs, and they were all H100s. I assume that the true FLOP/s number will be higher as I assume they will likely use some Blackwell chips. Unclear how many GPUs will be in each of the three stated data centers, so this (20k/3) is a lower bound on how large the largest cluster must be. Also, it is unclear what type of GPUs will be used (though presumable some H100s and Blackwells). | Private | Gamla Tunavägen, 811 42 Sandviken, Sweden | Cloud,Microsoft | Microsoft said it will place more than 20,000 graphic processing units (GPUs) at its data centers in Sandviken, Gävle, and Staffanstorp | Planned 2027 | Unclear how many GPUs will be in which location, and which type. FLOP/s count is lower bound on FLOP/s assuming that they're all H100s (and there are 20k/3 GPUs in each cluster) | 6,666.00 | Unknown | 18.82 | 1.3e+19 | 1.3e+19 | 6.6e+18 | https://web.archive.org/web/20240915193858/https://www.datacenterdynamics.com/en/news/microsoft-to-deploy-20000-gpus-in-sweden/ | 60.64 | 16.73 | ||||||||||||||||||||||||||||||||
| Microsoft Sweden Staffanstorp | Planned | Likely | Yes | 19.12 | 6.7e+3 | 6,666.00 | Sweden | Microsoft | The "Reported Manual Flop/s" column is the number of INT8 FLOPs that this cluster could do assuming it was using (20k/3) GPUs, and they were all H100s. I assume that the true FLOP/s number will be higher as I assume they will likely use some Blackwell chips. Unclear how many GPUs will be in each of the three stated data centers, so this (20k/3) is a lower bound on how large the largest cluster must be. Also, it is unclear what type of GPUs will be used (though presumable some H100s and Blackwells). | Private | Västanvägen 86, 245 42 Staffanstorp, Sweden | Cloud,Microsoft | Microsoft said it will place more than 20,000 graphic processing units (GPUs) at its data centers in Sandviken, Gävle, and Staffanstorp | Planned 2027 | Unclear how many GPUs will be in which location, and which type. FLOP/s count is lower bound on FLOP/s assuming that they're all H100s (and there are 20k/3 GPUs in each cluster) | 6,666.00 | Unknown | 18.82 | 1.3e+19 | 1.3e+19 | 6.6e+18 | https://web.archive.org/web/20240915193858/https://www.datacenterdynamics.com/en/news/microsoft-to-deploy-20000-gpus-in-sweden/ | 55.64 | 13.19 | ||||||||||||||||||||||||||||||||
| Alps Supercomputer Phase 1 | Existing | Confirmed | Yes | 19.10 | 6.4e+3 | NVIDIA GH200 | 6,400.00 | Switzerland | ETH Domain | 2024-06-01 | Public | 8.68 | 2.9e+8 | Swiss National Supercomputing Centre
Via Trevano 131, 6900 Lugano, Switzerland | Swiss Universities | GH200 | Calculated from Top500 | 2024-06-01 | 11.86 | Alps Supercomputer Phase 2 | 6,400.00 | NVIDIA | True | 15 | 18.80 | 1.3e+19 | 1.3e+19 | 6.3e+18 | 3.2e+18 | 9.13 | 8.68 | 7.3e+11 | 2.9e+8 | Meta GenAI 2024a | 0.26 | https://web.archive.org/web/20250215163401/https://www.top500.org/system/180259/ | 46.02 | 8.96 | ||||||||||||||||||
| AWS EC2 Trn1 | Existing | Likely | Unclear | 19.10 | 6.4e+3 | Amazon Trainium1 | 30,000.00 | United States of America | Amazon | 2022-10-11 | Private | Cloud | AWS Trainium | customers will be able to scale the training of machine learning models with up to 30,000 Trainium accelerators interconnected with EFA petabit-scale networking | 2022-10-11 | 30,000.00 | Amazon AWS | 18.76 | 1.3e+19 | 1.3e+19 | 5.7e+18 | 5.7e+18 | https://web.archive.org/web/20240531005626/https://press.aboutamazon.com/2022/10/aws-announces-general-availability-of-amazon-ec2-trn1-instances-powered-by-aws-designed-trainium-chips | 39.38 | -97.92 | |||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 19.00 | 6.0e+3 | China | Public | China | Anonymized | Anonymized | 18.78 | 1.0e+19 | 1.0e+19 | 6.0e+18 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Paper on Movie Gen | Existing | Confirmed | Yes | 19.08 | 6.1e+3 | NVIDIA H100 SXM5 80GB | 6,144.00 | Meta AI | 2024-10-04 | Private | 8.77 | 2.3e+8 | Meta | H100 | "We trained the media generation models using up to 6,144 H100 GPUs, each running at 700W TDP and with 80GB HBM3," | 2024-10-04 | 11.84 | True | Meta GenAI 2024a | 6,144.00 | NVIDIA | 24 | 18.78 | 1.2e+19 | 1.2e+19 | 6.1e+18 | 3.0e+18 | 8.77 | 6.9e+11 | 2.3e+8 | xAI Colossus Memphis Phase 1 | 0.06 | https://web.archive.org/web/20250211214423/https://ai.meta.com/static-resource/movie-gen-research-paper | |||||||||||||||||||||||
| AIST ABCI 3.0 | Existing | Confirmed | Yes | 19.08 | 6.1e+3 | NVIDIA H200 SXM | 6,128.00 | Japan | National Institute of Advanced Industrial Science and Technology (AIST) | 2025-01-20 | Public | 8.59 | 2.4e+8 | Kashiwa Campus, The University of Tokyo
Japan, 〒277-0882 Chiba, Kashiwa, Kashiwanoha, 5 Chome−1−5 | industry,academia,and government in Japan | H200 | ABCI 3.0 consists of computing servers equipped with 6128 of the NVIDIA H200 GPUs | 2025-01-20 | 11.85 | AIST ABCI 2.0 | 6,128.00 | NVIDIA | True | 32 | 18.78 | 1.2e+19 | 1.2e+19 | 6.1e+18 | 3.0e+18 | 8.59 | 7.1e+11 | 2.2e+8 | 2.3e+8 | 2.4e+8 | 36B yen | xAI Colossus Memphis Phase 1 | 0.06 | https://web.archive.org/web/20241115024655/https://arxiv.org/html/2411.09134v1 | https://web.archive.org/web/20250211075424/https://www.asahi.com/ajw/articles/15593572 | 35.69 | 139.69 | |||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 19.00 | 6.0e+3 | China | 2022-08-15 | Private | China | Anonymized | Anonymized | True | 3 | 19.00 | 1.0e+19 | 1.0e+19 | Microsoft GPT-4 cluster | 0.80 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| University of Bristol Isambard-AI | Existing | Confirmed | Yes | 19.03 | 5.4e+3 | NVIDIA GH200 | 5,448.00 | United Kingdom of Great Britain and Northern Ireland | UK Research and Innovation | 2025-06-15 | Public | 7.63 | 2.8e+8 | National Composites Centre
Bristol & Bath Science Park, Emersons Green, Bristol BS16 7FS, United Kingdom | U.K. government’s Frontier AI Taskforce | GH200 | When fully installed next year, it will pack 5,448 NVIDIA GH200 Grace Hopper Superchips | 2025-06 | In Top500 | 11.85 | 5,448.00 | NVIDIA | True | 18.73 | 1.1e+19 | 1.1e+19 | 5.4e+18 | 2.7e+18 | 7.63 | 7.1e+11 | 2.5e+8 | 2.7e+8 | 2.8e+8 | The UK government says it will cough up £225 million ($273 million) | https://web.archive.org/web/20240526090503/https://blogs.nvidia.com/blog/uk-largest-ai-supercomputer/ | https://web.archive.org/web/20240525234841/https://www.hpcwire.com/off-the-wire/uks-isambard-ai-supercomputer-goes-online-ranking-2nd-greenest-globally/ | https://www.theregister.com/2023/11/01/uk_isambard_supercomputer/ | 51.45 | -2.60 | |||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 19.00 | 5.0e+3 | China | 2025-02-15 | Public/Private | China | Anonymized | Anonymized | True | 36 | 19.00 | 1.0e+19 | 1.0e+19 | xAI Colossus Memphis Phase 2 | 0.03 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| KDDI Sharp Sakai | Planned | Likely | Yes | 19.00 | 5.1e+3 | NVIDIA GB200 NVL2 (per GPU) | 2,000.00 | Japan | KDDI | Unclear whether "1,000 units" means 1k GPUs, 1k chips, or 1k racks of 72 GPUs | Private | 4.80 | 1番地 Takumicho, Sakai Ward, Sakai, Osaka 590-0908, Japan | Cloud | GB200 | It is assumed that NVIDIA shall supply an estimated quantity of 1,000 units of GB200 NVL72 | Planned Q1 2026 | Very uncertain about GPU number | 12.02 | 2,000.00 | NVIDIA | 18.70 | 1.0e+19 | 1.0e+19 | 5.0e+18 | 2.5e+18 | 4.80 | 1.0e+12 | https://web.archive.org/web/20240917144105/https://newsroom.kddi.com/english/news/detail/kddi_nr_s-9_3387.html | 34.58 | 135.48 | |||||||||||||||||||||||||
| Meta Research SuperCluster (RSC-1) Phase 2 | Existing | Confirmed | Yes | 19.00 | 5.0e+3 | NVIDIA A100 | 16,000.00 | United States of America | Meta AI | 2023-05-18 | The location is found from triangulating from the promo video. I'm fairly confident in it, but it isn't guaranteed to be accurate | Private | 13.28 | 3.6e+8 | 6200 Technology Blvd, 23150 Sandston, Virginia, USA | Meta | A100 | The full system includes 2,000 DGX A100 systems, totaling a staggering 16,000 A100 GPUs | 2023-05-18 | 11.58 | Meta Research SuperCluster (RSC-1) Phase 1 | 16,000.00 | NVIDIA | True | 4 | 18.70 | 1.0e+19 | 1.0e+19 | 5.0e+18 | 2.5e+18 | 13.28 | 3.8e+11 | 3.6e+8 | Microsoft GPT-4 cluster | 0.64 | https://web.archive.org/web/20231215015022/http://www.hpcwire.com/2023/05/18/meta-completes-research-supercluster-announces-next-gen-datacenter/ | https://web.archive.org/web/20240828114058/https://ai.meta.com/blog/ai-rsc/ | 37.48 | -77.24 | |||||||||||||||||
| Inflection-2 training cluster | Existing | Likely | Yes | 19.00 | 5.0e+3 | NVIDIA H100 SXM5 80GB | 5,000.00 | United States of America | Inflection AI,CoreWeave | 2023-07-30 | Inflection lent 3,500 H100 GPUs to Corweave (who appears to be the party that built that cluster) for the June MLPerf submission. Given that this is just slightly larger and operational just after, it appears very likely that this cluster is an extension of the MLPerf cluster (or the MLPerf cluster was just a subset of this cluster).
It's possible that both of these subsets of the Eos-DFW cluster, but there isn't much evidence for this besides that Eos-DFW was submitted to MLPerf by CoreWeave later that year (I give it a ~50% chance this is Eos-DFW) | Private | 7.26 | 1.8e+8 | Inflection | H100 | Inflection-2 was trained on 5,000 NVIDIA H100 GPUs | 2023-07-30 | 11.83 | NVIDIA Coreweave MLPerf v3.0 Submission 2023 | True | NVIDIA CoreWeave Eos-DFW Phase 1 | 5,000.00 | NVIDIA | 5 | 18.69 | 9.9e+18 | 9.9e+18 | 4.9e+18 | 2.5e+18 | 7.26 | 6.8e+11 | 1.8e+8 | Microsoft GPT-4 cluster | 0.63 | https://web.archive.org/web/20241203070436/https://inflection.ai/blog/inflection-2 | https://archive.ph/fB8Io | 39.38 | -97.92 | |||||||||||||||||
| Sustainable Metal Cloud Singapore Phase 2 | Planned | Likely | Unclear | 19.00 | 5.0e+3 | NVIDIA H100 SXM5 80GB | 5,000.00 | Singapore | SMC - Sustainable Metal Cloud | Private | 7.01 | 2.4e+8 | Singapore | Cloud | H100 | Their ambitious projection is to have 5,000 GPUs operational in Singapore by the end of the year | Planned H2 2024 | 11.85 | Sustainable Metal Cloud Singapore Phase 1 | 5,000.00 | NVIDIA | 18.69 | 9.9e+18 | 9.9e+18 | 4.9e+18 | 2.5e+18 | 7.01 | 7.1e+11 | 2.4e+8 | https://web.archive.org/web/20250116210511/https://itbrief.asia/story/firmus-sustainable-metal-cloud-powerfully-cuts-ai-energy-usage | 1.35 | 103.81 | ||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.95 | 5.0e+3 | 40,000.00 | China | 2024-06-15 | Public/Private | China | 40,000.00 | Anonymized | Anonymized | True | 20 | 18.95 | 9.0e+18 | 9.0e+18 | Meta GenAI 2024a | 0.20 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||
| NVIDIA Eos Phase 2 | Existing | Confirmed | Yes | 18.96 | 4.6e+3 | NVIDIA H100 SXM5 80GB | 4,608.00 | United States of America | NVIDIA | 2023-11-01 | There's also a 10.7k cluster with the same name | Private | 6.69 | 1.5e+8 | United States of America | NVIDIA | H100 | Its “Eos” supercomputer is expected to be the world’s fastest AI system after it begins operations later this year, featuring a total of 576 DGX H100 systems with 4,608 DGX H100 GPUs. | 2023-11-01 | Stated in NVIDIA blog post | 11.83 | NVIDIA Eos Phase 1 | 4,608.00 | NVIDIA | True | 8 | 18.66 | 9.1e+18 | 9.1e+18 | 4.6e+18 | 2.3e+18 | 6.69 | 6.8e+11 | 1.5e+8 | Tesla 10k H100 Cluster | 0.46 | https://nvidianews.nvidia.com/news/nvidia-announces-dgx-h100-systems-worlds-most-advanced-enterprise-ai-infrastructure | https://web.archive.org/web/20240901052405/https://www.nextplatform.com/2024/03/06/a-tale-of-two-nvidia-eos-supercomputers/ | https://web.archive.org/web/20240901052405/https://www.nextplatform.com/2024/03/06/a-tale-of-two-nvidia-eos-supercomputers/ | 39.38 | -97.92 | |||||||||||||||
| Lawrence Livermore NL Tuolumne | Existing | Confirmed | Yes | 18.96 | 4.6e+3 | AMD Instinct MI300A | 4,608.00 | United States of America | US Department of Energy | 2024-11-18 | Public | 5.66 | 1.9e+8 | Lawrence Livermore National Laboratory
7000 East Ave, Livermore, CA 94550 | Lawrence Livermore NL,Scientific Research | AMD MI300A | Calculated from Top500 | 2024-11-18 | 11.90 | 4,608.00 | AMD | True | 35 | 18.66 | 9.0e+18 | 9.0e+18 | 4.5e+18 | 2.3e+18 | 7.14 | 5.66 | 8.0e+11 | 1.9e+8 | xAI Colossus Memphis Phase 1 | 0.05 | https://web.archive.org/web/20241109063437/https://www.top500.org/system/180282/ | 37.68 | -121.77 | |||||||||||||||||||
| Google Oklahoma TPU v4 Pods | Existing | Confirmed | Yes | 18.95 | 4.6e+3 | Google TPU v4 | 32,768.00 | United States of America | 2022-05-12 | 8 pods x (4096 chips/pod) = 32,768 chips | Private | 11.86 | 3.0e+8 | 4581 Webb St, Pryor, OK 74361 | TPUv4 | Google’s fourth iteration of its Tensor Processing Units launched at last year’s I/O and a single TPU pod consists of 4,096 of these chips. Each chip has a peak performance of 275 teraflops and each pod promises a combined compute power of up to 1.1 exaflops of compute power. Google now operates a full cluster of eight of these pods in its Oklahoma data center with up to 9 exaflops of peak aggregate performance | 2022-05-12 | 11.88 | 32,768.00 | True | 2 | 18.95 | 9.0e+18 | 9.0e+18 | 9.0e+18 | 11.86 | 7.6e+11 | 3.0e+8 | Microsoft GPT-4 cluster | 0.58 | https://web.archive.org/web/20240516184309/https://techcrunch.com/2022/05/11/google-launches-a-9-exaflop-cluster-of-cloud-tpu-v4-pods-into-public-preview/ | 36.24 | -95.33 | |||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Unclear | 18.95 | 4.0e+3 | 30,000.00 | China | 2024-11-15 | Private | 20.00 | 6.0e+8 | China | 11.30 | 30,000.00 | Anonymized | Anonymized | 18.60 | 9.0e+18 | 9.0e+18 | 4.0e+18 | 2.0e+18 | 20.00 | 2.0e+11 | 6.0e+8 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||
| Mare Nostrum 5 | Existing | Confirmed | Yes | 18.95 | 4.5e+3 | NVIDIA H100 SXM5 80GB | 4,480.00 | Spain | Barcelona Supercomputing Center,EuroHPC JU | 2023-11-01 | Public | 6.00 | 1.6e+8 | Barcelona Supercomputing Center
Plaça d'Eusebi Güell, 1-3, Les Corts, 08034 Barcelona, Spain | European science and industry communities | H100 | 1120 nodes with 4 Hopper GPUs each | 2023-11-01 | Specs on their official website | 11.87 | 4,480.00 | NVIDIA | True | 10 | 18.65 | 8.9e+18 | 8.9e+18 | 4.4e+18 | 2.2e+18 | 6.51 | 6.00 | 7.4e+11 | 1.5e+8 | 1.6e+8 | 1.6e+8 | The budget is set for €151.41 million, which is $159.4 million with current US dollar exchange rates to the euro. | Tesla 10k H100 Cluster | 0.45 | https://web.archive.org/web/20240710184832/https://www.bsc.es/es/marenostrum/marenostrum-5 | https://web.archive.org/web/20240421214717/https://www.hpcwire.com/off-the-wire/bsc-inaugurates-marenostrum-5-marking-a-historic-investment-in-spanish-and-european-science/ | https://www.nextplatform.com/2022/06/16/atos-wins-marenostrum-5-deal-at-barcelona-supercomputing-center/ | 41.39 | 2.12 | |||||||||||||
| Together AI H100 Cluster | Existing | Likely | Yes | 18.94 | 4.4e+3 | NVIDIA H100 SXM5 80GB | 4,424.00 | United States of America | Together | 2023-10-16 | Unclear if all the GPUs are networked together | Private | 6.43 | 1.5e+8 | Cloud | H100 | Our first large H100 cluster (4424 GPUs) starts coming online today! | 2023-10-16 | 11.83 | 4,424.00 | NVIDIA | True | 9 | 18.64 | 8.8e+18 | 8.8e+18 | 4.4e+18 | 2.2e+18 | 6.43 | 6.8e+11 | 1.5e+8 | Tesla 10k H100 Cluster | 0.44 | https://archive.ph/58l2A#selection-489.0-489.68 | 39.38 | -97.92 | ||||||||||||||||||||
| EuroHPC Leonardo | Existing | Confirmed | Yes | 18.94 | 4.4e+3 | NVIDIA A100 | 13,824.00 | Italy | EuroHPC JU | 2022-11-22 | Public | 12.60 | 2.8e+8 | Via Piero Gobetti, 101, 40129 Bologna BO, Italy | European researchers,academia,and industry | A100 | This result is achieved with 3456 computing nodes, each equipped with four NVIDIA A100 SXM6 64GB GPUs driven by a single 32-cores Intel Ice Lake CPU. | 2022-11-22 | 11.53 | 13,824.00 | NVIDIA | True | 5 | 18.63 | 8.6e+18 | 8.6e+18 | 4.3e+18 | 2.2e+18 | 11.77 | 12.60 | 3.4e+11 | 3.1e+8 | 2.7e+8 | 2.8e+8 | Cost 240M Euros in 2022 | Microsoft GPT-4 cluster | 0.55 | https://web.archive.org/web/20240830104624/https://www.theregister.com/2022/11/22/leonardo_supercomputer_goes_live/ | https://web.archive.org/web/20230720164245/https://en.wikipedia.org/wiki/Leonardo_(supercomputer) | https://web.archive.org/web/20250115051101/https://leonardo-supercomputer.cineca.eu/hpc-system/ | 44.52 | 11.34 | ||||||||||||||
| Amazon Titan training cluster | Existing | Likely | Yes | 18.93 | 4.3e+3 | NVIDIA A100 | 13,760.00 | United States of America | Amazon | 2023-01-15 | Model was released to large clients April 2023, took 48 days to train, so this cluster was operational at least three months beforehand | Private | 11.42 | 3.1e+8 | Amazon | A100 | Specifically, they trained a 200B dense model on 4T tokens of data across 13,760 NVIDIA A100 chips (using 1,720 P4d nodes) | 2023-01 | 11.58 | 13,760.00 | NVIDIA | True | 6 | 18.63 | 8.6e+18 | 8.6e+18 | 4.3e+18 | 2.1e+18 | 11.42 | 3.8e+11 | 3.1e+8 | Microsoft GPT-4 cluster | 0.55 | https://web.archive.org/web/20240824020111/https://lifearchitect.ai/titan/ | 39.38 | -97.92 | ||||||||||||||||||||
| Google Hypercomputer TPU v5p pod | Existing | Confirmed | Yes | 18.92 | 4.2e+3 | Google TPU v5p | 8,960.00 | 2023-12-06 | The "Hypercomputer" is a cloud network, a TPU v5p pod seems to just be one component. Versions of this pod likely exist in multiple locations, possibly around the world | Private | 5.8e+7 | Google,Cloud | TPUv5p | Each TPU v5p pod composes together 8,960 chips over our highest-bandwidth inter-chip interconnect (ICI) at 4,800 Gbps/chip in a 3D torus topology. | 2023-12-06 | 8,960.00 | True | 18 | 18.61 | 8.2e+18 | 8.2e+18 | 4.1e+18 | 5.8e+7 | Microsoft Azure Eagle | 0.29 | https://web.archive.org/web/20240917131707/https://cloud.google.com/blog/products/ai-machine-learning/introducing-cloud-tpu-v5p-and-ai-hypercomputer | https://web.archive.org/web/20240528131246/https://www.hpcwire.com/2023/12/28/google-addresses-the-mysteries-of-its-hypercomputer/ | |||||||||||||||||||||||||||||
| Sesterce H100s Phase 2 | Existing | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,096.00 | France | Sesterce | 2024-07-06 | Private | 5.84 | 1.5e+8 | Paris, France | Cloud | H100 | From a private correspondence | 2024-07-06 | 11.84 | Sesterce H100s Phase 1 | 4,096.00 | NVIDIA | True | 29 | 18.61 | 8.1e+18 | 8.1e+18 | 4.1e+18 | 2.0e+18 | 5.84 | 6.9e+11 | 1.5e+8 | Meta GenAI 2024a | 0.17 | 48.85 | 2.35 | ||||||||||||||||||||
| Sesterce Synapse Phase 2 | Existing | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H200 SXM | 4,096.00 | France | Sesterce | 2025-01-15 | Private | 5.74 | 1.4e+8 | Paris, France | Cloud | H200 | Private Correspondence | 2025-01-15 | 11.85 | Sesterce Synapse Phase 1 | 4,096.00 | NVIDIA | True | 44 | 18.61 | 8.1e+18 | 8.1e+18 | 4.1e+18 | 2.0e+18 | 5.74 | 7.1e+11 | 1.4e+8 | xAI Colossus Memphis Phase 1 | 0.04 | https://web.archive.org/web/20240823055517/https://www.sesterce.com/reserved-cloud | https://archive.ph/ZDjLF | 48.85 | 2.35 | ||||||||||||||||||
| Yotta Shakti Cloud NM1 Phase 1 | Existing | Confirmed | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,096.00 | India | Yotta Data Services | 2024-05-15 | There is evidence on GPU listings of this portion being operational. The date comes from an article a month earlier saying when they expect to officially go live | Private | 5.84 | 1.5e+8 | Yotta Datacenter Park - Panvel Hiranandani Fortune City. Survey No. 30, MH SH 76, Panvel, Navi Mumbai, Maharashtra 410206, India | Cloud | H100 | "Yotta’s first slot of more than 4,000 GPUs was already booked by enterprises and is expected to go live by May 15, 2024"
"Yotta will deploy the first cluster of 16,384 GPUs at NM1" | 2024-05-15 | 11.84 | 4,096.00 | NVIDIA | True | 25 | 18.61 | 8.1e+18 | 8.1e+18 | 4.1e+18 | 2.0e+18 | 5.84 | 6.9e+11 | 1.5e+8 | Meta GenAI 2024a | 0.17 | https://web.archive.org/web/20240527112515/https://shakticloud.ai/ | https://web.archive.org/web/20240423043934/https://yotta.com/media/press-release-yotta-receives-indias-first-ever-consignment-of-nvidia-h100-the-fastest-gpus-in-the-world-at-its-nm1-data-center-park/ | https://web.archive.org/web/20240521205830/https://yotta.com/media/press-release-yotta-data-services-collaborates-with-nvidia-to-catalyze-indias-ai-transformation/ | https://web.archive.org/web/20240407130017/https://www.business-standard.com/companies/news/data-centre-operator-yotta-looks-to-expand-its-gpu-capacity-march-2025-124040700389_1.html | 19.03 | 73.02 | ||||||||||||||||
| PanaAI AUS AISF | Planned | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H200 SXM | 4,088.00 | Australia | PanaAI | Private | 5.73 | 1.5e+8 | Australia | Cloud | H200 | It will host up to 4088 NVIDIA H200 Tensor Core GPUs | Planned Q1 2025 | 11.85 | 4,088.00 | NVIDIA | 18.61 | 8.1e+18 | 8.1e+18 | 4.0e+18 | 2.0e+18 | 5.73 | 7.1e+11 | 1.5e+8 | https://web.archive.org/web/20250116143118/https://www.technologydecisions.com.au/content/it-management/news/panaai-to-build-australia-s-fastest-ai-supercomputer-731317630 | -25.73 | 134.49 | |||||||||||||||||||||||||
| Voltage Park Location 5 | Planned | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,088.00 | United States of America | Voltage Park | They offer configurations up to 4,088 GPUs, which suggests that their largest cluster is 4k, therefore all clusters would be roughly of size 4k | Private | 5.73 | 1.9e+8 | Cloud | H100 | With 24,000 H100 GPUs spread across six geographically distinct data centers | Planned | Unclear if the 24k chips are divided equally among the six locations. They offer clusters up to size 4,088 GPUs, though | 11.85 | 4,088.00 | NVIDIA | 18.61 | 8.1e+18 | 8.1e+18 | 4.0e+18 | 2.0e+18 | 5.73 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20240913183724/https://finance.yahoo.com/news/penguin-solutions-selected-managed-services-150000685.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cucGVycGxleGl0eS5haS8&guce_referrer_sig=AQAAACbflXKvc2DyvuR2WPik8ksxhKTAz7auQoF9nqCdOM4yDPTL4jd-W2b-V8yRSH21fxe4Ccv4cv3WjdfSFteyrcb-e_fbqoJhuhCzDkUZMwLHbCvHLBwKEOw_g_JSilSt4zqyJ6cKRYvHZ-fOSzVztIaQMrRoiWkdsftOLBO6ZB7P | https://web.archive.org/web/20240420201628/https://www.datacenterdynamics.com/en/news/ai-cloud-computing-non-profit-buys-24000-nvidia-h100-chips/ | 39.38 | -97.92 | |||||||||||||||||||||||
| Voltage Park Location 6 | Planned | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,088.00 | United States of America | Voltage Park | They offer configurations up to 4,088 GPUs, which suggests that their largest cluster is 4k, therefore all clusters would be roughly of size 4k | Private | 5.73 | 1.9e+8 | Cloud | H100 | With 24,000 H100 GPUs spread across six geographically distinct data centers | Planned | Unclear if the 24k chips are divided equally among the six locations. They offer clusters up to size 4,088 GPUs, though | 11.85 | 4,088.00 | NVIDIA | 18.61 | 8.1e+18 | 8.1e+18 | 4.0e+18 | 2.0e+18 | 5.73 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20240913183724/https://finance.yahoo.com/news/penguin-solutions-selected-managed-services-150000685.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cucGVycGxleGl0eS5haS8&guce_referrer_sig=AQAAACbflXKvc2DyvuR2WPik8ksxhKTAz7auQoF9nqCdOM4yDPTL4jd-W2b-V8yRSH21fxe4Ccv4cv3WjdfSFteyrcb-e_fbqoJhuhCzDkUZMwLHbCvHLBwKEOw_g_JSilSt4zqyJ6cKRYvHZ-fOSzVztIaQMrRoiWkdsftOLBO6ZB7P | https://web.archive.org/web/20240420201628/https://www.datacenterdynamics.com/en/news/ai-cloud-computing-non-profit-buys-24000-nvidia-h100-chips/ | 39.38 | -97.92 | |||||||||||||||||||||||
| Voltage Park Texas Phase 2 | Planned | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,088.00 | United States of America | Voltage Park | They offer configurations up to 4,088 GPUs, which suggests that their largest cluster is 4k, therefore all clusters would be roughly of size 4k | Private | 5.73 | 1.9e+8 | Texas | Cloud | H100 | With 24,000 H100 GPUs spread across six geographically distinct data centers | Planned | Unclear if the 24k chips are divided equally among the six locations. They offer clusters up to size 4,088 GPUs, though | 11.85 | Voltage Park Texas Phase 1 | 4,088.00 | NVIDIA | 18.61 | 8.1e+18 | 8.1e+18 | 4.0e+18 | 2.0e+18 | 5.73 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20240913183724/https://finance.yahoo.com/news/penguin-solutions-selected-managed-services-150000685.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cucGVycGxleGl0eS5haS8&guce_referrer_sig=AQAAACbflXKvc2DyvuR2WPik8ksxhKTAz7auQoF9nqCdOM4yDPTL4jd-W2b-V8yRSH21fxe4Ccv4cv3WjdfSFteyrcb-e_fbqoJhuhCzDkUZMwLHbCvHLBwKEOw_g_JSilSt4zqyJ6cKRYvHZ-fOSzVztIaQMrRoiWkdsftOLBO6ZB7P | https://web.archive.org/web/20240420201628/https://www.datacenterdynamics.com/en/news/ai-cloud-computing-non-profit-buys-24000-nvidia-h100-chips/ | 31.80 | -98.82 | |||||||||||||||||||||
| Voltage Park Utah | Planned | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,088.00 | United States of America | Voltage Park | They offer configurations up to 4,088 GPUs, which suggests that their largest cluster is 4k, therefore all clusters would be roughly of size 4k | Private | 5.73 | 1.9e+8 | Utah | Cloud | H100 | With 24,000 H100 GPUs spread across six geographically distinct data centers | Planned | Unclear if the 24k chips are divided equally among the six locations. They offer clusters up to size 4,088 GPUs, though | 11.85 | 4,088.00 | NVIDIA | 18.61 | 8.1e+18 | 8.1e+18 | 4.0e+18 | 2.0e+18 | 5.73 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20240913183724/https://finance.yahoo.com/news/penguin-solutions-selected-managed-services-150000685.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cucGVycGxleGl0eS5haS8&guce_referrer_sig=AQAAACbflXKvc2DyvuR2WPik8ksxhKTAz7auQoF9nqCdOM4yDPTL4jd-W2b-V8yRSH21fxe4Ccv4cv3WjdfSFteyrcb-e_fbqoJhuhCzDkUZMwLHbCvHLBwKEOw_g_JSilSt4zqyJ6cKRYvHZ-fOSzVztIaQMrRoiWkdsftOLBO6ZB7P | https://web.archive.org/web/20240420201628/https://www.datacenterdynamics.com/en/news/ai-cloud-computing-non-profit-buys-24000-nvidia-h100-chips/ | 39.25 | -111.60 | ||||||||||||||||||||||
| Voltage Park Virginia | Planned | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,088.00 | United States of America | Voltage Park | They offer configurations up to 4,088 GPUs, which suggests that their largest cluster is 4k, therefore all clusters would be roughly of size 4k | Private | 5.73 | 1.9e+8 | Virginia | Cloud | H100 | With 24,000 H100 GPUs spread across six geographically distinct data centers | Planned | Unclear if the 24k chips are divided equally among the six locations. They offer clusters up to size 4,088 GPUs, though | 11.85 | 4,088.00 | NVIDIA | 18.61 | 8.1e+18 | 8.1e+18 | 4.0e+18 | 2.0e+18 | 5.73 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20240913183724/https://finance.yahoo.com/news/penguin-solutions-selected-managed-services-150000685.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cucGVycGxleGl0eS5haS8&guce_referrer_sig=AQAAACbflXKvc2DyvuR2WPik8ksxhKTAz7auQoF9nqCdOM4yDPTL4jd-W2b-V8yRSH21fxe4Ccv4cv3WjdfSFteyrcb-e_fbqoJhuhCzDkUZMwLHbCvHLBwKEOw_g_JSilSt4zqyJ6cKRYvHZ-fOSzVztIaQMrRoiWkdsftOLBO6ZB7P | https://web.archive.org/web/20240420201628/https://www.datacenterdynamics.com/en/news/ai-cloud-computing-non-profit-buys-24000-nvidia-h100-chips/ | 37.68 | -78.62 | ||||||||||||||||||||||
| Voltage Park Washington | Planned | Likely | Yes | 18.91 | 4.1e+3 | NVIDIA H100 SXM5 80GB | 4,088.00 | United States of America | Voltage Park | They offer configurations up to 4,088 GPUs, which suggests that their largest cluster is 4k, therefore all clusters would be roughly of size 4k | Private | 5.73 | 1.9e+8 | 1023 39th Ave SE, Puyallup, WA 98374 | Cloud | H100 | With 24,000 H100 GPUs spread across six geographically distinct data centers | Planned | Unclear if the 24k chips are divided equally among the six locations. They offer clusters up to size 4,088 GPUs, though | 11.85 | 4,088.00 | NVIDIA | 18.61 | 8.1e+18 | 8.1e+18 | 4.0e+18 | 2.0e+18 | 5.73 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20240913183724/https://finance.yahoo.com/news/penguin-solutions-selected-managed-services-150000685.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cucGVycGxleGl0eS5haS8&guce_referrer_sig=AQAAACbflXKvc2DyvuR2WPik8ksxhKTAz7auQoF9nqCdOM4yDPTL4jd-W2b-V8yRSH21fxe4Ccv4cv3WjdfSFteyrcb-e_fbqoJhuhCzDkUZMwLHbCvHLBwKEOw_g_JSilSt4zqyJ6cKRYvHZ-fOSzVztIaQMrRoiWkdsftOLBO6ZB7P | https://web.archive.org/web/20240420201628/https://www.datacenterdynamics.com/en/news/ai-cloud-computing-non-profit-buys-24000-nvidia-h100-chips/ | https://web.archive.org/web/20240226140823/https://choosetacomapierce.org/news-events/voltage-park-selects-centeris-data-center/ | 47.16 | -122.28 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.90 | 4.0e+3 | China | 2024-05-15 | Public/Private | China | Anonymized | Anonymized | True | 26 | 18.90 | 8.0e+18 | 8.0e+18 | Meta GenAI 2024a | 0.20 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| CoreWeave LiquidLab | Existing | Likely | Yes | 18.90 | 4.0e+3 | NVIDIA H100 SXM5 80GB | 4,000.00 | United States of America | CoreWeave | Unclear exactly what type of GPU they use. Likely operational by now | Private | 5.61 | 1.9e+8 | CoreWeave | H100 | CoreWeave has a Liquid Lab where we conduct extensive liquid cooling testing, and we’re getting ready to deploy our first “small” liquid deployment of 4,000 GPUs | Planned | 11.85 | 4,000.00 | NVIDIA | True | 18.60 | 7.9e+18 | 7.9e+18 | 4.0e+18 | 2.0e+18 | 5.61 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20241006003335/https://www.coreweave.com/blog/building-ai-clusters-for-enterprises-2025 | 39.38 | -97.92 | ||||||||||||||||||||||||
| Microsoft Azure ND H100 v5 VM | Existing | Confirmed | Yes | 18.90 | 4.0e+3 | NVIDIA H100 SXM5 80GB | 4,000.00 | United States of America | Microsoft | 2023-03-13 | Planned to scale up | Private | 5.81 | 1.9e+8 | Cloud,Azure | H100 | Today marks the general availability of our Azure ND H100 v5 Virtual Machine (VM) series, featuring the latest NVIDIA H100 Tensor Core GPUs and NVIDIA Quantum-2 InfiniBand networking... we are leveraging an AI optimized 4K GPU cluster and will be ramping to hundreds of thousands of the latest GPUs in the next year. | 2023-03-13 | 11.83 | 4,000.00 | NVIDIA | True | 7 | 18.60 | 7.9e+18 | 7.9e+18 | 4.0e+18 | 2.0e+18 | 5.81 | 6.8e+11 | 1.9e+8 | Microsoft GPT-4 cluster | 0.51 | https://web.archive.org/web/20240817024910/https://azure.microsoft.com/en-us/blog/scale-generative-ai-with-new-azure-ai-infrastructure-advancements-and-availability/ | https://web.archive.org/web/20240819113927/https://azure.microsoft.com/en-us/blog/azure-previews-powerful-and-scalable-virtual-machine-series-to-accelerate-generative-ai/ | 39.38 | -97.92 | |||||||||||||||||||
| Mistral Large Model Implied Cluster | Existing | Unlikely | Yes | 18.90 | 4.0e+3 | NVIDIA H100 SXM5 80GB | 4,000.00 | 2023-11-26 | Taking H100 price as $2/hr, and assuming they trained for 3 months max, this implies that the cluster they used was >4k GPUs | Private | 5.81 | 1.3e+8 | Cloud,Mistral | "Mensch said his new model costs less than 20 million [Euros] to train" | 2023-11-26 | GPU count implied from stated training cost | 11.83 | 4,000.00 | NVIDIA | 18.60 | 7.9e+18 | 7.9e+18 | 4.0e+18 | 2.0e+18 | 5.81 | 6.8e+11 | 1.3e+8 | https://archive.ph/aAkAp | ||||||||||||||||||||||||||||
| Nebius Kansas City Phase 1 | Existing | Likely | Yes | 18.90 | 4.0e+3 | NVIDIA H200 SXM | 4,000.00 | United States of America | Nebius AI | 2025-04-15 | Current power is 5MW our of potential expansion to 40MW (which would host 35k GPUs), so extrapolating, that would put it at ~4k GPUs
They claimed that it was on track to go live in Q1 2025 in March 2025, implying that it likely came online soon after March | Private | 5.00 | 1.4e+8 | Kansas City, USA | Cloud | H200 | will house thousands of state-of-the-art NVIDIA GPUs, primarily NVIDIA Hopper GPUs in the initial phase, with the energy-efficient NVIDIA Blackwell platform expected to arrive in 2025. The colocation can be expanded from an initial 5 MW up to 40 MW, or about 35 thousand GPUs, at full potential capacity | 2025-04 | 11.90 | 4,000.00 | NVIDIA | True | 52 | 18.60 | 7.9e+18 | 7.9e+18 | 4.0e+18 | 2.0e+18 | 5.61 | 5.00 | 7.9e+11 | 1.4e+8 | xAI Colossus Memphis Phase 2 | 0.02 | https://web.archive.org/web/20250109005023/https://group.nebius.com/newsroom/nebius-expands-in-us-with-first-gpu-cluster-in-kansas-city-offices-in-san-francisco-dallas-and-new-york | 39.10 | -94.58 | ||||||||||||||||||
| OneAsia OBON Clusters | Planned | Likely | Yes | 18.90 | 4.0e+3 | NVIDIA H100 SXM5 80GB | 4,000.00 | Thailand | OneAsia | Private | 5.61 | 1.9e+8 | Bangkok, Thailand | Cloud,OBON,SIAM AI | H100 | OBON will deploy multiple 4K supercomputing clusters at OneAsia's state-of-the-art data centre | Planned | 11.85 | 4,000.00 | NVIDIA | 18.60 | 7.9e+18 | 7.9e+18 | 4.0e+18 | 2.0e+18 | 5.61 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20241211224237/https://www.prnewswire.com/apac/news-releases/oneasia-and-obon-unveil-thailands-first-4k-supercomputing-clusters-revolutionizing-ai-roadmap-in-thailand-302135527.html?tc=eml_cleartime | 13.75 | 100.49 | |||||||||||||||||||||||||
| Sweden 4k H100 Cluster | Existing | Likely | Yes | 18.90 | 4.0e+3 | NVIDIA H100 SXM5 80GB | 4,000.00 | Sweden | 2025-01-14 | Operational by 1/14/25 at the latest, but likely meaningfully earlier | Private | 5.61 | 1.5e+8 | Sweden | Cloud | H100 | From a private correspondence | 2025-01-14 | From a private correspondence about GPU cluster availability | 11.85 | 4,000.00 | NVIDIA | True | 46 | 18.60 | 7.9e+18 | 7.9e+18 | 4.0e+18 | 2.0e+18 | 5.61 | 7.1e+11 | 1.5e+8 | xAI Colossus Memphis Phase 1 | 0.04 | 64.96 | 17.68 | ||||||||||||||||||||
Telangana Yotta H1 Hyderbad AI City Cluster Phase 1 | Planned | Likely | Yes | 18.90 | 4.0e+3 | NVIDIA H100 SXM5 80GB | 4,000.00 | India | Government of Telangana | Public | 5.61 | 1.9e+8 | Hyderabad, India | Cloud | H100 | The first phase of the Supercomputer consisting of 4,000 high performance GPUs and the first Data Center within the campus, Yotta H1 will be operational within the next 24 months | Planned 2026 | 11.85 | 4,000.00 | NVIDIA | 18.60 | 7.9e+18 | 7.9e+18 | 4.0e+18 | 2.0e+18 | 5.61 | 7.1e+11 | 1.9e+8 | https://web.archive.org/web/20240906230223/https://www.crn.in/news/government-of-telangana-partners-with-yotta-to-launch-indias-largest-ai-supercomputer-of-25000-high-performance-gpus-in-a-purpose-built-50-mw-ai-cloud-data-centre-campus-in-hyderabad/ | 17.36 | 78.47 | |||||||||||||||||||||||||
| Gemini 1.0 Ultra training cluster A | Existing | Likely | Yes | 18.90 | 4.0e+3 | Google TPU v4 | 28,672.00 | United States of America | 2023-08-20 | Believed to be trained on two sets of 7 pods of 4096 TPUs | Private | 10.11 | 2.6e+8 | TPUv4 | Gemini Ultra, the most compute-intensive model known to date, was trained across multiple data centers totaling 55,000 TPU v4s | 2023-08-20 | 11.89 | 28,672.00 | True | 10 | 18.90 | 7.9e+18 | 7.9e+18 | 7.9e+18 | 10.11 | 7.8e+11 | 2.6e+8 | Microsoft GPT-4 cluster | 0.51 | https://arxiv.org/abs/2312.11805 | https://web.archive.org/web/20250320051300/https://ifp.org/web/20250320051300/https://ifp.org/future-of-ai-compute/ | 39.38 | -97.92 | |||||||||||||||||||||||
| Gemini 1.0 Ultra training cluster B | Existing | Likely | Yes | 18.90 | 4.0e+3 | Google TPU v4 | 28,672.00 | United States of America | 2023-08-20 | Believed to be trained on two sets of 7 pods of 4096 TPUs | Private | 10.11 | 2.6e+8 | TPUv4 | Gemini Ultra, the most compute-intensive model known to date, was trained across multiple data centers totaling 55,000 TPU v4s | 2023-08-20 | 11.89 | True | Google Oklahoma TPU v4 Pods | 28,672.00 | 10 | 18.90 | 7.9e+18 | 7.9e+18 | 7.9e+18 | 10.11 | 7.8e+11 | 2.6e+8 | Microsoft GPT-4 cluster | 0.51 | https://arxiv.org/abs/2312.11805 | https://web.archive.org/web/20250320051300/https://ifp.org/web/20250320051300/https://ifp.org/future-of-ai-compute/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Vultr Chicago Cluster | Existing | Likely | Yes | 18.89 | 4.0e+3 | AMD Instinct MI300X | 3,000.00 | United States of America | Vultr | 2024-12-11 | Not totally certain that the "thousands" of MI300X's are all in this data center, but I'm pretty sure. I'm also estimating "thousands" as 3k | Private | 4.59 | 8.2e+7 | 4513 Western Ave, Lisle, IL 60532 | Cloud | MI300X | With thousands of MI300X GPUs available, clusters of any size can be deployed | 2024-12-11 | Uncertain about exact chip count | 11.93 | 3,000.00 | AMD | True | 49 | 18.59 | 7.8e+18 | 7.8e+18 | 3.9e+18 | 2.0e+18 | 4.59 | 8.6e+11 | 8.2e+7 | xAI Colossus Memphis Phase 1 | 0.04 | https://web.archive.org/web/20241222233226/https://blogs.vultr.com/Lisle-data-center | 41.80 | -88.09 | ||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 18.90 | 4.0e+3 | China | Public/Private | China | Anonymized | Anonymized | 18.90 | 8.0e+18 | 8.0e+18 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 18.90 | 4.0e+3 | 10,000.00 | China | 2023-09-15 | Private | 10.00 | 3.0e+8 | China | 11.60 | 10,000.00 | Anonymized | Anonymized | True | 14 | 18.60 | 8.0e+18 | 8.0e+18 | 4.0e+18 | 2.0e+18 | 10.00 | 4.0e+11 | 3.0e+8 | Tesla 10k H100 Cluster | 0.40 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| NFDG Andromeda Phase 2 | Existing | Confirmed | Yes | 18.88 | 3.9e+3 | NVIDIA H100 SXM5 80GB | 3,632.00 | United States of America | Nat Friedman and Daniel Gross | 2024-07-15 | Private | 5.81 | 1.5e+8 | Ashburn, Virginia | Startups they invest in | H100,A100 | 3,200 H100s on 400 nodes interlinked with 3.2Tbps infiniband
432 H100s on 54 nodes interlinked with 3.2Tbps infiniband
768 A100s for training and inference with 1.6Tbps infiniband | 2024-07-15 | Listed as available on their official website | 11.82 | NFDG Andromeda Phase 1 | NVIDIA A100 SXM4 40 GB | 768 | 4,400.00 | NVIDIA | NVIDIA | True | 36 | 18.58 | 7.7e+18 | 7.7e+18 | 3.8e+18 | 1.9e+18 | 5.81 | 6.6e+11 | 1.5e+8 | Meta GenAI 2024a | 0.16 | https://web.archive.org/web/20240814194924/https://andromeda.ai/ | https://web.archive.org/web/20240706081837/https://www.businessinsider.com/nvidia-gpu-venture-capitalists-buying-for-startups-2023-6 | 39.04 | -77.49 | ||||||||||||||
| NVIDIA Coreweave MLPerf v3.0 Submission 2023 | Existing | Confirmed | Yes | 18.85 | 3.6e+3 | NVIDIA H100 SXM5 80GB | 3,584.00 | United States of America | NVIDIA,CoreWeave,Inflection AI | 2023-06-01 | Inflection lent 3,500 H100 GPUs to Corweave (who appears to be the party that built that cluster) for the June MLPerf submission. Given that this is just slightly larger and operational just after, it appears very likely that this cluster is an extension of the MLPerf cluster (or the MLPerf cluster was just a subset of this cluster).
It's possible that both of these subsets of the Eos-DFW cluster, but there isn't evidence for this besides that Eos-DFW was submitted to MLPerf later that year | Private | 5.21 | 1.3e+8 | NVIDIA | H100 | The 10,752 H100 GPUs far surpassed the scaling in AI training in June, when NVIDIA used 3,584 Hopper GPUs. | 2023-06-01 | 11.83 | Inflection-2 training cluster | True | NVIDIA CoreWeave Eos-DFW Phase 1 | 3,584.00 | NVIDIA | 9 | 18.55 | 7.1e+18 | 7.1e+18 | 3.5e+18 | 1.8e+18 | 5.21 | 6.8e+11 | 1.3e+8 | Microsoft GPT-4 cluster | 0.45 | https://web.archive.org/web/20240915170526/https://developer.nvidia.com/blog/nvidia-sets-new-generative-ai-performance-and-scale-records-in-mlperf-training-v4-0/ | https://archive.ph/fB8Io | 39.38 | -97.92 | |||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.85 | 4.0e+3 | China | 2024-08-15 | Public/Private | China | 20,000.00 | Anonymized | Anonymized | True | 41 | 18.85 | 7.0e+18 | 7.0e+18 | CoreWeave H200s | 0.08 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | No | 18.85 | 3.0e+3 | China | 2024-06-15 | Public/Private | China | 20,000.00 | Anonymized | Anonymized | 18.85 | 7.0e+18 | 7.0e+18 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.78 | 3.0e+3 | 10,000.00 | China | 2021-07-15 | Private | 6.00 | 2.0e+8 | China | 11.70 | 10,000.00 | Anonymized | Anonymized | True | 1 | 18.48 | 6.0e+18 | 6.0e+18 | 3.0e+18 | 2.0e+18 | 6.00 | 5.0e+11 | 3.0e+8 | 1.0e+8 | 2.0e+8 | DeepSeek Fire-Flyer 2 | 1.00 | 35.49 | 101.90 | ||||||||||||||||||||||||
| XTX Markets Cluster | Existing | Likely | Yes | 18.80 | 3.2e+3 | NVIDIA A100 | 10,000.00 | XTX Markets | 2023-03-27 | Unclear how many of the GPUs are A100s vs V100s
The State of AI report indicates that they have 10k A100s, but it's unclear if they're just estimating this or have an actual source
Also, the earliest point in the waybackmaching that the State of AI report mentions this cluster is March 2023, which is where the date comes from | Private | 14.52 | 4.1e+8 | XTX Markets | A100,V100,20,000 GPUs Maybe 10k A100, 10k V100? | Our dedicated research cluster contains a hundred thousand cores and twenty thousand A/V100 GPUs | 2023-03-27 | Unclear how many of the GPUs are A100s vs V100s
The State of AI report indicates that they have 10k A100s, but it's unclear if they're just estimating this or have an actual source | 11.48 | NVIDIA V100 | 10,000 | 20,000.00 | NVIDIA | NVIDIA | True | 8 | 18.64 | 6.2e+18 | 6.2e+18 | 4.4e+18 | 1.7e+18 | 14.52 | 3.0e+11 | 4.1e+8 | Microsoft GPT-4 cluster | 0.40 | https://web.archive.org/web/20240222140115/https://www.xtxmarkets.com/career/xty-labs-ai-residency/ | https://web.archive.org/web/20230331093712/https://www.stateof.ai/compute | ||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.78 | 3.0e+3 | China | Public | 30.00 | China | 11.30 | 10,000.00 | Anonymized | Anonymized | True | 18.78 | 6.0e+18 | 6.0e+18 | 30.00 | 2.0e+11 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||
| Novo Nordisk Gefion | Existing | Confirmed | Yes | 18.78 | 3.0e+3 | NVIDIA H100 SXM5 80GB | 1,528.00 | Denmark | Novo Nordisk Foundation,Danish Centre for AI Innovation,Export and Investment Fund of Denmark | 2024-10-23 | The Export and Investment Fund of Denmark (which is government-affiliated) owns 15% of the company created by Novo Nordisk that owns this supercomputer | Private | 2.93 | 9.9e+7 | Denmark | Public and private research; cloud | H100 | The supercomputer will feature 191 DGX H100 systems with a total of 1,528 Nvidia H100 Tensor Core GPUs and 382 Intel Xeon Platinum CPUs connected by Quantum 2 InfiniBand | 2024-10-23 | 11.71 | 1,528.00 | NVIDIA | True | 49 | 18.18 | 6.0e+18 | 6.0e+18 | 1.5e+18 | 7.6e+17 | 2.18 | 2.93 | 5.2e+11 | 5.8e+7 | 9.8e+7 | 9.9e+7 | In terms of price, Novo Nordisk Foundation is putting up 600 million Danish krone (about 14 cents to the US dollar) and the Export and Investment Fund of Denmark (EIFO) us putting up another 100 million Danish krone. | xAI Colossus Memphis Phase 1 | 0.03 | https://web.archive.org/web/20240531144416/https://www.datacenterdynamics.com/en/news/novo-nordisk-foundation-to-build-danish-nvidia-dgx-superpod-supercomputer/ | https://web.archive.org/web/20250201135159/https://novonordiskfonden.dk/en/news/denmarks-first-ai-supercomputer-is-now-operational/ | https://www.nextplatform.com/2024/10/23/the-great-danes-get-a-supercomputer-for-ai-and-maybe-hpc/ | 55.96 | 10.05 | |||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 18.78 | 3.0e+3 | 100,000.00 | China | 2021-03-15 | Public | China | 100,000.00 | Anonymized | Anonymized | True | 1 | 18.78 | 6.0e+18 | 6.0e+18 | 3.0e+18 | Sunway OceanLight | 1.00 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||
| FPT AI Factory Japan | Planned | Likely | Yes | 18.77 | 3.0e+3 | NVIDIA H100 SXM5 80GB | 3,000.00 | Japan | FPT Corporation | Same investment size as the Vietnam cluster, which said it would have "thousands" of GPUs. Estimating "thousands" as 3k here. They were supposed to receive their first shipment of 1k H100s in December 2024. The facility is said to be $200 million, so assuming an H100 price of ~30k, this means there would be a max of 6k H100s here | Private | 4.20 | 1.4e+8 | Japan | Cloud | H100,H200 | The center, like a similar $200 million Vietnam facility announced this year, will run on Nvidia Corp | Planned Q1 2025 | 11.85 | NVIDIA H200 SXM | 3,000.00 | NVIDIA | NVIDIA | 18.47 | 5.9e+18 | 5.9e+18 | 3.0e+18 | 1.5e+18 | 4.20 | 7.1e+11 | 1.4e+8 | https://archive.ph/pqult | 36.39 | 138.59 | ||||||||||||||||||||||
| FPT AI Factory Vietnam | Planned | Likely | Yes | 18.77 | 3.0e+3 | NVIDIA H100 SXM5 80GB | 3,000.00 | Vietnam | FPT Corporation | Estimating "thousands" as 3k here. They were supposed to receive their first shipment of 1k H100s in December 2024. The facility is said to be $200 million, so assuming an H100 price of ~30k, this means there would be a max of 6k H100s here | Private | 4.20 | 1.4e+8 | Vietnam | Cloud | H100,H200 | FPT's AI Factory, the first of its kind in Vietnam, is equipped with thousands of NVIDIA GPU H100 graphics chips and will be ready to provide services starting January 2025 | Planned Q1 2025 | 11.85 | NVIDIA H200 SXM | 3,000.00 | NVIDIA | NVIDIA | 18.47 | 5.9e+18 | 5.9e+18 | 3.0e+18 | 1.5e+18 | 4.20 | 7.1e+11 | 1.4e+8 | https://web.archive.org/web/20250116004357/https://fpt.com/en/news/fpt-news/can-canh-sieu-chip-nvidia-tai-nha-may-ai-cua-fpt-tai-viet-nam | 14.32 | 108.34 | ||||||||||||||||||||||
| Paper on Gemma 2 27B | Existing | Confirmed | Yes | 18.75 | 2.9e+3 | Google TPU v5p | 6,144.00 | 2024-06-27 | 'Table 3 | Training infrastructure with sharding' in 27B row lists 6144 in #Chips column | Private | 3.9e+7 | TPUv5p | we train on an 8x24x32 configuration of
TPUv5p, totaling 6144 chips | 2024-06-27 | True | Google Hypercomputer TPU v5p pod | 6,144.00 | 38 | 18.45 | 5.6e+18 | 5.6e+18 | 2.8e+18 | 3.9e+7 | Meta GenAI 2024a | 0.12 | https://storage.googleapis.com/deepmind-media/gemma/gemma-2-report.pdf | ||||||||||||||||||||||||||||||
| KAUST Shaheen-III | Planned | Likely | Yes | 18.75 | 2.8e+3 | NVIDIA GH200 | 2,816.00 | Saudi Arabia | King Abdullah University of Science and Technology (KAUST) | The CPU portion became operational in November 2023. It's unclear if the GPU portion is already operational, is still under construction, or got cancelled | Public | 5.30 | 1.3e+8 | King Abdullah University Of Science And Technology 23955, Saudi Arabia | GH200 | Across these seven accelerated cabinets: 704 nodes, each with quadruple Nvidia “Grace Hopper” Superchips, each with a tightly coupled CPU and GPU. These 2,816 Superchips
The compute-only partition, spread across 18 liquid-cooled HPE Cray EX4000 cabinets and networked with HPE Slingshot, will comprise 4,608 nodes, each with two of AMD’s fourth-gen Epyc “Genoa” CPUs | Planned | 11.72 | 2,816.00 | NVIDIA | 18.45 | 5.6e+18 | 5.6e+18 | 2.8e+18 | 1.4e+18 | 3.95 | 5.30 | 5.3e+11 | 1.3e+8 | https://web.archive.org/web/20240707221559/https://www.kaust.edu.sa/en/news/kaust-s-shaheen-iii-confirmed-as-the-middle-east-s-most-powerful-supercomputer | https://web.archive.org/web/20221128092556/https://www.hpcwire.com/2022/09/27/hpe-to-build-100-petaflops-shaheen-iii-supercomputer/ | 24.76 | 46.74 | |||||||||||||||||||||||
| Eni HPC6 | Existing | Confirmed | Yes | 18.73 | 2.7e+3 | AMD Radeon Instinct MI250X | 13,888.00 | Italy | Eni | 2024-11-18 | Private | 14.15 | 3.5e+8 | Via per la Corradina, 35, 27032 Ferrera Erbognone PV, Italy | Eni | AMD MI250X | the system includes 3472 computing nodes housing a total of 13,888 GPU | 2024-11-18 | 11.57 | 13,888.00 | AMD | True | 57 | 18.73 | 5.3e+18 | 5.3e+18 | 5.3e+18 | 1.3e+18 | 14.15 | 3.8e+11 | 3.5e+8 | xAI Colossus Memphis Phase 1 | 0.03 | https://archive.ph/RIheG | 45.10 | 8.86 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 18.70 | 3.0e+3 | 10,000.00 | China | 2025-01-15 | Private | China | 10,000.00 | Anonymized | Anonymized | True | 63 | 5.0e+18 | 5.0e+18 | xAI Colossus Memphis Phase 1 | 0.03 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||
| Los Alamos NL Venado | Existing | Confirmed | Yes | 18.70 | 2.6e+3 | NVIDIA GH200 | 2,560.00 | United States of America | US Department of Energy | 2024-04-16 | Public | 2.78 | 1.2e+8 | Los Alamos, New Mexico, USA | Los Alamos National Laboratory | GH200 | The system consists of 2,560 direct, liquid-cooled Nvidia Grace Hopper Superchips; 920 Nvidia Grace CPU Superchips, each of which contains 144 Arm cores | 2024-04-16 | 11.96 | 2,560.00 | NVIDIA | True | 31 | 18.40 | 5.1e+18 | 5.1e+18 | 2.5e+18 | 1.3e+18 | 3.65 | 2.78 | 9.1e+11 | 1.2e+8 | Meta GenAI 2024a | 0.10 | https://web.archive.org/web/20240921074409/https://www.datacenterdynamics.com/en/news/los-alamos-national-laboratory-unveils-new-venado-supercomputer-reportedly-capable-of-10-exaflops-of-peak-ai-performance/ | 35.88 | -106.30 | |||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 18.70 | 3.0e+3 | 3,000.00 | China | Public/Private | 8.00 | 1.0e+8 | China | 11.80 | 7,000.00 | Anonymized | Anonymized | 18.70 | 5.0e+18 | 5.0e+18 | 1.0e+18 | 2.00 | 8.00 | 6.3e+11 | 1.0e+8 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.70 | 3.0e+3 | China | 2024-01-15 | Private | China | Anonymized | Anonymized | True | 27 | 18.70 | 5.0e+18 | 5.0e+18 | Microsoft Azure Eagle | 0.20 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Meta Research SuperCluster 2 (RSC-2) | Existing | Confirmed | Yes | 18.70 | 2.5e+3 | NVIDIA A100 | 8,000.00 | United States of America | Meta AI | 2023-10-31 | Private | 6.64 | 1.6e+8 | Meta,Primarily vision applications at Meta | A100 | RSC-1 is a general ML cluster (e.g., training some of the prominent LLMs) of 16k GPU size, while RSC-2 focuses on vision applications and is of 8k GPU size. | 2023-10-31 | Reported in publication | 11.58 | 8,000.00 | NVIDIA | True | 20 | 18.40 | 5.0e+18 | 5.0e+18 | 2.5e+18 | 1.2e+18 | 6.64 | 3.8e+11 | 1.6e+8 | Tesla 10k H100 Cluster | 0.25 | https://web.archive.org/web/20241108001043/https://arxiv.org/abs/2410.21680 | https://arxiv.org/pdf/2410.21680v1 | 39.38 | -97.92 | |||||||||||||||||||
| NFDG Andromeda Phase 1 | Existing | Confirmed | Yes | 18.70 | 2.5e+3 | NVIDIA H100 SXM5 80GB | 2,512.00 | United States of America | Nat Friedman and Daniel Gross | 2023-06-13 | They have since expanded this | Private | 3.65 | 9.1e+7 | Ashburn, Virginia | Startups they invest in | H100 | 2,512 H100s on 314 nodes interlinked with 3.2Tbps infiniband | 2023-06-13 | Listed as available on their official website | 11.83 | NFDG Andromeda Phase 2 | 2,512.00 | NVIDIA | True | 13 | 18.40 | 5.0e+18 | 5.0e+18 | 2.5e+18 | 1.2e+18 | 3.65 | 6.8e+11 | 9.1e+7 | Microsoft GPT-4 cluster | 0.32 | https://web.archive.org/web/20230907083238/https://andromeda.ai/ | https://web.archive.org/web/20240706081837/https://www.businessinsider.com/nvidia-gpu-venture-capitalists-buying-for-startups-2023-6 | 39.04 | -77.49 | ||||||||||||||||
| Tesla A100 Cluster Phase 2 | Existing | Confirmed | Yes | 18.66 | 2.3e+3 | NVIDIA A100 SXM4 80 GB | 7,360.00 | United States of America | Tesla | 2022-08-16 | Private | 6.27 | 1.7e+8 | Tesla | A100 | now has a total of 7,360 A100 GPUs | 2022-08-16 | 11.56 | Tesla A100 Cluster Phase 1 | 7,360.00 | NVIDIA | True | 6 | 18.36 | 4.6e+18 | 4.6e+18 | 2.3e+18 | 1.1e+18 | 6.27 | 3.7e+11 | 1.7e+8 | Microsoft GPT-4 cluster | 0.29 | https://web.archive.org/web/20230521065409/https://www.hpcwire.com/2022/08/16/tesla-gooses-its-gpu-powered-ai-super-is-dojo-next/ | 39.38 | -97.92 | ||||||||||||||||||||
| EuroHPC LUMI | Existing | Confirmed | Yes | 18.66 | 2.3e+3 | AMD Radeon Instinct MI250X | 11,912.00 | Finland | EuroHPC JU | 2022-09-15 | Came online in phases, so unclear exactly when it became operational. A portion of it first was innagurated as operational June 2022, and close to 10k GPUs were confirmed to operational by Nov 2022. Reports suggest that "phase two" began in August. Marking the operational date as September as an estimate | Public | 12.09 | 1.7e+8 | Tehdaskatu 15, 87100 Kajaani, Finland | Researchers,Cloud | AMD MI250X | The GPU partition consists of 2978 nodes, each node with one 64-core AMD Trento CPU and four AMD MI250X GPUs. A total of 11912 AMD GPUs. | 2022-09 | Confirmed by official sources | 11.58 | 11,912.00 | AMD | True | 8 | 18.66 | 4.6e+18 | 4.6e+18 | 4.6e+18 | 1.1e+18 | 12.68 | 12.09 | 3.8e+11 | 3.1e+8 | 1.7e+8 | 1.7e+8 | "The total budget of the product, which will be co-funded by the EuroHPC JU and the LUMI Consortium, will total €144.5 million (over $165 million). The total cost of ownership of the system from 2020 to 2026 will be €200 million ($237.1 million), which includes the cost of hardware." | Microsoft GPT-4 cluster | 0.29 | https://web.archive.org/web/20241003125301/https://www.lumi-supercomputer.eu/lumis-full-system-architecture-revealed/ | https://web.archive.org/web/20240908234416/https://en.wikipedia.org/wiki/LUMI | https://lumi-supercomputer.github.io/LUMI-training-materials/2day-20240502/01_Architecture/ | https://www.tomshardware.com/news/amd-based-finnish-supercomputer-will-be-one-of-worlds-most-powerful-in-mid-2021 | 64.23 | 27.70 | |||||||||||
| Lawrence Berkeley NL NERSC Perlmutter | Existing | Confirmed | Yes | 18.65 | 2.3e+3 | NVIDIA A100 SXM4 40 GB | 6,144.00 | United States of America | US Department of Energy | 2021-05-17 | Public | 6.90 | 1.5e+8 | Lawrence Berkeley National Laboratory, Building 59, 1 Cyclotron Rd, Berkeley, CA 94720 | Scientific research | A100 | 1536 nodes: 1x AMD EPYC 7763 4x NVIDIA A100 (40GB)
256 nodes: 1x AMD EPYC 7763 4x NVIDIA A100 (80GB) | 2021-05-17 | 11.51 | NVIDIA A100 SXM4 80 GB | 1,024 | 7,168.00 | NVIDIA | NVIDIA | True | 2 | 18.35 | 4.5e+18 | 4.5e+18 | 2.2e+18 | 1.1e+18 | 6.16 | 6.90 | 3.2e+11 | 2.0e+8 | 1.5e+8 | 1.5e+8 | $146 million for machine plus multiple years of service | Sunway OceanLight | 0.75 | https://web.archive.org/web/20240421214604/https://www.hpcwire.com/2021/05/27/nersc-debuts-perlmutter-worlds-fastest-ai-supercomputer/ | https://web.archive.org/web/20240927184852/https://docs.nersc.gov/systems/perlmutter/architecture/ | https://top500.org/news/crays-next-generation-supercomputer-headed-to-berkeley-lab-in-2020/ | 37.88 | -122.25 | |||||||||||
| AIST ABCI-Q | Existing | Confirmed | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | Japan | National Institute of Advanced Industrial Science and Technology (AIST) | 2025-06-15 | "more than 2,000": estimated as 2048 | Public | 2.87 | 9.6e+7 | AIST Tsukuba
1 Chome Higashi, Tsukuba, Ibaraki 305-0046, Japan | H100 | The supercomputer is powered by more than 2,000 NVIDIA H100 Tensor Core GPUs in 500+ nodes | 2025-06 | "more than 2,000": estimated as 2,200 | 11.85 | 2,048.00 | NVIDIA | True | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.87 | 7.1e+11 | 9.6e+7 | https://web.archive.org/web/20240729195553/https://nvidianews.nvidia.com/news/nvidia-powers-japans-abci-q-supercomputer-for-quantum-research | 36.08 | 140.08 | ||||||||||||||||||||||
| Horizon Compute Baobab Phase 2 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | United States of America | Horizon Compute | 2025-03-01 | Unclear exactly when this was expanded to 2k H100s, but at the latest by March 1st, 2025 | Private | 2.87 | 6.9e+7 | Cloud | H100 | 256 HGX H100 nodes (2048 GPUs) | 2025-03-01 | 11.85 | Horizon Compute Baobab Phase 1 | 2,048.00 | NVIDIA | True | 75 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.87 | 7.1e+11 | 6.9e+7 | xAI Colossus Memphis Phase 2 | 0.01 | https://web.archive.org/web/20241230203302/https://horizoncompute.com/ | https://archive.ph/VaYyj#selection-320.0-330.0 | 39.38 | -97.92 | ||||||||||||||||||
| Northern Data Group Taiga Cloud Island 3 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | Northern Data Group | 2024-03-30 | Private | 2.92 | 7.3e+7 | Europe | Cloud | H100 | "Taiga Cloud is building out its NVIDIA H100 GPU infrastructure into pods of 512 GPUs, connected into islands of four pods each (2,048 GPUs) using NVIDIA BlueField DPUs and the NVIDIA Quantum-2 InfiniBand platform"
"the purchase of 20 NVIDIA H100 GPU Pods – each made up of 512 H100 GPUs" | 2024-03-30 | 11.84 | 2,048.00 | NVIDIA | True | 36 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.3e+7 | Meta GenAI 2024a | 0.08 | https://web.archive.org/web/20241122185558/https://www.eqs-news.com/news/corporate/northern-data-group-europes-largest-deployment-of-nvidia-h100-tensor-core-gpus-to-expand-taiga-clouds-generative-ai-csp-offering/1903591 | |||||||||||||||||||||||
| Northern Data Group Taiga Cloud Island 4 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | Northern Data Group | 2024-03-30 | Private | 2.92 | 7.3e+7 | Europe | Cloud | H100 | "Taiga Cloud is building out its NVIDIA H100 GPU infrastructure into pods of 512 GPUs, connected into islands of four pods each (2,048 GPUs) using NVIDIA BlueField DPUs and the NVIDIA Quantum-2 InfiniBand platform"
"the purchase of 20 NVIDIA H100 GPU Pods – each made up of 512 H100 GPUs" | 2024-03-30 | 11.84 | 2,048.00 | NVIDIA | True | 36 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.3e+7 | Meta GenAI 2024a | 0.08 | https://web.archive.org/web/20241122185558/https://www.eqs-news.com/news/corporate/northern-data-group-europes-largest-deployment-of-nvidia-h100-tensor-core-gpus-to-expand-taiga-clouds-generative-ai-csp-offering/1903591 | |||||||||||||||||||||||
Northern Data Group Njored Taiga Cloud Island 5 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | United Kingdom of Great Britain and Northern Ireland | Northern Data Group | 2024-03-30 | Mentioned in 2025-06 Top500 as "Njored" with at least 1,952 H100s | Private | 2.92 | 7.3e+7 | United Kingdom | Cloud | H100 | "Taiga Cloud is building out its NVIDIA H100 GPU infrastructure into pods of 512 GPUs, connected into islands of four pods each (2,048 GPUs) using NVIDIA BlueField DPUs and the NVIDIA Quantum-2 InfiniBand platform"
"the purchase of 20 NVIDIA H100 GPU Pods – each made up of 512 H100 GPUs" | 2024-03-30 | 11.84 | 2,048.00 | NVIDIA | True | 36 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.3e+7 | Meta GenAI 2024a | 0.08 | https://web.archive.org/web/20241122185558/https://www.eqs-news.com/news/corporate/northern-data-group-europes-largest-deployment-of-nvidia-h100-tensor-core-gpus-to-expand-taiga-clouds-generative-ai-csp-offering/1903591 | 54.24 | -2.37 | |||||||||||||||||||
| Northern Data Group Taiga Cloud NO1 Island 1 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | Norway | Northern Data Group | 2024-03-30 | There are two 2,048 H100 islands at this location | Private | 2.92 | 7.3e+7 | Stølevegen 39, 4715 Øvrebø, Norway | Cloud | H100 | "Taiga Cloud is building out its NVIDIA H100 GPU infrastructure into pods of 512 GPUs, connected into islands of four pods each (2,048 GPUs) using NVIDIA BlueField DPUs and the NVIDIA Quantum-2 InfiniBand platform"
"The partnership provides Taiga Cloud with additional access to 100% hydro-powered data centers, with Bulk housing 2 of Taiga Cloud’s islands of NVIDIA H100 GPUs and customer availability starting from Q1 2024." | 2024-03-30 | 11.84 | 2,048.00 | NVIDIA | True | 36 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.3e+7 | Meta GenAI 2024a | 0.08 | https://web.archive.org/web/20241122185558/https://www.eqs-news.com/news/corporate/northern-data-group-europes-largest-deployment-of-nvidia-h100-tensor-core-gpus-to-expand-taiga-clouds-generative-ai-csp-offering/1903591 | https://web.archive.org/web/20240621132432/https://bulkinfrastructure.com/newsroom/taiga-cloud-continues-rollout-of-nvidia-h100-gpus | 58.26 | 7.89 | ||||||||||||||||||
| Northern Data Group Taiga Cloud NO1 Island 2 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | Norway | Northern Data Group | 2024-03-30 | There are two 2,048 H100 islands at this location | Private | 2.92 | 7.3e+7 | Stølevegen 39, 4715 Øvrebø, Norway | Cloud | H100 | "Taiga Cloud is building out its NVIDIA H100 GPU infrastructure into pods of 512 GPUs, connected into islands of four pods each (2,048 GPUs) using NVIDIA BlueField DPUs and the NVIDIA Quantum-2 InfiniBand platform"
"The partnership provides Taiga Cloud with additional access to 100% hydro-powered data centers, with Bulk housing 2 of Taiga Cloud’s islands of NVIDIA H100 GPUs and customer availability starting from Q1 2024." | 2024-03-30 | 11.84 | 2,048.00 | NVIDIA | True | 36 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.3e+7 | Meta GenAI 2024a | 0.08 | https://web.archive.org/web/20241122185558/https://www.eqs-news.com/news/corporate/northern-data-group-europes-largest-deployment-of-nvidia-h100-tensor-core-gpus-to-expand-taiga-clouds-generative-ai-csp-offering/1903591 | https://web.archive.org/web/20240621132432/https://bulkinfrastructure.com/newsroom/taiga-cloud-continues-rollout-of-nvidia-h100-gpus | 58.26 | 7.89 | ||||||||||||||||||
| NVIDIA Israel-1 Phase 2 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | Israel | NVIDIA | 2024-11-20 | Not totally sure when this became operational, but it seems like it was at least operational by November 2024 | Private | 2.92 | 7.8e+7 | Israel | NVIDIA | H100 | will feature 256 Nvidia HGX H100 systems, combining 2,048 Nvidia H100 80GB GPUs with more than 34 million CUDA cores and 1 million fourth-generation Tensor Cores, 2,560 BlueField-3 DPUs, and 80 Spectrum-4 switches | 2024-11-20 | The first half of the GPUs were confirmed to be operational Nov 2023, with the second half expected to be operational in the first half of 2024 | 11.84 | NVIDIA Israel-1 Phase 1 | 2,048.00 | NVIDIA | True | 65 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.8e+7 | xAI Colossus Memphis Phase 1 | 0.02 | https://web.archive.org/web/20240628053616/https://www.datacenterdynamics.com/en/news/nvidias-israel-1-supercomputer-starts-operations/ | 31.85 | 34.84 | |||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.60 | 2.0e+3 | 2,000.00 | China | 2024-09-15 | Private | 3.00 | 7.0e+7 | China | 11.82 | True | 2,000.00 | Anonymized | Anonymized | 54 | 18.30 | 4.0e+18 | 4.0e+18 | 2.0e+18 | 9.0e+17 | 3.00 | 6.7e+11 | 7.0e+7 | xAI Colossus Memphis Phase 1 | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Sesterce H100s Phase 1 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,048.00 | Sesterce | 2024-07-06 | Unclear exactly where this cluster is located. This is likely the early phases of the Paris 4k H100 cluster, but it's unclear | Private | 2.92 | 7.3e+7 | Cloud | H100 | 2024-07-06 | 11.84 | Sesterce H100s Phase 2 | 2,048.00 | NVIDIA | True | 48 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.3e+7 | Meta GenAI 2024a | 0.08 | https://archive.ph/ZDjLF | https://archive.ph/gJllE | ||||||||||||||||||||||
| Sesterce Synapse Phase 1 | Existing | Likely | Yes | 18.61 | 2.0e+3 | NVIDIA H200 SXM | 2,048.00 | France | Sesterce | 2024-09-30 | Private | 2.92 | 7.3e+7 | Paris, France | Cloud | H200 | 2048 x H200 SXM IB 3.2 available in early-september | 2024-09-30 | 11.84 | Sesterce Synapse Phase 2 | 2,048.00 | NVIDIA | True | 55 | 18.31 | 4.1e+18 | 4.1e+18 | 2.0e+18 | 1.0e+18 | 2.92 | 6.9e+11 | 7.3e+7 | xAI Colossus Memphis Phase 1 | 0.02 | https://web.archive.org/web/20240823055517/https://www.sesterce.com/reserved-cloud | https://archive.ph/ZDjLF | 48.85 | 2.35 | ||||||||||||||||||
| SoftBank CHIE-2 | Existing | Confirmed | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,040.00 | Japan | Softbank | 2024-10-31 | Private | 2.91 | 7.8e+7 | Japan | Softbank | H100 | Calculated from Top500 | 2024-10-31 | 11.84 | 2,040.00 | NVIDIA | True | 68 | 18.31 | 4.0e+18 | 4.0e+18 | 2.0e+18 | 1.0e+18 | 2.91 | 6.9e+11 | 7.8e+7 | xAI Colossus Memphis Phase 1 | 0.02 | https://web.archive.org/web/20241119082609/https://top500.org/system/180329/ | 36.39 | 138.59 | ||||||||||||||||||||
| SoftBank CHIE-3 | Existing | Confirmed | Yes | 18.61 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,040.00 | Japan | Softbank | 2024-10-31 | Private | 2.91 | 7.8e+7 | Japan | Softbank | H100 | Calculated from Top500 | 2024-10-31 | 11.84 | 2,040.00 | NVIDIA | True | 68 | 18.31 | 4.0e+18 | 4.0e+18 | 2.0e+18 | 1.0e+18 | 2.91 | 6.9e+11 | 7.8e+7 | xAI Colossus Memphis Phase 1 | 0.02 | https://web.archive.org/web/20241119062610/https://top500.org/system/180330/ | 36.39 | 138.59 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 18.60 | 2.0e+3 | China | 2021-04-15 | Private | China | Anonymized | Anonymized | True | 2 | 18.60 | 4.0e+18 | 4.0e+18 | Sunway OceanLight | 0.70 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.60 | 2.0e+3 | China | 2024-09-15 | Public/Private | China | Anonymized | Anonymized | True | 61 | 18.60 | 4.0e+18 | 4.0e+18 | xAI Colossus Memphis Phase 1 | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Sakura's H100s Phase 1 | Existing | Confirmed | Yes | 18.60 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,016.00 | Japan | Sakura Internet | 2024-07-31 | The Ministry of Economy, Trade and Industry (METI) will subsidize 6.8 billion yen ($48.2 million) of the 13.5 billion yen it will cost to build the machine | Public/Private | 2.88 | 9.6e+7 | Ishikari City, Hokkaido Japan | Cloud | H100 | we have completed the installation of 2,016 'NVIDIA H100 Tensor Core GPUs'. | 2024-07-31 | 11.84 | 2,016.00 | NVIDIA | True | 57 | 18.30 | 4.0e+18 | 4.0e+18 | 2.0e+18 | 1.0e+18 | 2.88 | 6.9e+11 | 7.7e+7 | 9.5e+7 | 9.6e+7 | The Ministry of Economy, Trade and Industry (METI) will subsidize 6.8 billion yen ($48.2 million) of the 13.5 billion yen it will cost to build the machine. | Meta GenAI 2024a | 0.08 | https://archive.ph/g8HTU | https://archive.ph/m4ZML# | 43.17 | 141.32 | |||||||||||||||
| Quebec 2k H100 Cluster | Existing | Likely | Yes | 18.60 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,000.00 | Canada | 2025-01-14 | Operational by 1/14/25 at the latest, but likely meaningfully earlier | Private | 2.80 | 7.5e+7 | Quebec, Canada | Cloud | H100 | From a private correspondence | 2025-01-14 | From a private correspondence about GPU cluster availability | 11.85 | 2,000.00 | NVIDIA | True | 82 | 18.30 | 4.0e+18 | 4.0e+18 | 2.0e+18 | 9.9e+17 | 2.80 | 7.1e+11 | 7.5e+7 | xAI Colossus Memphis Phase 1 | 0.02 | 46.81 | -71.20 | ||||||||||||||||||||
| Reka H100 Rental | Existing | Confirmed | Yes | 18.60 | 2.0e+3 | NVIDIA H100 SXM5 80GB | 2,000.00 | Reka AI | 2023-12-15 | Private | 2.90 | 8.7e+7 | Reka | H100 | "Our setup comprises of clusters from a mixture of vendors with our peak compute being approximately
2.5K H100s and 2.5K A100s"
"Provider A: 2000 chips" | 2023-12-15 | 11.83 | True | 2,000.00 | NVIDIA | 33 | 18.30 | 4.0e+18 | 4.0e+18 | 2.0e+18 | 9.9e+17 | 2.90 | 6.8e+11 | 8.7e+7 | Microsoft Azure Eagle | 0.14 | https://web.archive.org/web/20250108153733/https://publications.reka.ai/reka-core-tech-report.pdf | ||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 18.60 | 2.0e+3 | 2,000.00 | China | 2024-10-15 | Private | 3.00 | 7.0e+7 | China | 11.82 | 2,000.00 | Anonymized | Anonymized | True | 69 | 18.30 | 4.0e+18 | 4.0e+18 | 2.0e+18 | 9.0e+17 | 3.00 | 6.7e+11 | 7.0e+7 | xAI Colossus Memphis Phase 1 | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Meta Research SuperCluster (RSC-1) Phase 1 | Existing | Confirmed | Yes | 18.58 | 1.9e+3 | NVIDIA A100 | 6,080.00 | United States of America | Meta AI | 2022-01-24 | The location is found from triangulating from the promo video. I'm fairly confident in it, but it isn't guaranteed to be accurate | Private | 5.18 | 1.4e+8 | 6200 Technology Blvd, 23150 Sandston, Virginia, USA | Meta | A100 | At the time, “only” the first phase of the system was operational: 760 Nvidia DGX A100 nodes, totaling 6,080 Nvidia A100 GPUs | 2022-01-24 | 11.56 | Meta Research SuperCluster (RSC-1) Phase 2 | 6,080.00 | NVIDIA | True | 5 | 18.28 | 3.8e+18 | 3.8e+18 | 1.9e+18 | 9.5e+17 | 5.18 | 3.7e+11 | 1.4e+8 | DeepSeek Fire-Flyer 2 | 0.61 | https://web.archive.org/web/20231215015022/http://www.hpcwire.com/2023/05/18/meta-completes-research-supercluster-announces-next-gen-datacenter/ | https://web.archive.org/web/20240828114058/https://ai.meta.com/blog/ai-rsc/ | 37.48 | -77.24 | |||||||||||||||||
| CEA EXA1-HE Phase 2 | Existing | Confirmed | Yes | 18.58 | 1.9e+3 | NVIDIA GH200 | 1,908.00 | France | French Alternative Energies and Atomic Energy Commission (CEA) | 2024-04-17 | Phase 1 was from 2021 and didn't contain any GPUs | Public | 2.02 | 8.7e+7 | Military Applications Division of the French Alternative Energies and Atomic Energy Commission | GH200 | Calculated from Top500 | 2024-04-17 | 11.97 | 1,908.00 | NVIDIA | True | 46 | 18.28 | 3.8e+18 | 3.8e+18 | 1.9e+18 | 9.4e+17 | 2.72 | 2.02 | 9.4e+11 | 8.7e+7 | Meta GenAI 2024a | 0.08 | https://web.archive.org/web/20240430193812/https://www.datacenterdynamics.com/en/news/eviden-delivers-second-exa1-supercomputer-to-frances-cea/ | 47.82 | 2.62 | |||||||||||||||||||
| Tesla A100 Cluster Phase 1 | Existing | Confirmed | Yes | 18.56 | 1.8e+3 | NVIDIA A100 SXM4 80 GB | 5,760.00 | United States of America | Tesla | 2021-06-20 | Private | 4.95 | 1.8e+8 | Tesla | A100 | for a whopping 5,760 A100s throughout the system | 2021-06-20 | 11.56 | Tesla A100 Cluster Phase 2 | 5,760.00 | NVIDIA | True | 4 | 18.25 | 3.6e+18 | 3.6e+18 | 1.8e+18 | 9.0e+17 | 4.95 | 3.6e+11 | 1.8e+8 | Sunway OceanLight | 0.60 | https://web.archive.org/web/20230322111214/https://www.hpcwire.com/2021/06/22/ahead-of-dojo-tesla-reveals-its-massive-precursor-supercomputer/ | 39.38 | -97.92 | ||||||||||||||||||||
| Gcore data center Phase 2 | Planned | Likely | Yes | 18.54 | 1.8e+3 | NVIDIA H100 SXM5 80GB | 500.00 | Korea (Republic of) | Gcore,NHN Corporation | Private | 1.90 | 2.4e+7 | Incheon, South Korea | Cloud | H100,H200,GB200,>1,000 GPUs, including H200s and GB200s | Gcore plans to expand this to more than 1,000 GPUs by the end of the year and will include H200s and GB200s. | Planned Q4 2024 | 11.96 | Gcore data center Phase 1 | NVIDIA GB200 NVL2 (per GPU) | 500 | 1,000.00 | NVIDIA | NVIDIA | 18.24 | 3.5e+18 | 3.5e+18 | 1.7e+18 | 8.7e+17 | 1.90 | 9.2e+11 | 2.4e+7 | https://web.archive.org/web/20240723133814/https://www.datacenterdynamics.com/en/news/gcore-expands-into-south-korea-with-nhn-cloud/ | 37.46 | 126.71 | |||||||||||||||||||||
| Oak Ridge NL Summit | Decommissioned | Confirmed | Yes | 18.54 | 1.7e+3 | NVIDIA Tesla V100 SXM2 | 27,648.00 | United States of America | US Department of Energy | 2018-06-08 | Public | 13.00 | 2.2e+8 | Oak Ridge National Laboratory
5200, 1 Bethel Valley Rd, Oak Ridge, TN 37830 | US civilian research | V100 | "The IBM AC922 system consists of 4,608 compute servers, each containing two 22-core IBM Power9 processors and six NVIDIA Tesla V100 graphics processing unit accelerators" | 2018-06-08 | 11.42 | 27,648.00 | NVIDIA | 1 | 18.54 | 3.5e+18 | 3.5e+18 | 4.3e+17 | 19.02 | 13.00 | 2.7e+11 | 6.0e+8 | 2.0e+8 | 2.2e+8 | we estimate that Sierra represented $125 million of the $325 million of the CORAL procurement based on a flat flops cost between the two machines | True | 2024-11-15 | Oak Ridge NL Summit | 1.00 | https://web.archive.org/web/20240904145755/https://www.ornl.gov/news/ornl-launches-summit-supercomputer | https://web.archive.org/https://www.olcf.ornl.gov/olcf-resources/compute-systems/summit/summit-faqs | https://www.nextplatform.com/2017/10/05/clever-machinations-livermores-sierra-supercomputer/ | 36.01 | -84.27 | ||||||||||||||
| Microsoft Azure Meta AI Rental | Existing | Confirmed | Yes | 18.53 | 1.7e+3 | NVIDIA A100 | 5,400.00 | United States of America | Microsoft | 2022-05-27 | Not marked "possible duplicate" since we don't have any other Microsoft supercomputers existing at this time that are this large and aren't known to be used by anyone else (eg, the 25k A100 supercomputer used by OpenAI) | Private | 4.60 | 1.2e+8 | Meta | A100 | Meta AI did reveal that the company plans to utilize next-gen machine learning workloads on a reserved cluster on Microsoft Azure that would enlist 5,400 A100 GPUs from NVIDIA. | 2022-05-27 | 11.56 | 5,400.00 | NVIDIA | True | 10 | 18.23 | 3.4e+18 | 3.4e+18 | 1.7e+18 | 8.4e+17 | 4.60 | 3.7e+11 | 1.2e+8 | Microsoft GPT-4 cluster | 0.22 | https://web.archive.org/web/20220630142849/https://wccftech.com/microsoft-azure-upgraded-amd-instinct-mi200-gpu-clusters-for-large-scale-ai-training/ | 39.38 | -97.92 | ||||||||||||||||||||
| Argonne NL Aurora | Existing | Confirmed | Yes | 18.52 | 1.7e+3 | Intel Data Center GPU Max 1550 | 63,744.00 | United States of America | US Department of Energy | 2024-05-13 | Public | 60.00 | 5.0e+8 | Theory and Computing Sciences Building
Building 240, Argonne National Laboratory, 9700 S. Cass Ave, Argonne Ct, Lemont, IL 60439 | Argonne National Laboratory and general scientific community | Intel Data Center GPU Max Series 1550 | The Aurora supercomputer at Argonne National Laboratory is now fully equipped with all 10,624 compute blades, boasting 63,744 Intel® Data Center GPU Max Series and 21,248 Intel® Xeon® CPU Max Series processors. | 2024-05-13 | From official sources | 10.75 | 63,744.00 | Intel | True | 52 | 18.52 | 3.3e+18 | 3.3e+18 | 3.3e+18 | 77.96 | 60.00 | 5.6e+10 | 9.4e+8 | 5.0e+8 | 5.0e+8 | The Aurora system was budgeted for $500 million, and HPE is still getting paid to assemble the system and Intel is getting paid to help with the software stack, we suspect. | Meta GenAI 2024a | 0.07 | https://web.archive.org/web/20240824025729/https://www.intel.com/content/www/us/en/newsroom/news/intel-powered-aurora-supercomputer-breaks-exascale-barrier.html#gs.f9p1j0 | https://web.archive.org/web/20240524235355/https://www.extremetech.com/computing/intel-aurora-supercomputer-breaks-exascale-barrier-but-fails-to-topple | https://web.archive.org/web/20240604155426/https://extremecomputingtraining.anl.gov/wp-content/uploads/sites/96/2023/08/ATPESC-2023-Track-1-Talk-3-Servesh-Mulalidharan-Aurora.pdf | 41.72 | -87.98 | ||||||||||||||
| Jean Zay Supercomputer Phase 4 | Existing | Confirmed | Yes | 18.50 | 1.6e+3 | NVIDIA H100 SXM5 80GB | 1,456.00 | France | GENCI | 2024-09-15 | Public | 2.42 | 6.4e+7 | Campus universitaire d'Orsay, Batiment 506, Rue John Von Neumann, 91403 Orsay, France | Researchers in academia and industry | V100,A100,H100,1,832 V100s 416 A100s 1,456 H100s | In total after this extension, Jean Zay will be equipped with 1,456 NVIDIA H100 GPUs, in addition to the 416 NVIDIA A100 Tensor Core GPUs and 1,832 NVIDIA V100 Tensor Core GPUs remaining from the old configuration | 2024-09 | 11.81 | Jean Zay Supercomputer Phase 3 | NVIDIA A100 SXM4 80 GB | 416 | 3,704.00 | NVIDIA | NVIDIA | True | 71 | 18.20 | 3.1e+18 | 3.1e+18 | 1.6e+18 | 7.9e+17 | 2.42 | 6.5e+11 | 6.4e+7 | xAI Colossus Memphis Phase 1 | 0.02 | https://web.archive.org/web/20240528032205/http://www.idris.fr/eng/annonces/idris-extension-jean-zay-h100-eng.html | https://web.archive.org/web/20240523101724/https://www.cnrs.fr/en/press/genci-and-cnrs-choose-eviden-make-jean-zay-supercomputer-one-most-powerful-france | 48.71 | 2.18 | |||||||||||||||
| Nebius ISEG | Existing | Confirmed | Yes | 18.48 | 1.5e+3 | NVIDIA H100 SXM5 80GB | 1,520.00 | Finland | Nebius AI | 2023-11-01 | Private | 2.25 | 5.0e+7 | Moreenikatu 6, 04600 Mäntsälä, Finland | Cloud | H100 | "The Mäntsälä data center is also home to ISEG, among the most powerful supercomputers worldwide in both performance and energy efficiency."
Chip count calculated from Top500 | 2023-11-01 | 11.83 | 1,520.00 | NVIDIA | True | 32 | 18.18 | 3.0e+18 | 3.0e+18 | 1.5e+18 | 7.5e+17 | 2.21 | 2.25 | 6.7e+11 | 5.0e+7 | Tesla 10k H100 Cluster | 0.15 | https://web.archive.org/web/20241008144121/https://www.datacenterdynamics.com/en/news/nebius-deploys-ai-cluster-at-equinix-data-center-in-paris/ | https://web.archive.org/web/20241109070701/https://www.top500.org/system/180234/ | 60.63 | 25.26 | ||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | No | 18.48 | 2.0e+3 | China | 2022-08-15 | Private | China | Anonymized | Anonymized | 18.48 | 3.0e+18 | 3.0e+18 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.48 | 2.0e+3 | China | 2024-03-15 | Public | China | Anonymized | Anonymized | True | 43 | 18.48 | 3.0e+18 | 3.0e+18 | Meta GenAI 2024a | 0.06 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 18.48 | 2.0e+3 | China | 2024-06-15 | Public/Private | China | Anonymized | Anonymized | True | 59 | 18.48 | 3.0e+18 | 3.0e+18 | Meta GenAI 2024a | 0.06 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 18.48 | 2.0e+3 | Hong Kong | Public | Hong Kong | Anonymized | Anonymized | 18.30 | 3.0e+18 | 3.0e+18 | 2.0e+18 | 22.28 | 114.16 | ||||||||||||||||||||||||||||||||||||||||
| Sandia NL El Dorado | Existing | Confirmed | Yes | 18.47 | 1.5e+3 | AMD Instinct MI300A | 1,520.00 | United States of America | US Department of Energy | 2024-11-18 | Public | 1.86 | 6.2e+7 | Albuquerque, New Mexico | Scientific Research,Sandia NL | AMD MI300A | Calculated from Top500 | 2024-11-18 | 11.90 | 1,520.00 | AMD | True | 90 | 18.17 | 3.0e+18 | 3.0e+18 | 1.5e+18 | 7.5e+17 | 2.35 | 1.86 | 8.0e+11 | 6.2e+7 | xAI Colossus Memphis Phase 1 | 0.02 | https://web.archive.org/web/20241125211138/https://www.top500.org/system/180309/ | 35.09 | -106.65 | |||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Unclear | 18.48 | 1.0e+3 | China | Public | China | 20,000.00 | Anonymized | Anonymized | 18.00 | 3.0e+18 | 3.0e+18 | 1.0e+18 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||
| NVIDIA Selene Phase 2 | Existing | Confirmed | Yes | 18.45 | 1.4e+3 | NVIDIA A100 | 4,480.00 | United States of America | NVIDIA | 2020-11-30 | Private | 4.74 | 1.2e+8 | Santa Clara, California | NVIDIA | A100 | Configuration: 4480 NVIDIA A100 Tensor Core GPUs | 2020-11-30 | 11.47 | NVIDIA Selene Phase 1 | 4,480.00 | NVIDIA | True | 2 | 18.15 | 2.8e+18 | 2.8e+18 | 1.4e+18 | 7.0e+17 | 3.91 | 4.74 | 2.9e+11 | 1.2e+8 | Oak Ridge NL Summit | 0.81 | https://web.archive.org/web/20240406153716/https://www.hpcwire.com/2020/06/22/nvidia-nabs-7-spot-on-top500-with-selene-launches-a100-pcie-cards/ | https://web.archive.org/web/20240404062320/https://www.opensfs.org/wp-content/uploads/Accelerating-AI-at-Scale_Julie_Prethvi_updated051421.pdf | 37.35 | -121.96 | |||||||||||||||||
| Paper on Megatron-Turing | Existing | Confirmed | Yes | 18.45 | 1.4e+3 | NVIDIA A100 SXM4 80 GB | 4,480.00 | Microsoft,NVIDIA | 2022-01-28 | Private | 3.82 | 1.0e+8 | Microsoft,NVIDIA | A100 | "Model training is done with mixed precision using 16-bit bfloat on NVIDIA’s Selene [2] supercomputer with 560 DGX A100 nodes. Each cluster node has 8 NVIDIA 80-GB A100 GPUs" | 2022-01-28 | 11.56 | True | NVIDIA Selene Phase 2 | 4,480.00 | NVIDIA | 8 | 18.15 | 2.8e+18 | 2.8e+18 | 1.4e+18 | 7.0e+17 | 3.82 | 3.7e+11 | 1.0e+8 | DeepSeek Fire-Flyer 2 | 0.45 | https://arxiv.org/abs/2201.11990 | |||||||||||||||||||||||
| Meta 2017 V100 Cluster | Existing | Confirmed | Yes | 18.44 | 1.4e+3 | NVIDIA Tesla V100 SXM2 | 22,000.00 | United States of America | Meta AI | 2017-10-01 | Unsure if this cluster has since been decommissioned, since it's relatively old
Our only information on the date is sources saying it was "Built in 2017". The V100 wasn't released until June 2017, so I assume this cluster was built in the second half of 2017. October 1st is the middle of the second half of 2017, so I put that as the date | Private | 15.50 | 4.8e+8 | Meta | V100 | The first generation of this infrastructure, designed in 2017, has 22,000 NVIDIA V100 Tensor Core GPUs in a single cluster that performs 35,000 training jobs a day. | 2017-10-01 | Might be decommissioned by now | 11.25 | 22,000.00 | NVIDIA | True | 1 | 18.44 | 2.8e+18 | 2.8e+18 | 3.4e+17 | 15.50 | 1.8e+11 | 4.8e+8 | True | Meta 2017 V100 Cluster | 1.00 | https://web.archive.org/web/20240828114058/https://ai.meta.com/blog/ai-rsc/ | 39.38 | -97.92 | |||||||||||||||||||
| TensorWave MI300X Cluster 1 Phase 1 | Existing | Likely | Yes | 18.42 | 1.3e+3 | AMD Instinct MI300X | 1,000.00 | United States of America | TensorWave | 2024-04-01 | Private | 1.53 | 2.7e+7 | Cloud | MI300X | 1000 MI300Xs available from 04/01/2024 to 04/01/2025 (USA) | 2024-04-01 | 11.93 | 1,000.00 | AMD | True | 52 | 18.12 | 2.6e+18 | 2.6e+18 | 1.3e+18 | 6.5e+17 | 1.53 | 8.6e+11 | 2.7e+7 | Meta GenAI 2024a | 0.05 | https://web.archive.org/web/20240420190200/https://gpulist.ai/detail/8f759e7 | https://web.archive.org/web/20241009165502/https://www.theregister.com/2024/04/16/amd_tensorwave_mi300x/ | 39.38 | -97.92 | ||||||||||||||||||||
| Paper on Falcon 180B | Existing | Confirmed | Yes | 18.41 | 1.3e+3 | NVIDIA A100 | 4,096.00 | Amazon | 2023-11-28 | Private | 3.40 | 8.1e+7 | Cloud | A100 | pretrain these models on up to 4,096 A100s on cloud AWS infrastructure | 2023-11-28 | 11.58 | True | Amazon Titan training cluster | 4,096.00 | NVIDIA | 40 | 18.11 | 2.6e+18 | 2.6e+18 | 1.3e+18 | 6.4e+17 | 3.40 | 3.8e+11 | 8.1e+7 | Microsoft Azure Eagle | 0.09 | https://arxiv.org/abs/2311.16867 | https://web.archive.org/web/20250210123929/https://huggingface.co/tiiuae/falcon-180B | ||||||||||||||||||||||
| Core42 SuperPOD | Existing | Confirmed | Yes | 18.40 | 1.3e+3 | NVIDIA H100 SXM5 80GB | 1,272.00 | United Arab Emirates | G42 | 2024-10-30 | Public/Private | 1.81 | 4.8e+7 | United Arab Emirates | Cloud,G42 | H100 | Calculated from Top500 | 2024-10-30 | 11.84 | 1,272.00 | NVIDIA | True | 88 | 18.10 | 2.5e+18 | 2.5e+18 | 1.3e+18 | 6.3e+17 | 1.81 | 6.9e+11 | 4.8e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://web.archive.org/web/20241125214402/https://www.middleeastainews.com/p/core42s-nvidia-dgx-superpod-ranked-first | 23.85 | 54.26 | ||||||||||||||||||||
| Tesla Training Cluster | Existing | Likely | Yes | 18.40 | 1.3e+3 | NVIDIA A100 | 4,032.00 | United States of America | Tesla | 2021-08-20 | I'm assuming these are A100s because H100s weren't out yet at the time and that is what the other cluster mentioned has | Private | 3.46 | 1.4e+8 | Tesla | A100 | as well as the previously mentioned 5670 system which is used for training, it has a second, 4032 GPU system for training and a 1752 GPU system for auto-labeling | 2021-08-20 | Number of GPUs officially announced, but no other details about it released | 11.56 | 4,032.00 | NVIDIA | True | 9 | 18.10 | 2.5e+18 | 2.5e+18 | 1.3e+18 | 6.3e+17 | 3.46 | 3.6e+11 | 1.4e+8 | DeepSeek Fire-Flyer 2 | 0.40 | https://web.archive.org/web/20240221025227/https://www.datacenterdynamics.com/en/news/tesla-details-dojo-supercomputer-reveals-dojo-d1-chip-and-training-tile-module/ | 39.38 | -97.92 | |||||||||||||||||||
| AWS EC2 P4d | Existing | Likely | Unclear | 18.40 | 1.3e+3 | NVIDIA A100 SXM4 40 GB | 4,000.00 | United States of America | Amazon | 2020-11-02 | Private | 3.49 | 1.1e+8 | Cloud | A100 | EC2 UltraClusters are comprised of More than 4,000 latest NVIDIA A100 Tensor Core GPUs | 2020-11-02 | 11.55 | 4,000.00 | NVIDIA | 18.10 | 2.5e+18 | 2.5e+18 | 1.2e+18 | 6.2e+17 | 3.49 | 3.6e+11 | 1.1e+8 | https://web.archive.org/web/20240522231925/https://pages.awscloud.com/amazon-ec2-p4d.html | 39.38 | -97.92 | |||||||||||||||||||||||||
| Ezra-1 Stability AI AWS Cluster | Existing | Confirmed | Yes | 18.40 | 1.3e+3 | NVIDIA A100 | 4,000.00 | Amazon | 2022-07-20 | Private | 3.41 | 9.0e+7 | Stability AI | A100 | The model was trained on our 4,000 A100 Ezra-1 AI ultracluster over the last month | 2022-07-20 | 11.56 | 4,000.00 | NVIDIA | True | 16 | 18.10 | 2.5e+18 | 2.5e+18 | 1.2e+18 | 6.2e+17 | 3.41 | 3.7e+11 | 9.0e+7 | Microsoft GPT-4 cluster | 0.16 | https://web.archive.org/web/20241207204816/https://stability.ai/news/stable-diffusion-announcement | ||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | No | 18.30 | 1.0e+3 | China | 2022-11-15 | China | True | Anonymized | Anonymized | 18.30 | 2.0e+18 | 2.0e+18 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| JUWELS-Booster | Existing | Confirmed | Yes | 18.37 | 1.2e+3 | NVIDIA A100 | 3,744.00 | Germany | Julich Supercomputing Center | 2020-11-16 | Public | 3.16 | 1.0e+8 | 52428 Jülich, Germany | Academic research in Europe | A100 | total of 1872 CPUs (AMD EPYC Rome) + total of 3744 GPUs (NVIDIA A100) | 2020-11-16 | 11.57 | 3,744.00 | NVIDIA | True | 3 | 18.07 | 2.3e+18 | 2.3e+18 | 1.2e+18 | 5.8e+17 | 3.27 | 3.16 | 3.7e+11 | 1.0e+8 | Oak Ridge NL Summit | 0.68 | https://archive.ph/3RM36 | 50.92 | 6.36 | |||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 18.30 | 1.0e+3 | China | Private | China | Anonymized | Anonymized | 18.30 | 2.0e+18 | 2.0e+18 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Huawei Pangu Ultra MoE 910Bs | Existing | Confirmed | Yes | 18.35 | 1.1e+3 | Huawei Ascend 910B | 6,000.00 | 4.80 | Huawei | In the end, we achieve an MFU of 30.0% when training Pangu Ultra MoE, with performance comparable to that of DeepSeek R1, on 6K Ascend NPUs | 11.67 | 6,000.00 | Huawei | True | 18.35 | 2.3e+18 | 2.3e+18 | 5.6e+17 | 4.80 | 4.7e+11 | ||||||||||||||||||||||||||||||||||||
| Paper on AFM-server | Existing | Confirmed | Yes | 18.35 | 1.1e+3 | Google TPU v4 | 8,192.00 | United States of America | 2024-06-10 | Cloud TPU clusters implies they trained on Google infrastructure | Private | 2.84 | 7.3e+7 | Cloud,Google | TPUv4 | "We train AFM-server from scratch for 6.3T tokens on 8192
TPUv4 chips, using a sequence length of 4096 and a batch-size of 4096 sequences."
"The AFM models are pre-trained on v4 and v5p Cloud TPU clusters"
"AFM-server was trained on 8192 TPUv4 chips provisioned as 8 × 1024 chip slices, where slices are connected together by the data-center network (DCN)" | 2024-06-10 | 11.90 | True | Google Oklahoma TPU v4 Pods | 8,192.00 | 66 | 18.35 | 2.3e+18 | 2.3e+18 | 2.3e+18 | 2.84 | 7.9e+11 | 7.3e+7 | Meta GenAI 2024a | 0.05 | https://machinelearning.apple.com/research/apple-intelligence-foundation-language-models | https://web.archive.org/web/20250114124728/https://arxiv.org/pdf/2407.21075 | 39.38 | -97.92 | |||||||||||||||||||||
| JCAHPC Miyabi | Existing | Confirmed | Yes | 18.35 | 1.1e+3 | NVIDIA GH200 | 1,120.00 | Japan | Joint Center for Advanced High Performance Computing (JCAHPC) | 2025-01-15 | Public | 1.57 | 5.5e+7 | Japan | Scientific Research | GH200 | Miyabi-G (Accelerator-node group): 1,120 nodes, one NVIDIA GH200 Grace Hopper Superchip per node | 2025-01 | 11.85 | 1,120.00 | NVIDIA | True | 109 | 18.04 | 2.2e+18 | 2.2e+18 | 1.1e+18 | 5.5e+17 | 1.57 | 7.1e+11 | 5.5e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://web.archive.org/web/20240417155526/https://www.jcahpc.jp/eng/supercomputer/miyabi-e.html | 36.39 | 138.59 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 18.30 | 1.0e+3 | China | 2022-07-15 | Public/Private | China | Anonymized | Anonymized | True | 17 | 18.30 | 2.0e+18 | 2.0e+18 | Microsoft GPT-4 cluster | 0.10 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Lawrence Livermore NL Sierra | Existing | Confirmed | Yes | 18.33 | 1.1e+3 | NVIDIA V100 | 17,280.00 | United States of America | US Department of Energy | 2018-06-01 | Public | 11.50 | 1.4e+8 | Lawrence Livermore National Laboratory
7000 East Ave, Livermore, CA 94550 | Lawrence Livermore NL,Scientific Research | V100 | Total GPUs: 17,280 | 2018-06-01 | 11.27 | 17,280.00 | NVIDIA | True | 2 | 18.33 | 2.2e+18 | 2.2e+18 | 2.7e+17 | 11.89 | 11.50 | 1.9e+11 | 3.4e+8 | 1.3e+8 | 1.4e+8 | we estimate that Sierra represented $125 million of the $325 million of the CORAL procurement based on a flat flops cost between the two machines | True | Meta 2017 V100 Cluster | 0.79 | https://web.archive.org/web/20240926221231/https://en.wikipedia.org/wiki/Sierra_(supercomputer) | https://www.nextplatform.com/2017/10/05/clever-machinations-livermores-sierra-supercomputer/ | https://web.archive.org/web/20240927001630/https://hpc.llnl.gov/hardware/compute-platforms/sierra | 37.68 | -121.77 | ||||||||||||||
| Anonymized Chinese System | Planned | Likely | Unclear | 18.30 | 1.0e+3 | 4,000.00 | China | Public | 3.00 | China | 11.52 | 4,000.00 | Anonymized | Anonymized | 18.00 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 3.00 | 3.3e+11 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | 4,000.00 | China | 2020-09-15 | Public | 3.00 | China | 11.52 | 4,000.00 | Anonymized | Anonymized | True | 4 | 18.00 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 3.00 | 3.3e+11 | Oak Ridge NL Summit | 0.60 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Chan Zuckerberg Initiative GPU Cluster | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | United States of America | CZ Biohub Network,CoreWeave | 2025-01-15 | CoreWeave cloud based cluster | Public | 1.44 | 3.9e+7 | Chan Zuckerberg Initiative,Healthcare,Researchers | H100 | The high-performance computing cluster, which is planned to comprise 1,000+ GPUs, will enable AI and large language models for biomedicine at scale. | 2025-01 | 11.85 | True | 1,024.00 | NVIDIA | 113 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.44 | 7.1e+11 | 3.9e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://web.archive.org/web/20241201154848/https://chanzuckerberg.com/newsroom/czscience-builds-ai-gpu-cluster-predictive-cell-models/ | https://web.archive.org/web/20250221155133/https://chanzuckerberg.com/rfa/ai-computing-gpu/ | 39.38 | -97.92 | |||||||||||||||||||
| Denvr Dataworks H100 | Existing | Likely | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | United States of America | Denvr Dataworks | 2024-10-15 | It appears to be in their central US availablity zone, suggesting that it's in the United States. Other location information is unknown | Private | 1.46 | 3.9e+7 | Cloud | H100 | The cluster features Nvidia HGX H100 servers with 1,024 Hopper architecture-based SXM5 GPUs | 2024-10-15 | 11.84 | 1,024.00 | NVIDIA | True | 92 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.9e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://web.archive.org/web/20241214104602/https://www.datacenterdynamics.com/en/news/denvr-dataworks-expands-with-cloud-based-nvidia-h100-gpu-cluster/ | 39.38 | -97.92 | ||||||||||||||||||||
| GreenNode Bangkok Cluster | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | Thailand | VNG Corporation | 2024-06-25 | Private | 1.46 | 3.6e+7 | 1 Soi Ramkhamhaeng 28, Hua Mak, Khet Bang Kapi, Bangkok 10240, Thailand | Cloud | H100 | GreenNode's GPU cluster includes 128 bare-metal servers equipped with 1024 NVIDIA H100 Tensor Core GPUs | 2024-06-25 | 11.84 | 1,024.00 | NVIDIA | True | 73 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.6e+7 | Meta GenAI 2024a | 0.04 | https://web.archive.org/web/20250108210408/https://greennode.ai/blog/launch-hyper-scale-ai-gpu-cluster | https://archive.ph/pi5KL | 13.75 | 100.64 | |||||||||||||||||||
| Horizon Compute Baobab Phase 1 | Existing | Likely | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | United States of America | Horizon Compute | 2024-02-15 | Judging by their current offerings (March 2025), they have a 2k H100 cluster and a 1k H100 cluster, so I'm interpreting this as them saying that they had two 1k H100 clusters in Q1 2024 | Private | 1.46 | 4.5e+7 | Cloud | H100 | Since Q1 2024, we have already deployed two large-scale clusters (2000+ H100s) | Q1 2024 | 11.84 | Horizon Compute Baobab Phase 2 | 1,024.00 | NVIDIA | True | 50 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 4.5e+7 | Microsoft Azure Eagle | 0.07 | https://web.archive.org/web/20241230203302/https://horizoncompute.com/ | 39.38 | -97.92 | |||||||||||||||||||
| IBM Blue Vela | Existing | Likely | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | United States of America | IBM | 2024-07-07 | There's 128x8=1,024 GPUs per pod, and it's implied that there's multiple pods, implying that there are thousands of GPUs, but this is all the information I have | Private | 1.46 | 3.6e+7 | IBM | A100,H100 | It uses 128-node Compute Pods. These contain 4 x Scalable Units, each of which contain 32 nodes. The nodes contain [8] Nvidia H100 GPUs. | 2024-07-07 | Very uncertain on number of chips. | 11.84 | 1,024.00 | NVIDIA | True | 77 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.6e+7 | Meta GenAI 2024a | 0.04 | https://web.archive.org/web/20240804175848/https://blocksandfiles.com/2024/08/02/big-blues-storage-scale-using-blue-vela-ai-supercomputer/ | https://web.archive.org/web/20240516182554/https://www.eetimes.com/ibm-refreshes-its-vela-research-ai-supercomputer/ | 39.38 | -97.92 | ||||||||||||||||||
| LeptonAI H100 Cluster | Existing | Likely | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | United States of America | Lepton AI | 2024-08-20 | This cluster is implied by their blog post. Unclear when it was first operational | Private | 1.46 | 3.9e+7 | Cloud | H100 | We observe similar statistics in our operation of different-sized clusters as well. For example, a 128 nodes (1024 GPU) cluster experiences one interruption every 30 hours | 2024-08-20 | 11.84 | 1,024.00 | NVIDIA | True | 81 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.9e+7 | Meta GenAI 2024a | 0.04 | https://web.archive.org/web/20250128074456/https://blog.lepton.ai/introducing-gpud-the-missing-gpu-management-for-ai-0f0d026337e3 | 39.38 | -97.92 | ||||||||||||||||||||
| Neevcloud cluster 1 | Existing | Unlikely | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | India | NeevCloud | 2024-04-29 | Marked "unlikely" because an industry expert doubts it exists | Private | 1.46 | 3.6e+7 | Central India | Cloud | H100 | 1024 H100s available from 04/29/2024 | 2024-04-29 | 11.84 | 1,024.00 | NVIDIA | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.6e+7 | https://web.archive.org/web/20240420194021/https://gpulist.ai/detail/ea604d5 | https://web.archive.org/web/20250110180448/https://entrepreneurshipstudio.com/news/posts/neevcloud-launches-with-40-000-gpus-revolutionizing-ai-cloud-services-for-smes-in-india | 23.32 | 87.89 | ||||||||||||||||||||||
| NVIDIA Eos Phase 1 | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | United States of America | NVIDIA | 2023-05-22 | Full system = 4608 GPUs
Pre system = (128/576)x4608 = 1024 | Private | 1.49 | 4.9e+7 | United States of America | NVIDIA | H100 | Nvidia’s new “Pre-Eos 128 Node DGX SuperPOD” is another one we’ve been waiting to hear word on.... In its full configuration, Eos will span 18 32-DGX H100 Pods, for a total of 576 DGX H100 systems, 4,608 H100 GPUs | 2023-05-22 | Ranked in Top500 | 11.83 | NVIDIA Eos Phase 2 | 1,024.00 | NVIDIA | True | 29 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.49 | 6.8e+11 | 4.9e+7 | Microsoft GPT-4 cluster | 0.13 | https://web.archive.org/web/20240406153948/https://www.hpcwire.com/2023/05/22/top500-frontier-gains-92-petaflops-henri-gets-a-little-greener/ | 39.38 | -97.92 | |||||||||||||||||
| NVIDIA Helios | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA GH200 | 1,024.00 | NVIDIA | 2023-11-15 | They announced in May 2023 that they were building it and that it would be operational by the end of the year. They haven't publicly confirmed that it is operational, but it seems very likely that it is | Private | 1.49 | 4.7e+7 | NVIDIA | GH200 | "Helios will include 1,024 Grace Hopper Superchips and is expected to come online by the end of the year." | Q4 2023 | Likely finished by now | 11.83 | 1,024.00 | NVIDIA | True | 46 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.49 | 6.8e+11 | 4.7e+7 | Microsoft Azure Eagle | 0.07 | https://web.archive.org/web/20240825120701/https://nvidianews.nvidia.com/news/nvidia-announces-dgx-gh200-ai-supercomputer | ||||||||||||||||||||||
| NVIDIA Israel-1 Phase 1 | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | Israel | NVIDIA | 2023-11-23 | Private | 1.49 | 3.4e+7 | Israel | NVIDIA | H100 | The first phase includes Dell PowerEdge XE9680 servers based on 128 Nvidia HGX H100 systems
will feature 256 Nvidia HGX H100 systems, combining 2,048 Nvidia H100 80GB GPUs with more than 34 million CUDA cores and 1 million fourth-generation Tensor Cores, 2,560 BlueField-3 DPUs, and 80 Spectrum-4 switches | 2023-11-23 | The first half of the GPUs were confirmed to be operational Nov 2023, with the second half expected to be operational in the first half of 2024 | 11.83 | NVIDIA Israel-1 Phase 2 | 1,024.00 | NVIDIA | True | 46 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.49 | 6.8e+11 | 3.4e+7 | Microsoft Azure Eagle | 0.07 | https://web.archive.org/web/20240628053616/https://www.datacenterdynamics.com/en/news/nvidias-israel-1-supercomputer-starts-operations/ | 31.85 | 34.84 | ||||||||||||||||||
| Ori Global Cloud H100 Cluster | Existing | Likely | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | Ori Industries | 2024-10-16 | Private | 1.46 | 3.9e+7 | Cloud | H100 | The cluster features 1,024 Nvidia H100s | 2024-10-16 | Unclear where it is located | 11.84 | 1,024.00 | NVIDIA | True | 92 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.9e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://archive.ph/UwWFb | |||||||||||||||||||||||
| Paper on Mamba 2 Hybrid | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | NVIDIA | 2024-06-12 | Private | 1.46 | 3.7e+7 | NVIDIA | H100 | "When training on NVIDIA H100 GPUs (NVIDIA 2023), with a tensor-parallel
size of four and data-parallel size of 256 (1024 total GPUs)" | 2024-06-12 | 11.84 | True | NVIDIA CoreWeave Eos-DFW Phase 1 | 1,024.00 | NVIDIA | 71 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.7e+7 | Meta GenAI 2024a | 0.04 | https://arxiv.org/abs/2406.07887 | |||||||||||||||||||||||
| SIAM AI HGX | Existing | Likely | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | Thailand | SIAM AI | 2024-09-12 | Private | 1.46 | 3.9e+7 | Thailand | Cloud | H100 | currently operates a powerful cluster of 1,024 NVIDIA H100 GPUs | 2024-09-12 | 11.84 | 1,024.00 | NVIDIA | True | 84 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.9e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://web.archive.org/web/20241204114134/https://www.cmkl.ac.th/news/the-first-hgx-h200-poc--asias-first-nvidia-h200--was-presented-by-siam-ai-nvidia-cloud-partner-to-cmkl-university-and-aiei | 15.13 | 101.02 | ||||||||||||||||||||
| Sustainable Metal Cloud Singapore Phase 1 | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | Singapore | SMC - Sustainable Metal Cloud | 2024-05-30 | Private | 1.46 | 3.7e+7 | Singapore | Cloud | H100 | H100 GPUs, up to 1,024 per cluster | 2024-05-30 | 11.84 | 1,024.00 | NVIDIA | True | 68 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 6.9e+11 | 3.7e+7 | Meta GenAI 2024a | 0.04 | https://archive.ph/ElLD9 | 1.35 | 103.81 | ||||||||||||||||||||
| Ubilink.AI Supercomputer | Existing | Confirmed | Yes | 18.31 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,024.00 | Taiwan | Ubilink AI | 2024-11-18 | Private | 1.78 | 3.9e+7 | Tucheng, New Taipei, Taiwan | Cloud | H100 | Calculated from Top500 | 2024-11-18 | 11.75 | 1,024.00 | NVIDIA | True | 104 | 18.01 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.1e+17 | 1.46 | 1.78 | 5.7e+11 | 3.9e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://web.archive.org/web/20241125215925/https://ubitus.net/ubilink-opening/ | 24.97 | 121.44 | |||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 18.30 | 1.0e+3 | China | Public/Private | China | Anonymized | Anonymized | 18.30 | 2.0e+18 | 2.0e+18 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Scaleway Nabuchodonosor | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,016.00 | France | Iliad SA | 2023-10-05 | Private | 1.48 | 3.4e+7 | Paris, France | Scaleway,Cloud | H100 | Iliad has purchased an Nvidia DGX SuperPOD with 1,016 H100 GPUs | 2023-10-05 | 11.83 | 1,016.00 | NVIDIA | True | 38 | 18.00 | 2.0e+18 | 2.0e+18 | 1.0e+18 | 5.0e+17 | 1.48 | 6.8e+11 | 3.4e+7 | Tesla 10k H100 Cluster | 0.10 | https://web.archive.org/web/20240225054506/https://www.datacenterdynamics.com/en/news/french-telco-iliad-acquires-nvidia-dgx-superpod-with-1016-h100-gpus/ | https://web.archive.org/web/20240228100033/https://www.telecoms.com/digital-ecosystem/iliad-lays-claim-to-europe-s-most-powerful-ai-supercomputer | https://web.archive.org/web/20231005173700/https://www.digitaltveurope.com/2023/09/26/xavier-niels-iliad-sets-out-plan-to-create-european-ai-champion-with-supercomputer-investment/ | 48.85 | 2.35 | ||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | 1,000.00 | China | 2024-09-15 | Public/Private | 0.70 | China | 12.46 | 3,000.00 | Anonymized | Anonymized | True | 94 | 18.30 | 2.0e+18 | 2.0e+18 | 4.0e+17 | 0.70 | 2.9e+12 | xAI Colossus Memphis Phase 1 | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | China | 2024-01-15 | Public/Private | China | Anonymized | Anonymized | True | 54 | 18.30 | 2.0e+18 | 2.0e+18 | Microsoft Azure Eagle | 0.07 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | China | 2024-11-15 | Public | China | Anonymized | Anonymized | True | 120 | 18.30 | 2.0e+18 | 2.0e+18 | xAI Colossus Memphis Phase 1 | 0.01 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 18.30 | 1.0e+3 | China | Private | China | Anonymized | Anonymized | 18.30 | 2.0e+18 | 2.0e+18 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | China | 2024-12-15 | Public/Private | China | Anonymized | Anonymized | True | 121 | 18.30 | 2.0e+18 | 2.0e+18 | xAI Colossus Memphis Phase 1 | 0.01 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Hut 8 H100 Cluster | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,000.00 | United States of America | Hut 8 | 2024-09-26 | Private | 1.43 | 3.8e+7 | Chicago, USA | Cloud | H100 | The cluster, hosted at a tier-three data center in Chicago, comprises multiple Hewlett Packard Enterprise (“HPE”) Cray supercomputers powered by 1,000 NVIDIA H100 GPUs. | 2024-09-26 | 11.84 | 1,000.00 | NVIDIA | True | 101 | 18.00 | 2.0e+18 | 2.0e+18 | 9.9e+17 | 4.9e+17 | 1.43 | 6.9e+11 | 3.8e+7 | xAI Colossus Memphis Phase 1 | 0.01 | https://web.archive.org/web/20241217152625/https://hut8.com/2024/09/26/hut-8-gpu-as-a-service-vertical-goes-live-with-inaugural-deployment/ | 41.88 | -87.63 | ||||||||||||||||||||
| NHN Cloud's National AI Data Center | Existing | Confirmed | Yes | 18.30 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,000.00 | Korea (Republic of) | NHN Corporation | 2023-11-01 | Public/Private | 1.45 | 3.3e+7 | Oryong-dong, Buk-gu, Gwangju, South Korea | Cloud | H100 | some 1,000 Nvidia H100 graphics processing units (GPUs) are working, | 2023-11-01 | 11.83 | 1,000.00 | NVIDIA | True | 44 | 18.00 | 2.0e+18 | 2.0e+18 | 9.9e+17 | 4.9e+17 | 1.45 | 6.8e+11 | 3.3e+7 | Tesla 10k H100 Cluster | 0.10 | https://web.archive.org/web/20240721155544/https://koreajoongangdaily.joins.com/news/2024-03-25/business/industry/NHN-Cloud-aims-sky-high-with-AI-data-center-in-Gwangju/2010013 | 35.22 | 126.85 | ||||||||||||||||||||
| Voltage Park Texas Phase 1 | Existing | Likely | Yes | 18.30 | 1.0e+3 | NVIDIA H100 SXM5 80GB | 1,000.00 | United States of America | Voltage Park | 2024-02-27 | Private | 1.43 | 3.6e+7 | Texas | Cloud | H100 | 1000 H100s available from 02/27/2024 to 02/27/2025 (Texas) | 2024-02-27 | This is from a posting on gpulist.ai | 11.84 | 1,000.00 | NVIDIA | True | 56 | 18.00 | 2.0e+18 | 2.0e+18 | 9.9e+17 | 4.9e+17 | 1.43 | 6.9e+11 | 3.6e+7 | Microsoft Azure Eagle | 0.07 | https://web.archive.org/web/20240226194913/https://gpulist.ai/detail/2104b9d | https://web.archive.org/web/20240420201628/https://www.datacenterdynamics.com/en/news/ai-cloud-computing-non-profit-buys-24000-nvidia-h100-chips/ | https://web.archive.org/web/20240913183724/https://finance.yahoo.com/news/penguin-solutions-selected-managed-services-150000685.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cucGVycGxleGl0eS5haS8&guce_referrer_sig=AQAAACbflXKvc2DyvuR2WPik8ksxhKTAz7auQoF9nqCdOM4yDPTL4jd-W2b-V8yRSH21fxe4Ccv4cv3WjdfSFteyrcb-e_fbqoJhuhCzDkUZMwLHbCvHLBwKEOw_g_JSilSt4zqyJ6cKRYvHZ-fOSzVztIaQMrRoiWkdsftOLBO6ZB7P | 31.80 | -98.82 | |||||||||||||||||
| Paper on Chameleon | Existing | Confirmed | Yes | 18.28 | 9.7e+2 | NVIDIA A100 | 3,072.00 | Meta AI | 2024-05-16 | Table 2 Chemeleon Model Pre-Training Resource Usage lists 3072 Concurrent GPUs for the 34B model. | Private | 2.50 | 6.0e+7 | A100 | "NVIDIA A100 80GB GPUs power both environments." | 2024-05-16 | 11.58 | True | Meta Research SuperCluster (RSC-1) Phase 2 | 3,072.00 | NVIDIA | 76 | 17.98 | 1.9e+18 | 1.9e+18 | 9.6e+17 | 4.8e+17 | 2.50 | 3.8e+11 | 6.0e+7 | Meta GenAI 2024a | 0.04 | https://arxiv.org/abs/2405.09818v1 | |||||||||||||||||||||||
| TSUBAME4.0 | Existing | Confirmed | Yes | 18.28 | 9.6e+2 | NVIDIA H100 SXM5 80GB | 960.00 | Japan | Tokyo Institute of Technology | 2024-04-01 | Public | 1.36 | 3.4e+7 | 4259 Nagatsutacho, Midori Ward, Yokohama, Kanagawa 226-0026, Japan | Cloud | H100 | The Japanese system will pack in 240 nodes of HPE's Cray XD6500 hardware...
Each node has a pair of AMD's 4th generation Epyc processors with 768GB of memory and four of Nvidia's H100 Tensor Core GPUs | 2024-04-01 | 11.84 | 960.00 | NVIDIA | True | 68 | 17.98 | 1.9e+18 | 1.9e+18 | 9.5e+17 | 4.7e+17 | 1.37 | 1.36 | 7.0e+11 | 3.4e+7 | Meta GenAI 2024a | 0.04 | https://web.archive.org/web/20241002114126/https://www.theregister.com/2023/05/22/hpe_gets_to_build_new/ | https://web.archive.org/web/20240627054219/https://www.hpcwire.com/off-the-wire/tokyo-techs-tsubame4-0-supercomputer-now-operational/ | 35.52 | 139.53 | ||||||||||||||||||
| Paper on AFM-on-device | Existing | Confirmed | Yes | 18.27 | 9.5e+2 | Google TPU v5p | 2,048.00 | 2024-06-10 | Cloud TPU clusters implies they trained on Google infrastructure | Private | 1.3e+7 | Cloud | TPUv5p | "AFM-on-device was trained on one slice of 2048 TPUv5p chips."
"The AFM models are pre-trained on v4 and v5p Cloud TPU clusters" | 2024-06-10 | True | Google Hypercomputer TPU v5p pod | 2,048.00 | 81 | 17.97 | 1.9e+18 | 1.9e+18 | 9.4e+17 | 1.3e+7 | Meta GenAI 2024a | 0.04 | https://machinelearning.apple.com/research/apple-intelligence-foundation-language-models | https://web.archive.org/web/20250114124728/https://arxiv.org/pdf/2407.21075 | ||||||||||||||||||||||||||||
| Intel Stability Gaudi 2 | Existing | Confirmed | Yes | 18.26 | 9.1e+2 | Intel Habana Gaudi2 | 4,000.00 | United States of America | Intel | 2024-03-11 | Private | 4.89 | 3.4e+7 | Texas | Stability AI | Habana Gaudi2 | build a large AI supercomputer using Xeon processors and 4,000 Gaudi2 AI hardware accelerators | 2024-03-11 | 11.57 | 4,000.00 | Intel | True | 59 | 18.26 | 1.8e+18 | 1.8e+18 | 4.89 | 3.7e+11 | 3.4e+7 | Microsoft Azure Eagle | 0.06 | https://web.archive.org/web/20240806003107/https://www.datacenterdynamics.com/en/news/intel-and-dell-to-build-supercomputer-for-stability-ai-featuring-cpus-and-gaudi2-accelerators/ | https://web.archive.org/web/20241201154514/https://stability.ai/news/putting-the-ai-supercomputer-to-work | https://web.archive.org/web/20241202232149/https://www.databricks.com/blog/llm-training-and-inference-intel-gaudi2-ai-accelerators | 31.80 | -98.82 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.30 | 9.0e+2 | 7,000.00 | China | 2024-03-15 | Public | 4.00 | China | 11.35 | 7,000.00 | Anonymized | Anonymized | True | 61 | 17.95 | 2.0e+18 | 2.0e+18 | 9.0e+17 | 2.0e+17 | 4.00 | 2.3e+11 | Meta GenAI 2024a | 0.04 | 35.49 | 101.90 | ||||||||||||||||||||||||||||
| IBM Vela | Existing | Likely | Unclear | 18.25 | 9.1e+2 | NVIDIA A100 SXM4 80 GB | 2,880.00 | United States of America | IBM | 2022-05-30 | Private | 2.45 | 6.5e+7 | IBM | A100 | Vela, which has been online since May of last year, consists of 60 racks (per Forbes) and an unspecified number of nodes; however, one might be inclined to trust the above diagram – accurate in all other respects – and guess six nodes per rack, for a total of 360 nodes and 2,880 A100 GPUs. | 2022-05-30 | Number of chips is an estimate | 11.56 | 2,880.00 | NVIDIA | 17.95 | 1.8e+18 | 1.8e+18 | 9.0e+17 | 4.5e+17 | 2.45 | 3.7e+11 | 6.5e+7 | https://web.archive.org/web/20230320230907/https://www.hpcwire.com/2023/02/08/ibm-introduces-vela-cloud-ai-supercomputer-powered-by-intel-nvidia/ | 39.38 | -97.92 | ||||||||||||||||||||||||
| Paper on PaLM | Existing | Confirmed | Yes | 18.23 | 8.5e+2 | Google TPU v4 | 6,144.00 | United States of America | 2022-04-05 | Private | 2.22 | 5.6e+7 | TPUv4 | "We trained PaLM on 6144 TPU v4 chips using Pathways," | 2022-04-05 | 11.88 | 6,144.00 | True | 15 | 18.23 | 1.7e+18 | 1.7e+18 | 1.7e+18 | 2.22 | 7.6e+11 | 5.6e+7 | DeepSeek Fire-Flyer 2 | 0.27 | https://arxiv.org/abs/2204.02311 | 39.38 | -97.92 | |||||||||||||||||||||||||
| Microsoft Ares/Maia | Existing | Likely | Yes | 18.21 | 8.3e+2 | Maia 100 (M100) | 2,048.00 | United States of America | Microsoft | 2023-11-15 | Private | 2.12 | Cloud | Maia 100 (aka Athena aka M100) | The largest individual deployment for the backend network for Maia is 2,048, but there is nothing preventing them from scaling this up further. | 2023-11-15 | 11.89 | 2,048.00 | Microsoft | True | 51 | 18.21 | 1.6e+18 | 1.6e+18 | 2.12 | 7.7e+11 | Microsoft Azure Eagle | 0.06 | https://archive.ph/Iigef | https://web.archive.org/web/20250211085121/https://semianalysis.com/2023/11/15/microsoft-infrastructure-ai-and-cpu/ | 39.38 | -97.92 | ||||||||||||||||||||||||
| Iris Energy Prince George cluster | Existing | Confirmed | Yes | 18.21 | 8.2e+2 | NVIDIA H100 SXM5 80GB | 816.00 | Canada | IREN | 2024-04-30 | Private | 1.16 | 2.9e+7 | Prince George, Canada | Cloud | H100 | "AI Cloud Services offered by Iris Energy cater to AI customers, providing cloud compute capabilities with 816 NVIDIA H100 GPUs. This service has been operational since 2024 and is characterized by a performance-focused technology stack." | 2024-04-30 | 11.84 | 816.00 | NVIDIA | True | 77 | 17.91 | 1.6e+18 | 1.6e+18 | 8.1e+17 | 4.0e+17 | 1.16 | 6.9e+11 | 2.9e+7 | Meta GenAI 2024a | 0.03 | https://www.globenewswire.com/en/news-release/2023/12/21/2799869/0/en/10-EH-s-Equity-Raising-Program-Complete.html | https://irisenergy.gcs-web.com/news-releases/news-release-details/iris-energy-purchases-nvidia-h100-gpus-target-generative-ai | https://iris-energy.co/blog/iris-energy-expands-partnership-with-poolside-for-ai-cloud-services/428 | 53.92 | -122.75 | ||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Unclear | 18.30 | 8.0e+2 | China | Private | China | Anonymized | Anonymized | 18.30 | 2.0e+18 | 2.0e+18 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.30 | 8.0e+2 | China | 2024-01-15 | Public/Private | China | Anonymized | Anonymized | True | 56 | 18.30 | 2.0e+18 | 2.0e+18 | Microsoft Azure Eagle | 0.05 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Argonne NL Polaris | Existing | Confirmed | Yes | 18.15 | 7.1e+2 | NVIDIA A100 | 2,240.00 | United States of America | US Department of Energy | 2021-11-01 | Public | 1.92 | 7.6e+7 | 9700 S Cass Ave, Lemont, IL 60439 | Argonne National Laboratory and general scientific community | A100 | That includes a total of 560 AMD Epyc Milan CPUs and 2,240 Nvidia 40GB A100 GPUs, connected by HPE's Slingshot networking | 2021-11-01 | From gov website | 11.56 | 2,240.00 | NVIDIA | True | 13 | 17.84 | 1.4e+18 | 1.4e+18 | 7.0e+17 | 3.5e+17 | 1.92 | 3.6e+11 | 7.6e+7 | DeepSeek Fire-Flyer 2 | 0.22 | https://web.archive.org/web/20230512044406/https://www.hpcwire.com/2022/08/09/argonne-deploys-polaris-supercomputer-for-science-in-advance-of-aurora/ | https://web.archive.org/web/20240530041315/https://www.top500.org/system/180016/ | https://web.archive.org/web/20240228132238/https://www.datacenterdynamics.com/en/news/argonnes-44-petaflops-polaris-supercomputer-goes-live/ | 41.72 | -87.98 | |||||||||||||||||
| NAVER Corp Sejong | Existing | Confirmed | Yes | 18.15 | 7.1e+2 | NVIDIA A100 | 2,240.00 | Korea (Republic of) | NAVER | 2023-11-15 | Private | 1.86 | 4.4e+7 | 824, Haengbok-daero, Sejong-si, South Korea | NAVER,Cloud | A100 | Calculated from Top500 | 2023-11-15 | 11.58 | 2,240.00 | NVIDIA | True | 52 | 17.84 | 1.4e+18 | 1.4e+18 | 7.0e+17 | 3.5e+17 | 1.86 | 3.8e+11 | 4.4e+7 | Microsoft Azure Eagle | 0.05 | https://web.archive.org/web/20241109103411/https://www.top500.org/system/180222/ | 36.48 | 127.31 | ||||||||||||||||||||
| NVIDIA Selene Phase 1 | Existing | Confirmed | Yes | 18.15 | 7.1e+2 | NVIDIA A100 | 2,240.00 | United States of America | NVIDIA | 2020-06-22 | Private | 2.41 | 6.2e+7 | Santa Clara, California | NVIDIA | A100 | Altogether, Selene comprises 280 DGX A100s, housing a total 2,240 A100 GPUs and 494 Mellanox Quantum 200G InfiniBand switches, providing 56 TB/s network fabric | 2020-06-22 | 11.46 | NVIDIA Selene Phase 2 | 2,240.00 | NVIDIA | True | 4 | 17.84 | 1.4e+18 | 1.4e+18 | 7.0e+17 | 3.5e+17 | 1.96 | 2.41 | 2.9e+11 | 6.2e+7 | Oak Ridge NL Summit | 0.40 | https://web.archive.org/web/20240406153716/https://www.hpcwire.com/2020/06/22/nvidia-nabs-7-spot-on-top500-with-selene-launches-a100-pcie-cards/ | https://web.archive.org/web/20241229072511/https://top500.org/system/179842/ | 37.35 | -121.96 | |||||||||||||||||
| Microsoft Azure Voyager-EUS2 | Existing | Confirmed | Yes | 18.12 | 6.7e+2 | NVIDIA A100 | 2,112.00 | United States of America | Microsoft | 2021-11-01 | Private | 1.81 | 7.2e+7 | 101 Herbert Dr, Boydton, VA 23917 | Cloud,Azure | A100 | Calculated from Top500 | 2021-11-01 | Listed in Top500 | 11.56 | 2,112.00 | NVIDIA | True | 15 | 17.82 | 1.3e+18 | 1.3e+18 | 6.6e+17 | 3.3e+17 | 1.81 | 3.6e+11 | 7.2e+7 | DeepSeek Fire-Flyer 2 | 0.21 | https://web.archive.org/web/20240716100550/https://www.zdnet.com/article/microsoft-now-has-one-of-the-worlds-fastest-supercomputers-and-no-it-doesnt-run-on-windows/ | https://web.archive.org/web/20240808114950/https://www.top500.org/system/180024/ | 36.68 | -78.38 | ||||||||||||||||||
| Paper on LLaMA | Existing | Confirmed | Yes | 18.11 | 6.5e+2 | NVIDIA A100 SXM4 80 GB | 2,048.00 | Meta AI | 2023-02-27 | Private | 1.70 | 4.6e+7 | Meta | A100 | "Finally, we estimate that we used 2048 A100-80GB for a period of approximately 5 months to develop our models." | 2023-02-27 | 11.58 | True | Meta Research SuperCluster (RSC-1) Phase 1 | 2,048.00 | NVIDIA | 30 | 17.81 | 1.3e+18 | 1.3e+18 | 6.4e+17 | 3.2e+17 | 1.70 | 3.8e+11 | 4.6e+7 | Microsoft GPT-4 cluster | 0.08 | https://arxiv.org/abs/2302.13971 | |||||||||||||||||||||||
| Softbank SuperPOD | Existing | Confirmed | Yes | 18.11 | 6.5e+2 | NVIDIA A100 | 2,048.00 | Japan | Softbank | 2023-09-30 | Estimated "over 2,000" as 2,048 | Private | 1.70 | 4.0e+7 | Japan | Softbank | A100 | This Japan top-level*2 computing platform comprises an NVIDIA DGX SuperPOD(Open in a new window)™ AI supercomputer with over 2,000 NVIDIA Tensor Core GPUs(Open in a new window), NVIDIA Networking(Open in a new window) and NVIDIA AI Enterprise(Open in a new window) software | 2023-09-30 | 11.58 | 2,048.00 | NVIDIA | True | 42 | 17.81 | 1.3e+18 | 1.3e+18 | 6.4e+17 | 3.2e+17 | 1.70 | 3.8e+11 | 4.0e+7 | Tesla 10k H100 Cluster | 0.06 | https://web.archive.org/web/20240623042927/https://www.softbank.jp/en/corp/news/press/sbkk/2023/20231031_01/ | 36.39 | 138.59 | |||||||||||||||||||
| Petrobras Pegasus (Pégaso) | Existing | Confirmed | Yes | 18.10 | 6.4e+2 | NVIDIA A100 | 2,016.00 | Brazil | Petrobras | 2022-11-01 | Public/Private | 1.80 | 4.6e+7 | Rio de Janeiro, Brazil | Petrobras | A100 | With 21 petaflops of power, 678TB of RAM memory, 2,016 GPUs, and a 400GB/s Internet link, Pegasus will be the most powerful supercomputer in Latin America. | 2022-11-01 | 11.54 | 2,016.00 | NVIDIA | True | 28 | 17.80 | 1.3e+18 | 1.3e+18 | 6.3e+17 | 3.1e+17 | 1.72 | 1.80 | 3.5e+11 | 4.6e+7 | Microsoft GPT-4 cluster | 0.08 | https://web.archive.org/web/20240423211648/https://www.datacenterdynamics.com/en/news/petrobras-announces-new-pegasus-supercomputer-in-rio-de-janeiro/ | -22.91 | -43.21 | |||||||||||||||||||
| Azure OpenAI GPT-3 Cluster | Existing | Likely | Yes | 18.10 | 6.3e+2 | NVIDIA V100 | 10,000.00 | United States of America | Microsoft,OpenAI | 2020-04-15 | It has "since been upgraded to include tens of thousands of A100 chips". Rumors suggest that it is in Washington State | Private | 6.55 | 1.9e+8 | 1515 Port Industrial Way, Quincy, WA 98848 | OpenAI | V100 | "The supercomputer developed for OpenAI is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server. Compared with other machines listed on the TOP500 supercomputers in the world, it ranks in the top five, Microsoft says."
"We understand the supercomputer uses AMD second-generation Rome Epyc server processors, and Nvidia V100 GPUs, from sources in the industry." | 2020-04 | 11.28 | 10,000.00 | NVIDIA | True | 4 | 18.10 | 1.3e+18 | 1.3e+18 | 1.6e+17 | 6.55 | 1.9e+11 | 1.9e+8 | True | Oak Ridge NL Summit | 0.36 | https://web.archive.org/web/20240902024137/https://news.microsoft.com/source/features/ai/openai-azure-supercomputer/ | https://web.archive.org/web/20240307163254/https://www.theregister.com/2020/05/20/microsoft_openai_supercomputer/ | 47.24 | -119.88 | ||||||||||||||||||
| ExxonMobil Discovery 5 | Existing | Confirmed | Yes | 18.10 | 6.3e+2 | NVIDIA A100 SXM4 40 GB | 2,000.00 | United States of America | ExxonMobil | 2022-11-01 | Private | 1.68 | 4.4e+7 | 22777 Springwoods Village Pkwy, Spring, TX 77389 | ExxonMobil | A100 | Calculated from Top500 | 2022-11-01 | 11.57 | 2,000.00 | NVIDIA | True | 30 | 17.80 | 1.2e+18 | 1.2e+18 | 6.2e+17 | 3.1e+17 | 1.70 | 1.68 | 3.7e+11 | 4.4e+7 | Microsoft GPT-4 cluster | 0.08 | https://web.archive.org/web/20241109060139/https://www.top500.org/system/180131/ | 30.10 | -95.45 | |||||||||||||||||||
| ND A100 v4 | Existing | Likely | Yes | 18.10 | 6.3e+2 | NVIDIA A100 | 2,000.00 | United States of America | Microsoft | 2021-06-01 | They ran the high performance LINPACK benchmark on a pool of 164 8xA100 VMs, so the clusters can include at least 1312 A100s. This might be a slice of a larger cluster.
Estimating as 2k since it says "thousands" | Private | 1.72 | 5.4e+7 | Cloud | A100 | "The ND A100 v4 series starts with a single VM and eight NVIDIA Ampere A100 40GB Tensor Core GPUs. ND A100 v4-based deployments can scale up to thousands of GPUs" | 2021-06-01 | 11.56 | 2,000.00 | NVIDIA | True | 12 | 17.80 | 1.2e+18 | 1.2e+18 | 6.2e+17 | 3.1e+17 | 1.72 | 3.6e+11 | 5.4e+7 | Sunway OceanLight | 0.21 | https://web.archive.org/web/20241114112532/https://azure.microsoft.com/de-de/blog/azure-announces-general-availability-of-scaleup-scaleout-nvidia-a100-gpu-instances-claims-title-of-fastest-public-cloud-super/ | https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/gpu-accelerated/ndasra100v4-series?tabs=sizebasic | 39.38 | -97.92 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.00 | 6.0e+2 | 5,000.00 | China | 2019-12-15 | Public | 3.00 | China | 11.37 | 4,000 | 9,000.00 | Anonymized | Anonymized | True | 4 | 17.85 | 1.0e+18 | 1.0e+18 | 7.0e+17 | 3.00 | 2.3e+11 | Oak Ridge NL Summit | 0.30 | 35.49 | 101.90 | ||||||||||||||||||||||||||||
| Corvex B200s | Planned | Likely | Yes | 18.06 | 5.8e+2 | NVIDIA B200 | 256.00 | United States of America | Corvex.ai | 0.51 | Delaware | Planned Q2 2025 | 12.05 | 256.00 | NVIDIA | 17.76 | 1.2e+18 | 1.2e+18 | 5.8e+17 | 2.9e+17 | 0.51 | 1.1e+12 | Private correspondence with Corvex.ai staff | 39.00 | -75.51 | |||||||||||||||||||||||||||||||
| Google TPU v4 Pod | Existing | Confirmed | Yes | 18.05 | 5.7e+2 | Google TPU v4 | 4,096.00 | United States of America | 2021-05-19 | Used in July 2021 MLPerf benchmark, mentioned May 2021, likely operational beforehand | Private | 1.50 | 3.8e+7 | TPUv4 | "Google deploys the chips in its own data centers, combining them in pods of 4,096 TPUs"
"TPU v3. 4,096 of these TPU v4 chips are networked together to create a TPU v4 Pod" | 2021-05-19 | 11.88 | 4,096.00 | True | 13 | 18.05 | 1.1e+18 | 1.1e+18 | 1.1e+18 | 1.50 | 7.5e+11 | 3.8e+7 | Sunway OceanLight | 0.19 | https://web.archive.org/web/20250221082528/https://cloud.google.com/blog/products/ai-machine-learning/google-wins-mlperf-benchmarks-with-tpu-v4 | https://web.archive.org/web/20241005134423/https://www.datacenterdynamics.com/en/news/google-unveils-fourth-generation-tpu-ai-chip/ | 39.38 | -97.92 | |||||||||||||||||||||||
| Paper on Gemma 2 9B | Existing | Confirmed | Yes | 18.05 | 5.7e+2 | Google TPU v4 | 4,096.00 | United States of America | 2024-06-27 | Private | 1.42 | 3.7e+7 | TPUv4 | "For the 9B model, we train on an 8x16x32 configuration of TPUv4, totaling 4096 chips," | 2024-06-27 | 11.90 | True | Google Oklahoma TPU v4 Pods | 4,096.00 | 105 | 18.05 | 1.1e+18 | 1.1e+18 | 1.1e+18 | 1.42 | 7.9e+11 | 3.7e+7 | Meta GenAI 2024a | 0.02 | https://storage.googleapis.com/deepmind-media/gemma/gemma-2-report.pdf | 39.38 | -97.92 | ||||||||||||||||||||||||
| NVIDIA Taipei-1 | Existing | Confirmed | Yes | 18.04 | 5.6e+2 | NVIDIA H100 SXM5 80GB | 512.00 | Taiwan | NVIDIA | 2024-06-01 | Private | 0.89 | 1.8e+7 | Kaohsiung Software Technology Park
No. 10號, Fusing 4th Rd, Cianjhen District, Kaohsiung City, Taiwan 806 | Cloud,NVIDIA | H100,L40 | It will include 64 DGX H100 systems – each of which includes eight H100 GPUs (for a total of 512) – and 64 Omniverse-focused OVX systems, each of which includes four L40 GPUs and a BlueField-3 DPU...
the system is aimed at wide-ranging use cases like large language models (LLMs), healthcare, robotics, industrial digital twins and – like Taiwania 4 – climate science | 2024-06-01 | Listed as operational in Top500, details of system confirmed earlier | 11.79 | NVIDIA L40 | 256 | 768.00 | NVIDIA | NVIDIA | True | 98 | 17.74 | 1.1e+18 | 1.1e+18 | 5.5e+17 | 2.8e+17 | 0.89 | 6.2e+11 | 1.8e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240423000708/https://www.hpcwire.com/2023/05/29/nvidia-announces-four-supercomputers-with-two-in-taiwan/ | 22.60 | 120.30 | ||||||||||||||||
| Samsung SSC-21 | Existing | Confirmed | Yes | 18.04 | 5.5e+2 | NVIDIA A100 | 1,760.00 | Korea (Republic of) | Samsung | 2021-11-01 | Private | 1.51 | 6.0e+7 | Korea (Republic of) | Samsung | A100 | Calculated from Top500 | 2021-11-01 | Listed in Top500 | 11.56 | 1,760.00 | NVIDIA | True | 20 | 17.74 | 1.1e+18 | 1.1e+18 | 5.5e+17 | 2.7e+17 | 1.51 | 3.6e+11 | 6.0e+7 | DeepSeek Fire-Flyer 2 | 0.18 | https://web.archive.org/web/20240117215326/https://www.kedglobal.com/artificial-intelligence/newsView/ked202305310010#:~:text=Samsung%20Electronics%20Co.%2C%20the%20world's,with%201.19%20exaflops%20of%20performance. | https://web.archive.org/web/20240717134459/https://www.top500.org/system/180041/ | ||||||||||||||||||||
| Tesla Auto-Labeling Cluster | Existing | Likely | Yes | 18.04 | 5.5e+2 | NVIDIA A100 | 1,752.00 | United States of America | Tesla | 2021-08-20 | I'm assuming these are A100s because H100s weren't out yet at the time and that is what the other cluster mentioned has | Private | 1.51 | 6.0e+7 | Tesla | A100 | as well as the previously mentioned 5670 system which is used for training, it has a second, 4032 GPU system for training and a 1752 GPU system for auto-labeling | 2021-08-20 | Number of GPUs officially announced, but no other details about it released | 11.56 | 1,752.00 | NVIDIA | True | 18 | 17.74 | 1.1e+18 | 1.1e+18 | 5.5e+17 | 2.7e+17 | 1.51 | 3.6e+11 | 6.0e+7 | DeepSeek Fire-Flyer 2 | 0.18 | https://web.archive.org/web/20240221025227/https://www.datacenterdynamics.com/en/news/tesla-details-dojo-supercomputer-reveals-dojo-d1-chip-and-training-tile-module/ | 39.38 | -97.92 | |||||||||||||||||||
| Tesla Dojo 1 Phase 1 | Planned | Confirmed | Yes | 18.04 | 5.5e+2 | Tesla D1 Dojo | 3,000.00 | United States of America | Tesla | Private | 2.40 | San Jose, California | Tesla,but "“It's probably going to make more sense to have Dojo operate in like an Amazon Web Services manner than to try to sell it to someone else | Tesla Dojo D1 | Once complete, the entire system will house around 120 tiles, holding 3,000 custom D1 chips. | Planned | Musk plans for a scaled up version of this cluster to be operational by the end of 2024. Unclear if the smaller version ever ran | 11.66 | 3,000.00 | Tesla | 18.04 | 1.1e+18 | 1.1e+18 | 6.8e+16 | 2.40 | 4.5e+11 | $1B | https://web.archive.org/web/20241208081438/https://www.datacenterdynamics.com/en/news/tesla-begins-installing-dojo-supercomputer-cabinets-trips-local-substation/ | https://web.archive.org/web/20240903213050/https://techcrunch.com/2024/08/10/teslas-dojo-a-timeline/ | https://web.archive.org/web/20250125054824/https://dgtlinfra.com/elon-musk-data-centers/ | 37.34 | -121.89 | ||||||||||||||||||||||||
| DeepL Mercury | Existing | Confirmed | Yes | 18.03 | 5.4e+2 | NVIDIA H100 SXM5 80GB | 544.00 | Sweden | DeepL | 2023-08-02 | Private | 0.79 | 2.0e+7 | Slaggvarpsvägen 21, 791 77 Falun, Sweden | DeepL | H100 | DeepL has added a DGX SuperPOD with 68 NVIDIA DGX H100 systems which are supplied by DELTA Computer Products | 2023-08-02 | 11.83 | 544.00 | NVIDIA | True | 46 | 17.73 | 1.1e+18 | 1.1e+18 | 5.4e+17 | 2.7e+17 | 0.79 | 6.8e+11 | 2.0e+7 | Microsoft GPT-4 cluster | 0.07 | https://web.archive.org/web/20241120225551/https://digests.digitalisationworld.com/news/65928/deepl-deploys-largest-nvidia-dgx-h100-superpod-in-europe | 60.61 | 15.61 | ||||||||||||||||||||
| Paper on AlphaCode | Existing | Confirmed | Yes | 18.01 | 5.2e+2 | Google TPU v4 | 3,750.00 | United States of America | 2022-02-08 | Private | 1.36 | 3.5e+7 | TPUv4 | 2022-02-08 | 11.88 | True | Google TPU v4 Pod | 3,750.00 | 24 | 18.01 | 1.0e+18 | 1.0e+18 | 1.0e+18 | 1.36 | 7.6e+11 | 3.5e+7 | DeepSeek Fire-Flyer 2 | 0.17 | https://arxiv.org/abs/2203.07814 | 39.38 | -97.92 | |||||||||||||||||||||||||
| Paper on Phi-3.5-Mini | Existing | Confirmed | Yes | 18.01 | 5.1e+2 | NVIDIA H100 SXM5 80GB | 512.00 | Microsoft | 2024-04-22 | Private | 0.73 | 1.8e+7 | Cloud | H100 | 2024-04-22 | 11.84 | True | Microsoft Azure Eagle | 512.00 | NVIDIA | 94 | 17.70 | 1.0e+18 | 1.0e+18 | 5.1e+17 | 2.5e+17 | 0.73 | 6.9e+11 | 1.8e+7 | Meta GenAI 2024a | 0.02 | https://arxiv.org/abs/2404.14219 | ||||||||||||||||||||||||
| Lawrence Livermore NL El Capitan Phase 1 | Existing | Confirmed | Yes | 18.00 | 5.1e+2 | AMD Instinct MI300A | 512.00 | United States of America | US Department of Energy | 2024-06-01 | Public | 0.79 | 2.1e+7 | Lawrence Livermore National Laboratory
7000 East Ave, Livermore, CA 94550 | US Government,Lawrence Livermore NL,Scientific Research | AMD MI300A | Calculated from Top500 | 2024-06-01 | From Top500 | 11.80 | Lawrence Livermore NL El Capitan Phase 2 | 512.00 | AMD | True | 104 | 17.70 | 1.0e+18 | 1.0e+18 | 5.0e+17 | 2.5e+17 | 0.79 | 6.3e+11 | 2.1e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240406154137/https://www.hpcwire.com/2022/06/21/amds-mi300-apus-to-power-exascale-el-capitan-supercomputer/ | https://web.archive.org/web/20240517002417/https://en.wikipedia.org/wiki/El_Capitan_(supercomputer) | https://web.archive.org/web/20240806222453/https://www.nextplatform.com/2023/07/10/lining-up-the-el-capitan-supercomputer-against-the-ai-upstarts/ | https://web.archive.org/web/20240611014231/https://top500.org/system/180283/ | 37.68 | -121.77 | |||||||||||||||
| Lawrence Livermore NL RZAdams | Existing | Confirmed | Yes | 18.00 | 5.1e+2 | AMD Instinct MI300A | 512.00 | United States of America | US Department of Energy | 2024-06-01 | Public | 0.79 | 2.1e+7 | 7000 East Ave, Livermore, CA 94550 | Lawrence Livermore NL,Researchers | AMD MI300A | Calculated from Top500 | 2024-06-01 | 11.80 | 512.00 | AMD | True | 104 | 17.70 | 1.0e+18 | 1.0e+18 | 5.0e+17 | 2.5e+17 | 0.79 | 6.3e+11 | 2.1e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20250121085639/https://top500.org/system/180284/ | 37.68 | -121.71 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | China | 2023-01-15 | Private | China | Anonymized | Anonymized | True | 39 | 18.00 | 1.0e+18 | 1.0e+18 | Microsoft GPT-4 cluster | 0.06 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 18.00 | 5.0e+2 | Hong Kong | Private | Hong Kong | Anonymized | Anonymized | 18.00 | 1.0e+18 | 1.0e+18 | 22.28 | 114.16 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | 400.00 | China | 2021-08-15 | Public | 0.30 | 1.0e+7 | China | 11.52 | 2,000.00 | Anonymized | Anonymized | True | 17 | 17.00 | 1.0e+18 | 1.0e+18 | 1.0e+17 | 6.0e+16 | 0.30 | 3.3e+11 | 1.0e+7 | DeepSeek Fire-Flyer 2 | 0.20 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Ahrefs Yep1 | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | NVIDIA H100 SXM5 80GB | 504.00 | United States of America | Ahrefs | 2024-06-01 | Private | 0.72 | 1.8e+7 | Ashburn, Virginia | Ahrefs | H100 | Calculated from Top500 | 2024-06-01 | 11.84 | 504.00 | NVIDIA | True | 108 | 17.70 | 1.0e+18 | 1.0e+18 | 5.0e+17 | 2.5e+17 | 0.72 | 6.9e+11 | 1.8e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20250212094644/https://top500.org/system/180265/ | 39.04 | -77.49 | ||||||||||||||||||||
| Recursion BioHive-2 | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | NVIDIA H100 SXM5 80GB | 504.00 | United States of America | Recursion Pharmaceuticals | 2024-05-13 | Private | 0.72 | 1.8e+7 | 41 400 W, Salt Lake City, UT 84101 | Recursion Pharmaceuticals,Pharmaceutical Research | H100 | BioHive-2 is equipped with 504 Nvidia H100 Tensor Core GPUs | 2024-05-13 | 11.84 | 504.00 | NVIDIA | True | 100 | 17.70 | 1.0e+18 | 1.0e+18 | 5.0e+17 | 2.5e+17 | 0.72 | 6.9e+11 | 1.8e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240621074430/https://the-decoder.com/biohive-2-to-speed-up-drug-development-by-several-years/ | 40.77 | -111.90 | ||||||||||||||||||||
| Yandex Chervonenkis | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | NVIDIA A100 SXM4 80 GB | 1,592.00 | Russia | Yandex | 2021-11-01 | Private | 1.37 | 6.5e+7 | ул. Пушкина, 21, Sasovo, Ryazan Oblast, Russia, 391431 | Yandex | A100 | GPU 1,592 NVIDIA A100 80G | 2021-11-01 | 11.56 | 1,592.00 | NVIDIA | True | 23 | 17.70 | 9.9e+17 | 9.9e+17 | 5.0e+17 | 2.5e+17 | 1.37 | 3.6e+11 | 6.5e+7 | DeepSeek Fire-Flyer 2 | 0.16 | https://web.archive.org/web/20231115155013/https://yandex.com/supercomputers | 54.37 | 41.94 | ||||||||||||||||||||
| BNY Mellon Supercomputer | Existing | Likely | Yes | 18.00 | 5.0e+2 | NVIDIA H100 SXM5 80GB | 500.00 | BNY Mellon | 2024-03-18 | "Dozens" probably means at least 36 and less than 100. Multiply this by 8 since it's DGXs. I estimated around 500 GPUs for this | Private | 0.71 | 1.8e+7 | BNY Mellon | H100 | The system, equipped with dozens of NVIDIA DGX systems and NVIDIA InfiniBand networking | 2024-03-18 | Very uncertain about exact number | 11.84 | 500.00 | NVIDIA | True | 85 | 17.69 | 9.9e+17 | 9.9e+17 | 4.9e+17 | 2.5e+17 | 0.71 | 6.9e+11 | 1.8e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240725023803/https://blogs.nvidia.com/blog/bny-mellon-superpod/ | ||||||||||||||||||||||
| Saudi Aramco Dammam-7 | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | NVIDIA Tesla V100 SXM2 | 7,912.00 | Saudi Arabia | Saudi Aramco | 2021-01-21 | Private | 5.10 | 1.6e+8 | Dhahran, Saudi Arabia | Saudi Aramco | V100 | Calculated from Top500 | 2021-01-21 | Listed in Top500 | 11.29 | 7,912.00 | NVIDIA | True | 10 | 18.00 | 9.9e+17 | 9.9e+17 | 1.2e+17 | 5.10 | 1.9e+11 | 1.6e+8 | Oak Ridge NL Summit | 0.29 | https://web.archive.org/web/20240823132655/https://www.hpcwire.com/2021/01/21/saudi-aramco-unveils-dammam-7-its-new-top-ten-supercomputer/ | https://web.archive.org/web/20240423152513/https://www.top500.org/system/179885/ | 24.78 | 46.65 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | 2,000.00 | China | 2021-12-15 | Private | 1.00 | China | 11.70 | True | 2,000.00 | Anonymized | Anonymized | 25 | 17.70 | 1.0e+18 | 1.0e+18 | 5.0e+17 | 1.00 | 5.0e+11 | DeepSeek Fire-Flyer 2 | 0.20 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 18.00 | 5.0e+2 | 40,000.00 | China | 2020-11-15 | Public | 30.00 | China | 10.52 | 40,000.00 | Anonymized | Anonymized | True | 8 | 18.00 | 1.0e+18 | 1.0e+18 | 30.00 | 3.3e+10 | Oak Ridge NL Summit | 0.30 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||
| Condor Galaxy 3 (CG-3) | Planned | Confirmed | Yes | 17.98 | 4.9e+2 | Cerebras CS-3 | 64.00 | United States of America | G42,Cerebras Systems | Third of 9 planned supercomputers that will be interconnected
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.6e+8 | Dallas, Texas | Cloud | Cerebras CS3 | "• 8 ExaFLOPs
• 64 x CS-3s
• 108 TB of Memory"
"CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs"
"Upon completion in 2024, the nine inter-connected supercomputers will have 36 ExaFLOPS of AI compute, making it one of the most powerful cloud AI supercomputers in the world." | Planned | Reported on their official website | 64.00 | Cerebras | 17.98 | 9.6e+17 | 9.6e+17 | 2.6e+8 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | 32.78 | -96.80 | ||||||||||||||||||||||||||
| AGH Cyfronet Helios | Existing | Confirmed | Yes | 17.96 | 4.6e+2 | NVIDIA GH200 | 440.00 | Poland | AGH University of Krakow | 2024-06-01 | Public | 0.53 | 2.1e+7 | Nawojki 11, 30-950 Kraków, Poland | AGH University of Krakow,Researchers in Poland,Cloud | GH200,H100 | The GPU partition contains 440 Nvidia GH200 Grace Hopper Superchips and the final INT partition is equipped with 24 Nvidia H100 Tensor Core GPUs and high-speed NVMe local storage. | 2024-06-01 | 11.94 | NVIDIA H100 SXM5 80GB | 24 | 464.00 | NVIDIA | NVIDIA | True | 115 | 17.66 | 9.2e+17 | 9.2e+17 | 4.6e+17 | 2.3e+17 | 0.66 | 0.53 | 8.7e+11 | 2.1e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240514095456/https://www.datacenterdynamics.com/en/news/hpe-to-build-new-35-petaflop-supercomputer-in-poland/ | https://web.archive.org/web/20240527154516/https://www.cyfronet.pl/en/news/19869,42,komunikat,helios__athena_and_ares_-_3_supercomputers_from_cyfronet_among_the_fastest_and_most_energy-efficient_computers_in_the_world.html | https://web.archive.org/web/20240806125635/https://top500.org/system/180244/ | 50.07 | 19.91 | ||||||||||||||
| Eni HPC5 | Existing | Confirmed | Yes | 17.96 | 4.6e+2 | NVIDIA V100 | 7,280.00 | Italy | Eni | 2020-02-21 | Private | 4.03 | 1.4e+8 | Via per la Corradina, 35, 27032 Ferrera Erbognone PV, Italy | Eni | V100 | In total, the system comprises 7,280 NVIDIA V100 GPUs. | 2020-02-21 | 11.35 | 7,280.00 | NVIDIA | True | 5 | 17.96 | 9.1e+17 | 9.1e+17 | 1.1e+17 | 4.77 | 4.03 | 2.3e+11 | 1.4e+8 | Oak Ridge NL Summit | 0.26 | https://web.archive.org/web/20231120074252/https://energyindustryreview.com/tech/new-supercomputing-system-hpc5/ | https://web.archive.org/web/20240906185041/https://en.wikipedia.org/wiki/HPC5 | 45.10 | 8.86 | |||||||||||||||||||
| CSIRO Virga | Existing | Confirmed | Yes | 17.95 | 4.5e+2 | NVIDIA H100 SXM5 80GB | 448.00 | Australia | CSIRO | 2024-07-11 | Public | 0.64 | 1.1e+7 | 54 Sheppard St, Hume ACT 2620, Australia | Researchers,Academia,Healthcare | H100 | There are 448 H100 GPUs in total in Virga | 2024-07-11 | 11.84 | 448.00 | NVIDIA | True | 130 | 17.65 | 8.9e+17 | 8.9e+17 | 4.4e+17 | 2.2e+17 | 0.64 | 6.9e+11 | 1.6e+7 | 1.1e+7 | 1.1e+7 | "The system actually cost AU$16.3m (US$10.85m)." | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240720195802/https://blocksandfiles.com/2024/07/03/australia-dell-based-virga-ai-workload-cluster/ | https://www.datacenterdynamics.com/en/news/australias-csiro-unveils-virga-supercomputer/ | -35.39 | 149.17 | ||||||||||||||||
| SURF Snellius Phase 3 | Existing | Confirmed | Yes | 17.94 | 4.4e+2 | NVIDIA H100 SXM5 80GB | 352.00 | Netherlands | SURF | 2024-07-11 | Public | 0.74 | 1.8e+7 | Science Park 120, 1098 XG Amsterdam, Netherlands | Academia,Researchers | H100,A100 | The expansion includes the addition of 352 NVIDIA H100 (Hopper) GPUs | 2024-07-11 | 11.77 | SURF Snellius Phase 1 | NVIDIA A100 SXM4 80 GB | 288 | 640.00 | NVIDIA | NVIDIA | True | 131 | 17.64 | 8.8e+17 | 8.8e+17 | 4.4e+17 | 2.2e+17 | 0.74 | 5.9e+11 | 1.8e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240712185633/https://www.surf.nl/en/news/gpu-expansion-of-supercomputer-snellius-enables-even-faster-data-processing-for-dutch | https://web.archive.org/web/20240725104223/https://visualization.surf.nl/snellius-virtual-tour/#/ | 52.36 | 4.95 | |||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.94 | 4.0e+2 | 400.00 | Hong Kong | 2024-04-15 | Public | 0.60 | 2.0e+7 | Hong Kong | 11.82 | 400.00 | Anonymized | Anonymized | True | 104 | 17.60 | 8.7e+17 | 8.7e+17 | 4.0e+17 | 2.0e+17 | 0.60 | 6.7e+11 | 2.0e+7 | Meta GenAI 2024a | 0.02 | 22.28 | 114.16 | ||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.90 | 4.0e+2 | China | 2024-03-15 | Public/Private | China | Anonymized | Anonymized | True | 91 | 17.90 | 8.0e+17 | 8.0e+17 | Meta GenAI 2024a | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Saudi Aramco Tuwaiq-1 | Existing | Confirmed | Yes | 17.92 | 4.2e+2 | NVIDIA H100 SXM5 80GB | 416.00 | Saudi Arabia | Saudi Aramco | 2024-06-01 | Private | 0.59 | 1.5e+7 | Saudi Arabia | Saudi Aramco | H100 | Calculated from Top500 | 2024-06-01 | 11.84 | 416.00 | NVIDIA | True | 119 | 17.61 | 8.2e+17 | 8.2e+17 | 4.1e+17 | 2.1e+17 | 0.59 | 6.9e+11 | 1.5e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20241205003221/https://top500.org/system/180260/ | 23.38 | 44.65 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.90 | 4.0e+2 | 2,000.00 | China | 2023-05-15 | Public/Private | 1.00 | China | 11.60 | 2,000.00 | Anonymized | Anonymized | True | 49 | 17.60 | 8.0e+17 | 8.0e+17 | 4.0e+17 | 1.00 | 4.0e+11 | Microsoft GPT-4 cluster | 0.05 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Microsoft Azure Pioneer-EUS | Existing | Confirmed | Yes | 17.91 | 4.1e+2 | NVIDIA A100 | 1,312.00 | United States of America | Microsoft | 2021-06-01 | Private | 1.13 | 3.5e+7 | Virginia | Cloud,Azure | A100 | Calculated from Top500 | 2021-06-01 | Listed in Top500 | 11.56 | 1,312.00 | NVIDIA | True | 18 | 17.61 | 8.2e+17 | 8.2e+17 | 4.1e+17 | 2.0e+17 | 1.13 | 3.6e+11 | 3.5e+7 | Sunway OceanLight | 0.14 | https://web.archive.org/web/20241109163038/https://www.top500.org/system/179969/ | 37.68 | -78.62 | |||||||||||||||||||
| Microsoft Azure Pioneer-SCUS | Existing | Confirmed | Yes | 17.91 | 4.1e+2 | NVIDIA A100 | 1,312.00 | United States of America | Microsoft | 2021-06-01 | Private | 1.13 | 3.5e+7 | Texas | Cloud,Azure | A100 | Calculated from Top500 | 2021-06-01 | Listed in Top500 | 11.56 | 1,312.00 | NVIDIA | True | 18 | 17.61 | 8.2e+17 | 8.2e+17 | 4.1e+17 | 2.0e+17 | 1.13 | 3.6e+11 | 3.5e+7 | Sunway OceanLight | 0.14 | https://web.archive.org/web/20241109090838/https://www.top500.org/system/179970/ | 31.80 | -98.82 | |||||||||||||||||||
| Microsoft Azure Pioneer-WEU | Existing | Confirmed | Yes | 17.91 | 4.1e+2 | NVIDIA A100 | 1,312.00 | Netherlands | Microsoft | 2021-06-01 | Private | 1.13 | 3.5e+7 | Netherlands | Cloud,Azure | A100 | Calculated from Top500 | 2021-06-01 | Listed in Top500 | 11.56 | 1,312.00 | NVIDIA | True | 18 | 17.61 | 8.2e+17 | 8.2e+17 | 4.1e+17 | 2.0e+17 | 1.13 | 3.6e+11 | 3.5e+7 | Sunway OceanLight | 0.14 | https://web.archive.org/web/20241109142236/https://www.top500.org/system/179968/ | 51.97 | 5.61 | |||||||||||||||||||
| Microsoft Azure Pioneer-WUS2 | Existing | Confirmed | Yes | 17.91 | 4.1e+2 | NVIDIA A100 | 1,312.00 | United States of America | Microsoft | 2021-06-01 | Private | 1.13 | 3.5e+7 | Washington | Cloud,Azure | A100 | Calculated from Top500 | 2021-06-01 | Listed in Top500 | 11.56 | 1,312.00 | NVIDIA | True | 18 | 17.61 | 8.2e+17 | 8.2e+17 | 4.1e+17 | 2.0e+17 | 1.13 | 3.6e+11 | 3.5e+7 | Sunway OceanLight | 0.14 | https://web.archive.org/web/20241109144557/https://www.top500.org/system/179967/ | 48.03 | -120.09 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.90 | 4.0e+2 | China | 2021-07-15 | Public | China | Anonymized | Anonymized | True | 24 | 8.0e+17 | 8.0e+17 | DeepSeek Fire-Flyer 2 | 0.10 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.90 | 4.0e+2 | 30,000.00 | China | 2022-04-15 | Public/Private | China | 30,000.00 | Anonymized | Anonymized | True | 36 | 17.90 | 8.0e+17 | 8.0e+17 | DeepSeek Fire-Flyer 2 | 0.10 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.90 | 4.0e+2 | China | 2018-03-15 | Public | China | Anonymized | Anonymized | True | 2 | 17.90 | 8.0e+17 | 8.0e+17 | Meta 2017 V100 Cluster | 0.30 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.90 | 4.0e+2 | 400.00 | China | 2023-08-15 | 0.60 | 1.0e+7 | China | 11.82 | 400.00 | Anonymized | Anonymized | True | 63 | 17.60 | 8.0e+17 | 8.0e+17 | 4.0e+17 | 2.0e+17 | 0.60 | 6.7e+11 | 1.0e+7 | Microsoft GPT-4 cluster | 0.05 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Princeton Della Phase 2 | Existing | Confirmed | Yes | 17.89 | 4.0e+2 | NVIDIA H100 SXM5 80GB | 296.00 | United States of America | Princeton University | 2024-03-15 | Public | 0.68 | 1.7e+7 | 300 Forrestal Rd, Princeton, NJ 08544 | Academia | H100,A100 | Princeton University has invested in a new cluster of 300 Nvidia H100 GPUs to support AI research at the institution. | 2024-03-15 | 11.76 | Princeton Della Phase 1 | NVIDIA A100 SXM4 80 GB | 316 | 612.00 | NVIDIA | NVIDIA | True | 98 | 17.59 | 7.8e+17 | 7.8e+17 | 3.9e+17 | 2.0e+17 | 0.68 | 5.8e+11 | 1.7e+7 | Meta GenAI 2024a | 0.02 | https://web.archive.org/web/20240320225612/https://www.datacenterdynamics.com/en/news/princeton-university-acquires-300-strong-nvidia-h100-gpu-cluster/ | https://web.archive.org/web/20240521031936/https://researchcomputing.princeton.edu/systems/della | 40.34 | -74.62 | |||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.85 | 4.0e+2 | 1,000.00 | China | 2023-06-15 | 1.00 | 3.0e+7 | China | 11.60 | 1,000.00 | Anonymized | Anonymized | True | 61 | 17.60 | 7.0e+17 | 7.0e+17 | 4.0e+17 | 2.0e+17 | 1.00 | 4.0e+11 | 3.0e+7 | Microsoft GPT-4 cluster | 0.05 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Microsoft Explorer-WUS3 | Existing | Confirmed | Yes | 17.87 | 3.7e+2 | AMD Radeon Instinct MI250X | 1,920.00 | United States of America | Microsoft | 2023-06-01 | Chip count from Top500 | Private | 1.99 | 4.9e+7 | Phoenix, Arizona | Cloud,Azure | AMD MI250X | "The highest ranked new entry to the June 2023 Top500 list is #11 - a #Microsoft #Azure #AI supercomputer with nearly 2000 #AMD MI250X #GPUs and InfiniBand HDR interconnect: https://lnkd.in/e2WrGDGa"
"The cluster was spun up in Azure’s West US3 datacenter, and Microsoft has confirmed it is a permanent system that has or will run real HPC and AI workloads"
Chip count from Top500 | 2023-06-01 | 11.57 | 1,920.00 | AMD | True | 60 | 17.87 | 7.4e+17 | 7.4e+17 | 7.4e+17 | 1.8e+17 | 1.99 | 3.7e+11 | 4.9e+7 | Microsoft GPT-4 cluster | 0.05 | https://web.archive.org/web/20240406153948/https://www.hpcwire.com/2023/05/22/top500-frontier-gains-92-petaflops-henri-gets-a-little-greener/ | https://archive.ph/83Jbr | https://web.archive.org/web/20241109085342/https://www.top500.org/system/180171/ | 33.45 | -112.08 | |||||||||||||||||
| Naver DGX Superpod | Existing | Confirmed | Yes | 17.84 | 3.5e+2 | NVIDIA A100 | 1,120.00 | Korea (Republic of) | NAVER | 2020-10-05 | Private | 0.98 | 3.0e+7 | NAVER | A100 | NAVER CLOVA is using its DGX SuperPOD built with 140 DGX A100 | 2020-10-05 | 11.55 | 1,120.00 | NVIDIA | True | 10 | 17.54 | 7.0e+17 | 7.0e+17 | 3.5e+17 | 1.7e+17 | 0.98 | 3.6e+11 | 3.0e+7 | Oak Ridge NL Summit | 0.20 | https://web.archive.org/web/20240825043935/https://nvidianews.nvidia.com/news/nvidia-announces-ready-made-nvidia-dgx-superpods-offered-by-global-network-of-certified-partners | 36.54 | 128.17 | |||||||||||||||||||||
| University of Florida HiPerGator 3.0 Superpod | Existing | Confirmed | Yes | 17.84 | 3.5e+2 | NVIDIA A100 | 1,120.00 | United States of America | University of Florida | 2021-01-31 | Public | 1.03 | 3.0e+7 | Gainesville, FL 32611 | University of Florida | A100 | UF’s HiPerGator 3 supercomputer will integrate 140 NVIDIA DGX A100 systems powered by a combined 1,120 NVIDIA A100 Tensor Core GPUs | 2021-01-31 | 11.53 | 1,120.00 | NVIDIA | True | 14 | 17.54 | 7.0e+17 | 7.0e+17 | 3.5e+17 | 1.7e+17 | 0.96 | 1.03 | 3.4e+11 | 3.0e+7 | Oak Ridge NL Summit | 0.20 | https://web.archive.org/web/20240521065742/https://blogs.nvidia.com/blog/university-of-florida-nvidia-ai-supercomputer/ | 29.65 | -82.34 | |||||||||||||||||||
| PCSS Poznan Proxima | Existing | Confirmed | Yes | 17.84 | 3.5e+2 | NVIDIA H100 SXM5 80GB | 348.00 | Poland | Polish Academy of Sciences | 2024-06-01 | Public | 0.50 | 1.2e+7 | Poznań, Poland | Academia | H100 | Calculated from Top500 | 2024-06-01 | 11.84 | 348.00 | NVIDIA | True | 134 | 17.54 | 6.9e+17 | 6.9e+17 | 3.4e+17 | 1.7e+17 | 0.50 | 6.9e+11 | 1.2e+7 | Meta GenAI 2024a | 0.01 | https://web.archive.org/web/20250121170241/https://top500.org/system/180290/ | 52.41 | 16.93 | ||||||||||||||||||||
| Yandex Lyapunov | Existing | Confirmed | Yes | 17.83 | 3.5e+2 | NVIDIA A100 SXM4 40 GB | 1,096.00 | Russia | Yandex | 2021-11-01 | Private | 0.94 | 3.0e+7 | ул. Пушкина, 21, Sasovo, Ryazan Oblast, Russia, 391431 | Yandex | A100 | GPU 1,096 NVIDIA A100 40G | 2021-11-01 | 11.56 | 1,096.00 | NVIDIA | True | 35 | 17.53 | 6.8e+17 | 6.8e+17 | 3.4e+17 | 1.7e+17 | 0.94 | 3.6e+11 | 3.0e+7 | DeepSeek Fire-Flyer 2 | 0.11 | https://web.archive.org/web/20231115155013/https://yandex.com/supercomputers | 54.37 | 41.94 | ||||||||||||||||||||
| Yandex Galushkin | Existing | Confirmed | Yes | 17.83 | 3.4e+2 | NVIDIA A100 SXM4 80 GB | 1,088.00 | Russia | Yandex | 2021-11-01 | Private | 0.93 | 4.5e+7 | Ulitsa Energetikov, 34, Vladimir, Vladimir Oblast, Russia, 600902 | Yandex | A100 | GPU 1,088 NVIDIA A100 80G | 2021-11-01 | 11.56 | 1,088.00 | NVIDIA | True | 36 | 17.53 | 6.8e+17 | 6.8e+17 | 3.4e+17 | 1.7e+17 | 0.93 | 3.6e+11 | 4.5e+7 | DeepSeek Fire-Flyer 2 | 0.11 | https://web.archive.org/web/20231115155013/https://yandex.com/supercomputers | 56.13 | 40.40 | ||||||||||||||||||||
| NVIDIA SATURN V Phase 2 | Existing | Likely | Yes | 17.82 | 3.3e+2 | NVIDIA V100 | 5,280.00 | United States of America | NVIDIA | 2018-03-26 | This was first mentioned November 2017, but it appears to not be operational yet at that time
A later version of the supercomputer was actually four clustesrs, but its unclear if this was | Private | 3.63 | 1.0e+8 | NVIDIA | V100 | If you look at our Saturn V Supercomputer add-in video which we use for our internal training, that has 5,280 GPUs | 2018-03-26 | 11.26 | NVIDIA SATURN V Phase 1 | NVIDIA SATURN V Phase 3 | 5,280.00 | NVIDIA | True | 3 | 17.82 | 6.6e+17 | 6.6e+17 | 8.3e+16 | 3.63 | 1.8e+11 | 1.0e+8 | Meta 2017 V100 Cluster | 0.24 | https://web.archive.org/web/20240528104941/https://insidehpc.com/2018/04/inside-new-nvidia-dgx-2-supercomputer-nvswitch/ | https://top500.org/system/178928/ | 39.38 | -97.92 | ||||||||||||||||||
| SK Telecom Titan Phase 2 | Existing | Confirmed | Yes | 17.81 | 3.3e+2 | NVIDIA A100 | 1,040.00 | Korea (Republic of) | SK Telecom | 2023-02-14 | Private | 0.86 | 2.3e+7 | Korea (Republic of) | SK Telecom | A100 | by increasing its capacity to 1,040 NVIDIA A100 GPUs | 2023-02-14 | 11.58 | 1,040.00 | NVIDIA | True | 58 | 17.51 | 6.5e+17 | 6.5e+17 | 3.2e+17 | 1.6e+17 | 0.86 | 3.8e+11 | 2.3e+7 | Microsoft GPT-4 cluster | 0.04 | https://web.archive.org/web/20240725135908/https://insidehpc.com/2023/02/sk-telecom-doubles-capacity-of-titan-supercomputer/ | ||||||||||||||||||||||
| Nemotron-3-8B training cluster | Existing | Confirmed | Yes | 17.81 | 3.2e+2 | NVIDIA A100 | 1,024.00 | NVIDIA | 2024-04-04 | Private | 0.83 | 2.0e+7 | NVIDIA | A100 | "1,024 A100s were used for 19 days to train the model" | 2024-04-04 | 11.58 | True | NVIDIA Selene Phase 2 | 1,024.00 | NVIDIA | 117 | 17.50 | 6.4e+17 | 6.4e+17 | 3.2e+17 | 1.6e+17 | 0.83 | 3.8e+11 | 2.0e+7 | Meta GenAI 2024a | 0.01 | https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/nemotron-3-8b-base-4k | |||||||||||||||||||||||
| Paper on HyperCLOVA | Existing | Confirmed | Yes | 17.81 | 3.2e+2 | NVIDIA A100 | 1,024.00 | Korea (Republic of) | NAVER | 2021-09-10 | Private | 0.88 | 3.5e+7 | NAVER | A100 | "Our model is based on megatron-LM (Shoeybi et al., 2019) and trained on the NVIDIA Superpod, which includes 128 strongly clustered DGX servers with 1,024 A100 GPUs." | 2021-09-10 | 11.56 | True | Naver DGX Superpod | 1,024.00 | NVIDIA | 32 | 17.50 | 6.4e+17 | 6.4e+17 | 3.2e+17 | 1.6e+17 | 0.88 | 3.6e+11 | 3.5e+7 | DeepSeek Fire-Flyer 2 | 0.10 | https://arxiv.org/abs/2109.04650 | 36.54 | 128.17 | ||||||||||||||||||||
| Gcore data center Phase 1 | Existing | Confirmed | Yes | 17.80 | 3.2e+2 | NVIDIA H100 SXM5 80GB | 320.00 | Korea (Republic of) | Gcore,NHN Corporation | 2024-04-15 | Private | 0.46 | 1.1e+7 | Incheon, South Korea | Cloud | H100 | Gcore will initially have the data center equipped with 320 Nvidia H100 GPUs stacked onto 40 servers. Each server also has 2 terabytes of data storage and 112 CPU cores, with a bandwidth of 3.2 Tbps. | 2024-04-15 | 11.84 | 320.00 | NVIDIA | True | 120 | 17.50 | 6.3e+17 | 6.3e+17 | 3.2e+17 | 1.6e+17 | 0.46 | 6.9e+11 | 1.1e+7 | Meta GenAI 2024a | 0.01 | https://web.archive.org/web/20240723133814/https://www.datacenterdynamics.com/en/news/gcore-expands-into-south-korea-with-nhn-cloud/ | 37.46 | 126.71 | ||||||||||||||||||||
| Paper on OPT | Existing | Confirmed | Yes | 17.79 | 3.1e+2 | NVIDIA A100 SXM4 80 GB | 992.00 | Meta AI | 2022-05-02 | Private | 0.84 | 2.2e+7 | Meta | A100 | "We are also releasing both the logbook of our model creation as well as our codebase, metaseq,3 which enabled training OPT-175B on 992 80GB A100 GPUs," | 2022-05-02 | 11.56 | True | Meta Research SuperCluster (RSC-1) Phase 1 | 992.00 | NVIDIA | 46 | 17.49 | 6.2e+17 | 6.2e+17 | 3.1e+17 | 1.5e+17 | 0.84 | 3.7e+11 | 2.2e+7 | Microsoft GPT-4 cluster | 0.04 | https://arxiv.org/abs/2205.01068 | |||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 17.78 | 3.0e+2 | China | Private | China | Anonymized | Anonymized | 17.78 | 6.0e+17 | 6.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
Wroclaw Centre for Networking and Supercomputing Lem | Existing | Confirmed | Yes | 17.78 | 3.0e+2 | NVIDIA H100 SXM5 80GB | 304.00 | Poland | Wroclaw Tech (Wrocław University of Science and Technology) | 2024-06-01 | Public | 0.43 | 1.1e+7 | plac Grunwaldzki 9, 50-366 Wrocław, Poland | Academia,Researchers | H100 | Calculated from Top500 | 2024-06-01 | 11.84 | 304.00 | NVIDIA | True | 143 | 17.48 | 6.0e+17 | 6.0e+17 | 3.0e+17 | 1.5e+17 | 0.43 | 6.9e+11 | 1.1e+7 | Meta GenAI 2024a | 0.01 | https://web.archive.org/web/20250121115648/https://top500.org/system/180272/ | 51.11 | 17.06 | ||||||||||||||||||||
| AIST ABCI 2.0 | Existing | Confirmed | Yes | 17.78 | 3.0e+2 | NVIDIA A100 SXM4 40 GB | 960.00 | Japan | National Institute of Advanced Industrial Science and Technology (AIST) | 2021-05-30 | Builds off of ABCI 1.0, which was first operational in August 2018 | Public | 2.82 | 1.2e+8 | Kashiwa Campus, The University of Tokyo
Japan, 〒277-0882 Chiba, Kashiwa, Kashiwanoha, 5 Chome−1−5 | industry,academia,and government in Japan | A100,V100 | 120 Compute Nodes (A) that form in total 960 NVIDIA A100 GPU accelerators, 1,088 Compute Nodes (V) that form in total 4,352 NVIDIA GPU V100 accelerators | 2021-05-30 | Info from detailed official spec sheet | 11.48 | AIST ABCI 1.0 | AIST ABCI 3.0 | NVIDIA Tesla V100 SXM2 | 4,352 | 5,312.00 | NVIDIA | NVIDIA | True | 21 | 17.93 | 6.0e+17 | 6.0e+17 | 8.4e+17 | 2.2e+17 | 3.63 | 2.82 | 3.0e+11 | 1.2e+8 | Sunway OceanLight | 0.10 | https://web.archive.org/web/20240901124715/https://abci.ai/en/about_abci/computing_resource.html | 35.69 | 139.69 | ||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.78 | 3.0e+2 | 20,000.00 | China | 2020-09-15 | Public | 20.00 | China | 10.48 | 20,000.00 | Anonymized | Anonymized | True | 10 | 17.78 | 6.0e+17 | 6.0e+17 | 20.00 | 3.0e+10 | Oak Ridge NL Summit | 0.20 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||
| NEC Corp Japan Supercomputer | Existing | Confirmed | Yes | 17.76 | 2.9e+2 | NVIDIA A100 SXM4 80 GB | 928.00 | Japan | NEC Corporation | 2023-07-24 | Private | 0.77 | 2.1e+7 | Japan | NEC | A100 | 116 GPU servers, each equipped with 8 NVIDIA A100 80 GB Tensor Core GPUs per node | 2023-07-24 | 11.58 | 928.00 | NVIDIA | True | 73 | 17.46 | 5.8e+17 | 5.8e+17 | 2.9e+17 | 1.4e+17 | 0.77 | 3.8e+11 | 2.1e+7 | Microsoft GPT-4 cluster | 0.04 | https://web.archive.org/web/20240314221209/https://www.nec.com/en/global/rd/aisupercomputer/index.html | 36.39 | 138.59 | ||||||||||||||||||||
| Paper on CoCa | Existing | Confirmed | Yes | 17.75 | 2.8e+2 | Google TPU v4 | 2,048.00 | United States of America | 2022-05-04 | Private | 0.74 | 1.9e+7 | TPUv4 | "Pretraining CoCa takes about 5 days on 2,048 CloudTPUv4 chips" | 2022-05-04 | 11.88 | True | Paper on PaLM | 2,048.00 | 49 | 17.75 | 5.6e+17 | 5.6e+17 | 5.6e+17 | 0.74 | 7.6e+11 | 1.9e+7 | Microsoft GPT-4 cluster | 0.04 | https://arxiv.org/abs/2205.01917v2 | 39.38 | -97.92 | ||||||||||||||||||||||||
| AIST ABCI 1.0 | Existing | Confirmed | Yes | 17.74 | 2.7e+2 | NVIDIA Tesla V100 SXM2 | 4,352.00 | Japan | National Institute of Advanced Industrial Science and Technology (AIST) | 2018-06-01 | Public | 2.99 | 1.8e+8 | Kashiwa Campus, The University of Tokyo
Japan, 〒277-0882 Chiba, Kashiwa, Kashiwanoha, 5 Chome−1−5 | industry,academia,and government in Japan | V100 | The original ABCI system 4352 Nvidia V100 GPUs | 2018-06-01 | Confirmed by Top500 | 11.26 | AIST ABCI 2.0 | 4,352.00 | NVIDIA | True | 5 | 17.74 | 5.4e+17 | 5.4e+17 | 6.8e+16 | 2.99 | 1.8e+11 | 9.4e+7 | 1.6e+8 | 1.8e+8 | $172 for machine, prototype, and the associated datacenter (which was $10M), subtracting out cost of datacenter gives $162 | Meta 2017 V100 Cluster | 0.20 | https://web.archive.org/web/20240421015736/https://www.nextplatform.com/2021/07/08/inside-look-inside-japans-abci-ai-supercomputer-upgrade/ | 35.69 | 139.69 | ||||||||||||||||
| GENCI Adastra | Existing | Confirmed | Yes | 17.71 | 2.6e+2 | AMD Radeon Instinct MI250X | 1,352.00 | France | GENCI | 2022-06-01 | Public | 1.61 | 3.5e+7 | Montpellier, France | Academic research in Europe,Industry in France | AMD MI250X | Adastra contains a total of 1352 AMD MI250X on its 338 accelerated nodes. | 2022-06-01 | Listed in Top500 | 11.51 | 1,352.00 | AMD | True | 54 | 17.71 | 5.2e+17 | 5.2e+17 | 5.2e+17 | 1.3e+17 | 1.44 | 1.61 | 3.2e+11 | 3.5e+7 | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20240411122725/https://dci.dci-gitlab.cines.fr/webextranet/architecture/index.html | 43.61 | 3.88 | ||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.70 | 3.0e+2 | 2,000.00 | China | 2023-08-15 | 1.00 | China | 11.70 | 2,000.00 | Anonymized | Anonymized | True | 80 | 17.70 | 5.0e+17 | 5.0e+17 | 3.0e+17 | 1.00 | 5.0e+11 | Microsoft GPT-4 cluster | 0.03 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||
| MITRE Federal AI Sandbox | Planned | Confirmed | Yes | 17.70 | 2.6e+2 | NVIDIA H100 SXM5 80GB | 256.00 | United States of America | MITRE | Public/Private | 0.36 | 1.2e+7 | Ashburn, Virginia | MITRE,US government agencies | H100 | The organization's upcoming Federal AI Sandbox is basically an Nvidia DGX SuperPOD – a modular supercomputer built from 256 Nv H100 GPUs | Planned Q4 2024 | 11.85 | 256.00 | NVIDIA | 17.40 | 5.1e+17 | 5.1e+17 | 2.5e+17 | 1.3e+17 | 0.36 | 7.1e+11 | 1.2e+7 | https://web.archive.org/web/20240927024647/https://www.theregister.com/2024/05/08/mitre_ai_computer/ | 39.04 | -77.49 | |||||||||||||||||||||||||
| MPT-30B training cluster | Existing | Confirmed | Yes | 17.70 | 2.6e+2 | NVIDIA H100 SXM5 80GB | 256.00 | MosaicML | 2023-06-20 | 0.37 | 9.2e+6 | H100 | "Training was completed on 256 H100-80GBs" | 2023-06-20 | 11.83 | 256.00 | NVIDIA | True | 75 | 17.40 | 5.1e+17 | 5.1e+17 | 2.5e+17 | 1.3e+17 | 0.37 | 6.8e+11 | 9.2e+6 | Microsoft GPT-4 cluster | 0.03 | https://huggingface.co/mosaicml/mpt-30b/commits/main | ||||||||||||||||||||||||||
| Paper on Falcon Mamba 7B | Existing | Confirmed | Yes | 17.70 | 2.6e+2 | NVIDIA H100 SXM5 80GB | 256.00 | Technology Innovation Institute | 2024-07-24 | Possibly used cloud | Public | 0.37 | 9.1e+6 | Technology Innovation Institute | H100 | "Falcon-Mamba-7B was trained on 256 H100 80GB GPUs" | 2024-07-24 | 11.84 | True | 256.00 | NVIDIA | 168 | 17.40 | 5.1e+17 | 5.1e+17 | 2.5e+17 | 1.3e+17 | 0.37 | 6.9e+11 | 9.1e+6 | Meta GenAI 2024a | 0.01 | https://huggingface.co/tiiuae/falcon-mamba-7b/tree/main | |||||||||||||||||||||||
| Google MLPerf 0.7 Submission | Existing | Confirmed | Yes | 17.70 | 2.5e+2 | Google TPU v3 | 4,096.00 | United States of America | 2020-07-29 | Private | 1.97 | 3.9e+7 | TPU v3 | From MLPerf 0.7 | 2020-07-29 | 11.41 | 4,096.00 | True | 11 | 17.70 | 5.0e+17 | 5.0e+17 | 1.97 | 2.6e+11 | 3.9e+7 | Oak Ridge NL Summit | 0.15 | https://web.archive.org/web/20250217182530/https://github.com/mlcommons/training_results_v0.7/blob/master/all_hyper_params.md | 39.38 | -97.92 | ||||||||||||||||||||||||||
| Paper on Gopher | Existing | Confirmed | Yes | 17.70 | 2.5e+2 | Google TPU v3 | 4,096.00 | United States of America | Google DeepMind | 2021-12-08 | 4096 TPUV3 chips reported in table A27 | Private | 1.94 | 3.9e+7 | Georgia, USA | TPU v3 | 2021-12-08 | 11.42 | True | Google MLPerf 0.7 Submission | 4,096.00 | 42 | 17.70 | 5.0e+17 | 5.0e+17 | 1.94 | 2.6e+11 | 3.9e+7 | DeepSeek Fire-Flyer 2 | 0.08 | https://arxiv.org/abs/2112.11446 | 32.33 | -83.26 | |||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 17.70 | 3.0e+2 | China | 2023-12-15 | Public/Private | China | Anonymized | Anonymized | True | 108 | 17.70 | 5.0e+17 | 5.0e+17 | Microsoft Azure Eagle | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.70 | 3.0e+2 | China | 2024-03-15 | Public/Private | China | Anonymized | Anonymized | True | 117 | 17.70 | 5.0e+17 | 5.0e+17 | Meta GenAI 2024a | 0.01 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 17.70 | 3.0e+2 | China | Private | China | Anonymized | Anonymized | 17.70 | 5.0e+17 | 5.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.70 | 3.0e+2 | China | 2021-07-15 | Public/Private | China | Anonymized | Anonymized | True | 32 | 17.70 | 5.0e+17 | 5.0e+17 | DeepSeek Fire-Flyer 2 | 0.08 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 17.70 | 3.0e+2 | China | Public/Private | China | 500.00 | Anonymized | Anonymized | 17.70 | 5.0e+17 | 5.0e+17 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 17.70 | 3.0e+2 | China | Public/Private | China | Anonymized | Anonymized | 17.70 | 5.0e+17 | 5.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| EuroHPC MeluXina | Existing | Confirmed | Yes | 17.70 | 2.5e+2 | NVIDIA A100 | 800.00 | Luxembourg | EuroHPC JU | 2021-06-07 | Public | 0.69 | 3.8e+7 | 1 ZA et Commerciale Klengbusbierg 7795, 7795 Bissen, Luxembourg | European researchers,Cloud | A100 | 200 GPU nodes are part of the Accelerator Module, each featuring 2 AMD Rome CPUs (32 cores @ 2.35 GHz - 128HT cores total) and 4 NVIDIA A100-40 GPUs | 2021-06-07 | 11.56 | 800.00 | NVIDIA | True | 30 | 17.40 | 5.0e+17 | 5.0e+17 | 2.5e+17 | 1.2e+17 | 0.69 | 0.69 | 3.6e+11 | 2.2e+7 | 3.6e+7 | 3.8e+7 | "MeluXina is funded via a joint investment of about EUR 30 million from the European Union and Luxembourg" | Sunway OceanLight | 0.08 | https://web.archive.org/web/20240406154520/https://www.hpcwire.com/off-the-wire/meluxina-named-eus-greenest-supercomputer/ | https://web.archive.org/web/20241014163423/https://docs.lxp.lu/system/overview/ | https://eurohpc-ju.europa.eu/meluxina-new-eurohpc-world-class-supercomputer-luxembourg-2020-09-29_en | 49.79 | 6.09 | ||||||||||||||
| Paper on YaLM | Existing | Likely | Yes | 17.70 | 2.5e+2 | NVIDIA A100 | 800.00 | Yandex | 2022-06-23 | 0.68 | 1.8e+7 | A100 | "It took us 65 days to train the model on a pool of 800 A100 graphics cards" | 2022-06-23 | 11.56 | True | Yandex Chervonenkis | 800.00 | NVIDIA | 58 | 17.40 | 5.0e+17 | 5.0e+17 | 2.5e+17 | 1.2e+17 | 0.68 | 3.7e+11 | 1.8e+7 | Microsoft GPT-4 cluster | 0.03 | https://medium.com/yandex/yandex-publishes-yalm-100b-its-the-largest-gpt-like-neural-network-in-open-source-d1df53d0e9a6 | |||||||||||||||||||||||||
| KT SuperPOD | Existing | Confirmed | Yes | 17.69 | 2.5e+2 | NVIDIA A100 | 792.00 | Korea (Republic of) | KT | 2023-06-01 | Private | 0.66 | 1.8e+7 | Korea (Republic of) | KT | A100 | Calculated from Top500 | 2023-06-01 | 11.58 | 792.00 | NVIDIA | True | 79 | 17.39 | 4.9e+17 | 4.9e+17 | 2.5e+17 | 1.2e+17 | 0.66 | 3.8e+11 | 1.8e+7 | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20241205015244/https://top500.org/system/180163/ | ||||||||||||||||||||||
| SberCloud Christofari Neo | Existing | Confirmed | Yes | 17.69 | 2.5e+2 | NVIDIA A100 SXM4 80 GB | 792.00 | Russia | SberCloud | 2021-11-01 | SberCloud, which owns this supercomputer, was originally owned by Sber, but was sold off to Noviye Vozmozhnosti in March 2022
Sberbank is majority owned by the Russian Government. | Private | 0.68 | 3.3e+7 | Russia | Cloud | A100 | Calculated from Top500 | 2021-11-01 | 11.56 | 792.00 | NVIDIA | True | 45 | 17.39 | 4.9e+17 | 4.9e+17 | 2.5e+17 | 1.2e+17 | 0.68 | 3.6e+11 | 3.3e+7 | DeepSeek Fire-Flyer 2 | 0.08 | https://web.archive.org/web/20230517125435/https://en.wikipedia.org/wiki/Christofari | 61.99 | 96.67 | |||||||||||||||||||
| Cineca Marconi-100 | Existing | Confirmed | Yes | 17.69 | 2.5e+2 | NVIDIA V100 | 3,952.00 | Italy | Cineca | 2020-04-20 | Public | 2.64 | 7.6e+7 | Bologna, Italy | Academic research in Europe | V100 | Nodes: 980
Processors: 2x16 cores IBM POWER9 AC922 at 2.6(3.1) GHz
Accelerators: 4 x NVIDIA Volta V100 GPUs/node, Nvlink 2.0, 16GB | 2020-04-20 | 11.27 | 3,952.00 | NVIDIA | True | 10 | 17.69 | 4.9e+17 | 4.9e+17 | 6.2e+16 | 2.59 | 2.64 | 1.9e+11 | 7.6e+7 | Oak Ridge NL Summit | 0.14 | https://web.archive.org/web/20240523132015/https://wiki.u-gov.it/confluence/pages/viewpage.action?pageId=336727645 | https://web.archive.org/web/20240520090110/https://www.top500.org/system/179845/ | 44.50 | 11.34 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.70 | 2.0e+2 | 1,000.00 | China | 2022-05-15 | Public/Private | 0.60 | China | 11.52 | 1,000.00 | Anonymized | Anonymized | True | 58 | 17.30 | 5.0e+17 | 5.0e+17 | 2.0e+17 | 0.60 | 3.3e+11 | Microsoft GPT-4 cluster | 0.03 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.70 | 2.0e+2 | 1,000.00 | China | 2021-09-15 | Public/Private | 0.60 | China | 11.70 | 1,000.00 | Anonymized | Anonymized | True | 39 | 17.48 | 5.0e+17 | 5.0e+17 | 3.0e+17 | 0.60 | 5.0e+11 | DeepSeek Fire-Flyer 2 | 0.08 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Fastweb NeXXt AI Factory | Existing | Confirmed | Yes | 17.69 | 2.5e+2 | NVIDIA H100 SXM5 80GB | 248.00 | Italy | Fastweb | 2024-07-09 | 31x8=248 | Private | 0.35 | 8.8e+6 | Via di San Clemente, 53 24036 Ponte San Pietro Bergamo, Italy | Cloud,Fastweb | H100 | Fastweb entered an agreement with NVIDIA to acquire 31 NVIDIA DGX H100 systems | 2024-07-09 | 11.84 | 248.00 | NVIDIA | True | 177 | 17.39 | 4.9e+17 | 4.9e+17 | 2.5e+17 | 1.2e+17 | 0.35 | 6.9e+11 | 8.8e+6 | Meta GenAI 2024a | 0.01 | https://web.archive.org/web/20240714211301/https://www.hpcwire.com/off-the-wire/fastweb-unveils-italys-1st-nvidia-dgx-superpod-ai-supercomputer/ | 45.71 | 9.59 | |||||||||||||||||||
| Opera Iceland KEF-1 SuperPOD | Existing | Confirmed | Yes | 17.69 | 2.5e+2 | NVIDIA H100 SXM5 80GB | 248.00 | Iceland | Opera | 2024-02-28 | Private | 0.35 | 8.9e+6 | Sjónarhóli 6, 260 Reykjanesbæ, Iceland | Opera | H100 | Calculated from Top500 | 2024-02-28 | 11.84 | 248.00 | NVIDIA | True | 122 | 17.39 | 4.9e+17 | 4.9e+17 | 2.5e+17 | 1.2e+17 | 0.35 | 6.9e+11 | 8.9e+6 | Microsoft Azure Eagle | 0.02 | https://web.archive.org/web/20240705020431/https://press.opera.com/2024/02/07/opera-deploying-green-powered-ai-data-cluster-in-iceland/ | 63.96 | -22.54 | ||||||||||||||||||||
| Condor Galaxy 1 (CG-1) | Existing | Confirmed | Yes | 17.68 | 2.4e+2 | Cerebras CS-2 | 64.00 | United States of America | G42,Cerebras Systems | 2023-11-13 | First of 9 planned supercomputers that will be interconnected
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.2e+8 | Colovore LLC Data Center
1101 Space Park Dr, Santa Clara, CA 95054 | Cloud | Cerebras CS2 | CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs | 2023-11-13 | Reported on their official website | 64.00 | Cerebras | True | 107 | 17.68 | 4.8e+17 | 4.8e+17 | 2.2e+8 | NVIDIA CoreWeave Eos-DFW Phase 1 | 0.02 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | 37.38 | -121.95 | |||||||||||||||||||||
| Condor Galaxy 4 (CG-4) | Planned | Likely | Yes | 17.68 | 2.4e+2 | Cerebras CS-2 | 64.00 | G42,Cerebras Systems | Fourth of 9 planned supercomputers that will be interconnected
No news on this since the original announcement of the plan. Unclear if they will use CS2 or CS3. They originally planned to use CS2 for CG2, but ended up using CS3, suggesting they might also use CS3 for this
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.2e+8 | Undisclosed, could be anywhere "across the globe world" | Cloud | Cerebras CS2 | "CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs"
"Upon completion in 2024, the nine inter-connected supercomputers will have 36 ExaFLOPS of AI compute, making it one of the most powerful cloud AI supercomputers in the world." | Planned | No news on this since original announcement of plan. Unclear if they will use CS2 or CS3 | 64.00 | Cerebras | 17.68 | 4.8e+17 | 4.8e+17 | 2.2e+8 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | |||||||||||||||||||||||||||||
| Condor Galaxy 5 (CG-5) | Planned | Likely | Yes | 17.68 | 2.4e+2 | Cerebras CS-2 | 64.00 | G42,Cerebras Systems | Fifth of 9 planned supercomputers that will be interconnected
No news on this since the original announcement of the plan. Unclear if they will use CS2 or CS3. They originally planned to use CS2 for CG2, but ended up using CS3, suggesting they might also use CS3 for this
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.2e+8 | Undisclosed, could be anywhere "across the globe world" | Cloud | Cerebras CS2 | "CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs"
"Upon completion in 2024, the nine inter-connected supercomputers will have 36 ExaFLOPS of AI compute, making it one of the most powerful cloud AI supercomputers in the world." | Planned | No news on this since original announcement of plan. Unclear if they will use CS2 or CS3 | 64.00 | Cerebras | 17.68 | 4.8e+17 | 4.8e+17 | 2.2e+8 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | |||||||||||||||||||||||||||||
| Condor Galaxy 6 (CG-6) | Planned | Likely | Yes | 17.68 | 2.4e+2 | Cerebras CS-2 | 64.00 | G42,Cerebras Systems | Sixth of 9 planned supercomputers that will be interconnected
No news on this since the original announcement of the plan. Unclear if they will use CS2 or CS3. They originally planned to use CS2 for CG2, but ended up using CS3, suggesting they might also use CS3 for this
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.2e+8 | Undisclosed, could be anywhere "across the globe world" | Cloud | Cerebras CS2 | "CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs"
"Upon completion in 2024, the nine inter-connected supercomputers will have 36 ExaFLOPS of AI compute, making it one of the most powerful cloud AI supercomputers in the world." | Planned | No news on this since original announcement of plan. Unclear if they will use CS2 or CS3 | 64.00 | Cerebras | 17.68 | 4.8e+17 | 4.8e+17 | 2.2e+8 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | |||||||||||||||||||||||||||||
| Condor Galaxy 7 (CG-7) | Planned | Likely | Yes | 17.68 | 2.4e+2 | Cerebras CS-2 | 64.00 | G42,Cerebras Systems | Seventh of 9 planned supercomputers that will be interconnected
No news on this since the original announcement of the plan. Unclear if they will use CS2 or CS3. They originally planned to use CS2 for CG2, but ended up using CS3, suggesting they might also use CS3 for this
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.2e+8 | Undisclosed, could be anywhere "across the globe world" | Cloud | Cerebras CS2 | "CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs"
"Upon completion in 2024, the nine inter-connected supercomputers will have 36 ExaFLOPS of AI compute, making it one of the most powerful cloud AI supercomputers in the world." | Planned | No news on this since original announcement of plan. Unclear if they will use CS2 or CS3 | 64.00 | Cerebras | 17.68 | 4.8e+17 | 4.8e+17 | 2.2e+8 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | |||||||||||||||||||||||||||||
| Condor Galaxy 8 (CG-8) | Planned | Likely | Yes | 17.68 | 2.4e+2 | Cerebras CS-2 | 64.00 | G42,Cerebras Systems | Eighth of 9 planned supercomputers that will be interconnected
No news on this since the original announcement of the plan. Unclear if they will use CS2 or CS3. They originally planned to use CS2 for CG2, but ended up using CS3, suggesting they might also use CS3 for this
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.2e+8 | Undisclosed, could be anywhere "across the globe world" | Cloud | Cerebras CS2 | "CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs"
"Upon completion in 2024, the nine inter-connected supercomputers will have 36 ExaFLOPS of AI compute, making it one of the most powerful cloud AI supercomputers in the world." | Planned | No news on this since original announcement of plan. Unclear if they will use CS2 or CS3 | 64.00 | Cerebras | 17.68 | 4.8e+17 | 4.8e+17 | 2.2e+8 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | |||||||||||||||||||||||||||||
| Condor Galaxy 9 (CG-9) | Planned | Likely | Yes | 17.68 | 2.4e+2 | Cerebras CS-2 | 64.00 | G42,Cerebras Systems | Ninth of 9 planned supercomputers that will be interconnected
No news on this since the original announcement of the plan. Unclear if they will use CS2 or CS3. They originally planned to use CS2 for CG2, but ended up using CS3, suggesting they might also use CS3 for this
Unconventional hardware
Reported FLOP/s is with sparcity
A UAE government holds a significant but unknown stake in G42, possibly qualifying this as a government project | Public/Private | 2.2e+8 | Undisclosed, could be anywhere "across the globe world" | Cloud | Cerebras CS2 | "CG-1 links 64 Cerebras CS-2 systems together into a single, easy-to-use AI supercomputer, with an AI training capacity of 4 exaFLOPs"
"Upon completion in 2024, the nine inter-connected supercomputers will have 36 ExaFLOPS of AI compute, making it one of the most powerful cloud AI supercomputers in the world." | Planned | No news on this since original announcement of plan. Unclear if they will use CS2 or CS3 | 64.00 | Cerebras | 17.68 | 4.8e+17 | 4.8e+17 | 2.2e+8 | https://web.archive.org/web/20240831173119/https://cerebras.ai/blog/introducing-condor-galaxy-1-a-4-exaflop-supercomputer-for-generative-ai/ | https://web.archive.org/web/20240603001926/https://www.hpcwire.com/off-the-wire/cerebras-and-g42-unveil-condor-galaxy-to-accelerate-global-ai-advancement/ | https://web.archive.org/web/20240827205627/https://cerebras.ai/condor-galaxy | https://web.archive.org/web/20240519195141/https://www.servethehome.com/detail-of-the-giant-cerebras-wafer-scale-cluster-nvidia/ | |||||||||||||||||||||||||||||
| FZJ JURECA | Existing | Confirmed | Yes | 17.68 | 2.4e+2 | NVIDIA A100 SXM4 40 GB | 768.00 | Germany | Julich Supercomputing Center | 2021-06-23 | Public | 0.68 | 2.1e+7 | 52428 Jülich, Germany | Academia,Researchers | A100 | In total, 192 of the 768 nodes are equipped with four NVIDIA A100 graphics processing units (GPUs) | 2021-06-23 | 11.55 | 768.00 | NVIDIA | True | 33 | 17.38 | 4.8e+17 | 4.8e+17 | 2.4e+17 | 1.2e+17 | 0.66 | 0.68 | 3.5e+11 | 2.1e+7 | Sunway OceanLight | 0.08 | https://web.archive.org/web/20231211004104/https://www.hpcwire.com/off-the-wire/julich-upgrades-jureca-supercomputer-for-large-data-volumes/ | 50.92 | 6.36 | |||||||||||||||||||
| Max-Planck-Gesellschaft Raven | Existing | Confirmed | Yes | 17.68 | 2.4e+2 | NVIDIA A100 | 768.00 | Germany | Max Planck Society | 2021-06-01 | Public | 0.66 | 2.1e+7 | Gießenbachstraße 2, 85748 Garching bei München, Germany | Academia,Max Planck Society | A100 | is creating with Lenovo a system called Raven-GPU, powered by 768 NVIDIA A100 GPUs. | 2021-06-01 | 11.56 | 768.00 | NVIDIA | True | 31 | 17.38 | 4.8e+17 | 4.8e+17 | 2.4e+17 | 1.2e+17 | 0.66 | 0.66 | 3.6e+11 | 2.1e+7 | Sunway OceanLight | 0.08 | https://web.archive.org/web/20240520113356/https://www.datacenterdynamics.com/en/news/atos-to-build-215m-supercomputer-for-german-science-and-technology-organization/ | https://web.archive.org/web/20241212234331/https://blogs.nvidia.com/blog/hpc-supercomputers-a100-gpus/ | 48.26 | 11.67 | ||||||||||||||||||
| Paper on GLM-130B | Existing | Confirmed | Yes | 17.68 | 2.4e+2 | NVIDIA A100 | 768.00 | Zhipu AI | 2022-10-05 | Public/Private | 0.65 | 1.7e+7 | A100 | "Specifically, GLM-130B is a bilingual (English and Chinese) bidirectional dense model with 130 billion parameters, pre-trained over 400 billion tokens on a cluster of 96 NVIDIA DGX-A100 (8×40G) GPU nodes" | 2022-10-05 | 11.56 | True | 768.00 | NVIDIA | 69 | 17.38 | 4.8e+17 | 4.8e+17 | 2.4e+17 | 1.2e+17 | 0.65 | 3.7e+11 | 1.7e+7 | Microsoft GPT-4 cluster | 0.03 | https://arxiv.org/abs/2210.02414 | |||||||||||||||||||||||||
| Paper on xTrimoPGLM | Existing | Confirmed | Yes | 17.68 | 2.4e+2 | NVIDIA A100 | 768.00 | BioMap Research | 2023-07-06 | Public/Private | 0.64 | 1.7e+7 | A100 | xTrimoPGLM-100B is trained on a cluster of 96 DGX-A100 GPU (8×40G) servers. Calculation - 96 servers×8 GPUs/server= 768 GPUs
| 2023-07-06 | 11.58 | True | 768.00 | NVIDIA | 87 | 17.38 | 4.8e+17 | 4.8e+17 | 2.4e+17 | 1.2e+17 | 0.64 | 3.8e+11 | 1.7e+7 | Microsoft GPT-4 cluster | 0.03 | https://www.biorxiv.org/content/10.1101/2023.07.05.547496v4.article-info | |||||||||||||||||||||||||
| NSC Berzelius Phase 2 | Existing | Confirmed | Yes | 17.67 | 2.4e+2 | NVIDIA A100 SXM4 80 GB | 752.00 | Sweden | Linköping University | 2023-06-01 | Public | 0.62 | 1.7e+7 | 583 30 Linköping, Sweden | Swedish academic research groups | A100,752 A100s (480 40GB, 272 80GB) | addition will bring 34 additional nodes to the SuperPod, also based on the DGX A100 structure; this time, though, the A100s in the systems will be the 80GB variants rather than the 40GB GPUs in the remainder of the system | 2023-06-01 | 11.58 | NSC Berzelius Phase 1 | 752.00 | NVIDIA | True | 87 | 17.37 | 4.7e+17 | 4.7e+17 | 2.3e+17 | 1.2e+17 | 0.62 | 3.8e+11 | 1.7e+7 | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20230712130753/https://www.hpcwire.com/2023/01/26/sweden-plans-expansion-for-nvidia-powered-berzelius-supercomputer/ | 58.39 | 15.56 | |||||||||||||||||||
| Karlsruher Institut für Technologie HoreKa | Existing | Confirmed | Yes | 17.66 | 2.3e+2 | NVIDIA A100 | 740.00 | Germany | Karlsruhe Institute of Technology | 2021-06-01 | Public | 0.63 | 2.0e+7 | Karlsruhe, Germany | Academia | A100 | to build a new 17-petaflops system that will pack 740 A100 GPUs on an NVIDIA Mellanox 200 Gbit/s InfiniBand network | 2021-06-01 | 11.56 | 740.00 | NVIDIA | True | 32 | 17.36 | 4.6e+17 | 4.6e+17 | 2.3e+17 | 1.2e+17 | 0.64 | 0.63 | 3.7e+11 | 2.0e+7 | Sunway OceanLight | 0.08 | https://web.archive.org/web/20210816050117/https://www.datacenterdynamics.com/en/news/horeka-hybrid-supercomputer-inaugurated-at-karlsruhe-institute-of-technology/ | https://web.archive.org/web/20241212234331/https://blogs.nvidia.com/blog/hpc-supercomputers-a100-gpus/ | 49.01 | 8.40 | ||||||||||||||||||
| Paper on AlphaZero | Existing | Likely | Yes | 17.66 | 2.3e+2 | Google TPU v1 | 5,000.00 | United States of America | 2017-12-05 | Private | 0.88 | 5.1e+7 | TPU v1 | Training proceeded
for 700,000 steps (mini-batches of size 4,096) starting from randomly initialised parameters,
using 5,000 first-generation TPUs (15) to generate self-play games | 2017-12-05 | 5,000.00 | True | 2 | 4.6e+17 | 4.6e+17 | 0.88 | 5.1e+7 | Meta 2017 V100 Cluster | 0.17 | https://arxiv.org/pdf/1712.01815 | 39.38 | -97.92 | |||||||||||||||||||||||||||||
| KT Internal MI250 Cluster | Existing | Confirmed | Yes | 17.66 | 2.3e+2 | AMD Radeon Instinct MI250X | 1,200.00 | Korea (Republic of) | KT | 2022-12-15 | Private | 1.28 | 3.1e+7 | Korea (Republic of) | KT | AMD MI250X | 1,200 MI250s for KT's internal LLM development | 2022-12-15 | 11.56 | 1,200.00 | AMD | True | 77 | 17.66 | 4.6e+17 | 4.6e+17 | 4.6e+17 | 1.1e+17 | 1.28 | 3.6e+11 | 3.1e+7 | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20241005005645/https://moreh.io/blog/training-221b-parameter-korean-llm-on-1200-amd-mi250-gpu-cluster-230814 | ||||||||||||||||||||||
| NSTDA Supercomputer Center (ThaiSC) LANTA | Existing | Confirmed | Yes | 17.64 | 2.2e+2 | NVIDIA A100 | 704.00 | Thailand | National Science and Technology Development Agency (NSTDA) | 2022-11-01 | Public | 0.54 | 1.6e+7 | Pathum Thani, Thailand | Academia,NSTDA,Thai Government | A100 | With 704 A100 GPUs, the new public high performance computing system will be Southeast Asia’s biggest. | 2022-11-01 | 11.61 | 704.00 | NVIDIA | True | 76 | 17.34 | 4.4e+17 | 4.4e+17 | 2.2e+17 | 1.1e+17 | 0.60 | 0.54 | 4.1e+11 | 1.6e+7 | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20240523025236/https://www.nvidia.com/en-sg/news/nvidia-powers-thailand-research-agency-new-supercomputer-with-region-largest-gpu-cluster/ | 14.02 | 100.53 | |||||||||||||||||||
| Petrobras Gaia | Existing | Confirmed | Yes | 17.64 | 2.2e+2 | NVIDIA A100 | 704.00 | Brazil | Petrobras | 2023-06-01 | Public/Private | 0.98 | 1.6e+7 | Rio de Janeiro, Brazil | Petrobras | A100 | Calculated from Top500 | 2023-06-01 | 11.35 | 704.00 | NVIDIA | True | 91 | 17.34 | 4.4e+17 | 4.4e+17 | 2.2e+17 | 1.1e+17 | 0.58 | 0.98 | 2.2e+11 | 1.6e+7 | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20240813050918/https://www.top500.org/system/180173/ | -22.91 | -43.21 | |||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 17.60 | 2.0e+2 | China | Public/Private | China | Anonymized | Anonymized | 17.60 | 4.0e+17 | 4.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| TotalEnergies Pangea III | Existing | Confirmed | Yes | 17.63 | 2.1e+2 | NVIDIA V100 | 3,384.00 | France | TotalEnergies | 2019-06-18 | Private | 2.49 | 6.7e+7 | Av. Larribau, 64000 Pau, France | TotalEnergies | V100 | Calculated from Top500 | 2019-06-18 | Listed in Top500 | 11.23 | 3,384.00 | NVIDIA | True | 8 | 17.63 | 4.2e+17 | 4.2e+17 | 5.3e+16 | 2.25 | 2.49 | 1.7e+11 | 6.7e+7 | Oak Ridge NL Summit | 0.12 | https://web.archive.org/web/20240518130419/https://www.top500.org/system/179689/ | 43.32 | -0.32 | |||||||||||||||||||
| Paper on Flamingo | Existing | Confirmed | Yes | 17.63 | 2.1e+2 | Google TPU v4 | 1,536.00 | United States of America | 2022-04-29 | Private | 0.56 | 1.4e+7 | TPUv4 | "All training and evaluation was performed on TPUv4 instances. The
largest model containing 80 billion parameters is trained on 1536 chips" | 2022-04-29 | 11.88 | True | Paper on PaLM | 1,536.00 | 60 | 17.63 | 4.2e+17 | 4.2e+17 | 4.2e+17 | 0.56 | 7.6e+11 | 1.4e+7 | DeepSeek Fire-Flyer 2 | 0.07 | https://arxiv.org/abs/2204.14198 | 39.38 | -97.92 | ||||||||||||||||||||||||
| University of Illinois NCSA Delta | Existing | Confirmed | Yes | 17.62 | 2.1e+2 | NVIDIA A100 | 480.00 | United States of America | University of Illinois Urbana-Champaign (UIUC) | 2022-10-11 | Public | 0.66 | 1.0e+7 | 1205 W Clark St, Urbana, IL 61801 | University of Illinois | A100,A40 | 100 quad A100 GPU nodes consisting of: Four NVIDIA A100 GPUs with 40 GB HBM2 RAM and NVLink
Five eight-way A100 GPU nodes consisting of: Eight NVIDIA A100 GPUs with 40 GB HBM2 RAM and NVLink
100 quad A40 GPU nodes consisting of: Four NVIDIA A40 GPUs with 48 GB GDDR6 RAM | 2022-10-11 | 11.50 | NVIDIA A40 PCIe | 400 | 880.00 | NVIDIA | NVIDIA | True | 76 | 17.32 | 4.2e+17 | 4.2e+17 | 2.1e+17 | 1.0e+17 | 0.66 | 3.2e+11 | 1.5e+7 | 1.0e+7 | 1.0e+7 | The original Delta machine, which we profiled here and which cost $10 million as well | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20240503170712/https://www.datacenterdynamics.com/en/news/ncsa-announces-delta-supercomputer-is-ready-for-operations/ | https://web.archive.org/web/20240927183658/https://www.ncsa.illinois.edu/research/project-highlights/delta/ | https://www.nextplatform.com/2023/07/11/ncsa-builds-out-delta-supercomputer-with-an-ai-extension/#:~:text=The%20original%20Delta%20machine%2C%20which,and%20now%20controlled%20by%20HPE. | 40.11 | -88.22 | ||||||||||||
| AIRAWAT-PSAI Phase 2 | Existing | Confirmed | Yes | 17.61 | 2.1e+2 | NVIDIA A100 | 656.00 | India | Center for Development of Advanced Computing (C-DAC) | 2023-05-23 | This consists of an original portion of 336 GPUs (PARAM Siddhi - AI) and a newer portion of 320 GPUs (AIRAWAT PoC) | Public | 0.54 | 1.5e+7 | Innovation Park, 34/B/1, Panchawati Rd, Mansarovar, Panchawati, Pashan, Pune, Maharashtra 411008, India | Cloud | A100 | 656 (82 nodes * 8 gpus per node) | 2023-05-23 | 11.58 | PARAM Siddhi-AI Phase 1 | 656.00 | NVIDIA | True | 91 | 17.31 | 4.1e+17 | 4.1e+17 | 2.0e+17 | 1.0e+17 | 0.54 | 3.8e+11 | 1.5e+7 | Microsoft GPT-4 cluster | 0.03 | https://drive.google.com/file/d/1bLDkmBo4dklKX5Psch-ifNpzNDcbELXA/view?usp=sharing | https://web.archive.org/web/20240720151011/https://cdac.in/index.aspx?id=hpc_nsf_siddhi-spec | 18.53 | 73.81 | |||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 17.60 | 2.0e+2 | China | Public | China | Anonymized | Anonymized | 17.60 | 4.0e+17 | 4.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| MSU-270 | Existing | Likely | Yes | 17.60 | 2.0e+2 | Russia | Moscow State Univeristy | 2023-09-02 | Public | Moscow, Russia | Academia,Researchers | with a peak computational power of 400 'AI' PetaFLOPS | 2023-09-02 | 100.00 | Unknown | True | 109 | 17.60 | 4.0e+17 | 4.0e+17 | Tesla 10k H100 Cluster | 0.02 | https://web.archive.org/web/20240909123929/https://www.tomshardware.com/news/russian-400-petaflops-supercomputer-for-ai-comes-online | 55.75 | 37.62 | |||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 17.60 | 2.0e+2 | China | Public | China | Anonymized | Anonymized | 17.60 | 4.0e+17 | 4.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| NVIDIA Cambridge-1 | Existing | Confirmed | Yes | 17.60 | 2.0e+2 | NVIDIA A100 | 640.00 | United Kingdom of Great Britain and Northern Ireland | NVIDIA | 2021-06-01 | Private | 0.55 | 5.4e+7 | London Rd, Harlow CM17 9NA, United Kingdom | Biomedical scientific community,including companies and the UK gov,NVIDIA | A100 | The Cambridge-A1 consists of 80 DGX A100 systems connected by Infiniband networking. Each DGX A100 has 8 A100 GPUs, 320GB GPU RAM, two AMD EPYC 7742 CPUs (64 cores each), up to 2TB system RAM, up to 30TB NVME data cache drives, and 2 1.92TB NVME SSDs. | 2021-06-01 | Reported by official NVIDIA post | 11.56 | 640.00 | NVIDIA | True | 35 | 17.30 | 4.0e+17 | 4.0e+17 | 2.0e+17 | 1.0e+17 | 0.55 | 3.6e+11 | 1.7e+7 | 5.2e+7 | 5.4e+7 | NVIDIA will invest around £40 million ($51.7 million) in Cambridge-1. | Sunway OceanLight | 0.07 | https://web.archive.org/web/20240617042217/https://www.theregister.com/2021/07/07/nvidia_launches_cambridge1_supercomputer_in/ | https://nvidianews.nvidia.com/news/nvidia-building-uks-most-powerful-supercomputer-dedicated-to-ai-research-in-healthcare | 51.74 | 0.13 | |||||||||||||||
| Calcul Québec Narval | Existing | Confirmed | Yes | 17.60 | 2.0e+2 | NVIDIA A100 | 636.00 | Canada | Compute Canada | 2021-11-01 | From Top500 | Public | 0.55 | 2.2e+7 | 1100 Notre-Dame St W, Montreal, Quebec H3C 1K3, Canada | Academia | A100 | it features more than 632 state-of-the-art graphics processing units (GPUs) that are particularly well-suited to work in artificial intelligence. | 2021-11-01 | In Top500 | 11.56 | 636.00 | NVIDIA | True | 54 | 17.30 | 4.0e+17 | 4.0e+17 | 2.0e+17 | 9.9e+16 | 0.55 | 0.55 | 3.6e+11 | 2.2e+7 | DeepSeek Fire-Flyer 2 | 0.06 | https://web.archive.org/web/20240714041506/https://www.calculquebec.ca/en/nouvelle/narval-a-new-supercomputer-dedicated-to-scientific-research/ | 45.49 | -73.56 | |||||||||||||||||
| Lawrence Livermore NL Lassen Phase 2 | Existing | Confirmed | Yes | 17.60 | 2.0e+2 | NVIDIA V100 | 3,168.00 | United States of America | US Department of Energy | 2020-08-19 | Public | 2.08 | 6.3e+7 | 7000 East Ave, Livermore, CA 94550 | Lawrence Livermore NL | V100,CS-1 | "At Livermore, the Cerebras CS-1 machine was recently integrated into the National Nuclear Security Administration’s Lassen supercomputer, the unclassified companion system to Sierra."
V100 count is from Top500 | 2020-08-19 | Listed in Top500 | 11.28 | Lawrence Livermore NL Lassen Phase 1 | Cerebras CS-1 | 1 | 3,169.00 | NVIDIA | Cerebras | True | 15 | 17.60 | 4.0e+17 | 4.0e+17 | 5.0e+16 | 2.08 | 1.9e+11 | 6.3e+7 | Oak Ridge NL Summit | 0.11 | https://web.archive.org/web/20240327161123/https://www.hpcwire.com/2020/08/19/cerebras-llnl-deployment-teases-second-gen-wafer-scale-ai-chip/ | https://web.archive.org/web/20240613070121/https://www.top500.org/system/179567/ | 37.68 | -121.71 | |||||||||||||||
| hessian.AI fortytwo | Existing | Confirmed | Yes | 17.60 | 2.0e+2 | NVIDIA A100 | 632.00 | Germany | hessian.AI | 2023-06-15 | Public | 0.52 | 1.6e+7 | Planckstraße 1, 64291 Darmstadt, Germany | Academia,Industry | A100 | In fortytwo‘s engine room, a total of 632 NVIDIA A100 Tensor Core GPUs, each with 80GB of memory, as well as four IPU units from Graphcore are performing their duties | 2023-06-15 | 11.58 | 632.00 | NVIDIA | True | 101 | 17.29 | 3.9e+17 | 3.9e+17 | 2.0e+17 | 9.9e+16 | 0.52 | 3.8e+11 | 1.4e+7 | 1.5e+7 | 1.6e+7 | "The 14.5 million project" presumably this is Euros | Microsoft GPT-4 cluster | 0.03 | https://web.archive.org/web/20240623043843/https://hessian.ai/supercomputer-for-cutting-edge-ai-research-in-hesse/ | 49.93 | 8.68 | |||||||||||||||||
| Japanese Research Institute 2 Supercomputer 1 | Existing | Confirmed | Yes | 17.59 | 2.0e+2 | NVIDIA A100 | 624.00 | Japan | 2022-06-01 | 0.53 | 1.4e+7 | Japan | A100 | Calculated from Top500 | 2022-06-01 | 11.56 | 624.00 | NVIDIA | True | 72 | 17.29 | 3.9e+17 | 3.9e+17 | 1.9e+17 | 9.7e+16 | 0.53 | 3.7e+11 | 1.4e+7 | Microsoft GPT-4 cluster | 0.02 | https://web.archive.org/web/20241210014053/https://top500.org/system/180083/ | 36.39 | 138.59 | |||||||||||||||||||||||
| EuroHPC Karolina | Existing | Confirmed | Yes | 17.56 | 1.8e+2 | NVIDIA A100 | 576.00 | Czechia | EuroHPC JU | 2021-06-01 | Public | 0.55 | 1.8e+7 | Studentská 6231/1b, 708 00 Ostrava 8, Czechia | European researchers | A100 | an accelerated part with 72 servers, and each of them is equipped with 8 GPU accelerators providing a performance of 11.6 PFlop/s for standard HPC simulations and up to 360 PFlop/s for artificial intelligence computations, | 2021-06-01 | 11.52 | 576.00 | NVIDIA | True | 37 | 17.25 | 3.6e+17 | 3.6e+17 | 1.8e+17 | 9.0e+16 | 0.49 | 0.55 | 3.3e+11 | 1.6e+7 | 1.8e+7 | 1.8e+7 | The cost of the procured system was almost EUR 15 million, where 35%, i.e. EUR 5.13 million, of which was paid by the EuroHPC JU. The remaining costs amounting to EUR 9.73 million was funded using the resources of the European Structural and Investment Funds | Sunway OceanLight | 0.06 | https://web.archive.org/web/20241118204224/https://www.it4i.cz/en/infrastructure/karolina | https://www.acrossproject.eu/karolina-supercomputer-belongs-to-the-worlds-top-energy-efficient-supercomputers/ | 49.83 | 18.16 | |||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 2.0e+2 | China | 2023-04-15 | Public/Private | China | Anonymized | Anonymized | True | 93 | 17.48 | 3.0e+17 | 3.0e+17 | Microsoft GPT-4 cluster | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Lawrence Livermore NL Lassen Phase 1 | Existing | Confirmed | Yes | 17.53 | 1.7e+2 | NVIDIA V100 | 2,736.00 | United States of America | US Department of Energy | 2018-11-01 | Public | 1.88 | 5.4e+7 | 7000 East Ave, Livermore, CA 94550 | Lawrence Livermore NL | V100 | Chip count from Top500 | 2018-11-01 | Listed in Top500 | 11.26 | Lawrence Livermore NL Lassen Phase 2 | 2,736.00 | NVIDIA | True | 8 | 17.53 | 3.4e+17 | 3.4e+17 | 4.3e+16 | 1.88 | 1.8e+11 | 5.4e+7 | Oak Ridge NL Summit | 0.10 | https://web.archive.org/web/20240327161123/https://www.hpcwire.com/2020/08/19/cerebras-llnl-deployment-teases-second-gen-wafer-scale-ai-chip/ | https://web.archive.org/web/20240613070121/https://www.top500.org/system/179567/ | 37.68 | -121.71 | ||||||||||||||||||
| Jean Zay Supercomputer Phase 2 | Existing | Confirmed | Yes | 17.53 | 1.7e+2 | NVIDIA Tesla V100 SXM2 | 2,696.00 | France | GENCI | 2020-06-01 | =261*4+31*8+351*4 | Public | 1.77 | 5.6e+7 | Campus universitaire d'Orsay, Batiment 506, Rue John Von Neumann, 91403 Orsay, France | Researchers in academia and industry | V100 | 261 four-GPU accelerated compute nodes with:
2 Intel Cascade Lake 6248 processors (20 cores at 2.5 GHz), namely 40 cores per node
192 GB of memory per node
4 Nvidia Tesla V100 SXM2 GPUs (32 GB)
31 eight-GPU accelerated compute nodes, currently dedicated to the AI community with:
2 Intel Cascade Lake 6226 processors (12 cores at 2.7 GHz), namely 24 cores per node
20 nodes with 384 GB of memory and 11 nodes with 768 GB of memory
8 Nvidia Tesla V100 SXM2 GPUs (32 GB)
Extension in the summer of 2020, 351 four-GPU accelerated compute nodes with:
2 Intel Cascade Lake 6248 processors (20 cores at 2.5 GHz), namely 40 cores per node
192 GB of memory per node
4 Nvidia Tesla V100 SXM2 GPUs (16 GB) | 2020-06-01 | 11.28 | Jean Zay Supercomputer Phase 1 | Jean Zay Supercomputer Phase 3 | 2,696.00 | NVIDIA | True | 14 | 17.53 | 3.4e+17 | 3.4e+17 | 4.2e+16 | 1.77 | 1.9e+11 | 5.6e+7 | Oak Ridge NL Summit | 0.10 | https://web.archive.org/web/20240528032205/http://www.idris.fr/eng/annonces/idris-extension-jean-zay-h100-eng.html | https://web.archive.org/web/20220319163248/http://www.idris.fr/eng/jean-zay/cpu/jean-zay-cpu-hw-eng.html | 48.71 | 2.18 | |||||||||||||||||
| Aleph Alpha alpha ONE | Existing | Confirmed | Yes | 17.50 | 1.6e+2 | NVIDIA A100 | 512.00 | Germany | Aleph Alpha | 2022-06-01 | Private | 0.44 | 1.2e+7 | Max-Urich-Straße 3, 13355 Berlin, Germany | German Government Agencies,Private Sector,Aleph Alpha | A100 | Calculated from Top500 | 2022-06-01 | 11.56 | 512.00 | NVIDIA | True | 76 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.44 | 3.7e+11 | 1.2e+7 | Microsoft GPT-4 cluster | 0.02 | https://web.archive.org/web/20241210021004/https://top500.org/system/180080/ | 52.54 | 13.38 | ||||||||||||||||||||
| Los Alamos NL Chicoma | Existing | Likely | Yes | 17.50 | 1.6e+2 | NVIDIA A100 | 512.00 | United States of America | US Department of Energy | 2021-12-20 | Public | 0.44 | 1.8e+7 | Los Alamos, New Mexico | Los Alamos National Laboratory | A100 | the remaining 128 are equipped with individual AMD Epyc 7713 CPUs and quadruple Nvidia A100 GPUs | 2021-12-20 | Slightly unclear when the GPUs were installed, but they seem to be installed by December 2021 at the latest | 11.56 | 512.00 | NVIDIA | True | 60 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.44 | 3.6e+11 | 1.8e+7 | DeepSeek Fire-Flyer 2 | 0.05 | https://web.archive.org/web/20240722055614/https://www.hpcwire.com/2022/03/17/los-alamos-chicoma-supercomputer-to-host-75-new-projects/ | 35.88 | -106.30 | |||||||||||||||||||
| Oracle 2020 A100 Cluster | Existing | Likely | Yes | 17.50 | 1.6e+2 | NVIDIA A100 | 512.00 | United States of America | Oracle | 2020-09-30 | Slightly uncertain if they were just saying that they could build clusters up to 512 A100 in size, or if this was actually already available and built. It seems likely it was available, though | Private | 0.45 | 1.4e+7 | Cloud | A100 | allowing customers to scale up to 512 GPUs in a single cluster | 2020-09-30 | 11.55 | 512.00 | NVIDIA | True | 20 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.45 | 3.6e+11 | 1.4e+7 | Oak Ridge NL Summit | 0.09 | https://web.archive.org/web/20240219152835/https://blogs.oracle.com/cloud-infrastructure/post/oracle-cloud-infrastructure-compute-and-high-performance-computing-roadmap-update | 39.38 | -97.92 | ||||||||||||||||||||
| Paper on BloombergGPT | Existing | Confirmed | Yes | 17.50 | 1.6e+2 | NVIDIA A100 | 512.00 | Bloomberg | 2023-03-30 | Private | 0.42 | 1.1e+7 | A100 | "This yields a total of 512 40GB A100 GPUs" | 2023-03-30 | 11.58 | True | 512.00 | NVIDIA | 95 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.42 | 3.8e+11 | 1.1e+7 | Microsoft GPT-4 cluster | 0.02 | https://arxiv.org/abs/2303.17564 | |||||||||||||||||||||||||
| Paper on CodeFuse | Existing | Confirmed | Yes | 17.50 | 1.6e+2 | NVIDIA A100 | 512.00 | Ant Group | 2023-10-10 | Private | 0.42 | 1.0e+7 | A100 | CodeFuse-13B was trained using 512 Nvidia A100 GPU cards, with a Hardware FLOPs Utilization (HFU) of approximately 60%. The training process took approximately 40 days to complete. | 2023-10-10 | 11.58 | True | 512.00 | NVIDIA | 123 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.42 | 3.8e+11 | 1.0e+7 | Tesla 10k H100 Cluster | 0.02 | https://arxiv.org/abs/2310.06266 | |||||||||||||||||||||||||
| Paper on Florence | Existing | Confirmed | Yes | 17.50 | 1.6e+2 | NVIDIA A100 SXM4 40 GB | 512.00 | Microsoft | 2021-11-22 | Private | 0.44 | 1.4e+7 | Microsoft | A100 | "The model takes 10 days to train on 512 NVIDIA-A100 GPUs
with 40GB memory per GPU." | 2021-11-22 | 11.56 | True | Microsoft Azure Voyager-EUS2 | 512.00 | NVIDIA | 59 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.44 | 3.6e+11 | 1.4e+7 | DeepSeek Fire-Flyer 2 | 0.05 | https://arxiv.org/abs/2111.11432v1 | |||||||||||||||||||||||
| Paper on Stable LM | Existing | Confirmed | Yes | 17.50 | 1.6e+2 | NVIDIA A100 SXM4 40 GB | 512.00 | Stability AI | 2023-01-18 | Private | 0.42 | 1.1e+7 | A100 | "Hardware: Stable LM 2 1.6B was trained on the Stability AI cluster across 512 NVIDIA A100 40GB GPUs (AWS P4d instances)." | 2023-01-18 | 11.58 | True | 512.00 | NVIDIA | 91 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.42 | 3.8e+11 | 1.1e+7 | Microsoft GPT-4 cluster | 0.02 | https://web.archive.org/web/20241218081859/https://huggingface.co/stabilityai/stablelm-2-1_6b | |||||||||||||||||||||||||
| Paper on StarCoder | Existing | Confirmed | Yes | 17.50 | 1.6e+2 | NVIDIA A100 | 512.00 | 2023-05-09 | 0.42 | 1.1e+7 | A100 | "We trained our model on a GPU cluster with 512 A100 80 GB GPUs distributed across 64 nodes." | 2023-05-09 | 11.58 | True | 512.00 | NVIDIA | 96 | 17.20 | 3.2e+17 | 3.2e+17 | 1.6e+17 | 8.0e+16 | 0.42 | 3.8e+11 | 1.1e+7 | Microsoft GPT-4 cluster | 0.02 | https://arxiv.org/abs/2305.06161 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 2.0e+2 | 500.00 | China | 2023-09-15 | Private | 0.40 | 1.0e+7 | China | 11.70 | 500.00 | Anonymized | Anonymized | True | 119 | 17.30 | 3.0e+17 | 3.0e+17 | 2.0e+17 | 8.0e+16 | 0.40 | 5.0e+11 | 1.0e+7 | Tesla 10k H100 Cluster | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 2.0e+2 | 1,000.00 | China | 2023-07-15 | Private | 0.50 | China | 11.78 | 1,000.00 | Anonymized | Anonymized | True | 117 | 17.48 | 3.0e+17 | 3.0e+17 | 2.0e+17 | 0.50 | 6.0e+11 | Microsoft GPT-4 cluster | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 2.0e+2 | China | 2022-05-15 | Public/Private | China | Anonymized | Anonymized | True | 73 | 17.48 | 3.0e+17 | 3.0e+17 | Microsoft GPT-4 cluster | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | 17.48 | 2.0e+2 | China | Public | China | Anonymized | Anonymized | 17.48 | 3.0e+17 | 3.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 17.48 | 2.0e+2 | China | Public/Private | China | Anonymized | Anonymized | 17.48 | 3.0e+17 | 3.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 17.48 | 2.0e+2 | China | Public | China | Anonymized | Anonymized | 17.48 | 3.0e+17 | 3.0e+17 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 2.0e+2 | China | 2023-06-15 | Public | China | Anonymized | Anonymized | True | 114 | 17.48 | 3.0e+17 | 3.0e+17 | Microsoft GPT-4 cluster | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| NSC Berzelius Phase 1 | Existing | Confirmed | Yes | 17.48 | 1.5e+2 | NVIDIA A100 SXM4 40 GB | 480.00 | Sweden | Linköping University | 2021-03-23 | Public | 0.41 | 1.3e+7 | 583 30 Linköping, Sweden | Swedish academic research groups | A100 | Berzelius, as it stands, is an Nvidia SuperPod consisting of 60 DGX A100 nodes. Each node, in turn, carries eight A100 (40GB) GPUs | 2021-03-23 | 11.56 | NSC Berzelius Phase 2 | 480.00 | NVIDIA | True | 28 | 17.18 | 3.0e+17 | 3.0e+17 | 1.5e+17 | 7.5e+16 | 0.41 | 3.6e+11 | 1.3e+7 | Sunway OceanLight | 0.05 | https://web.archive.org/web/20230712130753/https://www.hpcwire.com/2023/01/26/sweden-plans-expansion-for-nvidia-powered-berzelius-supercomputer/ | 58.39 | 15.56 | |||||||||||||||||||
| PLaMO-13B training cluster | Existing | Likely | Yes | 17.48 | 1.5e+2 | NVIDIA A100 | 480.00 | Japan | Preferred Networks Inc | 2023-09-25 | Private | 0.40 | 9.5e+6 | A100 | A100 40GB x480 | 2023-09-25 | 11.58 | True | AIST ABCI 2.0 | 480.00 | NVIDIA | 132 | 17.18 | 3.0e+17 | 3.0e+17 | 1.5e+17 | 7.5e+16 | 0.40 | 3.8e+11 | 9.5e+6 | Tesla 10k H100 Cluster | 0.02 | https://huggingface.co/pfnet/plamo-13b | https://web.archive.org/web/20250216174632/https://tech.preferred.jp/en/blog/llm-plamo/ | https://www.preferred.jp/en/news/pr20230928/ | 36.39 | 138.59 | |||||||||||||||||||
| Pawsey Supercomputing Centre Setonix | Existing | Confirmed | Yes | 17.47 | 1.5e+2 | AMD Radeon Instinct MI250X | 768.00 | Australia | Pawsey Supercomputing Centre | 2022-07-01 | Public | 0.83 | 2.0e+7 | 1 Bryce Ave, Kensington WA 6151, Australia | Academia | AMD MI250X | Calculated from Top500 | 2022-07-01 | 11.55 | 768.00 | AMD | True | 83 | 17.47 | 2.9e+17 | 2.9e+17 | 2.9e+17 | 7.3e+16 | 0.82 | 0.83 | 3.5e+11 | 2.0e+7 | Microsoft GPT-4 cluster | 0.02 | https://web.archive.org/web/20240616091315/https://www.hpcwire.com/off-the-wire/pawseys-setonix-supercomputer-fires-up-for-researchers/ | -31.99 | 115.88 | |||||||||||||||||||
| Paper on GLaM | Existing | Confirmed | Yes | 17.45 | 1.4e+2 | Google TPU v4 | 1,024.00 | United States of America | 2021-12-13 | Private | 0.37 | 9.5e+6 | TPUv4 | "Moreover, to reach similar (and slightly exceeded) scores as GPT-3, we
train using 1,024 TPU-v4 chips for 574 hours (with 280B
tokens)." | 2021-12-13 | 11.88 | True | Google TPU v4 Pod | 1,024.00 | 63 | 17.45 | 2.8e+17 | 2.8e+17 | 2.8e+17 | 0.37 | 7.5e+11 | 9.5e+6 | DeepSeek Fire-Flyer 2 | 0.05 | https://arxiv.org/abs/2112.06905 | 39.38 | -97.92 | ||||||||||||||||||||||||
| Paper on Minerva | Existing | Confirmed | Yes | 17.45 | 1.4e+2 | Google TPU v4 | 1,024.00 | United States of America | 2022-06-29 | Private | 0.37 | 9.4e+6 | TPUv4 | "the 540B model was trained for 29 days on a v4-1024." | 2022-06-29 | 11.88 | True | Google Oklahoma TPU v4 Pods | 1,024.00 | 83 | 17.45 | 2.8e+17 | 2.8e+17 | 2.8e+17 | 0.37 | 7.6e+11 | 9.4e+6 | Microsoft GPT-4 cluster | 0.02 | https://arxiv.org/abs/2206.14858 | 39.38 | -97.92 | ||||||||||||||||||||||||
| Paper on PaLI | Existing | Confirmed | Yes | 17.45 | 1.4e+2 | Google TPU v4 | 1,024.00 | United States of America | 2022-09-14 | Private | 0.37 | 9.4e+6 | TPUv4 | "The largest model, PaLI-17B, is pretrained using 1,024 GCP-TPUv4 chips for 7 days." | 2022-09-14 | 11.88 | True | Google Oklahoma TPU v4 Pods | 1,024.00 | 88 | 17.45 | 2.8e+17 | 2.8e+17 | 2.8e+17 | 0.37 | 7.6e+11 | 9.4e+6 | Microsoft GPT-4 cluster | 0.02 | https://arxiv.org/abs/2209.06794v4 | 39.38 | -97.92 | ||||||||||||||||||||||||
| Paper on ViT-22B | Existing | Confirmed | Yes | 17.45 | 1.4e+2 | Google TPU v4 | 1,024.00 | United States of America | 2023-02-10 | Private | 0.36 | 9.3e+6 | TPUv4 | "ViT-22B was trained on 1024 TPU V4 chips for 177K steps" | 2023-02-10 | 11.89 | True | Google Oklahoma TPU v4 Pods | 1,024.00 | 99 | 17.45 | 2.8e+17 | 2.8e+17 | 2.8e+17 | 0.36 | 7.8e+11 | 9.3e+6 | Microsoft GPT-4 cluster | 0.02 | https://arxiv.org/abs/2302.05442v1 | 39.38 | -97.92 | ||||||||||||||||||||||||
| University of Edinburgh DiRAC Tursa | Existing | Confirmed | Yes | 17.45 | 1.4e+2 | NVIDIA A100 | 448.00 | United Kingdom of Great Britain and Northern Ireland | Distributed Research using Advanced Computing (DiRAC) | 2021-11-01 | Public | 0.43 | 1.5e+7 | Edinburgh, Scotland | Academia | A100 | 114 GPU nodes: 4x Nvidia RedStone A100-40 per node, 640 Tensor cores, 6,912 CUDA cores | 2021-11-01 | 11.51 | 448.00 | NVIDIA | True | 61 | 17.15 | 2.8e+17 | 2.8e+17 | 1.4e+17 | 7.0e+16 | 0.38 | 0.43 | 3.3e+11 | 1.5e+7 | DeepSeek Fire-Flyer 2 | 0.04 | https://web.archive.org/web/20220121113452/https://www.epcc.ed.ac.uk/hpc-services/dirac-tursa-gpu | 55.95 | -3.19 | |||||||||||||||||||
| MosaicML MPT training cluster | Existing | Confirmed | Yes | 17.44 | 1.4e+2 | NVIDIA A100 | 440.00 | MosaicML | 2023-03-05 | 0.37 | 9.9e+6 | A100 | "over 9.5 days on 440 GPUs," | 2023-03-05 | 11.58 | 440.00 | NVIDIA | True | 106 | 17.14 | 2.7e+17 | 2.7e+17 | 1.4e+17 | 6.9e+16 | 0.37 | 3.8e+11 | 9.9e+6 | Microsoft GPT-4 cluster | 0.02 | https://www.databricks.com/blog/mpt-7b | ||||||||||||||||||||||||||
| Petrobras Dragão | Existing | Confirmed | Yes | 17.43 | 1.4e+2 | NVIDIA V100 | 2,176.00 | Brazil | Petrobras | 2021-06-01 | Public/Private | 1.66 | 4.1e+7 | Rio de Janeiro, Brazil | Petrobras | V100 | Calculated from Top500 | 2021-06-01 | 11.21 | 2,176.00 | NVIDIA | True | 42 | 17.43 | 2.7e+17 | 2.7e+17 | 3.4e+16 | 1.40 | 1.66 | 1.6e+11 | 4.1e+7 | Sunway OceanLight | 0.05 | https://web.archive.org/web/20241126215645/https://top500.org/system/179941/ | -22.91 | -43.21 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 1.0e+2 | 500.00 | China | 2022-12-15 | Private | 0.30 | China | 11.52 | True | 500.00 | Anonymized | Anonymized | 101 | 17.00 | 3.0e+17 | 3.0e+17 | 1.0e+17 | 0.30 | 3.3e+11 | Microsoft GPT-4 cluster | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 1.0e+2 | 2,000.00 | China | 2019-11-15 | Private | 1.00 | 4.0e+7 | China | 11.48 | 2,000.00 | Anonymized | Anonymized | True | 10 | 17.48 | 3.0e+17 | 3.0e+17 | 3.0e+16 | 1.00 | 3.0e+11 | 4.0e+7 | Oak Ridge NL Summit | 0.08 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Jean Zay Supercomputer Phase 3 | Existing | Confirmed | Yes | 17.41 | 1.3e+2 | NVIDIA A100 | 416.00 | France | GENCI | 2022-06-01 | Public | 2.08 | 6.4e+7 | Campus universitaire d'Orsay, Batiment 506, Rue John Von Neumann, 91403 Orsay, France | Researchers in academia and industry | V100,A100 | The power boost will come from an additional 52 nodes made up of HPE Apollo 6500 Gen10 nodes, each with eight Nvidia A100 80GB GPUs – for a total of 640GB of high-bandwidth memory (HBM2) per server – and some minor additional updates that include three Apollo 6500 Gen10 nodes, each with eight Nvidia A100 40GB PCIe GPUs | 2022-06-01 | 11.35 | Jean Zay Supercomputer Phase 2 | NVIDIA Tesla V100 SXM2 | 2,696 | 3,112.00 | NVIDIA | NVIDIA | True | 86 | 17.67 | 2.6e+17 | 2.6e+17 | 4.7e+17 | 1.1e+17 | 2.08 | 2.2e+11 | 6.4e+7 | Microsoft GPT-4 cluster | 0.02 | https://web.archive.org/web/20240528032205/http://www.idris.fr/eng/annonces/idris-extension-jean-zay-h100-eng.html | https://web.archive.org/web/20240425051414/https://www.hpcwire.com/2021/11/17/frances-jean-zay-supercomputer-gets-ai-boost-from-hpe-nvidia/ | 48.71 | 2.18 | |||||||||||||||
| Paper on BLOOM | Existing | Confirmed | Yes | 17.41 | 1.3e+2 | NVIDIA A100 | 416.00 | 2022-11-09 | 0.35 | 9.4e+6 | A100 | Training was conducted on 48 nodes, each
having 8 NVIDIA A100 80GB GPUs (a total of 384 GPUs); due to possible hardware failures during training, we also maintained a reserve of 4 spare nodes. | 2022-11-09 | 11.56 | True | 416.00 | NVIDIA | 100 | 17.11 | 2.6e+17 | 2.6e+17 | 1.3e+17 | 6.5e+16 | 0.35 | 3.7e+11 | 9.4e+6 | Microsoft GPT-4 cluster | 0.02 | https://arxiv.org/abs/2211.05100 | |||||||||||||||||||||||||||
| AWS Fast BERT Training | Existing | Confirmed | Yes | 17.41 | 1.3e+2 | NVIDIA V100 | 2,048.00 | United States of America | Amazon | 2019-12-03 | Private | 1.36 | 4.0e+7 | Amazon | V100 | With TensorFlow, we achieved unprecedented scale with 2,048 GPUs on 256 P3dn.24xlarge instances to train BERT in 62 minutes | 2019-12-03 | 11.27 | 2,048.00 | NVIDIA | True | 11 | 17.41 | 2.6e+17 | 2.6e+17 | 3.2e+16 | 1.36 | 1.9e+11 | 4.0e+7 | Oak Ridge NL Summit | 0.07 | https://web.archive.org/web/20241003202056/https://aws.amazon.com/blogs/machine-learning/amazon-web-services-achieves-fastest-training-times-for-bert-and-mask-r-cnn/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.48 | 1.0e+2 | China | 2021-06-15 | Public/Private | China | Anonymized | Anonymized | True | 45 | 3.0e+17 | 3.0e+17 | Sunway OceanLight | 0.04 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||
| Taiwania 2 | Existing | Confirmed | Yes | 17.40 | 1.3e+2 | NVIDIA Tesla V100 SXM2 | 2,016.00 | Taiwan | National Applied Research Laboratories Taiwan | 2018-11-01 | Public | 1.50 | 1.5e+7 | Hsinchu, Taiwan | Taiwan Cloud | V100 | Powered by 2,016 NVIDIA V100 Tensor Core GPUs, TAIWANIA 2 can deliver 9 petaflops of computing power to support high performance computing and AI workloads at scale | 2018-11-01 | 11.23 | 2,016.00 | NVIDIA | True | 9 | 17.40 | 2.5e+17 | 2.5e+17 | 3.2e+16 | 1.39 | 1.50 | 1.7e+11 | 4.4e+7 | 1.4e+7 | 1.5e+7 | "“Taiwania 2” system, an NT$430 million" | Oak Ridge NL Summit | 0.07 | https://web.archive.org/web/20240420074136/https://blogs.nvidia.com/blog/taiwania-2/ | https://web.archive.org/web/20240303090755/https://www.nchc.org.tw/Message/MessageView/288?mid=92&page=4 | https://www.nextplatform.com/2024/10/09/taiwans-fastest-ai-supercomputer-goes-to-foxconn/#:~:text=The%20NCHC%2C%20as%20the%20center,supercomputer%20rankings%20at%20number%2020. | 24.81 | 120.97 | |||||||||||||||
| Paper on ViT | Existing | Confirmed | Yes | 17.40 | 1.3e+2 | Google TPU v3 | 2,048.00 | 2021-06-08 | Private | 0.97 | 1.9e+7 | TPU v3 | "For models ViT-g and ViT-G, to speed up the training, we scale up the batch size at most to 32 768 and distribute the training to 2048 TPUv3 chips." | 2021-06-08 | 11.42 | True | Google MLPerf 0.7 Submission | 2,048.00 | 47 | 17.40 | 2.5e+17 | 2.5e+17 | 0.97 | 2.6e+11 | 1.9e+7 | Sunway OceanLight | 0.04 | https://arxiv.org/abs/2106.04560 | ||||||||||||||||||||||||||||
| NVIDIA In-house DGX A100 Cluster | Existing | Likely | Yes | 17.40 | 1.3e+2 | NVIDIA A100 | 400.00 | United States of America | NVIDIA | 2020-05-14 | 50 nodes x 8 A100 per node | Private | 0.35 | 1.1e+7 | NVIDIA | A100 | The tech company also built its own in-house DGX A100 supercomputer, consisting of 50 nodes | 2020-05-14 | 11.55 | 400.00 | NVIDIA | True | 17 | 17.10 | 2.5e+17 | 2.5e+17 | 1.2e+17 | 6.2e+16 | 0.35 | 3.6e+11 | 1.1e+7 | Oak Ridge NL Summit | 0.07 | https://web.archive.org/web/20241129110307/https://fedscoop.com/argonne-national-lab-ai-supercomputer/ | 39.38 | -97.92 | ||||||||||||||||||||
| Paper on Code Llama | Existing | Likely | Yes | 17.40 | 1.3e+2 | NVIDIA A100 | 400.00 | Meta AI | 2023-08-24 | Private | 0.33 | 8.9e+6 | Meta | A100 | 2023-08-24 | 11.58 | True | Meta Research SuperCluster (RSC-1) Phase 2 | 400.00 | NVIDIA | 144 | 17.10 | 2.5e+17 | 2.5e+17 | 1.2e+17 | 6.2e+16 | 0.33 | 3.8e+11 | 8.9e+6 | Microsoft GPT-4 cluster | 0.02 | https://arxiv.org/abs/2308.12950 | ||||||||||||||||||||||||
| AGH Cyfronet Athena | Existing | Confirmed | Yes | 17.38 | 1.2e+2 | NVIDIA A100 SXM4 40 GB | 384.00 | Poland | AGH University of Krakow | 2022-06-01 | Public | 0.33 | 4.2e+6 | Podole 60, 30-394 Kraków, Poland | AGH University of Krakow,Researchers in Poland,Cloud | A100 | Athena's configuration includes: 48 servers with AMD EPYC processors and 1 TB of RAM (6,144 CPU compute cores in total) as well as 384 NVIDIA A100 GPGPU cards. | 2022-06-01 | 11.56 | 384.00 | NVIDIA | True | 92 | 17.08 | 2.4e+17 | 2.4e+17 | 1.2e+17 | 6.0e+16 | 0.33 | 3.7e+11 | 1.0e+7 | 4.0e+6 | 4.2e+6 | the new supercomputer, which cost 20 million zloty (€4.15 million) to build, not including infrastructure costs | Microsoft GPT-4 cluster | 0.02 | https://web.archive.org/web/20230205120608/https://www.hpcwire.com/2022/10/10/poland-inaugurates-athena-supercomputer/ | https://web.archive.org/web/20240426185305/https://www.datacenterdynamics.com/en/news/polands-agh-launches-athena-supercomputer-in-krakow/ | https://web.archive.org/web/20240715093943/https://top500.org/system/180055/ | https://notesfrompoland.com/2022/10/05/polands-fastest-supercomputer-launched-with-blessing-from-archbishop/ | 50.02 | 19.89 | ||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | 400.00 | China | 2023-06-15 | Public/Private | 0.30 | China | 11.52 | 400.00 | Anonymized | Anonymized | True | 136 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 0.30 | 3.3e+11 | Microsoft GPT-4 cluster | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Indiana University Bloomington Jetstream2 | Existing | Confirmed | Yes | 17.35 | 1.1e+2 | NVIDIA A100 SXM4 40 GB | 360.00 | United States of America | Indiana University Bloomington | 2022-02-20 | Jetstream2 is a larger project across multiple universities. Indiana University has the largest portion. | Public | 0.31 | 1.1e+7 | Bloomington, Indiana | Academia,Researchers | A100 | GPUs: 360 (NVIDIA A100 SXM4 40GB) | 2022-02-20 | 11.56 | 360.00 | NVIDIA | True | 77 | 17.05 | 2.2e+17 | 2.2e+17 | 1.1e+17 | 5.6e+16 | 0.31 | 3.7e+11 | 9.8e+6 | 1.0e+7 | 1.1e+7 | Indiana University has been awarded a $10 million NSF grant to build ‘Jetstream 2,’ | DeepSeek Fire-Flyer 2 | 0.04 | https://web.archive.org/web/20211207065233/https://docs.jetstream-cloud.org/overview/config/ | https://web.archive.org/web/20240721082615/https://news.iu.edu/it/live/news/30451-jetstream2-ai-for-everyone | 39.17 | -86.53 | |||||||||||||||
University of Tokyo Wisteria/BDEC-01 (Aquarius) | Existing | Confirmed | Yes | 17.35 | 1.1e+2 | NVIDIA A100 | 360.00 | Japan | The University of Tokyo | 2021-06-01 | Public | 0.32 | 9.7e+6 | 7 Chome-3-1 Hongo, Bunkyo City, Tokyo 113-8654, Japan | Academia | A100 | Calculated from Top500 | 2021-06-01 | 11.54 | 360.00 | NVIDIA | True | 47 | 17.05 | 2.2e+17 | 2.2e+17 | 1.1e+17 | 5.6e+16 | 0.31 | 0.32 | 3.5e+11 | 9.7e+6 | Sunway OceanLight | 0.04 | https://web.archive.org/web/20241126224540/https://top500.org/system/179963/ | 35.72 | 139.74 | |||||||||||||||||||
| NSCC ASPIRE 2A Phase 2 | Existing | Confirmed | Yes | 17.34 | 1.1e+2 | NVIDIA A100 | 352.00 | Singapore | National Supercomputing Center Singapore | 2022-11-01 | Public | 0.30 | 8.0e+6 | Singapore | Cloud | A100 | Calculated from Top500 | 2022-11-01 | Top500 | 11.56 | NSCC ASPIRE 2A Phase 1 | 352.00 | NVIDIA | True | 109 | 17.04 | 2.2e+17 | 2.2e+17 | 1.1e+17 | 5.5e+16 | 0.30 | 3.7e+11 | 8.0e+6 | Microsoft GPT-4 cluster | 0.01 | https://drive.google.com/file/d/1ARed1E6KguYXjkL3Qta2ZKv_isej3xNn/view?usp=drive_link | https://web.archive.org/web/20240530103453/https://www.top500.org/system/180077/ | https://web.archive.org/web/20231004100738/https://www.hpcwire.com/2021/04/28/hpe-will-build-singapores-new-national-supercomputer/ | 1.35 | 103.81 | ||||||||||||||||
| Osaka University SQUID | Existing | Confirmed | Yes | 17.32 | 1.1e+2 | NVIDIA A100 SXM4 80 GB | 336.00 | Japan | Osaka University | 2021-05-01 | 42x8=336 | Public | 0.29 | 1.1e+7 | Osaka, Japan | Osaka University,Academia,Researchers | A100 | 42 Nodes; GPU:NVIDIA A100 8 units | 2021-05-01 | 11.56 | 336.00 | NVIDIA | True | 34 | 17.02 | 2.1e+17 | 2.1e+17 | 1.0e+17 | 5.2e+16 | 0.29 | 3.6e+11 | 1.1e+7 | Sunway OceanLight | 0.04 | https://archive.ph/fcLwW#selection-705.1-705.24 | 34.68 | 135.50 | |||||||||||||||||||
| PARAM Siddhi-AI Phase 1 | Existing | Confirmed | Yes | 17.32 | 1.1e+2 | NVIDIA A100 | 336.00 | India | Center for Development of Advanced Computing (C-DAC) | 2020-11-01 | 8 GPUs per node | Public | 0.29 | 9.1e+6 | Innovation Park, 34/B/1, Panchawati Rd, Mansarovar, Panchawati, Pashan, Pune, Maharashtra 411008, India | Cloud | A100 | 42 x DGX A100 Nodes | 2020-11-01 | 11.55 | AIRAWAT-PSAI Phase 2 | 336.00 | NVIDIA | True | 26 | 17.02 | 2.1e+17 | 2.1e+17 | 1.0e+17 | 5.2e+16 | 0.29 | 3.6e+11 | 9.1e+6 | Oak Ridge NL Summit | 0.06 | https://drive.google.com/file/d/1bLDkmBo4dklKX5Psch-ifNpzNDcbELXA/view?usp=sharing | https://web.archive.org/web/20240720151011/https://cdac.in/index.aspx?id=hpc_nsf_siddhi-spec | 18.53 | 73.81 | |||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | China | Public/Private | China | Anonymized | Anonymized | True | 17.30 | 2.0e+17 | 2.0e+17 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | 400.00 | China | 2023-05-15 | Public | 0.30 | China | 11.52 | 400.00 | Anonymized | Anonymized | True | 132 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 0.30 | 3.3e+11 | Microsoft GPT-4 cluster | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | 400.00 | China | 2021-05-15 | Public | 0.30 | China | 11.52 | 400.00 | Anonymized | Anonymized | True | 36 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 0.30 | 3.3e+11 | Sunway OceanLight | 0.03 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | China | 2022-09-15 | Private | China | Anonymized | Anonymized | True | 106 | 17.30 | 2.0e+17 | 2.0e+17 | Microsoft GPT-4 cluster | 0.01 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | 400.00 | China | 2018-03-15 | Public | China | 400.00 | Anonymized | Anonymized | True | 4 | 17.30 | 2.0e+17 | 2.0e+17 | Meta 2017 V100 Cluster | 0.07 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||
| Leonardo SpA davinci-1 | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | NVIDIA A100 | 320.00 | Italy | Leonardo SpA | 2020-11-01 | Public/Private | 0.28 | 8.7e+6 | Genoa, Italy | Aerospace, Defense, and Security,Leonardo SpA | A100 | Calculated from Top500 | 2020-11-01 | 11.55 | 320.00 | NVIDIA | True | 28 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 5.0e+16 | 0.28 | 3.6e+11 | 8.7e+6 | Oak Ridge NL Summit | 0.06 | https://web.archive.org/web/20240512034721/https://www.leonardo.com/en/press-release-detail/-/detail//02-12-2020-davinci-1-is-the-name-of-the-new-leonardo-supercomputer | 44.41 | 8.93 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | 300.00 | China | 2022-10-15 | Private | 0.30 | 7.0e+6 | China | 11.52 | 300.00 | Anonymized | Anonymized | True | 111 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 5.0e+16 | 0.30 | 3.3e+11 | 7.0e+6 | Microsoft GPT-4 cluster | 0.01 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Recursion BioHive-1 | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | NVIDIA A100 | 320.00 | United States of America | Recursion Pharmaceuticals | 2021-04-15 | Private | 0.27 | 8.6e+6 | 41 400 W, Salt Lake City, UT 84101 | Recursion Pharmaceuticals,Pharmaceutical Research | A100 | BioHive-1 consists of 40 NVIDIA DGX A100 640GB nodes | 2021-04-15 | 11.56 | 320.00 | NVIDIA | True | 36 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 5.0e+16 | 0.27 | 3.6e+11 | 8.6e+6 | Sunway OceanLight | 0.03 | https://web.archive.org/web/20240908095141/https://www.genengnews.com/gen-edge/from-model-to-molecule-nvidia-doubles-down-on-ai-drug-discovery/ | 40.77 | -111.90 | ||||||||||||||||||||
| University of Cambridge Wilkes-3 | Existing | Confirmed | Yes | 17.30 | 1.0e+2 | NVIDIA A100 | 320.00 | United Kingdom of Great Britain and Northern Ireland | University of Cambridge | 2021-06-01 | Public | 0.26 | 8.6e+6 | Cambridge, England | Academia,Researchers | A100 | Calculated from Top500 | 2021-06-01 | 11.59 | 320.00 | NVIDIA | True | 52 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 5.0e+16 | 0.27 | 0.26 | 3.9e+11 | 8.6e+6 | Sunway OceanLight | 0.03 | https://web.archive.org/web/20241210022223/https://top500.org/system/179930/ | 52.20 | 0.12 | |||||||||||||||||||
| Princeton Della Phase 1 | Existing | Confirmed | Yes | 17.29 | 1.0e+2 | NVIDIA A100 | 316.00 | United States of America | Princeton University | 2023-02-01 | Public | 0.26 | 7.1e+6 | 300 Forrestal Rd, Princeton, NJ 08544 | Academia | A100 | Specifications on website | 2023-02-01 | 11.58 | Princeton Della Phase 2 | 316.00 | NVIDIA | True | 126 | 16.99 | 2.0e+17 | 2.0e+17 | 9.9e+16 | 4.9e+16 | 0.26 | 3.8e+11 | 7.1e+6 | Microsoft GPT-4 cluster | 0.01 | https://web.archive.org/web/20230201020753/https://researchcomputing.princeton.edu/systems/della | 40.34 | -74.62 | |||||||||||||||||||
| NVIDIA DGX SuperPOD 2019 | Existing | Confirmed | Yes | 17.28 | 9.7e+1 | NVIDIA Tesla V100 DGXS 32 GB | 1,536.00 | NVIDIA | 2019-06-17 | Private | 0.85 | 4.7e+7 | NVIDIA | V100 | NVIDIA sets the bar once again in supercomputing, building a well-balanced system with 96 NVIDIA® DGX-2H™ servers containing 1,536 NVIDIA Tesla® V100 SXM3 GPUs. The DGX SuperPOD has earned the 22nd spot on the June 2019 TOP500 list. | 2019-06-17 | 11.35 | 1,536.00 | NVIDIA | True | 11 | 17.28 | 1.9e+17 | 1.9e+17 | 2.4e+16 | 0.85 | 2.3e+11 | 4.7e+7 | Oak Ridge NL Summit | 0.06 | https://web.archive.org/web/20240530072101/https://developer.nvidia.com/blog/dgx-superpod-world-record-supercomputing-enterprise/ | https://web.archive.org/web/20240704024932/https://developer.nvidia.com/blog/training-bert-with-gpus/ | ||||||||||||||||||||||||
| RPI AiMOS | Existing | Confirmed | Yes | 17.28 | 9.6e+1 | NVIDIA V100 | 1,512.00 | United States of America | Rensselaer Polytechnic Institute | 2019-12-11 | Public | 0.93 | 2.9e+7 | Troy, New York | Researchers,Academia | V100 | AiMOS is comprised of 252 compute nodes with a total of 504 IBM Power9 processors and 1,512 Nvidia Volta GPUs. | 2019-12-11 | Listed in Top500 | 11.31 | 1,512.00 | NVIDIA | True | 15 | 17.28 | 1.9e+17 | 1.9e+17 | 2.4e+16 | 1.01 | 0.93 | 2.0e+11 | 2.9e+7 | Oak Ridge NL Summit | 0.05 | https://web.archive.org/web/20201206000554/https://www.hpcwire.com/2019/12/11/rpi-powers-up-aimos-ai-supercomputer/ | 42.73 | -73.69 | |||||||||||||||||||
| Google TensorFlow Research Cloud | Existing | Likely | Yes | 17.27 | 9.3e+1 | Google TPU v2 | 4,096.00 | United States of America | 2017-05-17 | 1 cloud TPU = 4 TPU chips, according to the Forbes doccument and the FLOP counts we know for TPU v2 | Private | 2.69 | 3.9e+7 | Google,Cloud | TPUv2 | "Google also announced the TensorFlow Research Cloud, a 1,000-TPU (4,000 Cloud TPU Chip) supercomputer delivering 180 PetaFlops"
"the TensorFlow Research Cloud (TFRC), a cluster of 1,000 Cloud TPUs"..."Up to 180 teraflops of floating-point performance per Cloud TPU" | 2017-05-17 | 10.84 | 4,096.00 | True | 1 | 17.27 | 1.8e+17 | 1.8e+17 | 1.2e+16 | 2.69 | 6.8e+10 | 3.9e+7 | Google TensorFlow Research Cloud | 1.00 | https://web.archive.org/web/20240724203521/https://research.google/blog/introducing-the-tensorflow-research-cloud/ | https://web.archive.org/web/20240619124934/https://www.forbes.com/sites/moorinsights/2017/05/22/google-cloud-tpu-strategic-implications-for-google-nvidia-and-the-machine-learning-industry/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 9.0e+1 | 1,000.00 | China | 2018-11-15 | Private | 1.00 | 3.0e+7 | China | 11.30 | 1,000.00 | Anonymized | Anonymized | True | 12 | 17.30 | 2.0e+17 | 2.0e+17 | 2.0e+16 | 1.00 | 2.0e+11 | 3.0e+7 | Oak Ridge NL Summit | 0.05 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 9.0e+1 | 1,000.00 | China | 2018-11-15 | Private | 1.00 | 3.0e+7 | China | 11.30 | 1,000.00 | Anonymized | Anonymized | True | 12 | 17.30 | 2.0e+17 | 2.0e+17 | 2.0e+16 | 1.00 | 2.0e+11 | 3.0e+7 | Oak Ridge NL Summit | 0.05 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 9.0e+1 | 1,000.00 | China | 2018-11-15 | Private | 1.00 | 3.0e+7 | China | 11.30 | 1,000.00 | Anonymized | Anonymized | True | 12 | 17.30 | 2.0e+17 | 2.0e+17 | 2.0e+16 | 1.00 | 2.0e+11 | 3.0e+7 | Oak Ridge NL Summit | 0.05 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| SURF Snellius Phase 1 | Existing | Confirmed | Yes | 17.25 | 9.1e+1 | NVIDIA A100 | 288.00 | Netherlands | SURF | 2021-09-16 | Public | 0.25 | 9.8e+6 | Science Park 120, 1098 XG Amsterdam, Netherlands | Academia,Researchers | A100 | The 72 Phase 1 GPU nodes each contain an NVIDIA HGX A100 baseboard with 4 GPUs | 2021-09-16 | 11.56 | SURF Snellius Phase 3 | 288.00 | NVIDIA | True | 74 | 16.95 | 1.8e+17 | 1.8e+17 | 9.0e+16 | 4.5e+16 | 0.25 | 3.6e+11 | 9.8e+6 | DeepSeek Fire-Flyer 2 | 0.03 | https://web.archive.org/web/20240712185633/https://www.surf.nl/en/news/gpu-expansion-of-supercomputer-snellius-enables-even-faster-data-processing-for-dutch | https://web.archive.org/web/20240725104223/https://visualization.surf.nl/snellius-virtual-tour/#/ | https://web.archive.org/web/20240716063628/https://www.cwi.nl/en/stories/hurray-a-new-national-supercomputer-snellius/ | 52.36 | 4.95 | |||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 17.30 | 9.0e+1 | 40,000.00 | China | 2017-11-15 | Public | 20.00 | China | 40,000.00 | Anonymized | Anonymized | True | 3 | 2.0e+17 | 2.0e+17 | 20.00 | Meta 2017 V100 Cluster | 0.06 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||
| NVIDIA SATURN V Phase 3 | Existing | Unlikely | Yes | 17.24 | 8.8e+1 | NVIDIA A100 | 280.00 | United States of America | NVIDIA | 2020-05-14 | This is actually four clusters, so the minimum size of the maximum constituent cluster is 1/4 the total size
This builds off earlier versions of the SaturnV, which has been around in different forms since 2016
Very uncertain about number of chips here | Private | 3.70 | 1.1e+8 | NVIDIA | A100,V100 | "Nvidia has updated its own internal supercomputer known as SaturnV with four DGX SuperPODS - or 1,120 A100s - to add another 2.8 exaFLOPS of power"
"Saturn-V is not really one system. It sits in four different locations under one management interface" | 2020-05-14 | Uncertain about what chips are in which of the four clusters | 11.30 | NVIDIA SATURN V Phase 2 | NVIDIA V100 | 5,280 | 5,560.00 | NVIDIA | NVIDIA | 17.87 | 1.7e+17 | 1.7e+17 | 7.5e+17 | 1.3e+17 | 3.70 | 2.0e+11 | 1.1e+8 | https://web.archive.org/web/20240416013949/https://www.hpcwire.com/2020/05/14/nvidias-ampere-a100-gpu-2-5x-the-hpc-20x-the-training/ | https://web.archive.org/web/20240307200601/https://www.theregister.com/2020/05/14/ampere_nvidias_gpu/ | https://web.archive.org/web/20240730095809/https://blogs.nvidia.com/blog/dgx-superpod-a100/ | https://web.archive.org/web/20240416004832/https://www.nextplatform.com/2017/11/30/inside-nvidias-next-gen-saturn-v-ai-cluster/ | 39.38 | -97.92 | ||||||||||||||||
| U.S. Army Jean | Existing | Confirmed | Yes | 17.24 | 8.8e+1 | NVIDIA A100 | 280.00 | United States of America | US Department of Defense | 2021-05-30 | Public | 0.24 | 7.6e+6 | Aberdeen Proving Ground, Maryland, 21005 | US Department of Defense,US Army | A100 | Jean incorporates 1,202 Cascade Lake-AP CPUs, 280 A100 GPUs | 2021-05-30 | No exact confirmation on when it was operational, but it said "mid fiscal 2021" for when it was expected to be operational. Known to be operational by 2023 | 11.56 | 280.00 | NVIDIA | True | 50 | 16.94 | 1.7e+17 | 1.7e+17 | 8.7e+16 | 4.4e+16 | 0.24 | 3.6e+11 | 7.6e+6 | Sunway OceanLight | 0.03 | https://web.archive.org/web/20231004094910/https://www.hpcwire.com/2020/08/24/dod-orders-two-ai-focused-supercomputers-from-liqid/ | 39.48 | -76.14 | |||||||||||||||||||
| TU Dresden AlphaCentauri | Existing | Confirmed | Yes | 17.23 | 8.6e+1 | NVIDIA A100 SXM4 40 GB | 272.00 | Germany | Technische Universität Dresden | 2021-02-28 | Public | 0.29 | 7.3e+6 | Dresden, Germany | Academia,Researchers | A100 | At the heart of the system and essential for the computing power are a total of 272 NVIDIA A100 GPUs | 2021-02-28 | 11.47 | 272.00 | NVIDIA | True | 41 | 16.93 | 1.7e+17 | 1.7e+17 | 8.5e+16 | 4.2e+16 | 0.23 | 0.29 | 2.9e+11 | 7.3e+6 | Oak Ridge NL Summit | 0.05 | https://web.archive.org/web/20241202181003/https://tu-dresden.de/tu-dresden/newsportal/news/alpha-centauri-platzierung?set_language=en | 51.05 | 13.74 | |||||||||||||||||||
| University of Minnesota Agate | Existing | Confirmed | Yes | 17.22 | 8.3e+1 | NVIDIA A100 SXM4 40 GB | 264.00 | United States of America | University of Minnesota | 2021-11-01 | Public | 0.23 | 7.1e+6 | 117 Pleasant St SE # 5, Minneapolis, MN 55455 | Academia,Researchers | A100 | 264 NVIDIA A100 Tensor Core GPUs | 2021-11-01 | 11.56 | 264.00 | NVIDIA | True | 87 | 16.92 | 1.6e+17 | 1.6e+17 | 8.2e+16 | 4.1e+16 | 0.23 | 3.6e+11 | 7.1e+6 | DeepSeek Fire-Flyer 2 | 0.03 | https://web.archive.org/web/20241202175121/https://www-archive.msi.umn.edu/content/agate | 44.98 | -93.24 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 8.0e+1 | 300.00 | China | 2023-04-15 | Public | 0.20 | China | 11.70 | 400.00 | Anonymized | Anonymized | True | 146 | 17.00 | 2.0e+17 | 2.0e+17 | 1.0e+17 | 0.20 | 5.0e+11 | Microsoft GPT-4 cluster | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| G42 Artemis | Existing | Confirmed | Yes | 17.21 | 8.2e+1 | NVIDIA V100 | 1,296.00 | United Arab Emirates | G42 | 2019-11-01 | Public/Private | 1.23 | 2.5e+7 | Abu Dhabi, United Arab Emirates | Scientific research and cloud | V100 | Calculated from Top500 | 2019-11-01 | 11.12 | 1,296.00 | NVIDIA | True | 19 | 17.21 | 1.6e+17 | 1.6e+17 | 2.0e+16 | 0.86 | 1.23 | 1.3e+11 | 2.5e+7 | Oak Ridge NL Summit | 0.05 | https://web.archive.org/web/20240530181044/https://blogs.nvidia.com/blog/artemis-genomics/ | 24.45 | 54.38 | ||||||||||||||||||||
| JAMSTEC ZettaScaler-2.2 Gyoukou | Existing | Likely | Yes | 17.21 | 8.1e+1 | PEZY-SC2 | 10,000.00 | Japan | Japan Agency for Marine-Earth Science and Technology | 2017-11-15 | Public | 2.60 | Japan Agency for Marine-Earth Science and Technology | PEZY-SC2 | 10,000 PEZY-SC2 processor modules | 2017-11 | 10.79 | JAMSTEC ZettaScaler-2.0 Gyoukou | 10,000.00 | PEZY | True | 4 | 17.21 | 1.6e+17 | 1.6e+17 | 8.2e+16 | 4.23 | 2.60 | 6.2e+10 | Meta 2017 V100 Cluster | 0.06 | https://web.archive.org/web/20241222180018/https://en.wikipedia.org/wiki/Gyoukou | https://www.top500.org/system/179102/ | 36.39 | 138.59 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 8.0e+1 | 1,000.00 | China | 2020-07-15 | Private | 0.80 | 2.0e+7 | China | 11.40 | 1,000.00 | Anonymized | Anonymized | True | 31 | 17.30 | 2.0e+17 | 2.0e+17 | 2.0e+16 | 0.80 | 2.5e+11 | 2.0e+7 | Oak Ridge NL Summit | 0.05 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.30 | 8.0e+1 | China | Public/Private | China | Anonymized | Anonymized | True | 17.30 | 2.0e+17 | 2.0e+17 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Indiana University Big Red 200 | Existing | Confirmed | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | United States of America | Indiana University Bloomington | 2021-11-30 | Unclear exactly when this was first operational. It was first listed in Top500 in November 2021, but it was planned to be operational Fall 2020 | Public | 0.22 | 1.0e+7 | Bloomington, Indiana | Indiana University | A100 | Big Red 200 is accelerated by 256 NVIDIA A100 Tensor Core GPUs | 2021-11-30 | 11.56 | 256.00 | NVIDIA | True | 92 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.22 | 3.6e+11 | 8.7e+6 | 9.6e+6 | 1.0e+7 | is a $9.6 million Cray Shasta system | DeepSeek Fire-Flyer 2 | 0.03 | https://web.archive.org/web/20240709091139/https://news.iu.edu/it/live/news/30305-indiana-university-unveils-supercomputer-big-red | https://web.archive.org/web/20230530192726/https://www.hpcwire.com/off-the-wire/big-red-200-is-a-leap-forward-for-indiana-university-students-and-researchers/ | https://web.archive.org/web/20240915161038/https://www.top500.org/system/180020/ | https://www.hpcwire.com/2020/01/24/indiana-university-dedicates-big-red-200-cray-shasta-supercomputer/ | 39.17 | -86.53 | |||||||||||||
| NSCC ASPIRE 2A Phase 1 | Existing | Confirmed | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | Singapore | National Supercomputing Center Singapore | 2022-06-01 | Public | 0.22 | 5.8e+6 | Singapore | Cloud | A100 | Calculated from Top500 | 2022-06-01 | Top500 | 11.56 | NSCC ASPIRE 2A Phase 2 | 256.00 | NVIDIA | True | 116 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.22 | 3.7e+11 | 5.8e+6 | Microsoft GPT-4 cluster | 0.01 | https://drive.google.com/file/d/1ARed1E6KguYXjkL3Qta2ZKv_isej3xNn/view?usp=drive_link | https://web.archive.org/web/20240530103453/https://www.top500.org/system/180077/ | https://web.archive.org/web/20231004100738/https://www.hpcwire.com/2021/04/28/hpe-will-build-singapores-new-national-supercomputer/ | 1.35 | 103.81 | ||||||||||||||||
| Paper on EVA | Existing | Confirmed | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | 2022-11-14 | Public/Private | 0.22 | 5.8e+6 | A100 | "These modifications allow us to train a 1.1B CLIP with a batch size of
41k on 256× NVIDIA A100 40GB GPUs." | 2022-11-14 | 11.56 | True | 256.00 | NVIDIA | 134 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.22 | 3.7e+11 | 5.8e+6 | Microsoft GPT-4 cluster | 0.01 | https://arxiv.org/abs/2211.07636 | ||||||||||||||||||||||||||
| Paper on Polyglot | Existing | Confirmed | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | EleutherAI,Stability AI | 2023-06-04 | Private | 0.21 | 5.7e+6 | A100 | "Additionally, we received invaluable assistance from Stability AI2 , who provided access to 256 A100s (8 * 32 nodes) on HPC clusters" | 2023-06-04 | 11.58 | True | Paper on Stable Diffusion | 256.00 | NVIDIA | 161 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.21 | 3.8e+11 | 5.7e+6 | Microsoft GPT-4 cluster | 0.01 | https://arxiv.org/abs/2306.02254 | ||||||||||||||||||||||||
| Paper on Pythia | Existing | Confirmed | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | EleutherAI | 2023-04-03 | Appendix D table with GPU counts for each model size | Private | 0.21 | 5.7e+6 | Cloud | A100 | "All GPUs are A100s with 40GB of memory", | 2023-04-03 | 11.58 | True | 256.00 | NVIDIA | 148 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.21 | 3.8e+11 | 5.7e+6 | Microsoft GPT-4 cluster | 0.01 | https://arxiv.org/abs/2304.01373 | |||||||||||||||||||||||
| Paper on SAM | Existing | Confirmed | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | Meta AI | 2023-04-05 | Private | 0.21 | 5.7e+6 | A100 | "Wedistribute training across 256 GPUs," | 2023-04-05 | 11.58 | True | Meta Research SuperCluster (RSC-1) Phase 1 | 256.00 | NVIDIA | 148 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.21 | 3.8e+11 | 5.7e+6 | Microsoft GPT-4 cluster | 0.01 | https://arxiv.org/abs/2304.02643 | ||||||||||||||||||||||||
| Paper on Stable Diffusion | Existing | Likely | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | Stability AI | 2021-12-20 | Private | 0.22 | 8.8e+6 | Cloud | A100 | 2021-12-20 | 11.56 | True | 256.00 | NVIDIA | 95 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.22 | 3.6e+11 | 8.8e+6 | DeepSeek Fire-Flyer 2 | 0.03 | https://arxiv.org/abs/2112.10752 | |||||||||||||||||||||||||
| Paper on XGLM | Existing | Confirmed | Yes | 17.20 | 8.1e+1 | NVIDIA A100 | 256.00 | Meta AI | 2021-12-20 | Private | 0.22 | 8.8e+6 | Meta | A100 | "The XGLM 7.5B model was trained on 256 A100 GPUs for about 3 weeks, at a speed of 311.6k words per second" | 2021-12-20 | 11.56 | True | Meta Research SuperCluster (RSC-1) Phase 1 | 256.00 | NVIDIA | 95 | 16.90 | 1.6e+17 | 1.6e+17 | 8.0e+16 | 4.0e+16 | 0.22 | 3.6e+11 | 8.8e+6 | DeepSeek Fire-Flyer 2 | 0.03 | https://arxiv.org/abs/2112.10668 | |||||||||||||||||||||||
| Universitaet Frankfurt Goethe-NHR | Existing | Confirmed | Yes | 17.20 | 8.0e+1 | AMD Instinct MI210 | 880.00 | Germany | Goethe University Frankfurt | 2023-06-01 | Public | 0.33 | Brüningstraße 50, 65926 Frankfurt am Main, Germany | Academia,Researchers | AMD MI210 | of 880 AMD MI210 graphics cards | 2023-06-01 | 11.68 | 880.00 | AMD | True | 168 | 17.20 | 1.6e+17 | 1.6e+17 | 0.33 | 4.8e+11 | Microsoft GPT-4 cluster | 0.01 | https://archive.ph/cFXgi | 50.10 | 8.54 | ||||||||||||||||||||||||
| TACC Lonestar6 | Existing | Confirmed | Yes | 17.20 | 7.9e+1 | NVIDIA A100 PCIe | 252.00 | United States of America | University of Texas at Austin | 2022-11-14 | Public | 0.16 | 5.6e+6 | 10100 Burnet Rd, Austin, TX 78758 | Academia,Researchers | A100 | Lonestar6 hosts 84 A100 GPU nodes that are configured identically to the compute nodes with the addition of 3 NVIDIA A100 GPUs | 2022-11-14 | 11.69 | 252.00 | NVIDIA | True | 139 | 16.90 | 1.6e+17 | 1.6e+17 | 7.9e+16 | 3.9e+16 | 0.16 | 4.9e+11 | 5.6e+6 | Microsoft GPT-4 cluster | 0.01 | https://web.archive.org/web/20240808090149/https://www.hpcwire.com/off-the-wire/taccs-lonestar6-supercomputer-gets-gpu-and-server-boost/ | https://web.archive.org/web/20241006113309/https://docs.tacc.utexas.edu/hpc/lonestar6/ | 30.39 | -97.73 | |||||||||||||||||||
| Microsoft Azure Immunity Bio | Existing | Confirmed | Yes | 17.19 | 7.9e+1 | NVIDIA V100 | 1,250.00 | United States of America | Microsoft | 2020-04-01 | Private | 0.82 | 2.4e+7 | Cloud | V100 | The cluster contains over 1,250 NVIDIA V100 Tensor Core high performance graphics processing units (GPUs) | 2020-04-01 | 11.28 | 1,250.00 | NVIDIA | True | 25 | 17.19 | 1.6e+17 | 1.6e+17 | 2.0e+16 | 0.82 | 1.9e+11 | 2.4e+7 | Oak Ridge NL Summit | 0.05 | https://web.archive.org/web/20240910193454/https://immunitybio.com/immunitybio-combines-supercomputing-power-with-microsoft-azure-to-target-infection-doorway-of-the-coronavirus/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Eni HPC4 Phase 2 | Existing | Confirmed | Yes | 17.19 | 7.9e+1 | NVIDIA A100 | 250.00 | Italy | Eni | 2021-10-30 | Private | 1.99 | 6.0e+7 | Via per la Corradina, 35, 27032 Ferrera Erbognone PV, Italy | Eni | V100,A100 | Comprising the bulk of the system, there are 1,375 nodes that house two Nvidia “V100 O&G” GPUs; 125 nodes have two Nvidia A100s | 2021-10-30 | 11.33 | Eni HPC4 Phase 1 | NVIDIA V100 | 2,750 | 3,000.00 | NVIDIA | NVIDIA | True | 82 | 17.63 | 1.6e+17 | 1.6e+17 | 4.2e+17 | 8.2e+16 | 1.99 | 2.1e+11 | 6.0e+7 | DeepSeek Fire-Flyer 2 | 0.02 | https://web.archive.org/web/20220124003756/https://www.hpcwire.com/2021/10/13/eni-goes-back-to-hpe-for-hpc4-refresh-via-greenlake/ | https://web.archive.org/web/20250213075947/https://top500.org/system/179444/ | 45.10 | 8.86 | |||||||||||||||
| SberCloud Christofari | Existing | Confirmed | Yes | 17.18 | 7.6e+1 | NVIDIA V100 | 1,200.00 | Russia | SberCloud | 2019-11-09 | SberCloud, which owns this supercomputer, was originally owned by Sber, but was sold off to Noviye Vozmozhnosti in March 2022
Sberbank is majority owned by the Russian Government. | Private | 0.80 | 2.3e+7 | Skolkovo Innovation Center
Инновационный центр Сколково
Moscow Oblast
Russia | Cloud | V100 | Calculated from Top500 | 2019-11-09 | 11.27 | 1,200.00 | NVIDIA | True | 21 | 17.18 | 1.5e+17 | 1.5e+17 | 1.9e+16 | 0.80 | 1.9e+11 | 2.3e+7 | Oak Ridge NL Summit | 0.04 | https://web.archive.org/web/20230517125435/https://en.wikipedia.org/wiki/Christofari | 55.53 | 37.77 | ||||||||||||||||||||
| EuroHPC Vega | Existing | Confirmed | Yes | 17.18 | 7.6e+1 | NVIDIA A100 PCIe | 240.00 | Slovenia | EuroHPC JU,Institute of Information Science (IZUM) | 2021-04-01 | Public | 0.15 | 2.1e+7 | Prešernova ulica 17, 2000 Maribor, Slovenia | Cloud,European researchers | A100 | With 960 CPU nodes (overall 1920 CPUs AMD Epyc 7H12 – 122000 cores) and 60 GPU nodes (overall 240 GPUs NVidia A100) | 2021-04-01 | 11.69 | 240.00 | NVIDIA | True | 49 | 16.87 | 1.5e+17 | 1.5e+17 | 7.5e+16 | 3.7e+16 | 0.15 | 4.8e+11 | 6.5e+6 | 2.0e+7 | 2.1e+7 | "The Vega supercomputer was jointly procured by Slovenia and the EU with an investment of €17.2 million"
This supercomputer contains a lot of CPUs, so the price might be higher than normal | Sunway OceanLight | 0.03 | https://web.archive.org/web/20210420154820/https://www.hpcwire.com/off-the-wire/vega-first-new-eurohpc-supercomputer-to-be-delivered-in-the-eu-will-be-operational-april-2021/ | https://archive.ph/fCnkQ | https://www.hpcwire.com/off-the-wire/vega-online-the-eu-first-eurohpc-supercomputer-is-operational/ | 46.56 | 15.65 | |||||||||||||||
Japan Atomic Energy Agency and Quantum and Radiological Science and Technology HPE SGI8600 | Existing | Confirmed | Yes | 17.13 | 6.9e+1 | NVIDIA Tesla V100 SXM2 | 1,088.00 | Japan | Japan Atomic Energy Agency | 2020-11-01 | Public | 0.71 | 2.3e+7 | Japan | Japan Atomic Energy Agency,National Institutes for Quantum and Radiological Science and Technology | V100 | Calculated from Top500 | 2020-11-01 | 11.28 | 1,088.00 | NVIDIA | True | 41 | 17.13 | 1.4e+17 | 1.4e+17 | 1.7e+16 | 0.71 | 1.9e+11 | 2.3e+7 | Oak Ridge NL Summit | 0.04 | https://web.archive.org/web/20240331015933/https://ccse.jaea.go.jp/computer/index_eng.html | 36.39 | 138.59 | |||||||||||||||||||||
| Petrobras Atlas | Existing | Confirmed | Yes | 17.13 | 6.9e+1 | NVIDIA V100 | 1,088.00 | Brazil | Petrobras | 2020-06-01 | Public/Private | 0.98 | 2.1e+7 | Rio de Janeiro, Brazil | Petrobras | V100 | Calculated from Top500 | 2020-06-01 | 11.14 | 1,088.00 | NVIDIA | True | 31 | 17.13 | 1.4e+17 | 1.4e+17 | 1.7e+16 | 0.71 | 0.98 | 1.4e+11 | 2.1e+7 | Oak Ridge NL Summit | 0.04 | https://web.archive.org/web/20241210015105/https://www.top500.org/system/179854/ | -22.91 | -43.21 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.00 | 7.0e+1 | 500.00 | China | 2018-12-15 | Public | 0.50 | 2.0e+7 | China | 10.08 | 200 | 700.00 | Anonymized | Anonymized | True | 17 | 15.78 | 1.0e+17 | 1.0e+17 | 6.0e+15 | 4.0e+16 | 0.50 | 1.2e+10 | 2.0e+7 | Oak Ridge NL Summit | 0.04 | 35.49 | 101.90 | |||||||||||||||||||||||||
| Jean Zay Supercomputer Phase 1 | Existing | Confirmed | Yes | 17.12 | 6.6e+1 | NVIDIA Tesla V100 SXM2 32 GB | 1,044.00 | France | GENCI | 2019-06-01 | Public | 0.70 | 3.1e+7 | Campus universitaire d'Orsay, Batiment 506, Rue John Von Neumann, 91403 Orsay, France | Researchers in academia and industry | V100 | the system encompasses 1,528 Intel next-generation Xeon nodes and 261 GPU nodes, each with four Nvidia Tesla V100 (32GB) GPUs, 1,044 in all. | 2019-06-01 | 11.27 | Jean Zay Supercomputer Phase 2 | 1,044.00 | NVIDIA | True | 18 | 17.12 | 1.3e+17 | 1.3e+17 | 1.6e+16 | 0.70 | 1.9e+11 | 2.2e+7 | 2.8e+7 | 3.1e+7 | approximately €25 million for the first partition of the Jean-Zay machine | Oak Ridge NL Summit | 0.04 | https://web.archive.org/web/20230123052742/https://www.hpcwire.com/2019/01/22/france-to-deploy-ai-focused-supercomputer-jean-zay/ | https://news.cnrs.fr/opinions/computing-the-cost-of-computation | 48.71 | 2.18 | ||||||||||||||||
| Paper on DALL-E | Existing | Confirmed | Yes | 17.11 | 6.5e+1 | NVIDIA V100 | 1,024.00 | OpenAI | 2021-02-24 | Private | 0.66 | 1.9e+7 | OpenAI | V100 | We trained the model using 1024, 16 GB NVIDIA V100 GPUs | 2021-02-24 | 11.29 | True | Azure OpenAI GPT-3 Cluster | 1,024.00 | NVIDIA | 50 | 17.11 | 1.3e+17 | 1.3e+17 | 1.6e+16 | 0.66 | 1.9e+11 | 1.9e+7 | Oak Ridge NL Summit | 0.04 | https://arxiv.org/abs/2102.12092 | ||||||||||||||||||||||||
| Paper on RoBERTa | Existing | Confirmed | Yes | 17.11 | 6.5e+1 | NVIDIA Tesla V100 DGXS 32 GB | 1,024.00 | United States of America | Meta AI | 2019-07-26 | Private | 0.57 | 3.2e+7 | We pretrain our model using
1024 V100 GPUs for approximately one day | 2019-07-26 | 11.35 | True | Meta 2017 V100 Cluster | 1,024.00 | NVIDIA | 21 | 17.11 | 1.3e+17 | 1.3e+17 | 1.6e+16 | 0.57 | 2.3e+11 | 3.2e+7 | Oak Ridge NL Summit | 0.04 | https://arxiv.org/abs/1907.11692 | 39.38 | -97.92 | |||||||||||||||||||||||
| Preferred Networks MN-2 | Existing | Confirmed | Yes | 17.11 | 6.5e+1 | NVIDIA V100 | 1,024.00 | Japan | Preferred Networks Inc | 2019-07-30 | Private | 0.68 | 2.0e+7 | Japan | Preferred Networks | V100 | GPUs: 1,024 | 2019-07-30 | 11.27 | 1,024.00 | NVIDIA | True | 21 | 17.11 | 1.3e+17 | 1.3e+17 | 1.6e+16 | 0.68 | 1.9e+11 | 2.0e+7 | Oak Ridge NL Summit | 0.04 | https://web.archive.org/web/20241126234046/https://fuse.wikichip.org/news/3063/japanese-ai-startup-preferred-networks-designed-a-custom-half-petaflops-training-chip/ | 36.39 | 138.59 | |||||||||||||||||||||
| Google TPUv3 POD Generic | Existing | Confirmed | Yes | 17.10 | 6.4e+1 | Google TPU v3 | 1,024.00 | 2019-05-07 | Refers to the general commercially available pod, not to a specific cluster using the pod | Private | 0.50 | 1.0e+7 | TPU v3 | A TPU v3 Pod is composed of 1024 chips interconnected with high-speed links | 2019-05-07 | 11.40 | 1,024.00 | True | 18 | 17.10 | 1.3e+17 | 1.3e+17 | 0.50 | 2.5e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.04 | https://web.archive.org/web/20231128061217/https://cloud.google.com/blog/products/ai-machine-learning/googles-scalable-supercomputers-for-machine-learning-cloud-tpu-pods-are-now-publicly-available-in-beta | https://web.archive.org/web/20240626015135/https://cloud.google.com/tpu/docs/v3 | |||||||||||||||||||||||||||
| Paper on GShard | Existing | Confirmed | Yes | 17.10 | 6.4e+1 | Google TPU v3 | 1,024.00 | 2020-10-30 | Private | 0.49 | 9.8e+6 | TPU v3 | 2020-10-30 | 11.41 | True | Google MLPerf 0.7 Submission | 1,024.00 | 44 | 17.10 | 1.3e+17 | 1.3e+17 | 0.49 | 2.6e+11 | 9.8e+6 | Oak Ridge NL Summit | 0.04 | https://arxiv.org/abs/2006.16668 | |||||||||||||||||||||||||||||
| Paper on LaMDA | Existing | Confirmed | Yes | 17.10 | 6.4e+1 | Google TPU v3 | 1,024.00 | 2022-01-20 | Private | 0.48 | 9.7e+6 | TPU v3 | "We pre-trained LaMDA on 1024 TPU-v3 chips" | 2022-01-20 | 11.42 | True | Google MLPerf 0.7 Submission | 1,024.00 | 110 | 17.10 | 1.3e+17 | 1.3e+17 | 0.48 | 2.6e+11 | 9.7e+6 | DeepSeek Fire-Flyer 2 | 0.02 | https://arxiv.org/abs/2201.08239 | ||||||||||||||||||||||||||||
| Paper on Meena | Existing | Confirmed | Yes | 17.10 | 6.4e+1 | Google TPU v3 | 1,024.00 | 2020-01-27 | Private | 0.49 | 9.9e+6 | TPU v3 | "We trained our best model for 30 days on a TPUv3 Pod (2,048 TPU cores)" | 2020-01-27 | 11.41 | True | Google TPUv3 POD Generic | 1,024.00 | 29 | 17.10 | 1.3e+17 | 1.3e+17 | 0.49 | 2.6e+11 | 9.9e+6 | Oak Ridge NL Summit | 0.04 | https://arxiv.org/abs/2001.09977 | ||||||||||||||||||||||||||||
| Paper on Noisy Student | Existing | Confirmed | Yes | 17.10 | 6.4e+1 | Google TPU v3 | 1,024.00 | 2019-11-11 | Private | 0.50 | 1.0e+7 | TPU v3 | 2019-11-11 | 11.40 | True | Google TPUv3 POD Generic | 1,024.00 | 26 | 17.10 | 1.3e+17 | 1.3e+17 | 0.50 | 2.5e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.04 | https://arxiv.org/abs/1911.04252v4 | |||||||||||||||||||||||||||||
| Paper on Pseudo Labels | Existing | Confirmed | Yes | 17.10 | 6.4e+1 | Google TPU v3 | 1,024.00 | 2020-03-23 | Private | 0.49 | 9.9e+6 | TPU v3 | "Specifically, our training process runs on a cluster of 2,048
TPUv3 cores." | 2020-03-23 | 11.41 | True | Google TPUv3 POD Generic | 1,024.00 | 30 | 17.10 | 1.3e+17 | 1.3e+17 | 0.49 | 2.6e+11 | 9.9e+6 | Oak Ridge NL Summit | 0.04 | https://arxiv.org/abs/2003.10580 | ||||||||||||||||||||||||||||
| Paper on Switch transformer | Existing | Likely | Yes | 17.10 | 6.4e+1 | Google TPU v3 | 1,024.00 | 2021-01-11 | The TPU count is from the expert parallelism (1 expert per core) with 2048 experts and 2 tensor cores per TPU v3. | Private | 0.48 | 9.8e+6 | TPU v3 | 2021-01-11 | 11.42 | True | Google MLPerf 0.7 Submission | 1,024.00 | 50 | 17.10 | 1.3e+17 | 1.3e+17 | 0.48 | 2.6e+11 | 9.8e+6 | Oak Ridge NL Summit | 0.04 | https://arxiv.org/abs/2101.03961 | ||||||||||||||||||||||||||||
| SiDi IARA | Existing | Confirmed | Yes | 17.10 | 6.3e+1 | NVIDIA A100 | 200.00 | Brazil | SiDi | 2021-06-01 | Private | 0.17 | 5.4e+6 | São Paulo, Brazil | SiDi | A100 | Calculated from Top500 | 2021-06-01 | 11.56 | 200.00 | NVIDIA | True | 83 | 16.80 | 1.2e+17 | 1.2e+17 | 6.2e+16 | 3.1e+16 | 0.17 | 3.6e+11 | 5.4e+6 | Sunway OceanLight | 0.02 | https://web.archive.org/web/20250131005940/https://top500.org/system/179931/ | -23.55 | -46.65 | ||||||||||||||||||||
| Texas A&M Grace | Existing | Confirmed | Yes | 17.10 | 6.3e+1 | NVIDIA A100 | 200.00 | United States of America | Texas A&M | 2021-11-01 | Public | 0.17 | 6.8e+6 | College Station, Texas | Academia | A100 | Calculated from Top500 | 2021-11-01 | 11.56 | 200.00 | NVIDIA | True | 108 | 16.80 | 1.2e+17 | 1.2e+17 | 6.2e+16 | 3.1e+16 | 0.17 | 3.6e+11 | 6.8e+6 | DeepSeek Fire-Flyer 2 | 0.02 | https://web.archive.org/web/20241203212138/https://campustechnology.com/articles/2020/11/19/new-texas-am-supercomputer-grace-goes-online-in-december.aspx?s=ct_it_031220 | 30.62 | -96.35 | ||||||||||||||||||||
| Argonne NL Sophia (Formerly Theta) | Existing | Confirmed | Yes | 17.08 | 6.1e+1 | NVIDIA A100 | 192.00 | United States of America | US Department of Energy | 2020-05-30 | This was originally an addition of a GPU portion of the Theta supercomputer. The Theta supercomputer has since been retired, and these GPUs were shifted to the Sophia supercomputer | Public | 0.17 | 5.3e+6 | Argonne National Laboratory and general scientific community | A100 | the augmented Theta architecture adds 24 NVIDIA DGX A100 nodes to the existing system. Each DGX A100 node comprises eight NVIDIA A100 Tensor Core GPUs | 2020-05-30 | 11.55 | 192.00 | NVIDIA | True | 38 | 16.78 | 1.2e+17 | 1.2e+17 | 6.0e+16 | 3.0e+16 | 0.17 | 3.6e+11 | 5.3e+6 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20240926172236/https://docs.alcf.anl.gov/sophia/hardware-overview/machine-overview/ | https://web.archive.org/web/20240930034827/https://www.datacenterdynamics.com/en/news/argonne-national-lab-retires-theta-supercomputer/ | https://web.archive.org/web/20240927032645/https://www.hpcwire.com/off-the-wire/argonne-augments-theta-supercomputer-with-gpus-to-accelerate-coronavirus-research/ | 39.38 | -97.92 | ||||||||||||||||||
| Atos Spartan2 | Existing | Confirmed | Yes | 17.08 | 6.1e+1 | NVIDIA A100 | 192.00 | France | 2020-11-01 | Private | 0.19 | 5.2e+6 | France | Cloud | A100 | Calculated from Top500 | 2020-11-01 | 11.50 | 192.00 | NVIDIA | True | 52 | 16.78 | 1.2e+17 | 1.2e+17 | 6.0e+16 | 3.0e+16 | 0.17 | 0.19 | 3.2e+11 | 5.2e+6 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20241210023753/https://top500.org/system/179893/ | 47.82 | 2.62 | ||||||||||||||||||||
| Commissariat a l'Energie Atomique Topaze | Existing | Confirmed | Yes | 17.08 | 6.1e+1 | NVIDIA A100 | 192.00 | France | French Alternative Energies and Atomic Energy Commission (CEA) | 2021-06-23 | Public | 0.16 | 5.2e+6 | Essonne, France | Researchers,Industry | A100 | Calculated from Top500 | 2021-06-23 | 11.56 | 192.00 | NVIDIA | True | 89 | 16.78 | 1.2e+17 | 1.2e+17 | 6.0e+16 | 3.0e+16 | 0.16 | 3.6e+11 | 5.2e+6 | Sunway OceanLight | 0.02 | https://web.archive.org/web/20240617010640/https://atos.net/en/2021/press-release_2021_06_23/topaze-a-new-computer-at-the-ccrt-co-designed-by-atos-and-the-cea-to-meet-the-challenges-of-high-performance-computing-and-data-processing | 48.62 | 2.43 | ||||||||||||||||||||
| Dell Rattler | Existing | Confirmed | Yes | 17.08 | 6.1e+1 | NVIDIA A100 | 192.00 | United States of America | Dell Technologies | 2021-11-01 | Private | 0.16 | 6.6e+6 | Austin, Texas | Dell | A100 | Calculated from Top500 | 2021-11-01 | 11.56 | 192.00 | NVIDIA | True | 110 | 16.78 | 1.2e+17 | 1.2e+17 | 6.0e+16 | 3.0e+16 | 0.16 | 3.6e+11 | 6.6e+6 | DeepSeek Fire-Flyer 2 | 0.02 | https://web.archive.org/web/20240716193131/https://www.top500.org/system/180040/ | 30.27 | -97.74 | ||||||||||||||||||||
High-Performance Computing Center Stuttgart Hawk | Existing | Confirmed | Yes | 17.08 | 6.1e+1 | NVIDIA A100 | 192.00 | Germany | University of Stuttgart | 2021-09-20 | Public | 0.16 | 6.6e+6 | Nobelstraße 19, 70569 Stuttgart, Germany | Academia,Industry | A100 | Hawk's upgrade consists of 24 HPE Apollo 6500 Gen10 Plus systems with 192 NVIDIA A100 GPUs based on the NVIDIA Ampere architecture | 2021-09-20 | 11.56 | 192.00 | NVIDIA | True | 98 | 16.78 | 1.2e+17 | 1.2e+17 | 6.0e+16 | 3.0e+16 | 0.16 | 3.6e+11 | 6.6e+6 | DeepSeek Fire-Flyer 2 | 0.02 | https://web.archive.org/web/20240714005409/https://www.hlrs.de/news/detail/hawk-upgrade-artificial-intelligence | 48.74 | 9.10 | ||||||||||||||||||||
| MIT LLSC TX-GAIA | Existing | Confirmed | Yes | 17.05 | 5.7e+1 | NVIDIA V100 | 896.00 | United States of America | US Department of Defense | 2019-09-26 | Public | 2.57 | 1.7e+7 | MIT Lincoln Laboratory Supercomputing Center
50 Water St, Holyoke, MA 01040 | MIT | V100 | has just installed a new GPU-accelerated supercomputer, powered by 896 NVIDIA Tensor Core V100 GPUs | 2019-09-26 | Confirmed by NVIDIA blog | 10.64 | 896.00 | NVIDIA | True | 24 | 17.05 | 1.1e+17 | 1.1e+17 | 1.4e+16 | 0.60 | 2.57 | 4.4e+10 | 1.7e+7 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20240805184855/https://developer.nvidia.com/blog/mit-lincoln-laboratory-supercomputing-center-tx-gaia/ | 42.21 | -72.61 | |||||||||||||||||||
| Nagoya University Flow Type II (Furo) | Existing | Confirmed | Yes | 17.04 | 5.6e+1 | NVIDIA Tesla V100 SXM2 | 884.00 | Japan | Nagoya University | 2020-07-30 | Public | 0.58 | 1.8e+7 | Nagoya, Japan | Academia | V100 | Calculated from Top500 | 2020-07-30 | 11.28 | 884.00 | NVIDIA | True | 45 | 17.04 | 1.1e+17 | 1.1e+17 | 1.4e+16 | 0.58 | 1.9e+11 | 1.8e+7 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20231001220341/https://developer.nvidia.com/blog/nagoya-university-to-install-new-15-petaflop-gpu-accelerated-supercomputer/ | 35.17 | 136.90 | |||||||||||||||||||||
| CSCS Piz Daint Phase 2 | Existing | Confirmed | Yes | 17.04 | 5.5e+1 | NVIDIA Tesla P100 PCIe 16GB | 5,704.00 | Switzerland | ETH Domain | 2017-06-01 | Public | 4.37 | 1.2e+8 | Swiss Universities,Academia | P100 | NVIDIA® Tesla® P100 16GB - 5704 Nodes | 2017-06-01 | 10.40 | CSCS Piz Daint Phase 1 | 5,704.00 | NVIDIA | True | 2 | 17.04 | 1.1e+17 | 1.1e+17 | 5.4e+16 | 3.35 | 4.37 | 2.5e+10 | 1.2e+8 | Google TensorFlow Research Cloud | 0.59 | https://web.archive.org/web/20240925211711/https://www.cscs.ch/computers/piz-daint | https://web.archive.org/web/20240519145824/https://www.cscs.ch/publications/news/2017/piz-daint-one-of-the-most-powerful-supercomputers-in-the-world | 46.80 | 8.23 | |||||||||||||||||||
| Japanese Research Institute 1 Supercomputer 1 | Existing | Confirmed | Yes | 17.02 | 5.3e+1 | NVIDIA Tesla V100 SXM2 | 832.00 | Japan | 2019-06-01 | 0.55 | 1.8e+7 | Japan | V100 | Calculated from Top500 | 2019-06-01 | 11.27 | 832.00 | NVIDIA | True | 21 | 17.02 | 1.0e+17 | 1.0e+17 | 1.3e+16 | 0.55 | 1.9e+11 | 1.8e+7 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20241205003621/https://top500.org/system/179698/ | 36.39 | 138.59 | ||||||||||||||||||||||||
| Microsoft Azure ND v2 Largest Stated | Existing | Confirmed | Yes | 17.00 | 5.1e+1 | NVIDIA V100 | 800.00 | United States of America | Microsoft | 2019-11-18 | This was the largest size mentioned. There were likely smaller ones as well, and possibly multiple of this size | Private | 0.53 | 1.6e+7 | Cloud,Azure | V100 | Built to handle the most demanding AI and high performance computing applications, the largest deployments of Azure’s new NDv2 instance rank among the world’s fastest supercomputers, offering up to 800 NVIDIA V100 Tensor Core GPUs | 2019-11-18 | 11.27 | 800.00 | NVIDIA | True | 31 | 17.00 | 1.0e+17 | 1.0e+17 | 1.3e+16 | 0.53 | 1.9e+11 | 1.6e+7 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20240523151746/https://nvidianews.nvidia.com/news/nvidia-announces-scalable-gpu-accelerated-supercomputer-in-the-microsoft-azure-cloud | 39.38 | -97.92 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 17.00 | 5.0e+1 | China | 2021-10-15 | Public | China | Anonymized | Anonymized | True | 106 | 17.00 | 1.0e+17 | 1.0e+17 | DeepSeek Fire-Flyer 2 | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||
| MTS Grom | Existing | Confirmed | Yes | 17.00 | 5.0e+1 | NVIDIA A100 | 160.00 | Russia | MTS Systems Corporation (MTS) | 2021-06-01 | Private | 0.14 | 4.3e+6 | Russia | MTS,Cloud | A100 | Calculated from Top500 | 2021-06-01 | 11.56 | 160.00 | NVIDIA | True | 91 | 16.70 | 1.0e+17 | 1.0e+17 | 5.0e+16 | 2.5e+16 | 0.14 | 3.6e+11 | 4.3e+6 | Sunway OceanLight | 0.02 | https://web.archive.org/web/20240915111238/https://www.top500.org/system/179955/ | 61.99 | 96.67 | ||||||||||||||||||||
| NVIDIA Tethys | Existing | Confirmed | Yes | 17.00 | 5.0e+1 | NVIDIA A100 | 160.00 | United States of America | NVIDIA | 2021-11-01 | Private | 0.13 | 5.5e+6 | NVIDIA | A100 | Calculated from Top500 | 2021-11-01 | 11.60 | 160.00 | NVIDIA | True | 121 | 16.70 | 1.0e+17 | 1.0e+17 | 5.0e+16 | 2.5e+16 | 0.14 | 0.13 | 4.0e+11 | 5.5e+6 | DeepSeek Fire-Flyer 2 | 0.02 | https://web.archive.org/web/20250126160950/https://top500.org/system/180034/ | 39.38 | -97.92 | ||||||||||||||||||||
| Vingroup VinAI Research Superpod | Existing | Likely | Yes | 17.00 | 5.0e+1 | NVIDIA A100 | 160.00 | Vietnam | Vingroup | 2021-05-30 | Private | 0.14 | 4.3e+6 | Vietnam | VinAI Research | A100 | who hopes to have in May the new cluster of 20 DGX A100 systems linked together with an NVIDIA Mellanox HDR 200Gb/s InfiniBand network | 2021-05-30 | 11.56 | 160.00 | NVIDIA | True | 78 | 16.70 | 1.0e+17 | 1.0e+17 | 5.0e+16 | 2.5e+16 | 0.14 | 3.6e+11 | 4.3e+6 | Sunway OceanLight | 0.02 | https://web.archive.org/web/20241213185718/https://blogs.nvidia.com/blog/ai-superpod-vinai/ | 14.32 | 108.34 | ||||||||||||||||||||
| SAKURA Internet 2019 V100 Supercomputer | Existing | Confirmed | Yes | 16.98 | 4.9e+1 | NVIDIA Tesla V100 SXM2 | 768.00 | Japan | Sakura Internet | 2019-06-01 | Private | 0.51 | 1.7e+7 | Cloud | V100 | Calculated from Top500 | 2019-06-01 | 11.27 | 768.00 | NVIDIA | True | 22 | 16.98 | 9.6e+16 | 9.6e+16 | 1.2e+16 | 0.51 | 1.9e+11 | 1.7e+7 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20241213053529/https://top500.org/system/179696/ | 36.39 | 138.59 | ||||||||||||||||||||||
| Simon Fraser University/Compute Canada Cedar | Existing | Confirmed | Yes | 16.98 | 4.9e+1 | NVIDIA V100 | 768.00 | Canada | Compute Canada | 2020-06-01 | Public | 0.56 | 1.5e+7 | 8888 University Dr W, Burnaby, BC V5A 1S6, Canada | Academia,Researchers | V100 | Calculated from Top500 | 2020-06-01 | 11.24 | 768.00 | NVIDIA | True | 45 | 16.98 | 9.6e+16 | 9.6e+16 | 1.2e+16 | 0.50 | 0.56 | 1.7e+11 | 1.5e+7 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20240915102455/https://top500.org/system/179859/ | 49.28 | -122.91 | ||||||||||||||||||||
| Petrobras Fênix Phase 2 | Existing | Confirmed | Yes | 16.95 | 4.5e+1 | NVIDIA V100 | 720.00 | Brazil | Petrobras | 2020-06-01 | Public/Private | 0.70 | 1.4e+7 | Rio de Janeiro, Brazil | Petrobras | V100 | Calculated from Top500 | 2020-06-01 | 11.11 | Petrobras Fênix Phase 1 | 720.00 | NVIDIA | True | 47 | 16.95 | 9.0e+16 | 9.0e+16 | 1.1e+16 | 0.47 | 0.70 | 1.3e+11 | 1.4e+7 | Oak Ridge NL Summit | 0.03 | https://web.archive.org/web/20241119033423/https://top500.org/system/179681/ | -22.91 | -43.21 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.95 | 5.0e+1 | 700.00 | China | 2019-11-15 | Private | 0.50 | 1.0e+7 | China | 11.26 | 700.00 | Anonymized | Anonymized | True | 30 | 16.95 | 9.0e+16 | 9.0e+16 | 1.0e+16 | 0.50 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.03 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Samsung SSC-21 Scalable Module | Existing | Confirmed | Yes | 16.95 | 4.5e+1 | NVIDIA A100 | 144.00 | Korea (Republic of) | Samsung | 2021-11-01 | This is different than Samsung SSC-21 | Private | 0.18 | 4.9e+6 | Korea (Republic of) | Samsung | A100 | Calculated from Top500 | 2021-11-01 | Listed in Top500 | 11.39 | 144.00 | NVIDIA | True | 128 | 16.65 | 9.0e+16 | 9.0e+16 | 4.5e+16 | 2.2e+16 | 0.12 | 0.18 | 2.5e+11 | 4.9e+6 | DeepSeek Fire-Flyer 2 | 0.01 | https://web.archive.org/web/20241210015858/https://top500.org/system/180042/ | |||||||||||||||||||
| Calcul Québec Béluga | Existing | Confirmed | Yes | 16.93 | 4.3e+1 | NVIDIA V100 | 688.00 | Canada | Compute Canada | 2019-04-26 | Public | 0.44 | 1.4e+7 | 1100 Notre-Dame St W, Montreal, Quebec H3C 1K3, Canada | Academia | V100 | The GPU subsection, meanwhile, was measured at 2.278 Linpack petaflops and consists of 172 Intel-based servers utilizing 688 Nvidia Volta GPUs | 2019-04-26 | 11.29 | 688.00 | NVIDIA | True | 19 | 16.93 | 8.6e+16 | 8.6e+16 | 1.1e+16 | 0.46 | 0.44 | 2.0e+11 | 1.4e+7 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20201024200651/https://www.hpcwire.com/2019/04/26/beluga-supercomputer-now-serving-canadian-researchers/ | 45.49 | -73.56 | ||||||||||||||||||||
| Preferred Networks MN-1b | Decommissioned | Confirmed | Yes | 16.92 | 4.2e+1 | NVIDIA V100 | 512.00 | Japan | Preferred Networks Inc | 2018-07-30 | Decommissioned July 2022 | Private | 0.94 | 1.0e+7 | Japan | Preferred Networks | V100,P100 | The system featured 1,024 Nvidia Tesla P100 GPUs achieving a peak compute of 1.39 petaFLOPS and 9.3 petaFLOPS of SP from the GPU. At the time, the MN-1 ranked 1st in Japan and 12th in the world on the TOP500 among industrial supercomputers. In July 2018 PFN enhanced the MN-1 by adding 512 additional Tesla V100 GPUs | 2018-07-30 | 10.95 | NVIDIA P100 | 1,024 | 1,536.00 | NVIDIA | NVIDIA | 13 | 16.92 | 8.3e+16 | 8.3e+16 | 1.8e+16 | 0.94 | 8.9e+10 | 1.0e+7 | 2021-07 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20241126234046/https://fuse.wikichip.org/news/3063/japanese-ai-startup-preferred-networks-designed-a-custom-half-petaflops-training-chip/ | https://web.archive.org/web/20241119124332/https://www.preferred.jp/en/projects/supercomputers/ | 36.39 | 138.59 | ||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 700.00 | China | 2018-06-15 | Private | 0.50 | 1.0e+7 | China | 11.20 | 700.00 | Anonymized | Anonymized | True | 12 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.50 | 1.6e+11 | 1.0e+7 | Meta 2017 V100 Cluster | 0.03 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2018-07-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 14 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-07-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 28 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-06-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 26 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 52 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 52 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 52 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| NCI Australia Gadi | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | NVIDIA Tesla V100 SXM2 | 640.00 | Australia | National Computational Infrastructure Australia | 2020-01-23 | Public | 0.42 | 1.3e+7 | Canberra ACT 2601, Australia | Academia,Researchers | V100 | 160 nodes each containing four Nvidia V100 GPUs and two 24-core Intel Xeon Scalable 'Cascade Lake' processor | 2020-01-23 | 11.28 | 640.00 | NVIDIA | True | 40 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.42 | 1.9e+11 | 1.3e+7 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20241014023433/https://www.nci.org.au/our-systems/hpc-systems | -28.05 | 153.42 | |||||||||||||||||||||
| PLUG training cluster | Existing | Likely | Yes | 16.90 | 4.0e+1 | NVIDIA A100 | 128.00 | 2021-04-19 | 0.11 | 3.5e+6 | A100 | 2021-04-19 | 11.56 | 128.00 | NVIDIA | True | 86 | 16.60 | 8.0e+16 | 8.0e+16 | 4.0e+16 | 2.0e+16 | 0.11 | 3.6e+11 | 3.5e+6 | Sunway OceanLight | 0.01 | https://mp.weixin.qq.com/s/DAQomIkDa52Sef-ruyH5qg | ||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-07-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | True | 600.00 | Anonymized | Anonymized | 31 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 37 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 37 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-11-15 | Public | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 40 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2020-07-15 | Public | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 64 | 16.90 | 8.0e+16 | 8.0e+16 | 1.0e+16 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| RPI Supercomputer 2 | Existing | Confirmed | Yes | 16.88 | 3.8e+1 | NVIDIA V100 | 608.00 | United States of America | Rensselaer Polytechnic Institute | 2020-06-01 | Public | 0.40 | 1.2e+7 | Troy, New York | Researchers,Academia | V100 | Calculated from Top500 | 2020-06-01 | 11.28 | 608.00 | NVIDIA | True | 63 | 16.88 | 7.6e+16 | 7.6e+16 | 9.5e+15 | 0.40 | 1.9e+11 | 1.2e+7 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20241008115946/https://www.top500.org/system/179852/ | 42.73 | -73.69 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-07-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | True | 600.00 | Anonymized | Anonymized | 32 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-07-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | True | 600.00 | Anonymized | Anonymized | 32 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 41 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 41 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 41 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2020-07-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | True | 600.00 | Anonymized | Anonymized | 66 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 64 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.90 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.30 | 600.00 | Anonymized | Anonymized | True | 64 | 16.90 | 8.0e+16 | 8.0e+16 | 9.0e+15 | 0.40 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-06-15 | Public/Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 71 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Oak Ridge NL Titan | Decommissioned | Confirmed | Yes | 16.87 | 3.7e+1 | NVIDIA Tesla K20X | 18,688.00 | United States of America | US Department of Energy | 2012-10-29 | Public | 15.81 | 2.8e+8 | Oak Ridge, Tennessee, United States | Oak Ridge National Laboratory | K20X | It used 18,688 CPUs paired with an equal number of GPUs | 2012-10-29 | 18,688.00 | NVIDIA | 1 | 7.4e+16 | 7.4e+16 | 8.79 | 15.81 | 2.8e+8 | 2019-08-02 | Oak Ridge NL Titan | 1.00 | https://web.archive.org/web/20241123132125/https://en.wikipedia.org/wiki/Titan_(supercomputer) | 36.01 | -84.27 | ||||||||||||||||||||||||
| Microsoft Research Hypercluster | Existing | Confirmed | Yes | 16.86 | 3.6e+1 | NVIDIA V100 | 576.00 | United States of America | Microsoft | 2019-11-01 | Private | 0.38 | 1.1e+7 | Microsoft | V100 | Calculated from Top500 | 2019-11-01 | Listed in Top500 | 11.27 | 576.00 | NVIDIA | True | 47 | 16.86 | 7.2e+16 | 7.2e+16 | 9.0e+15 | 0.38 | 1.9e+11 | 1.1e+7 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20240530033239/https://www.top500.org/system/179775/ | 39.38 | -97.92 | |||||||||||||||||||||
| NVIDIA Circe | Existing | Confirmed | Yes | 16.86 | 3.6e+1 | NVIDIA Tesla V100 SXM2 | 576.00 | United States of America | NVIDIA | 2018-11-01 | Private | 0.40 | 1.3e+7 | NVIDIA | V100 | Calculated from Top500 | 2018-11-01 | 11.26 | 576.00 | NVIDIA | True | 22 | 16.86 | 7.2e+16 | 7.2e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.3e+7 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20241224052126/https://www.top500.org/system/179564/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Petrobras Fênix Phase 1 | Existing | Confirmed | Yes | 16.86 | 3.6e+1 | NVIDIA V100 | 576.00 | Brazil | Petrobras | 2019-06-01 | Public/Private | 0.52 | 1.1e+7 | Rio de Janeiro, Brazil | Petrobras | V100 | Calculated from Top500 | 2019-06-01 | 11.14 | Petrobras Fênix Phase 2 | 576.00 | NVIDIA | True | 29 | 16.86 | 7.2e+16 | 7.2e+16 | 9.0e+15 | 0.38 | 0.52 | 1.4e+11 | 1.1e+7 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20241119033423/https://top500.org/system/179681/ | -22.91 | -43.21 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-07-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | 600.00 | Anonymized | Anonymized | True | 79 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-11-15 | Public/Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 93 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-07-15 | Public/Private | 0.40 | 1.0e+7 | China | 11.24 | True | 600.00 | Anonymized | Anonymized | 80 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-07-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | True | 600.00 | Anonymized | Anonymized | 80 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2019-07-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | True | 600.00 | Anonymized | Anonymized | 37 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 76 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 76 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 50 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 50 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 50 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 50 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2019-07-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | True | 600.00 | Anonymized | Anonymized | 37 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2019-07-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | True | 600.00 | Anonymized | Anonymized | 37 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 76 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 76 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 76 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 4.0e+1 | 600.00 | China | 2020-06-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 600.00 | Anonymized | Anonymized | True | 76 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 500.00 | Anonymized | Anonymized | True | 57 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 500.00 | Anonymized | Anonymized | True | 57 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.40 | 1.0e+7 | China | 11.24 | 500.00 | Anonymized | Anonymized | True | 57 | 16.85 | 7.0e+16 | 7.0e+16 | 9.0e+15 | 0.40 | 1.8e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-11-15 | Public/Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 113 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-07-15 | Public/Private | 0.30 | 1.0e+7 | China | 11.37 | True | 500.00 | Anonymized | Anonymized | 98 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-06-15 | Public/Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 92 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2019-07-15 | Public/Private | 0.30 | 1.0e+7 | China | 11.37 | True | 500.00 | Anonymized | Anonymized | 40 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-11-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 113 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-07-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | True | 500.00 | Anonymized | Anonymized | 98 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-07-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | True | 500.00 | Anonymized | Anonymized | 98 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 60 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 60 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-06-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 92 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-06-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 92 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2020-06-15 | Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 92 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.85 | 3.0e+1 | 500.00 | China | 2019-11-15 | Public/Private | 0.30 | 1.0e+7 | China | 11.37 | 500.00 | Anonymized | Anonymized | True | 60 | 16.85 | 7.0e+16 | 7.0e+16 | 8.0e+15 | 0.30 | 2.3e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| TACC Frontera | Existing | Confirmed | Yes | 16.81 | 3.2e+1 | NVIDIA Quadro RTX 5000 | 360.00 | United States of America | University of Texas at Austin | 2019-08-31 | Public | 0.48 | 8.7e+6 | 10100 Burnet Rd, Austin, TX 78758 | Academia,Researchers | V100,360 RTX 5000s 448 V100s | A 360 NVIDIA Quadro RTX 5000 GPU (graphics processing unit) system submerged in liquid coolant racks...
An IBM POWER9-hosted system with 448 NVIDIA V100 GPUs will provide additional performance | 2019-08-31 | 11.12 | NVIDIA V100 | 448 | 808.00 | NVIDIA | NVIDIA | True | 43 | 16.81 | 6.4e+16 | 6.4e+16 | 1.1e+16 | 0.48 | 1.3e+11 | 8.7e+6 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20240816171341/https://new.nsf.gov/news/nsf-funded-leadership-class-computing-center | https://web.archive.org/web/20230714051808/https://texascale.org/2019/feature-stories/operation-frontera/ | 30.39 | -97.73 | |||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 65 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | True | 500.00 | Anonymized | Anonymized | 65 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 65 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-06-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 31 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Paper on CogView | Existing | Confirmed | Yes | 16.81 | 3.2e+1 | NVIDIA Tesla V100 SXM2 32 GB | 512.00 | Alibaba | 2021-05-26 | Private | 0.33 | 1.1e+7 | V100 | We train the model with batch size of 6,144 sequences (6.7 million tokens per batch) for 144,000 steps on 512 V100 GPUs (32GB). | 2021-05-26 | 11.29 | True | Alibaba MLPerf 0.7 Submission | 512.00 | NVIDIA | 145 | 16.81 | 6.4e+16 | 6.4e+16 | 8.0e+15 | 0.33 | 1.9e+11 | 1.1e+7 | Sunway OceanLight | 0.01 | https://arxiv.org/abs/2105.13290 | |||||||||||||||||||||||||
| Paper on Megatron-LM | Existing | Confirmed | Yes | 16.81 | 3.2e+1 | NVIDIA Tesla V100 DGXS 32 GB | 512.00 | United States of America | NVIDIA | 2019-09-17 | Private | 0.28 | 1.6e+7 | V100 | 2019-09-17 | 11.35 | True | NVIDIA Circe | 512.00 | NVIDIA | 44 | 16.81 | 6.4e+16 | 6.4e+16 | 8.0e+15 | 0.28 | 2.3e+11 | 1.6e+7 | Oak Ridge NL Summit | 0.02 | https://arxiv.org/pdf/1909.08053 | 39.38 | -97.92 | |||||||||||||||||||||||
| Paper on SEER | Existing | Confirmed | Yes | 16.81 | 3.2e+1 | NVIDIA V100 | 512.00 | Meta AI | 2021-03-02 | Private | 0.33 | 9.7e+6 | Meta | V100 | "Training this model on 1 billion images requires 114,890 training iterations for a batch size of 8, 704 images, summing to 8 days of training over 512 GPUs." | 2021-03-02 | 11.29 | True | Meta 2017 V100 Cluster | 512.00 | NVIDIA | 136 | 16.81 | 6.4e+16 | 6.4e+16 | 8.0e+15 | 0.33 | 1.9e+11 | 9.7e+6 | Sunway OceanLight | 0.01 | https://arxiv.org/abs/2103.01988 | ||||||||||||||||||||||||
| Paper on SEER Training | Existing | Confirmed | Yes | 16.81 | 3.2e+1 | NVIDIA Tesla V100 DGXS 32 GB | 512.00 | Meta AI | 2021-03-05 | Private | 0.27 | 1.5e+7 | 2021-03-05 | 11.37 | True | Meta 2017 V100 Cluster | 512.00 | NVIDIA | 136 | 16.81 | 6.4e+16 | 6.4e+16 | 8.0e+15 | 0.27 | 2.3e+11 | 1.5e+7 | Sunway OceanLight | 0.01 | https://web.archive.org/web/20240921010425/https://developer.nvidia.com/blog/facebook-self-supervised-ai/ | Cluster performance database | ||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-07-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 41 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| University of Oxford Jade2 | Existing | Confirmed | Yes | 16.80 | 3.2e+1 | NVIDIA Tesla V100 SXM2 | 504.00 | United Kingdom of Great Britain and Northern Ireland | University of Oxford | 2020-11-01 | Planned to be decommissioned in 2025 | Public | 0.20 | 1.0e+7 | The Hartree Centre STFC Laboratory Sci-Tech Daresbury Warrington, Warrington WA4 4AD, United Kingdom | Academia | V100 | Calculated from Top500 | 2020-11-01 | 11.49 | 504.00 | NVIDIA | True | 133 | 16.80 | 6.3e+16 | 6.3e+16 | 7.9e+15 | 0.33 | 0.20 | 3.1e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20241119014213/https://top500.org/system/179867/ | 53.39 | -2.59 | |||||||||||||||||||
| Paper on ALBERT | Existing | Confirmed | Yes | 16.80 | 3.2e+1 | Google TPU v3 | 512.00 | 2019-09-26 | Private | 0.25 | 5.0e+6 | TPU v3 | 2019-09-26 | 11.40 | True | Google TPUv3 POD Generic | 512.00 | 48 | 16.80 | 6.3e+16 | 6.3e+16 | 0.25 | 2.5e+11 | 5.0e+6 | Oak Ridge NL Summit | 0.02 | https://arxiv.org/abs/1909.11942 | |||||||||||||||||||||||||||||
| Paper on ALIGN | Existing | Likely | Yes | 16.80 | 3.2e+1 | Google TPU v3 | 512.00 | 2021-02-11 | Private | 0.24 | 4.9e+6 | TPU v3 | 2021-02-11 | 11.42 | True | Google MLPerf 0.7 Submission | 512.00 | 140 | 16.80 | 6.3e+16 | 6.3e+16 | 0.24 | 2.6e+11 | 4.9e+6 | Oak Ridge NL Summit | 0.02 | https://arxiv.org/abs/2102.05918 | |||||||||||||||||||||||||||||
| Paper on T5 | Existing | Likely | Yes | 16.80 | 3.2e+1 | Google TPU v3 | 512.00 | 2019-10-23 | Private | 0.25 | 5.0e+6 | TPU v3 | 2019-10-23 | 11.40 | True | Google TPUv3 POD Generic | 512.00 | 48 | 16.80 | 6.3e+16 | 6.3e+16 | 0.25 | 2.5e+11 | 5.0e+6 | Oak Ridge NL Summit | 0.02 | https://arxiv.org/abs/1910.10683 | |||||||||||||||||||||||||||||
| Paper on XLM-R | Existing | Confirmed | Yes | 16.80 | 3.2e+1 | NVIDIA Tesla V100 SXM2 32 GB | 500.00 | Meta AI | 2020-04-08 | Private | 0.33 | 1.0e+7 | 2020-04-08 | 11.28 | True | Meta 2017 V100 Cluster | 500.00 | NVIDIA | 84 | 16.80 | 6.3e+16 | 6.3e+16 | 7.8e+15 | 0.33 | 1.9e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | https://arxiv.org/pdf/1911.02116.pdf | Cluster performance database | ||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-07-15 | Public | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 43 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 74 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| MIT Supercloud | Existing | Confirmed | Yes | 16.79 | 3.1e+1 | NVIDIA V100 | 488.00 | United States of America | Massachusetts Institute of Technology (MIT) | 2020-09-21 | Public | 0.32 | 9.3e+6 | 50 Water St, Holyoke, MA 01040 | Academia,MIT | V100 | Total GPUs: 488 | 2020-09-21 | 11.28 | 488.00 | NVIDIA | True | 126 | 16.79 | 6.1e+16 | 6.1e+16 | 7.6e+15 | 0.32 | 1.9e+11 | 9.3e+6 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20200921072313/https://supercloud.mit.edu/systems-and-software | 42.20 | -72.60 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2020-11-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 140 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 75 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2020-07-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | True | 500.00 | Anonymized | Anonymized | 121 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2018-11-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 23 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 75 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2020-06-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 112 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 75 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-06-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 32 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 75 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 75 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2020-06-15 | Private | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 112 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Paper on M6-T | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | NVIDIA Tesla V100 SXM2 32 GB | 480.00 | Alibaba | 2021-05-31 | Private | 0.31 | 9.9e+6 | V100 | Weadvance our models to 1 trillion parameters and successfully implement it on solely 480 NVIDIA V100-32GB GPUs, in comparison with the recent SOTAs on 2048 TPU cores. | 2021-05-31 | 11.29 | True | Alibaba MLPerf 0.7 Submission | 480.00 | NVIDIA | 165 | 16.78 | 6.0e+16 | 6.0e+16 | 7.5e+15 | 0.31 | 1.9e+11 | 9.9e+6 | Sunway OceanLight | 0.01 | https://arxiv.org/abs/2105.15082 | |||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-07-15 | Public | 0.30 | 1.0e+7 | China | 11.30 | True | 500.00 | Anonymized | Anonymized | 44 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2020-11-15 | Public | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 140 | 16.78 | 6.0e+16 | 6.0e+16 | 8.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Eni HPC4 Phase 1 | Existing | Confirmed | Yes | 16.77 | 3.0e+1 | NVIDIA P100 | 3,170.00 | Italy | Eni | 2018-01-18 | Private | 2.48 | Via per la Corradina, 35, 27032 Ferrera Erbognone PV, Italy | Eni | P100 | Calculated from Top500 | 2018-01-18 | 10.38 | Eni HPC4 Phase 2 | 3,170.00 | NVIDIA | True | 8 | 16.77 | 5.9e+16 | 5.9e+16 | 2.9e+16 | 1.82 | 2.48 | 2.4e+10 | Meta 2017 V100 Cluster | 0.02 | https://web.archive.org/web/20250213075947/https://top500.org/system/179444/ | https://web.archive.org/web/20241223213852/https://www.top500.org/news/eni-launches-186-petaflop-supercomputer/ | 45.10 | 8.86 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 16.78 | 3.0e+1 | China | 2017-11-15 | Private | 1.00 | China | 5,000.00 | Anonymized | Anonymized | True | 5 | 6.0e+16 | 6.0e+16 | 1.00 | Meta 2017 V100 Cluster | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2020-07-15 | Private | 0.30 | 1.0e+7 | China | 11.30 | True | 500.00 | Anonymized | Anonymized | 134 | 16.78 | 6.0e+16 | 6.0e+16 | 7.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2020-07-15 | Public | 0.30 | 1.0e+7 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 135 | 16.78 | 6.0e+16 | 6.0e+16 | 7.0e+15 | 0.30 | 2.0e+11 | 1.0e+7 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| CSCS Piz Daint Phase 1 | Existing | Confirmed | Yes | 16.76 | 2.9e+1 | NVIDIA Tesla P100 PCIe 16GB | 3,040.00 | Switzerland | ETH Domain | 2016-11-01 | Public | 1.81 | 3.2e+7 | Swiss Universities,Academia | P100 | Calculated from Top500 | 2016-11-01 | 10.50 | CSCS Piz Daint Phase 2 | 3,040.00 | NVIDIA | True | 2 | 16.76 | 5.8e+16 | 5.8e+16 | 2.9e+16 | 1.81 | 3.2e+10 | 3.2e+7 | Oak Ridge NL Titan | 0.79 | https://web.archive.org/web/20240925211711/https://www.cscs.ch/computers/piz-daint | https://web.archive.org/web/20240519145824/https://www.cscs.ch/publications/news/2017/piz-daint-one-of-the-most-powerful-supercomputers-in-the-world | 46.80 | 8.23 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 500.00 | China | 2019-11-15 | Public | 0.30 | 9.0e+6 | China | 11.30 | 500.00 | Anonymized | Anonymized | True | 86 | 16.78 | 6.0e+16 | 6.0e+16 | 7.0e+15 | 0.30 | 2.0e+11 | 9.0e+6 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.78 | 3.0e+1 | 1,000.00 | China | 2018-08-15 | Public | 0.30 | China | 11.30 | 1,000.00 | Anonymized | Anonymized | True | 20 | 16.78 | 6.0e+16 | 6.0e+16 | 3.0e+16 | 0.30 | 2.0e+11 | Oak Ridge NL Summit | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||||
| Continental DGX Supercomputer | Existing | Confirmed | Yes | 16.75 | 2.8e+1 | NVIDIA V100 | 448.00 | Germany | Continental AG | 2020-07-28 | Private | 0.29 | 8.5e+6 | Frankfurt, Germany | Continental | V100 | 53 Nvidia DGX server, 448 NVIDIA Tesla V100 GPUs | 2020-07-28 | 11.28 | 448.00 | NVIDIA | True | 139 | 16.75 | 5.6e+16 | 5.6e+16 | 7.0e+15 | 0.29 | 1.9e+11 | 8.5e+6 | Oak Ridge NL Summit | 0.02 | https://archive.ph/Ms4E5 | https://web.archive.org/web/20240417205902/https://www.continental.com/en-us/press/press-releases/continental-supercomputer-nvidia/ | 50.11 | 8.68 | ||||||||||||||||||||
| TACC Longhorn | Decommissioned | Confirmed | Yes | 16.75 | 2.8e+1 | NVIDIA Tesla V100 SXM2 | 448.00 | United States of America | University of Texas at Austin | 2019-11-01 | Decommissioned in November 2022 | Public | 0.30 | 9.6e+6 | Austin, Texas | Academia,Researchers | V100 | The researchers ran simulations of the coronavirus spike protein on 100 out of 448 of Longhorn’s NVIDIA V100 GPUs. | 2019-11-01 | 11.27 | 448.00 | NVIDIA | 88 | 16.75 | 5.6e+16 | 5.6e+16 | 7.0e+15 | 0.30 | 1.9e+11 | 9.6e+6 | 2022-11-30 | Oak Ridge NL Summit | 0.02 | https://web.archive.org/web/20240804184834/https://insidehpc.com/2021/08/taccs-longhorn-gpu-subsystem-helps-researchers-detect-potential-covid-19-virus-vulnerability/ | 30.27 | -97.74 | ||||||||||||||||||||
"Center for Advanced Intelligence Project, RIKEN, RAIDEN" | Existing | Confirmed | Yes | 16.72 | 2.6e+1 | NVIDIA V100 | 416.00 | Japan | RIKEN | 2018-06-01 | Public | 0.20 | 8.2e+6 | Japan, 〒103-0027 Tokyo, Chuo City, Nihonbashi, 1 Chome−4−1, Nihonbashi 1-chome Mitsui Building, 15階 | RIKEN,Researchers | V100 | Calculated from Top500 | 2018-06-01 | 11.41 | 416.00 | NVIDIA | True | 17 | 16.72 | 5.2e+16 | 5.2e+16 | 6.5e+15 | 0.29 | 0.20 | 2.6e+11 | 8.2e+6 | Meta 2017 V100 Cluster | 0.02 | https://web.archive.org/web/20240528101359/https://www.top500.org/system/179413/ | 35.67 | 139.78 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 16.70 | 3.0e+1 | China | 2017-11-15 | Private | 1.00 | China | 4,000.00 | Anonymized | Anonymized | True | 7 | 5.0e+16 | 5.0e+16 | 1.00 | Meta 2017 V100 Cluster | 0.02 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 3.0e+1 | 400.00 | China | 2018-06-15 | Private | 0.30 | 8.0e+6 | China | 11.22 | 400.00 | Anonymized | Anonymized | True | 19 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.30 | 1.7e+11 | 8.0e+6 | Meta 2017 V100 Cluster | 0.02 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 3.0e+1 | 400.00 | China | 2019-11-15 | Private | 0.30 | 8.0e+6 | China | 11.22 | 400.00 | Anonymized | Anonymized | True | 91 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.30 | 1.7e+11 | 8.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 3.0e+1 | 400.00 | China | 2019-11-15 | Private | 0.30 | 8.0e+6 | China | 11.22 | 400.00 | Anonymized | Anonymized | True | 91 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.30 | 1.7e+11 | 8.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 3.0e+1 | 400.00 | China | 2019-11-15 | Private | 0.30 | 8.0e+6 | China | 11.22 | 400.00 | Anonymized | Anonymized | True | 91 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.30 | 1.7e+11 | 8.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 3.0e+1 | 400.00 | China | 2019-11-15 | Private | 0.30 | 8.0e+6 | China | 11.22 | 400.00 | Anonymized | Anonymized | True | 91 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.30 | 1.7e+11 | 8.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 3.0e+1 | 400.00 | China | 2019-11-15 | Private | 0.30 | 8.0e+6 | China | 11.22 | 400.00 | Anonymized | Anonymized | True | 91 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.30 | 1.7e+11 | 8.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 3.0e+1 | 400.00 | China | 2019-06-15 | Private | 0.30 | 8.0e+6 | China | 11.22 | 400.00 | Anonymized | Anonymized | True | 40 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.30 | 1.7e+11 | 8.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Paper on AlphaStar | Existing | Confirmed | Yes | 16.67 | 2.4e+1 | Google TPU v3 | 384.00 | DeepMind | 2019-10-30 | Private | 0.19 | 3.8e+6 | TPU v3 | 2019-10-30 | 11.40 | True | Google TPUv3 POD Generic | 384.00 | 62 | 16.67 | 4.7e+16 | 4.7e+16 | 0.19 | 2.5e+11 | 3.8e+6 | Oak Ridge NL Summit | 0.01 | https://www.nature.com/articles/s41586-019-1724-z | ||||||||||||||||||||||||||||
Laboratório Nacional de Computação Científica Santos Dumont | Existing | Confirmed | Yes | 16.66 | 2.3e+1 | NVIDIA Tesla V100 SXM2 | 368.00 | Brazil | Laboratório Nacional de Computação Científica | 2019-11-01 | Public | 0.25 | 7.8e+6 | Av. Getulio Vargas, 333 - Quitandinha, Petrópolis - RJ, 25651-075, Brazil | Academia,Researchers | V100 | Calculated from Top500 | 2019-11-01 | 11.27 | 368.00 | NVIDIA | True | 99 | 16.66 | 4.6e+16 | 4.6e+16 | 5.8e+15 | 0.25 | 1.9e+11 | 7.8e+6 | Oak Ridge NL Summit | 0.01 | https://web.archive.org/web/20240516065301/https://en.wikipedia.org/wiki/Santos_Dumont_(supercomputer) | -22.53 | -43.22 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 2.0e+1 | 3,000.00 | China | 2012-03-15 | Public | 1.00 | 1.0e+7 | China | 3,000.00 | Anonymized | Anonymized | True | 1 | 5.0e+16 | 5.0e+16 | 3.0e+15 | 1.00 | 1.0e+7 | Sugon 6000 in NSC Shenzhen | 1.00 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||
| GSIC TSUBAME 3.0 | Existing | Confirmed | Yes | 16.66 | 2.3e+1 | NVIDIA Tesla P100 SXM2 | 2,156.00 | Japan | Tokyo Institute of Technology | 2017-11-01 | Public | 1.53 | 4.3e+7 | Ookayama, Japan | Academia,Tokyo Tech | P100 | Calculated from Top500 | 2017-11-01 | 10.48 | 2,156.00 | NVIDIA | True | 9 | 16.66 | 4.6e+16 | 4.6e+16 | 2.3e+16 | 1.52 | 1.53 | 3.0e+10 | 4.3e+7 | Meta 2017 V100 Cluster | 0.02 | https://web.archive.org/web/20230716170424/https://www.hpcwire.com/2017/02/16/tokyo-techs-tsubame-3-0-supercomputer/ | 34.66 | 133.92 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 2.0e+1 | 400.00 | China | 2018-11-15 | Private | 0.20 | 7.0e+6 | China | 11.40 | 400.00 | Anonymized | Anonymized | True | 33 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.20 | 2.5e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.70 | 2.0e+1 | 400.00 | China | 2019-06-15 | Private | 0.20 | 7.0e+6 | China | 11.40 | 400.00 | Anonymized | Anonymized | True | 44 | 16.70 | 5.0e+16 | 5.0e+16 | 6.0e+15 | 0.20 | 2.5e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 400.00 | China | 2018-11-15 | Private | 0.30 | 7.0e+6 | China | 11.12 | 400.00 | Anonymized | Anonymized | True | 34 | 16.60 | 4.0e+16 | 4.0e+16 | 6.0e+15 | 0.20 | 0.30 | 1.3e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2019-11-15 | Public/Private | 0.30 | 7.0e+6 | China | 11.12 | 300.00 | Anonymized | Anonymized | True | 105 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 0.30 | 1.3e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | ||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 7.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 35 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Public | 0.20 | 7.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 36 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| CSC Puhti | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | NVIDIA V100 | 320.00 | Finland | CSC – IT Center for Science | 2019-09-03 | Public | 0.21 | 6.2e+6 | Tehdaskatu 15, 87100 Kajaani, Finland | Academia,Researchers | V100 | has 80 nodes with a total peak performance of 2.7 Petaflops. Each node has four Nvidia Volta V100 GPUs | 2019-09-03 | 11.27 | 320.00 | NVIDIA | True | 65 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.21 | 1.9e+11 | 6.2e+6 | Oak Ridge NL Summit | 0.01 | https://archive.ph/2BTmI | 64.23 | 27.70 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Public | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Immunity Bio Cluster | Existing | Likely | Yes | 16.60 | 2.0e+1 | NVIDIA V100 | 320.00 | United States of America | ImmunityBio | 2020-04-01 | Private | 0.21 | 6.1e+6 | ImmunityBio | V100 | Similarly, ImmunityBio has deployed its 320 GPU cluster, which has always been optimized for and dedicated to molecular modeling of proteins, antibodies, antivirals, and targeted small molecule drugs. | 2020-04-01 | Uncertain exactly what type of GPU it is | 11.28 | 320.00 | NVIDIA | True | 119 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.21 | 1.9e+11 | 6.1e+6 | Oak Ridge NL Summit | 0.01 | https://web.archive.org/web/20240910193454/https://immunitybio.com/immunitybio-combines-supercomputing-power-with-microsoft-azure-to-target-infection-doorway-of-the-coronavirus/ | 39.38 | -97.92 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-06-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 22 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Meta 2017 V100 Cluster | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2019-06-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 49 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2020-07-15 | Public | 0.20 | 7.0e+6 | China | 11.30 | True | 300.00 | Anonymized | Anonymized | 159 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2019-07-15 | Public | 0.20 | 7.0e+6 | China | 11.30 | True | 300.00 | Anonymized | Anonymized | 62 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 7.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| RPI AiMOSx NPL Cluster | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | NVIDIA V100 | 320.00 | United States of America | Rensselaer Polytechnic Institute | 2020-06-01 | Public | 0.21 | 6.1e+6 | Troy, New York | Researchers,Academia | V100 | 40 nodes, 2x20 Core Xeon processors with 8 v100 Nvidia GPUS and 768 GB of RAM each | 2020-06-01 | Possibly actually has 608 V100s | 11.28 | True | RPI AiMOS | 320.00 | NVIDIA | 147 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.21 | 1.9e+11 | 6.1e+6 | Oak Ridge NL Summit | 0.01 | https://web.archive.org/web/20210814143702/https://cci.rpi.edu/center-capabilities | 42.73 | -73.69 | |||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 37 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| University of Tsukuba Cygnus | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | NVIDIA V100 | 320.00 | Japan | University of Tsukuba | 2019-04-30 | Public | 0.21 | 6.3e+6 | Tsukuba, Japan | Academia | V100 | Calculated from Top500 | 2019-04-30 | 11.27 | 320.00 | NVIDIA | True | 39 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.21 | 1.9e+11 | 6.3e+6 | Oak Ridge NL Summit | 0.01 | https://web.archive.org/web/20241006142717/https://www.top500.org/system/179688/ | 36.08 | 140.08 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Public | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 52 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| SENAI CIMATEC Ogbon Cimatec/Petrobras | Existing | Confirmed | Yes | 16.59 | 2.0e+1 | NVIDIA Tesla V100 SXM2 | 312.00 | Brazil | SENAI CIMATEC | 2019-11-01 | Public | 0.21 | 6.7e+6 | Salvador, Brazil | Researchers | V100 | Calculated from Top500 | 2019-11-01 | 11.27 | 312.00 | NVIDIA | True | 127 | 16.59 | 3.9e+16 | 3.9e+16 | 4.9e+15 | 0.21 | 1.9e+11 | 6.7e+6 | Oak Ridge NL Summit | 0.01 | https://web.archive.org/web/20241210024406/https://www.top500.org/system/179703/ | -12.98 | -38.48 | |||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Public | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 53 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2019-06-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 68 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2019-06-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 68 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Public | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 54 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| IBM DYEUS | Existing | Confirmed | Yes | 16.57 | 1.9e+1 | NVIDIA V100 | 296.00 | United States of America | IBM | 2019-11-18 | Private | 0.20 | 5.8e+6 | The Weather Company | V100 | the IBM POWER9 supercomputer sports 296 NVIDIA V100 GPUs to chew through so much atmospheric data | 2019-11-18 | 11.27 | 296.00 | NVIDIA | True | 136 | 16.57 | 3.7e+16 | 3.7e+16 | 4.6e+15 | 0.20 | 1.9e+11 | 5.8e+6 | Oak Ridge NL Summit | 0.01 | https://web.archive.org/web/20240807082029/https://blogs.nvidia.com/blog/new-ibm-supercomputer-optimized-for-gpus-to-bring-better-weather-predictions-worldwide/ | 39.38 | -97.92 | ||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Public | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 55 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Public | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 55 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.60 | 2.0e+1 | 300.00 | China | 2018-11-15 | Private | 0.20 | 6.0e+6 | China | 11.30 | 300.00 | Anonymized | Anonymized | True | 55 | 16.60 | 4.0e+16 | 4.0e+16 | 5.0e+15 | 0.20 | 2.0e+11 | 6.0e+6 | Oak Ridge NL Summit | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| NVIDIA SATURN V Volta | Existing | Confirmed | Yes | 16.52 | 1.7e+1 | NVIDIA V100 | 264.00 | United States of America | NVIDIA | 2017-11-01 | Seems likely that this cluster was folded into Saturn V Phase 2 | Private | 0.19 | 5.2e+6 | NVIDIA | V100 | Calculated from Top500 | 2017-11-01 | 11.25 | NVIDIA SATURN V Phase 2 | 264.00 | NVIDIA | True | 10 | 16.52 | 3.3e+16 | 3.3e+16 | 4.1e+15 | 0.19 | 1.8e+11 | 5.2e+6 | Meta 2017 V100 Cluster | 0.01 | https://top500.org/system/179166/ | 39.38 | -97.92 | ||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 16.48 | 2.0e+1 | 200.00 | China | 2018-06-15 | Private | 0.20 | 5.0e+6 | China | 11.18 | 200.00 | Anonymized | Anonymized | True | 24 | 16.48 | 3.0e+16 | 3.0e+16 | 4.0e+15 | 0.20 | 1.5e+11 | 5.0e+6 | Meta 2017 V100 Cluster | 0.01 | 35.49 | 101.90 | |||||||||||||||||||||||||||
| JAMSTEC ZettaScaler-2.0 Gyoukou | Existing | Likely | Yes | 16.41 | 1.3e+1 | PEZY-SC2 | 1,600.00 | Japan | Japan Agency for Marine-Earth Science and Technology | 2017-06-15 | We know that the ZettaScaler 2.2 has 10k processors, can can calculate the number of processors in this supercomputer by scaling by the number of accelerator cores listed in Top500 | Public | 0.32 | Japan Agency for Marine-Earth Science and Technology | PEZY-SC2 | Calculated from Top500 | 2017-06 | 10.91 | JAMSTEC ZettaScaler-2.2 Gyoukou | 1,600.00 | PEZY | True | 6 | 16.41 | 2.6e+16 | 2.6e+16 | 1.3e+16 | 0.68 | 0.32 | 8.1e+10 | Google TensorFlow Research Cloud | 0.14 | https://web.archive.org/web/20241222180018/https://en.wikipedia.org/wiki/Gyoukou | https://www.top500.org/system/179102/ | 36.39 | 138.59 | ||||||||||||||||||||
| US Government Supercomputer 1 | Existing | Confirmed | Yes | 16.32 | 1.1e+1 | NVIDIA Tesla K40c | 4,160.00 | United States of America | United States Government | 2014-11-01 | Public | 2.89 | US Government | K40 | Calculated from Top500 | 2014-11-01 | 4,160.00 | NVIDIA | True | 3 | 2.1e+16 | 2.1e+16 | 2.04 | 2.89 | Oak Ridge NL Titan | 0.29 | https://top500.org/system/178445/ | 39.38 | -97.92 | |||||||||||||||||||||||||||
| US Government Supercomputer 2 | Existing | Confirmed | Yes | 16.32 | 1.1e+1 | NVIDIA Tesla K40c | 4,160.00 | United States of America | United States Government | 2015-06-01 | Public | 2.89 | US Government | K40 | Calculated from Top500 | 2015-06-01 | 4,160.00 | NVIDIA | True | 3 | 2.1e+16 | 2.1e+16 | 2.04 | 2.89 | Oak Ridge NL Titan | 0.29 | https://top500.org/system/178519/ | 39.38 | -97.92 | |||||||||||||||||||||||||||
| Meta 2017 P100 Cluster | Existing | Confirmed | Yes | 16.27 | 9.4e+0 | NVIDIA P100 | 992.00 | United States of America | Meta AI | 2017-06-01 | Private | 0.67 | Meta | P100 | Calculated from Top500 | 2017-06-01 | 10.44 | 992.00 | NVIDIA | True | 8 | 16.27 | 1.9e+16 | 1.9e+16 | 9.2e+15 | 0.58 | 0.67 | 2.8e+10 | Google TensorFlow Research Cloud | 0.10 | https://top500.org/system/179068/ | 39.38 | -97.92 | |||||||||||||||||||||||
| NVIDIA SATURN V Phase 1 | Existing | Confirmed | Yes | 16.27 | 9.4e+0 | NVIDIA P100 | 992.00 | United States of America | NVIDIA | 2016-11-01 | Private | 0.59 | 1.2e+8 | NVIDIA | P100 | Calculated from Top500 | 2016-11-01 | 10.50 | NVIDIA SATURN V Phase 2 | 992.00 | NVIDIA | True | 6 | 16.27 | 1.9e+16 | 1.9e+16 | 9.2e+15 | 0.59 | 3.1e+10 | 1.2e+8 | Oak Ridge NL Titan | 0.25 | https://web.archive.org/web/20240528104941/https://insidehpc.com/2018/04/inside-new-nvidia-dgx-2-supercomputer-nvswitch/ | https://top500.org/system/178928/ | 39.38 | -97.92 | ||||||||||||||||||||
| GSIC TSUBAME 2.5 | Existing | Confirmed | Yes | 16.22 | 8.4e+0 | NVIDIA Tesla K20X | 4,224.00 | Japan | Tokyo Institute of Technology | 2014-06-01 | 1408*3 = 4224
Top500 implies that there are at least 4140 | Public | 2.69 | 6.3e+7 | Ookayama, Japan | Academia,Tokyo Tech | K20X | 1,408 Compute Nodes; 3 GPUs [per compute node] | 2014-06-01 | 4,224.00 | NVIDIA | True | 3 | 1.7e+16 | 1.7e+16 | 1.99 | 2.69 | 6.3e+7 | Oak Ridge NL Titan | 0.23 | https://web.archive.org/web/20241008131001/https://www.gsic.titech.ac.jp/sites/default/files/99-yagura.pdf | https://top500.org/system/178249/ | 34.66 | 133.92 | ||||||||||||||||||||||
| Energy Company DD | Existing | Confirmed | Yes | 16.15 | 7.1e+0 | NVIDIA Tesla K80 | 1,728.00 | United States of America | 2016-06-15 | Private | 1.24 | 1.6e+7 | K80 | Calculated from Top500 | 2016-06 | 1,728.00 | NVIDIA | True | 6 | 1.4e+16 | 1.4e+16 | 1.24 | 1.6e+7 | Oak Ridge NL Titan | 0.19 | https://top500.org/system/178838/ | 39.38 | -97.92 | ||||||||||||||||||||||||||||
| Eni HPC2 | Existing | Confirmed | Yes | 16.07 | 6.0e+0 | NVIDIA Tesla K20X | 3,000.00 | Italy | Eni | 2014-11-15 | Private | 1.41 | 4.5e+7 | Eni | K20X | Calculated from Top500 | 2014-11 | 3,000.00 | NVIDIA | True | 5 | 1.2e+16 | 1.2e+16 | 1.41 | 4.5e+7 | Oak Ridge NL Titan | 0.16 | https://top500.org/system/178425/ | 42.50 | 12.65 | ||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 16.00 | 6.0e+0 | 2,000.00 | China | 2016-06-15 | Public | 1.00 | China | 2,000.00 | Anonymized | Anonymized | True | 8 | 1.0e+16 | 1.0e+16 | 1.00 | Oak Ridge NL Titan | 0.10 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||
| P3.16xlarge | Existing | Confirmed | 15.90 | 4.0e+0 | NVIDIA V100 | 64.00 | 2018-03-27 | They use 8 P13.16xlarge instances together. | Private | 0.04 | V100 | 2018-03-27 | 11.26 | 64.00 | NVIDIA | 15.90 | 8.0e+15 | 8.0e+15 | 1.0e+15 | 0.04 | 1.8e+11 | https://aws.amazon.com/blogs/machine-learning/scalable-multi-node-deep-learning-training-using-gpus-in-the-aws-cloud/ | Cluster performance database | https://instances.vantage.sh/aws/ec2/p3.16xlarge | ||||||||||||||||||||||||||||||||
| Moscow State University Lomonosov 2 | Existing | Confirmed | Yes | 15.87 | 3.8e+0 | NVIDIA Tesla K40m | 1,472.00 | Russia | Moscow State Univeristy | 2016-06-15 | Public | 0.86 | Moscow State University | K40m | Calculated from Top500 | 2016-06 | 1,472.00 | NVIDIA | True | 9 | 7.4e+15 | 7.4e+15 | 0.86 | Oak Ridge NL Titan | 0.10 | https://top500.org/system/178444/ | 61.99 | 96.67 | ||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 15.85 | 4.0e+0 | 7,000.00 | China | 2010-11-15 | Public | 8.00 | China | 7,000.00 | Anonymized | Anonymized | True | 1 | 7.0e+15 | 7.0e+15 | 3.00 | 8.00 | Tianhe-1A | 1.00 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||
| Paper on Minibatch SGD | Existing | Confirmed | Yes | 15.68 | 2.4e+0 | NVIDIA P100 | 256.00 | United States of America | Meta AI | 2017-06-08 | Private | 0.15 | P100 | The training time of ResNet-101 is 92.5 minutes in our
implementation using 256 Tesla P100 GPUs and a minibatch size of 8k | 2017-06-08 | 10.50 | True | 256.00 | NVIDIA | 16 | 15.68 | 4.8e+15 | 4.8e+15 | 2.4e+15 | 0.15 | 3.2e+10 | Google TensorFlow Research Cloud | 0.03 | https://arxiv.org/abs/1706.02677 | 39.38 | -97.92 | |||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 15.30 | 1.0e+0 | 2,000.00 | China | 2011-11-15 | Public | 2.00 | China | 2,000.00 | Anonymized | Anonymized | True | 2 | 2.0e+15 | 2.0e+15 | 0.90 | 2.00 | Tianhe-1A | 0.30 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | China | Private | China | Anonymized | Anonymized | True | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Unclear | China | Private | China | Anonymized | Anonymized | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||||
| Brazil Scala AI City Phase 1 | Planned | Likely | Unclear | Brazil | Scala Data Centers | Private | 54.00 | Bom Retiro, Eldorado do Sul - State of Rio Grande do Sul, 92990-000, Brazil | It will see an initial 3 billion real investment (US$490m) and will offer 54MW of capacity | Planned 2026 | Very few details about anything | Unknown | 54.00 | https://web.archive.org/web/20241227122627/https://www.datacenterdynamics.com/en/news/brazilian-local-government-passes-law-for-scalas-ai-city/ | -30.13 | -51.50 | ||||||||||||||||||||||||||||||||||||||||
| Brazil Scala AI City Phase 2 | Planned | Unlikely | Unclear | Brazil | Scala Data Centers | Planned date is a very rough guess | Private | 4,750.00 | Bom Retiro, Eldorado do Sul - State of Rio Grande do Sul, 92990-000, Brazil | Uncertain | the entire campus is expected to have a capacity of 4.75GW at full build-out, once all phases are complete | Planned 2033 | Very few details about anything | Brazil Scala AI City Phase 1 | Unknown | 4,750.00 | https://web.archive.org/web/20241227122627/https://www.datacenterdynamics.com/en/news/brazilian-local-government-passes-law-for-scalas-ai-city/ | -30.13 | -51.50 | |||||||||||||||||||||||||||||||||||||
| Cambridge Dawn Phase 2 | Planned | Likely | Yes | United Kingdom of Great Britain and Northern Ireland | UK Research and Innovation | Unclear if Phase 2 will actually happen | Public | University of Cambridge, West Data Centre, Ada Lovelace Rd, Cambridge CB3 0QX, United Kingdom | scientists within Cambridge and across the UK in critical research fields such as clean energy,personalised medicine and climate | aiming to deliver a Phase 2 supercomputer in 2024 which will boast 10 times the level of performance. If taken forward, Dawn Phase 2 would significantly boost the UK AI capability | Planned Q4 2024 | Whether or not they're moving forward with phase 2 remains "up in the air" | Cambridge Dawn Phase 1 | Unknown | https://web.archive.org/web/20240913113027/https://www.datacenterdynamics.com/en/news/fastest-ai-supercomputer-in-the-uk-is-now-operational/ | https://www.theregister.com/2023/11/13/intel_dawn_ai_uk/?utm_source=chatgpt.com | 52.21 | 0.08 | ||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | China | 2024-01-15 | Public/Private | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 2,000.00 | China | 2023-04-15 | Public/Private | China | 2,000.00 | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | China | 2023-09-15 | Public/Private | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Coreweave EcoDataCenter | Planned | Unlikely | Yes | NVIDIA GB200 NVL2 (per GPU) | Sweden | CoreWeave | Unclear how many GPUs will be in this cluster, but likely thousands | Private | Slaggvarpsvägen 21, 791 77 Falun, Sweden | Cloud | GB200 | As part of its European expansion strategy, CoreWeave will leverage thousands of NVIDIA Blackwell GPUs | Planned 2025 | NVIDIA | https://web.archive.org/web/20241216210211/https://ecodatacenter.tech/press/ecodatacenter-brings-the-worlds-most-advanced-ai-technology-to-sweden-3339095 | 60.61 | 15.61 | |||||||||||||||||||||||||||||||||||||||
| CoreWeave Hillsboro Oregon | Planned | Likely | Yes | United States of America | CoreWeave,Digital Realty | Unclear when this became operational or will be, and what type of GPUs are used | Private | Hillsboro, Oregon | CoreWeave | Uncertain | CoreWeave has secured a 36 MW lease with Digital Realty at one of its data centers in Hillsboro, Oregon, to deploy tens of thousands of GPUs in a single facility | Planned | Unknown | https://web.archive.org/web/20250126143127/https://dgtlinfra.com/coreweave-data-center-locations/ | 45.52 | -122.99 | ||||||||||||||||||||||||||||||||||||||||
| Google Council Bluff Iowa | Planned | Likely | Yes | United States of America | Private | 10410 Bunge Ave, Council Bluffs, IA 51503 | Planned | Unknown | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 41.17 | -95.80 | |||||||||||||||||||||||||||||||||||||||||||||
| Google Lancaster Ohio | Planned | Likely | Yes | United States of America | Private | 231 Whiley Rd, Lancaster, OH 43130 | Planned | Unknown | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 39.72 | -82.68 | |||||||||||||||||||||||||||||||||||||||||||||
| Google Lincoln Nebraska | Planned | Likely | Yes | United States of America | Private | W933+CH4 Lincoln, Nebraska | The other two sites are not as large yet but are ramping up fast: combining all four campuses will form a GW-scale AI training cluster by 2026. The Lincoln datacenter that is ~50 miles away will be Google’s largest individual site. | Planned | Unknown | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 40.81 | -96.70 | ||||||||||||||||||||||||||||||||||||||||||||
| Google New Albany Ohio | Planned | Likely | Yes | United States of America | Private | 450.00 | 1101 Beech Rd SW, New Albany, OH 43054 | "450MW campus in Ohio to train Gemini, to expand further to 540MW"
"The New Albany cluster, shown below, is set to become one of Google’s largest and is already hosting TPU v4, v5, v6." | Planned 2025 | Unknown | 450.00 | https://www.youtube.com/watch?v=hobvps-H38o | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 40.06 | -82.76 | |||||||||||||||||||||||||||||||||||||||||
| Google Omaha Nebraska | Planned | Likely | Yes | United States of America | Private | 11110 State St, Omaha, NE 68142 | Planned | Unknown | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 41.34 | -96.09 | |||||||||||||||||||||||||||||||||||||||||||||
| Google Papillion Nebraska | Planned | Likely | Yes | United States of America | Private | 250.00 | 14706 Schram Rd, Omaha, NE 68138 | The Papillion campus shown below adds >250MW of capacity to Google’s operations around Omaha and Council Bluffs, which combined with the above totals north of 500MW of capacity in 2023, of which a large portion is allocated to TPUs | Planned | Unknown | 250.00 | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 41.13 | -96.14 | ||||||||||||||||||||||||||||||||||||||||||
| Google south Columbus Ohio | Planned | Likely | Yes | United States of America | Private | 5076 S High St, Lockbourne, OH 43137 | Planned | Unknown | https://web.archive.org/web/20241119062654/https://semianalysis.com/2024/09/04/multi-datacenter-training-openais/ | 39.86 | -83.00 | |||||||||||||||||||||||||||||||||||||||||||||
| Graphcore Good Computer | Planned | Unlikely | Yes | 8,192.00 | Graphcore | Unclear if this project is still happening, no news since 2022, Graphcore recently acquired by Softbank | Private | Bow IPU | "This machine will contain 8,192 IPUs of a generation beyond the Bow processor, but further leveraging 3D wafer-on-wafer stacking technology"
"adding that the machine will deliver “over 10 exaflops” of floating-point performance" | completion unclear | Unclear if this project is still happening, no news since 2022, Graphcore recently acquired by Softbank | 8,192.00 | Unknown | $130M | https://web.archive.org/web/20220523232610/https://www.enterpriseai.news/2022/03/04/graphcore-announces-wafer-on-wafer-ipu-good-computer/ | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | China | 2021-09-15 | Private | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | Hong Kong | 2024-09-15 | Public/Private | Hong Kong | Anonymized | Anonymized | True | 22.28 | 114.16 | ||||||||||||||||||||||||||||||||||||||||||||
| HPE GreenLake for LLMs Q01 | Existing | Unlikely | Unclear | 1,000.00 | Canada | Hewlett Packard Enterprise | Private | 2280 Rue Albert-Dion, Lévis, QC G7A 5M9, Canada | Cloud | H100,L40,L4,"thousands" | GreenLake for LLMs gives users access to AI-native architecture specifically designed for AI and simulation workloads on hundreds or thousands of GPUs and CPUs at once | Planned for end of 2023 | Unclear what type of GPUs they use and how many | 1,000.00 | Unknown | https://web.archive.org/web/20240918133952/https://www.datacenterdynamics.com/en/news/hpe-greenlake-gets-ai-and-hpc-offering/ | 46.68 | -71.33 | ||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | China | 2023-10-15 | Private | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | China | 2024-10-15 | Private | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| IndiaAI Mission Supercomputer | Planned | Likely | Yes | 10,000.00 | India | There were earlier talk about potentially getting 25k GPUs, which still seems to be on the table. Unclear if this will be one cluster or several | Public/Private | Cloud | "The ecosystem will comprise AI compute infrastructure of 10,000 or more Graphics Processing Units (GPUs), built through public-private partnership."
"In October 2023, the Indian government initiated a discussion regarding a proposal to set up 25,000 GPUs in the country, with the vision to make compute accessible to Indian companies engaged in the development and use of AI.... Subsequently, in March 2024, the cabinet approved a Rs. 10,372-crore outlay for the IndiaAI Mission, under which the government aims to establish compute capacity of at least 10,000 GPUs through the public-private partnership model." | Planned | 10,000.00 | Unknown | https://web.archive.org/web/20240902090402/https://pib.gov.in/PressReleaseIframePage.aspx?PRID=2012355 | https://web.archive.org/web/20240919153825/https://carnegieendowment.org/posts/2024/04/a-primer-on-compute?lang=en | 22.20 | 78.48 | ||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | China | 2022-05-15 | Public | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Confirmed | Yes | China | Public/Private | China | Anonymized | Anonymized | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||||
| Meta Temple Texas | Planned | Likely | Yes | United States of America | Meta AI | They were building one design for awhile then stopped, tore it down, and started building a new design | Private | NW H K Dodgen Loop & Industrial Blvd
76502 Temple
Texas, USA | Meta | Planned | Unknown | https://www.youtube.com/watch?v=hobvps-H38o | 31.14 | -97.37 | ||||||||||||||||||||||||||||||||||||||||||
| Microsoft Air Gapped Supercomputer | Existing | Confirmed | Yes | United States of America | Microsoft | 2024-05-07 | Private | Iowa, USA | US intelligence agencies | Microsoft Corp. has deployed a generative AI model entirely divorced from the internet, saying US intelligence agencies can now safely harness the powerful technology to analyze top-secret information. | 2024-05-07 | No details on specifics | Unknown | True | https://web.archive.org/web/20240626134614/https://qz.com/microsoft-offline-ai-service-chatgpt-gpt4-us-spies-1851462266 | 42.09 | -93.50 | |||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | China | Public/Private | China | Anonymized | Anonymized | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | China | Public | China | Anonymized | Anonymized | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||||
| Neevcloud planned GPUs | Planned | Likely | No | 40,000.00 | India | NeevCloud | Seems likely that this will be several clusters. Unclear how many GPUs will be in the largest cluster, | Private | India | Cloud | NeevCloud, aims to build an AI cloud infrastructure customized for Indian users, with a plan to incorporate 40,000 GPUs by 2026 | Planned | 40,000.00 | Unknown | https://web.archive.org/web/20250110180448/https://entrepreneurshipstudio.com/news/posts/neevcloud-launches-with-40-000-gpus-revolutionizing-ai-cloud-services-for-smes-in-india | 22.20 | 78.48 | |||||||||||||||||||||||||||||||||||||||
| NHRI Taiwan Supercomputer (NHRI-1) | Planned | Likely | Yes | Taiwan | National Health Research Institutes Taiwan | Very few details about any specifics | Public | Taiwan | Biomedical research | The National Health Research Institutes (NHRI) in Taiwan has announced a partnership with Nvidia and Asus to deliver the nation’s first biomedical supercomputer | Planned | Very few details | Unknown | https://web.archive.org/web/20220605020815/https://www.hpcwire.com/2022/04/05/nvidia-asus-cloud-team-for-taiwanese-biomedical-supercomputer/ | 23.78 | 120.93 | ||||||||||||||||||||||||||||||||||||||||
| NVIDIA Israel Blackwell Supercomputer | Planned | Likely | Yes | NVIDIA GB200 NVL2 (per GPU) | Israel | NVIDIA | Very unclear how big this will actually be | Private | 30.00 | J3P7+8P Eliakim, Israel | GB200 | is expected to include "thousands" of graphics processing units | Planned | NVIDIA | 30.00 | $500 million | https://web.archive.org/web/20250116151639/https://www.aibase.com/news/14757 | 32.08 | 34.92 | |||||||||||||||||||||||||||||||||||||
| Oracle $10B Saudi Arabia Investment | Planned | Likely | No | Saudi Arabia | Oracle | Unclear how this money will be invested | Private | Cloud | Uncertain | Oracle has committed to investing $14 billion in Saudi Arabia over the next 10 years to expand its cloud and AI offerings in the region. | Planned | Unclear what this investment will entail | Unknown | 1.4e+7 | http://web.archive.org/web/20250515192746/https://www.datacenterdynamics.com/en/news/oracle-commits-to-invest-14bn-in-saudi-arabia-over-next-10-years/ | 23.38 | 44.65 | |||||||||||||||||||||||||||||||||||||||
| OTP SambaNova Supercomputer | Planned | Unlikely | Very unclear if this ever happened or how big it is planned on being. They say "Europe's fastest AI supercomputer" | Private | Europe | Planned | Unknown | https://www.datacenterdynamics.com/en/news/otp-bank-selects-sambanova-systems-for-ai-supercomputer/ | ||||||||||||||||||||||||||||||||||||||||||||||||
| S. Korea 7th national supercomputer | Planned | Likely | Yes | Korea (Republic of) | Ministry of Science and ICT | Unclear what precision they're using, or what type of chips | Public | Korea (Republic of) | Researchers | The country plans to start the development of its seventh supercomputer in 2025 with an exaflop of performance | Planned | Only able to find one news report | Unknown | https://web.archive.org/web/20240117215326/https://www.kedglobal.com/artificial-intelligence/newsView/ked202305310010 | ||||||||||||||||||||||||||||||||||||||||||
| Singtel DC Tuas | Planned | Likely | Yes | NVIDIA H100 SXM5 80GB | Singapore | Singtel | No information on how large this cluster will be, but the datacenter will be 58MW | Private | 58.00 | 9 Tuas Ave 3 639408 Singapore | Cloud | H100,GB200 | Singtel’s GPUaaS will be powered by Nvidia H100 Tensor Core GPU-powered clusters that are operated in existing upgraded data centers in Singapore | Planned | NVIDIA | 58.00 | https://web.archive.org/web/20241210030636/https://www.datacenterdynamics.com/en/news/singtel-and-nscale-announce-partnership-to-boost-gpu-capacity-across-europe-and-southeast-asia/ | 1.32 | 103.66 | |||||||||||||||||||||||||||||||||||||
| SingularityNET | Planned | Likely | No | SingularityNET | Seems like it's decentralized and spread around the world | Private | SingularityNET plans to build the first of several modular compute containers that will act as decentralized hubs for a network of devices from fellow AI developers Fetch.ai, Ocean Protocol, NuNet, HyperCycle, as well as itself. Thanks to their design, the containers can be placed and relocated worldwide. | Planned | Unknown | https://web.archive.org/web/20240724001101/https://decrypt.co/241118/singularitynet-53-million-ai-supercomputer-data-centers | ||||||||||||||||||||||||||||||||||||||||||||||
| SoftBank Planned GB200 | Planned | Likely | Yes | NVIDIA GB200 NVL2 (per GPU) | Japan | Softbank | Private | Japan | Softbank | GB200 | In addition to its DGX SuperPOD, SoftBank plans to build another NVIDIA-accelerated supercomputer to run extremely compute-intensive workloads. Initial plans for the supercomputer are based on an NVIDIA Grace Blackwell platform design featuring NVIDIA GB200 NVL72 multi-node, liquid-cooled, rack-scale systems that combine NVIDIA Blackwell GPUs with power-efficient Arm-based NVIDIA Grace™ CPUs | Planned | NVIDIA | https://web.archive.org/web/20241116120654/https://nvidianews.nvidia.com/news/nvidia-and-softbank-accelerate-japans-journey-to-global-ai-powerhouse | 36.39 | 138.59 | ||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | China | 2020-11-15 | Public | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 500.00 | China | 2018-07-15 | Public | China | 500.00 | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||
| TACC Horizon | Planned | Likely | Yes | United States of America | University of Texas at Austin | Frontera currently has 800 GPUs, which are presumably V100s. For Horizon to have 100x it's AI compute, Horizon would need several thousand GPUs | Public | 1300 Louis Henna Blvd, Round Rock, TX 78664 | Academia,Researchers | thousands | Horizon, will provide 10x performance improvement for simulation over the current NSF Leadership-Class Computing system, Frontera, and meet the unique scientific requirements of the NSF community. For AI applications, the leap forward will be even larger, with more than 100x improvement over Frontera | Planned 2026 | No clear data yet about number of chips or what type | Unknown | https://web.archive.org/web/20240911080245/https://new.nsf.gov/news/nsf-announces-groundbreaking-computing-facility | https://web.archive.org/web/20240905170840/https://www.datacenterdynamics.com/en/news/tacc-chooses-sabey-data-centers-facility-for-new-supercomputer/ | 30.49 | -97.65 | ||||||||||||||||||||||||||||||||||||||
| Tata Group GH200 supercomputer | Planned | Unlikely | Yes | NVIDIA GH200 | India | Tata Group | Private | India | Tata Group,Cloud | GH200 | The first phase of this large-scale deployment is set to commence by the end of 2024, establishing one of the largest Nvidia Hopper GPU-based cloud supercomputers in India | Planned 2024 | Very little information about GPU quantity | NVIDIA | https://web.archive.org/web/20241216183257/https://www.businessworld.in/article/tata-communications-to-set-up-nvidia-hopper-gpu-supercomputer-by-2024e-plans-blackwell-gpu-expansion-in-2025-537162 | 22.20 | 78.48 | |||||||||||||||||||||||||||||||||||||||
Telangana Yotta Hyderbad AI City Cluster Phase 2 | Planned | Likely | Yes | 25,000.00 | India | Government of Telangana | Unclear exactly which chips they plan to use | Public | 50.00 | Hyderabad, India | Cloud | Once fully built, the AI Supercomputer powered by 25,000 GPUs and the 50 MW AI Cloud Data Centre campus | Planned | Telangana Yotta H1 Hyderbad AI City Cluster Phase 1 | 25,000.00 | Unknown | 50.00 | https://web.archive.org/web/20240906230223/https://www.crn.in/news/government-of-telangana-partners-with-yotta-to-launch-indias-largest-ai-supercomputer-of-25000-high-performance-gpus-in-a-purpose-built-50-mw-ai-cloud-data-centre-campus-in-hyderabad/ | 17.36 | 78.47 | ||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | China | 2022-09-15 | Private | China | Anonymized | Anonymized | True | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Unclear | China | Private | China | Anonymized | Anonymized | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||||
| Tesla Buffalo Dojo | Planned | Confirmed | Yes | Tesla D1 Dojo | United States of America | Tesla | Private | 1339 South Park Ave, Buffalo, NY 14220 | Tesla | Dojo | The governor is correct that this is a Dojo Supercomputer, but $500M, while a large sum, is only equivalent to a 10k H100 system from Nvidia. Tesla will spend more than that on Nvidia hardware this year. | Planned | NY governor announced plans for this in official press conference | Tesla | $500M | https://web.archive.org/web/20240127130129/https://teslanorth.com/2024/01/27/tesla-ai-computer-project-buffalo/ | 42.86 | -78.84 | ||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Confirmed | Yes | 50,000.00 | China | 2013-07-15 | Public | 20.00 | 2.0e+8 | China | 50,000.00 | Anonymized | Anonymized | True | 30.00 | 20.00 | 2.0e+8 | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Yes | 200,000.00 | China | 2023-12-15 | Public | 80.00 | China | 200,000.00 | Anonymized | Anonymized | True | 80.00 | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Unlikely | Unclear | China | Private | China | Anonymized | Anonymized | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||||
| Wonder Valley Datacenter Phase 1 | Planned | Unlikely | Unclear | Canada | Private | 1,400.00 | Alberta, Canada | Uncertain | "the project's first phase will include developing a 1.4GW redundant power solution and subsequent annual rollout of redundant power in 1GW increments"
"generate and offer 7.5GW of low-cost power to hyperscalers over the next five-10 years" | Planned 2027 | Unknown | 1,400.00 | https://archive.ph/c5kTp | 54.64 | -114.76 | |||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Existing | Likely | Unclear | 7,000.00 | China | 2023-04-15 | Private | China | 7,000.00 | Anonymized | Anonymized | 35.49 | 101.90 | |||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | 10,000.00 | China | Private | China | 10,000.00 | Anonymized | Anonymized | 35.49 | 101.90 | ||||||||||||||||||||||||||||||||||||||||||||
| Anonymized Chinese System | Planned | Likely | Yes | China | Public/Private | China | Anonymized | Anonymized | 35.49 | 101.90 |
Epoch's work is free to use, distribute, and reproduce provided the source and authors are credited under the Creative Commons BY license.
Learn more about this graph
Data come from our AI Supercomputers dataset, which collects information on 728 supercomputers with dedicated AI accelerators, spanning from 2010 to the present. We estimate that these supercomputers represent approximately 10-20% (by performance) of all AI chips produced before 2025. We focus on the 502 AI supercomputers which became operational in 2019 or later, since these are most relevant to modern AI training.
When cost figures are not explicitly reported, we estimate the cost of a data center by first estimating the hardware costs from chip quantities and public pricing information. We then factor in the expected cost of additional hardware such as networking switches and CPUs. Notably, we do not include the cost of power generation or data center construction. All cost figures are inflation-adjusted to January 2025 dollars.
For more information about the data, see Pilz et al., which describes the supercomputers dataset and analyzes key trends, and the dataset documentation.
Analysis
To estimate a trend in costs for leading AI supercomputers, we first identify 57 “frontier” AI supercomputers, defined as those in the top-10 highest 16-bit computing performance at the date they became operational. We then use a log-linear regression to analyze how costs have changed over time, calculating both the growth rate and its confidence interval.
We find that the cost of frontier AI supercomputers has been growing by 1.9x per year, with a 90% confidence interval of 1.8x to 2.1x.
Assumptions and limitations
- Our analysis does not include the costs of power generation or data center construction. These expenses could be substantial and may represent a growing share of total costs, especially as obtaining sufficient power at single locations for hyperscale supercomputers becomes more challenging. If that’s the case, our estimates may underestimate the true growth in supercomputer costs. Additionally, costs reported directly by organizations may vary in whether these overhead costs are included, adding further uncertainty to our estimates.






