What Is a GPU, and Why Is Everyone Fighting Over Them?

A single Nvidia H100 chip — the kind of processor training today's biggest AI models — can cost more than a small hatchback in India, and even at that price there's often a waiting list. A few years ago, the same class of chip was mostly known for making video games look good. What happened, and what exactly is this thing everyone suddenly can't get enough of?

The Chip That Does One Thing, a Million Times at Once

A CPU (central processing unit — the main "brain" in every laptop, phone, and server) is built to handle a wide variety of tasks one after another, very quickly, in a fairly strict order. It has a handful of powerful cores, each capable of complex decision-making. That's great for running an operating system or opening a spreadsheet.

A GPU (graphics processing unit) takes the opposite approach. Instead of a few powerful cores, it packs in thousands of smaller, simpler ones that all work on tiny pieces of the same problem simultaneously. It was originally built to do exactly that for video games — calculate the colour of every pixel on your screen, sixty times a second, which is really just the same math repeated millions of times in parallel.

That "same math, many times over" pattern turns out to be exactly what's needed to train an AI model. Teaching a neural network involves multiplying enormous grids of numbers against each other, over and over, until the model's internal weights settle into something useful. A GPU can chew through that kind of workload far faster than a CPU ever could, which is the whole reason gaming hardware quietly became the backbone of the AI industry.

Why They Became So Hard to Get

Once companies like OpenAI, Google, and Anthropic showed that bigger models trained on more GPUs kept getting more capable, every large tech company started buying as many as it could. Add in the fact that a handful of firms — mainly Nvidia — design the chips everyone wants, and that manufacturing advanced semiconductors takes specialised factories that can't simply be built overnight, and you get a supply crunch that's lasted years, not months.

There's also a split worth knowing about: training a model (teaching it from scratch, which needs huge clusters of GPUs running for weeks) and inference (actually using the trained model to answer your questions, which needs far less power per request but happens constantly, at massive scale). Both eat into the same limited GPU supply, which is part of why cloud computing bills for AI-heavy companies keep climbing.

Compute has quietly become as strategic a resource as oil or electricity — except unlike a barrel of crude, you can't stockpile a chip that a rival company hasn't finished designing yet.

What It Means Closer to Home

For Indian developers and startups, this shortage used to mean either paying steep cloud rental rates to foreign providers or simply not having access to serious training hardware at all. That's part of why the government's IndiaAI Mission set aside funding to build a shared pool of GPU capacity that Indian startups, researchers, and academic institutions can rent at subsidised rates rather than each having to import and own chips outright — a workaround for a market where import duties and currency exchange make buying top-tier GPUs directly an expensive proposition.

It shows up in consumer terms too. If you've shopped for a gaming laptop or a graphics card in an Indian electronics store recently, you've probably noticed prices that don't quite match what you'd expect from the dollar sticker price abroad — import costs and demand from data centres both push in the same direction. And for Indian gamers and video editors, that's meant longer waits between affordable upgrades.

A few practical things worth knowing if you're trying to make sense of GPUs in everyday use:

• You don't need to own one to build with AI. Cloud platforms rent GPU time by the hour, which is usually cheaper than buying hardware unless you're running workloads constantly.
• Not all GPUs are equal for AI work. Consumer gaming cards can run smaller models fine, but serious training needs data-centre-grade chips with far more memory.
• CUDA matters more than most buyers realise. It's Nvidia's software layer that lets programs talk to its GPUs efficiently — a big reason Nvidia's chips remain the default choice even when competitors are technically capable.
• "VRAM" is the number to watch. It's the GPU's own dedicated memory, and running out of it is usually what stops a large AI model or a heavy game from running smoothly, regardless of how fast the chip itself is.

The Takeaway

None of this requires becoming a hardware expert. But the next time you hear that a company has "bought thousands of GPUs" or announced a new AI model, it's worth remembering that the real bottleneck isn't cleverness or code — it's physical chips, built in a handful of factories worldwide, that everyone from a Bengaluru startup to a Silicon Valley giant is competing for. If you're building something that needs serious compute, renting through a cloud provider or a scheme like IndiaAI Mission's shared GPU access will almost always make more sense than trying to buy your way in.