To get started with cloud infrastructure, it's essential to understand its nomenclature. Terminology can vary depending on the provider, however the terms listed below are industry standards.
This guide is a cloud computing 101, breaking down key terms you'll encounter when working with modern GPU clouds.
Cloud Computing Terms
| Term | Definition | Further Reading |
|---|---|---|
| Hyperscaler | Large cloud providers offering massive global infrastructure (e.g., AWS, Azure, Google Cloud). | |
| Instance | A virtual machine running in the cloud with allocated compute resources. | |
| On Demand Instances | Instances billed per second or minute with no long-term commitment. | Read more |
| Reserved Instances | Instances purchased at a discount in advance and for a certain period. | |
| Spot Instances | Discounted instances using spare capacity, which can be interrupted with little notice. | Read more |
| GPU-as-a-Service (GPUaaS) | Cloud model where GPUs are rented on demand for compute workloads. | Read more |
| Neocloud | A newer category of GPU cloud providers focused on AI workloads. | Read more |
| Egress Fees | Charges for transferring data out of a cloud provider’s network. | Read more |
| LLM Inference | The process of running a trained large language model to generate outputs. | |
| Supervised Fine-Tuning (SFT) | Training a model on labeled data to improve task-specific performance. | Read more |
| Computer Vision | A field of AI that enables machines to interpret and analyze images and video. | Read more |
Graphic Processing Unit (GPU) Concepts
| Term | Definition | Further Reading |
|---|---|---|
| CUDA Cores | Programmable processing units in NVIDIA GPUs used for general computation. | Read more |
| DGX | NVIDIA's line of purpose-built AI supercomputing systems that integrate multiple GPUs, high-speed interconnects, and optimized software into a single turnkey platform. | Read more |
| HGX | NVIDIA's reference server board platform that allows OEM partners to build GPU-accelerated servers by integrating multiple high-end GPUs with NVLink and NVSwitch. | Read more |
| NVLink | NVIDIA's high-bandwidth, low-latency GPU-to-GPU interconnect that enables faster data sharing between GPUs than PCIe allows. | Read more |
| NVSwitch | NVIDIA's high-speed switch chip that enables full all-to-all NVLink connectivity across multiple GPUs within a node or across nodes. | Read more |
| PCIe | A standard high-speed interface used to connect GPUs, storage, and other peripherals to a computer's CPU and motherboard. | Read more |
| ROCm | AMD's open-source software platform for GPU computing, providing an alternative to NVIDIA's CUDA ecosystem for running AI and HPC workloads on AMD GPUs. | Read more |
| Transformer Engine | A dedicated hardware and software component in NVIDIA GPUs that was first introduced with the Hopper architecture (H100). It accelerates transformer-based AI models by automatically switching between FP8 and FP16 precision. | Read more |
| SXM | NVIDIA's high-bandwidth socket form factor for mounting data center GPUs directly onto a baseboard, enabling faster NVLink connections compared to standard PCIe cards. | Read more |
| Shannon Entropy | A mathematical measure of the average uncertainty, randomness, or information content inherent in a data source or probability distribution. | Read more |
| Tensor Cores | Specialized GPU cores optimized for AI and matrix operations. | Read more |
| Taiwan Semiconductor Manufacturing Company (TSMC) | The world's largest semiconductor producer, controlling nearly 70% of the advanced microchips global market. | Read more |
Takeaway
This glossary is just the starting point. For deeper dives into pricing, infrastructure, and performance comparisons, explore the Thunder Compute blog.
By building a strong foundation in cloud computing terminology, you'll be better equipped to navigate the rapidly evolving landscape of GPU cloud infrastructure.