How to Learn CUDA with hands on?

CUDA is a parallel computing platform and application programming interface (API) model created by NVIDIA. When it was first introduced, the name was an acronym for Compute Unified Device Architecture, but now it's only called CUDA.

Install CUDA

Windows

To use CUDA on your system, you will need the following installed:

A CUDA-capable GPU

A supported version of Microsoft Windows

A supported version of Microsoft Visual Studio

the NVIDIA CUDA Toolkit

Linux

Use Ubuntu, get the NVIDIA driver, and install CUDA Toolkit.

apt-get install cuda

An Interactive Course

If you don’t want to spend money to buy time on the AWS (e.g. Linux Accelerated Computing Instances) or Google compute engine, you can locally use GPGPUSim and Docker (see the CUDA Programming Essentials example https://school.scientificprogramming.io/course/lesson/cuda-programming-essentials/7/219) and use up to CUDA Toolkit 4.0 which is just enough to learn CUDA! You can get GPGPUSim with a detailed documented project from online. It was developed at the University of British Columbia by Tor Aamodt along with his graduate students
The best part of it is that you can use the ready-made version on a Docker and detailed information from CUDA GPU Simulator Container, check it! Docker GPGPUSIM.

To start with CUDA, you'll need a course that shows and tells you the CUDA programming by developing simple examples with a growing degree of difficulty starting from the CUDA toolkit installation to coding with the help of block and threads and so on. This course covers: