How to start programming with CUDA

I used to help undergraduate students as a mentor for the GPGPU team, one of the VIP (Vertically Integrated Project) project teams. Since most of students who joined the team had not done GPU programming, I guided them like below:

• Read chapter 2 and chapter 3 of the CUDA programming guide to understand the CUDA programming model execution models and programming interfaces. Now a days, there many videos for introduction to CUDA programming. (See the useful links below)
• Improve your understanding on GPGPU programing through hands-on experience
• Write a program to increase all elements of an input array by 1 for their understanding about the execution model and programming model
• Modify an example of the matrix multiplication to support non-power of two sized matrices for their understanding about the memory model (global, shared, and lobal) and the execution model
• Learn about the prefix-sum since it is commonly used for the data parallel programming (e.g., reduction): 1, 2.
• Start class projects since they are ready for it.

Useful links are below:

• An Introduction to CUDA (pdf) at NVIDIA GTC 2010
• NVIDIA CUDA Developer zone
• NVIDIA CUDA GPU Computing Forum
• NVIDIA CUDA Education and Training
• GPGPU.org
• CUDPP
• Programming Massively Parallel Processors with CUDA