For the full press release see here and the landing page is here.

NVIDIA has released version 3.0 of their CUDA Toolkit. As well as some significant new features this release also  provides tools to prepare for the upcoming Fermi-based GPUs. Highlights of this release include:

• Support for the Fermi architecture, with:
  • Native 64-bit GPU support
  • Multiple Copy Engine support
  • ECC reporting
  • Concurrent Kernel Execution
  • Fermi HW debugging support in cuda-gdb
  • Fermi HW profiling support for CUDA C and OpenCL in Visual Profiler
• C++ Class Inheritance and Template Inheritance support for increased programmer productivity
• A new unified interoperability API for Direct3D and OpenGL, with support for:
  • OpenGL texture interop
  • Direct3D 11 interop support
  • CUDA Driver / Runtime Buffer Interoperability, which allows applications using the CUDA Driver API to also use libraries implemented using the CUDA C Runtime such as CUFFT and CUBLAS.

• CUBLAS now supports all BLAS1, 2, and 3 routines including those for single and double precision complex numbers
• Up to 100x performance improvement while debugging applications with cuda-gdb
• cuda-gdb hardware debugging support for applications that use the CUDA Driver API
• cuda-gdb support for JIT-compiled kernels
• New CUDA Memory Checker reports misalignment and out of bounds errors, available as a stand-alone utility and debugging mode within cuda-gdb
• CUDA Toolkit libraries are now versioned, enabling applications to require a specific version, support multiple versions explicitly, etc.
• CUDA C/C++ kernels are now compiled to standard ELF format
• Support for device emulation mode has been packaged in a separate version of the CUDA C Runtime (CUDART), and is deprecated in this release. Now that more sophisticated hardware debugging tools are available and more are on the way, NVIDIA will be focusing on supporting these tools instead of the legacy device emulation functionality.
  • On Windows, use the new Parallel Nsight development environment for Visual Studio, with integrated GPU debugging and profiling tools (was code-named “Nexus”). Please see www.nvidia.com/nsight for details.
  • On Linux, use cuda-gdb and cuda-memcheck, and check out the solutions from Allinea and TotalView that will be available soon.

• Support for all the OpenCL features in the latest R195 production driver package:
  • Double Precision
  • Graphics Interoperability with OpenCL, Direc3D9, Direct3D10, and Direct3D11 for high performance visualization
  • Query for Compute Capability, so you can target optimizations for GPU architectures (cl_nv_device_attribute_query)
  • Ability to control compiler optimization settings via support for pragma unroll in OpenCL kernels and an extension that allows programmers to set compiler flags. (cl_nv_compiler_options)
  • OpenCL Images support, for better/faster image filtering
  • 32-bit global and local atomics for fast, convenient data manipulation
  • Byte Addressable Stores, for faster video/image processing and compression algorithms
  • Support for the latest OpenCL spec revision 1.0.48 and latest official Khronos OpenCL headers as of 2010-02-17

The toolkit, drivers, tools and documentation are available from http://developer.nvidia.com/object/cuda_3_0_downloads.html.

• Update release for ATI Stream SDK v2.0.
• Support for Red Hat® Enterprise Linux® 5.3.
• Support for Ubuntu® 9.10.
• Support for debugging OpenCL™ with GDB on x86 CPUs under Linux® (see application note for more details).
• Preview: Support for OpenCL™ / Microsoft® DirectX® 9 interoperability. Please see this knowledge base article for more information about this preview feature.
• Additional OpenCL™ samples:
  • BoxFilter
  • FFT
  • GaussianNoise (under cpp_cl)
  • URNG
• Stream KernelAnalyzer with OpenCL™ support (available for download separately from Stream KernelAnalyzer Product Page).
• Various OpenCL™ compiler and runtime fixes and enhancements (see developer release notes for more details).
• Support for ATI Radeon™ HD 5670 GPU and ATI Radeon™ HD 5570 GPU.

The development was undertaken in collaboration with AMD who have the first CPU + GPU OpenCL implementation and provides an interesting chance to compare and contrast performance of CPUs and GPUs on the same workloads.

Jon spends much more time looking at CAGRs, IHV marketing decks and sales projections than I do, but having read the report I do wonder if he’s missed a trick or two while staring at his tea leaves. Reading the report it’s clear that JPR’s remit was to explore multi-GPU from the perspective of scaling the graphics performance by using multi-AIB and multi-GPU systems. Not too surprising given that the report was at least partially sponsored by LucidLogix and to a lesser extent AMD and NVIDIA.

As an aside, LucidLogix are an interesting entrant into the graphics market, as they are producing an IHV agnostic chip that potentially allows for multi-GPU scaling to take a very interesting turn indeed. It’s not clear yet how their product will be greeted by the market (or indeed the IHVs), but if it works as well as they say it does (and I haven’t seen it in action yet) it has the potential to break the current state of single vendor (and mostly single device variant) multi-GPU systems (SLI vs Crossfire).It also has the ability to give habitual AIB buyers a far longer working life for their previous purchases, as in theory, a motherboard enhanced with a LucidLogix device means you can run your latest GPU in parallel with your previous primary graphics card.

ATI/AMD promised this a while back but I’m not sure it was ever delivered in any meaningful way (they were promoting it as a potential route for physics acceleration around the time NVIDIA acquired PhysX) and NVIDIA have shown systems where an NVIDIA IGP and an NVIDIA AIB co-exist and the driver selects the most appropriate device for a given workload (Hybrid SLI). With a LucidLogix Hydra device acting as a bridge, the theory is that the user sees something approaching additive scaling (certainly not a given from existing solutions) from a mix of different GPUs and best of all you won’t necessarily be tied to a single vendor either. Of course there’s likely to be a very long list of caveats to achieving this multi-GPU nirvana but that’s another (very interesting) blog entry and I digress.

My immediate thought while reading the JPR report was that the analysis curiously excluded IGPs (Integrated Graphics Processors) and thus I presume, devices coming from AMD and Intel with graphics integrated into the same package as the CPU (e.g. Fusion from AMD). The current trend to integrate graphics on the CPU package is a cost driven evolution from North Bridge IGPs (memory controllers already having moved to the CPU), with both AMD and Intel wanting to leverage their position as ‘platform’ vendors, and to offer price reductions to PC OEMs (but coincidentally increase the proportion of the PC’s BOM they see).

In terms of the number of people who only ever see an integrated graphics solution in action (be it in the chipset or in the future next to the CPU) we’re probably approaching 50%, given Intel’s current dominance in the mobile and mid to low cost desktop markets. If AMD’s Fusion range succeeds it will mostly likely take market share away from Intel rather than the AIB market. So this suggests one way in which JPR sees the market evolving, but apart from the continuing drive to improve rendering performance he doesn’t really offer any other market drivers. Should he have?

In recent years what has driven the evolution of the PC and the growth of the GPU vendors, has really been games. THis isn’t going to stop any time soon (even with consoles accounting for an increasing proportion of that market). At the same time the move to use COTS (Commercial Off The Shelf) systems in HPC (High Performance Compute) has also been a significant evolutionary driver for x86 as a platform.

So where does that leave this new wave of heterogeneous compute that the GPU vendors are so keen to exploite these days? Speaking as a person who already works with heterogeneous systems (i.e. accelerated using GPUs and other specialist co-processors) to solve engineering and scientific problems for clients, the quest for better performance for ISV (and custom) codes will drive adoption of in the personal workstation market, but that still leaves the consumer side of the equation.

This is where being involved in the Khronos Group and watching the evolution of OpenCL from the inside, has shown me just how wide a reach OpenCL (and other related APIs) potentially could have. It won’t just affect the software we write but it also has the potential to shape the direction that the future of PC architecture will take and ultimately this is bound up with the software that we run on them. This leads me back to my cause and effect observation: Just what would drive 50% of all PC owners by 2012 to have bought at least one additional AIB (or bought a machine that shipped with two AIBs)?

We have already seen that the market wasn’t quite ready for a single source, vendor specific and cost option as far as physics acceleration was concerned. This is not to say that PhysX has been a failure, but it’s not currently achieved the market penetration to really drive sales of NVIDIA AIBs in its own right. AMD have lately played smart, and offered a potential counter to the PhysX marketing bullet point that also addresses the issue of vendor specific solutions, by porting the Intel owned Havok physics engine to OpenCL (an interesting move in itself from a marketing perspective).

All of this leads to more questions than answers at the moment. Will the relatively recent existence of an open, cross platform and most importantly cross vendor programming target in OpenCL feed the growth of a non-game based software ecosystem, that is not just able to take advantage of heterogeneous acceleration, but will actually drive it in some quite remarkable directions? Will OpenCL be able to meet the Microsoft juggernaut (in the form of DirectX11 Compute shaders) head on?

I actually think these two approaches to tapping the compute horsepower available from GPUs are actually complementary rather than necessarily in direct opposition. DX11 Compute Shaders (and their evolutionary descendants) will undoubtedly enable performance improvements for games and open up a wealth of new options for game developers, but I think the smart money for ISVs wanting to develop other applications for heterogeneous systems will be with OpenCL. When the OpenCL standard matures a little and the IHVs work out some of the current interoperability issues, we will start to see truly heterogeneous software solutions arrive and this, I think, could be really important. A healthy software ecosystem will sustain, though it may not significantly grow the current AIB market (much as JPR predicts), but a lot still depends on where the gamers end up because at the moment this is what’s driving the year on year improvement in GPU performance.

As luck would have it Neil Trevett, the Khronos Group President, and incidentally an NVIDIA VP, has an interesting interview on The TechReport today along these lines.

In the case of questions about the the specification, or queries about forthcoming revisions of the standard (or even suggestions), one should definitely go to the Khronos Group’s own OpenCL forum. Here you will find forum members who are actually involved with the OpenCL standard body as well as developers and users of vendor implementations. It’s a fair bet that you will get most specification related questions answered here.

In the case that you believe you are observing something that does not meet the specification as stated, in a vendor implementation, then your first port of call should be the vendor’s own support forum. If it’s an obvious bug or known performance issue the odds are good that it will have already been documented, either in an answer to a prior forum posting or in a FAQ or Release Note.  It should go without saying that you should take care to follow their procedure for submitting queries or bugs before posting to a forum (i.e. do a forum search and read release notes – we don’t compile them just for fun).

Here’s a list of OpenCL related forum resources I have to hand:

• Khronos Group OpenCL forum
• NVIDIA OpenCL forum
• AMD Stream forum (AMD don’t have a public OpenCL forum at time of writing)

• Beyond3D GPGPU Technology & Programming forum
• GPGPU General GPGPU Discussion forum

I’m sure there are more out there, if you find them please send them to me and I’ll start keeping an OpenCL Resource page.