ZLUDA on Top of Vulkan ?

[RhynarAI]

Instead of relying on ROCm / HIP, could ZLUDA use Vulkan? This would open up A LOT of GPUs to run CUDA Code (Starting from GCN 1, Qualcom Chips, APPLE Ecosystem, Intel GPUs, Android.....)

Thanx for your work, this is great !

[superuser-miguel]

This would open up A LOT of GPUs to run CUDA Code

All of the GCN / Polaris GPU's which are still quite popular, including the RDNA 1st gen.

[deuszex]

It would, but if you look in the FAQ, there are no monetary backers so the project is run solely by the dev. Any more work would be on their account only, and switching or inforporating Vulkan would be a huge undertaking even if paid.

[RhynarAI]

If its all just a matter of money - several people proposed crouwdfunding or other ways to raise money for the project. I would contribute some money to the project.

How difficult or if possible it would be to switch to vulkan is something where the DEV could give one or two hints. After all, this project seemed to have switched from ONEAPI to ROCm at least once, I have not checked what the first ZLUDA was based on (maybe even OPENCL?)

[vricosti]

I thought Vulkan was only used for graphics, u mean it also supports gpu computing ?

[RhynarAI]

yes, vulkan compute is a thing. There are several "AI upscalers" made on top of vulkan and a lot of other stuff.. You can also look up geekbench scores - aside from cuda and openCL, Vulkan compute is seen...

[vosen]

Not going to happen. Ignoring host-side for now, the GPU-side language is just too lacking in features.
Vulkan Compute kernels are in SPIR-V (https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html) and here's a small sample of features supported by ZLUDA today, but not supported by Vulkan Compute flavor of SPIR-V (GLCompute as I understand):

• Disabling fp contraction
• Explicit alignment
• Some subgroup and group operators (not 100% certain)
• Bindless images
• Pointer casts
• Arbitrary virtual calls
• Inline assembly
• Rounding modes (not 100% certain)

Note, that Intel GPU on Level Zero/OneAPI also use SPIR-V, but the Intel GPU SPIR-V flavor for compute (Kernel) is different and much richer

Not sure if you can realisticaly get even the simplest CUDA app on Vulkan Compute as CUDA compiler loves this sequence:

ld.param.u64 %rd1, [foobar_param_0];
...
cvta.to.global.u64 %rd2, %rd1;

which is roughly in SPIR-V terms:

%rd1= OpLoad %i64 %foobar_param_0
...
%temp1 = OpConvertUToPtr %i8ptr_generic  %rd1
%temp2 = OpGenericCastToPtr %i8ptr_global %temp1
%rd2   = OpConvertPtrToU %i64 %temp2

OpGenericCastToPtr is not part of Vulkan Compute SPIR-V.

Not to even mention that you are losing performance libraries: cuDNN, cuBLAS etc courtesy of MIOpen, rocBLAS. Maybe you can interop your Vulkan with their HIP, but at this point why even try

[pixelcluster]

various limitations in Vulkan Compute stem from HW limitations of mobile GPUs

Limitations of certain types of GPUs actually isn't too big of a deal - You usually gate functionality only supported on certain hardware behind (optional) features and/or extensions. It would then be perfectly reasonable for ZLUDA to require that implementations expose these features once they're available.

[Hugobros3]

https://github.com/shady-gang/shady makes considerable effort to raise the bar of Vulkan shaders by clever compiling tricks and emulation, you might be interested in leveraging it. I have been using it to build Vcc and also got it to convert OpenCL SPIR-V kernels to run on Vulkan.

[vosen]

Wrt. shady: it's a good attempt, but they still have some painful (for CUDA) limitations: "Function and data pointers are not portable between host and device. Taking addresses and passing them across in either direction is UB - since Vulkan lacks a unified addressing extension this is a limitation we cannot address at this time."
I wonder how some of their workarounds work (generic pointers: do they tag high bits or do fat pointers?, etc).

[Hugobros3]

You can learn more about shady in this talk from Vulkanised 2024: https://www.youtube.com/watch?v=bRyculBbjMA.

Regarding sharing function pointers I don't think this is available in OpenCL either ?
For data pointers, it's just not in the scope of what a compiler can fix, this needs to be addressed by a Vulkan extension. I know some Mesa developers who might be interested in building such a thing though :)

[vosen]

I think Intel is looking into adding limited function pointer support (where you can use a function pointer but the pointer has to point into one of X declared functions) in SPIR-V and perhaps also in OpenCL?
CUDA and HIP function pointers just work (well, they are buggy on HIP, but it's fixable)

[RhynarAI]

OPENCL would be great and open up even more possibilities :) Imagine using older XEON Phis to run CUDA Code :) or some of the AMD OPENCL Exclusive enterprise GPUs :)

[ethanc8]

I think OpenCL might be viable. The chipStar project is able to compile CUDA/HIP to SPIR-V and run it with OpenCL, so it will probably be mostly complete. They use both Intel SPIR-V and OpenCL SPIR-V, and as far as I know they have similar capabilities, but I have not looked into it too deeply.