[SPARSE] Add support for rocSPARSE backend #544

Rbiessy · 2024-07-24T13:57:02Z

Description

Add support for the rocSPARSE backend.

Depends on #527 and #532.

Rendered docs: docs.zip

Checklist

All Submissions

Do all unit tests pass locally? amd_log.txt nvidia_log.txt intel_log.txt
- Tested the rocSPARSE backend on MI210 with oneAPI 2024.2 compiler + Codeplay AMD plugin + ROCm 6.1.0
Have you formatted the code using clang-format?

Rbiessy · 2024-07-30T15:58:25Z

Tests log on W6800: amd_w6800_log.txt

gajanan-choudhary · 2024-10-29T12:38:20Z

@Rbiessy, now that #527 is merged in, could you please rebase this branch against main, resolve conflicts, and force-push when you can so that I can start reviewing this PR?

Rbiessy · 2024-10-29T12:42:27Z

Yes, this is in progress! I expect I will need a few days at least. I'm aiming to merge the PR by the end of the year.

Rbiessy · 2024-11-01T16:32:27Z

@gajanan-choudhary I have updated the PR with recent changes from cuSPARSE. Note that it moved almost all the content from cusparse_task.hpp to common_launch_task.hpp with almost no changes.
New test logs:
log_mi210.txt
log_a100.txt
log_pvc.txt

README.md

gajanan-choudhary · 2024-11-24T17:28:49Z

docs/domains/sparse_linear_algebra.rst

+- The same sparse matrix handle cannot be reused for multiple operations
+  ``spmm``, ``spmv``, or ``spsv``. Doing so will throw a
+  ``oneapi::mkl::unimplemented`` exception. See `#332
+  <https://github.com/ROCm/rocSPARSE/issues/332>`_.


Wow, this is quite severe, but seems to be a legitimate issue on rocSPARSE side right now.

If and when they fix this issue, though, will it be easy for us to make changes (with a version check of course) that correctly performs the operations rather than throwing an unimplemented exception?

Yes, it's easy to fix on oneMath side. The issue is also referenced in this comment: https://github.com/oneapi-src/oneMKL/pull/544/files#diff-3b8c1c2c71abd54f8f90f43415c2f17b2a7fdb81c2b882c210f3cba56b4679adR63
One would just need to remove the used member, its 2 usages below as well as the mark_used method.

src/sparse_blas/backends/cusparse/cusparse_handles.cpp

src/sparse_blas/backends/rocsparse/rocsparse_task.hpp

gajanan-choudhary · 2024-11-26T11:02:27Z

src/sparse_blas/backends/rocsparse/rocsparse_handles.hpp

+        if (this->format == detail::sparse_format::COO &&
+            !this->has_matrix_property(matrix_property::sorted)) {


In the documentation, docs/domains/sparse_linear_algebra.rst, you've written:

The CSR format requires the column indices to be sorted within each row.

So shouldn't we be handling both CSR and COO formats here instead of just COO?

Good point, I added this check in b6518c9
I've also made sure we also have tests on AMD and Nvidia which are skipped when a property is not set. The runtime example also need to set this property now.

gajanan-choudhary · 2024-11-26T11:17:38Z

src/sparse_blas/backends/rocsparse/rocsparse_scope_handle.cpp

+    handle_helper.rocsparse_handle_container_mapper_.insert(
+        std::make_pair(piPlacedContext_, atomic_ptr));
+
+    sycl::detail::pi::contextSetExtendedDeleter(*placedContext_, ContextCallback, atomic_ptr);


Is it unchanged between the removed PI versus new UR APIs? Isn't there a sycl::detail::ur::contextSetExtendedDeleter somewhere that you need to place under #ifdef ONEAPI_ONEMKL_PI_INTERFACE_REMOVED #else #endif... ?

The contextSetExtendedDeleter is still in the pi namespace see https://github.com/intel/llvm/blob/sycl/sycl/include/sycl/detail/ur.hpp#L100. As the comment says there are plans to deprecate it. The oneAPI core team at Codeplay started to change the scopeContextHandle mechanism for BLAS in #609. They will make sure that oneMKL Interface will keep working if/when contextSetExtendedDeleter is deprecated.
If we ever need to change the scopeContextHandle mechanism, I would rather that we do it for both cuSPARSE and rocSPARSE backends in a separate PR.

gajanan-choudhary · 2024-11-26T11:23:41Z

src/sparse_blas/backends/rocsparse/rocsparse_handles.cpp

+    auto event = queue.submit([&](sycl::handler& cgh) {
+        auto acc = val.template get_access<sycl::access::mode::read_write>(cgh);
+        detail::submit_host_task(cgh, queue, [=](sycl::interop_handle ih) {
+            if (dvhandle->size != size) {


I wonder if we should throw an exception if dvhandle->size < size instead of !=. E.g., users may want to pad thesycl::buffer for some reason with zeros in the end, which wouldn't work with the current if condition.

Nevermind, this is set_dense_vector_data used for replacing/switching out arrays in an existing handle. The condition is fine then.

Although maybe it doesn't hurt to add the check both here and in init_xxx_data

The check to verify that the buffer size is compatible with the handle size is done elsewhere, see https://github.com/oneapi-src/oneMKL/blob/develop/src/sparse_blas/generic_container.hpp#L164
This is a common place that the other backends also use. That check will run for init_*_data functions.

gajanan-choudhary · 2024-11-26T11:27:31Z

src/sparse_blas/backends/rocsparse/rocsparse_handles.cpp

+template <typename fpType>
+void init_dense_vector(sycl::queue& queue, dense_vector_handle_t* p_dvhandle, std::int64_t size,
+                       sycl::buffer<fpType, 1> val) {
+    auto event = queue.submit([&](sycl::handler& cgh) {


Can we add a check here for sycl::buffer case that checks buffer->size() >= size and throws an exception otherwise?

This is already done, see my comment above: #544 (comment)

gajanan-choudhary · 2024-11-26T12:25:10Z

src/sparse_blas/backends/rocsparse/operations/rocsparse_spmv.cpp

+    CHECK_DESCR_MATCH(spmv_descr, alg, "spmv_optimize");
+
+    A_handle->mark_used();
+    auto& buffer_size = spmv_descr->temp_buffer_size;


Is there a reason why this variable specifically is kept and captured by reference outside compute_functor while other members of spmv_descr are created inside the functor? If it is because buffer_size is used later in an if condition, I'd prefer it if we change this approach and capture everything by [=] in compute_functor, even though it would mean replacing the last buffer_size > 0 with spmv_descr->temp_buffer_size > 0. I know that what you have right now is expected to work in this particular case (because of the condition that spmv_descr must live as long as spmv is being called), it is normally a bad idea to capture variables by reference in SYCL functors that are going to be running asynchronously unless you have an immediate event.wait() (e.g., like what we are doing in spmv_buffer_size() function).

Sure that's fine, done in c53ef8d

gajanan-choudhary

LGTM other than some minor change requests. This PR was a lot easier to review having reviewed #527. Thanks for the fantastic work!

Rbiessy · 2024-12-02T17:25:43Z

Thanks for the review Gajanan!

For completeness, logs on MI210: log_mi210.txt
and A100: log_a100.txt

FYI we will get a second approval and I aim to merge this after the oneMKL Interface renaming PR.

Rbiessy requested a review from a team July 24, 2024 13:57

Rbiessy changed the title ~~[SPARSE] Add support for rocsparse backend~~ [SPARSE] Add support for rocSPARSE backend Jul 24, 2024

Rbiessy mentioned this pull request Jul 30, 2024

[oneMKL][spblas] Restrict features not supported by any backends uxlfoundation/oneAPI-spec#542

Merged

gajanan-choudhary assigned Rbiessy Sep 12, 2024

gajanan-choudhary added backend A request to enable new implementation behind API Sparse BLAS domain Sparse BLAS domain issue/request feature A request to add a new feature labels Sep 12, 2024

[SPARSE] Add support for rocSPARSE backend

c4b952c

Rbiessy force-pushed the romain/rocsparse branch from af87004 to c4b952c Compare November 1, 2024 16:28

Rbiessy requested a review from a team as a code owner November 1, 2024 16:28

Rbiessy mentioned this pull request Nov 5, 2024

Rename oneMKL Interface to oneMath #602

Merged

2 tasks

gajanan-choudhary reviewed Nov 24, 2024

View reviewed changes

README.md Show resolved Hide resolved

gajanan-choudhary reviewed Nov 24, 2024

View reviewed changes

src/sparse_blas/backends/cusparse/cusparse_handles.cpp Show resolved Hide resolved

gajanan-choudhary reviewed Nov 24, 2024

View reviewed changes

src/sparse_blas/backends/rocsparse/rocsparse_task.hpp Show resolved Hide resolved

Rbiessy added 2 commits November 25, 2024 15:32

Add rocSPARSE product and version

dc22051

Add newline at end of file

633c655

gajanan-choudhary reviewed Nov 26, 2024

View reviewed changes

gajanan-choudhary approved these changes Nov 26, 2024

View reviewed changes

Rbiessy added 2 commits December 2, 2024 16:59

Throw exceptions on unsorted CSR format + tests this

b6518c9

Avoid capture by reference

c53ef8d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARSE] Add support for rocSPARSE backend #544

[SPARSE] Add support for rocSPARSE backend #544

Rbiessy commented Jul 24, 2024

Rbiessy commented Jul 30, 2024

gajanan-choudhary commented Oct 29, 2024 •

edited

Loading

Rbiessy commented Oct 29, 2024

Rbiessy commented Nov 1, 2024

gajanan-choudhary Nov 24, 2024

gajanan-choudhary Nov 24, 2024

Rbiessy Nov 25, 2024

gajanan-choudhary Nov 26, 2024 •

edited

Loading

Rbiessy Dec 2, 2024

gajanan-choudhary Nov 26, 2024 •

edited

Loading

Rbiessy Nov 29, 2024

gajanan-choudhary Nov 26, 2024 •

edited

Loading

gajanan-choudhary Nov 26, 2024

gajanan-choudhary Nov 26, 2024

Rbiessy Nov 29, 2024 •

edited

Loading

gajanan-choudhary Nov 26, 2024

Rbiessy Nov 29, 2024

gajanan-choudhary Nov 26, 2024

Rbiessy Dec 2, 2024

gajanan-choudhary left a comment

Rbiessy commented Dec 2, 2024

		if (this->format == detail::sparse_format::COO &&
		!this->has_matrix_property(matrix_property::sorted)) {

[SPARSE] Add support for rocSPARSE backend #544

Are you sure you want to change the base?

[SPARSE] Add support for rocSPARSE backend #544

Conversation

Rbiessy commented Jul 24, 2024

Description

Checklist

All Submissions

Rbiessy commented Jul 30, 2024

gajanan-choudhary commented Oct 29, 2024 • edited Loading

Rbiessy commented Oct 29, 2024

Rbiessy commented Nov 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gajanan-choudhary Nov 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gajanan-choudhary Nov 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gajanan-choudhary Nov 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Rbiessy Nov 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gajanan-choudhary left a comment

Choose a reason for hiding this comment

Rbiessy commented Dec 2, 2024

gajanan-choudhary commented Oct 29, 2024 •

edited

Loading

gajanan-choudhary Nov 26, 2024 •

edited

Loading

gajanan-choudhary Nov 26, 2024 •

edited

Loading

gajanan-choudhary Nov 26, 2024 •

edited

Loading

Rbiessy Nov 29, 2024 •

edited

Loading