community: Add support for SAP HANA Vector hnsw index creation and advanced filtering

[cinqisap]

Description
This PR synchronizes the hanavector integration with updates made in the Python version of LangChain (PR #27884 and PR #20821). The following changes have been introduced:

1. Index Creation Support:

   • Added a new function, createHnswIndex, to hanavector.ts. This function facilitates the creation of indexes for SAP HANA Vector, improving usability and alignment with Python capabilities.

2. Advanced Filtering Support:

   • Introduced support for advanced filtering operations. The newly supported operators are:
     • Equality and Comparison: $eq, $ne, $gt, $gte, $lt, $lte
     • Range and Membership: $between, $in, $nin
     • Pattern Matching: $like
     • Logical Operations: $and, $or
   • These operators enable users to perform more complex filter operations on metadata column.

Issue: No associated issue.

Dependencies: No new dependencies were introduced in this PR.

Twitter handle: @sapopensource

---

Changed integration main file

• libs/langchain-community/src/vectorstores/hanavector.ts

Integration tests

• libs/langchain-community/src/vectorstores/tests/hanavector.int.test.ts

Examples

• examples/src/indexes/vector_stores/hana_vector/

Documentation

• docs/core_docs/docs/integrations/vectorstores/hanavector.mdx

Testing

• Integration tests and examples were run successfully.
• Access credentials for running the integration tests can be provided to maintainers upon request.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-docs	✅ Ready (Inspect)	Visit Preview		Nov 27, 2024 8:48am

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-api-refs	⬜️ Ignored (Inspect)			Nov 27, 2024 8:48am

[jacoblee93]

You'll still want this for methods on the superclass

[cinqisap]

Thank you for pointing it out! I added this declaration and used it in methods.

[jacoblee93]

Looks nice! One small comment. I'm also not familiar with SAP Hana syntax but would just ask that you scan for any injection-type attacks in the code.

[jacoblee93]

Seems you're failing the new unit tests:

Summary of all failing tests
FAIL src/vectorstores/tests/hanavector.test.ts
  ● Sanity check tests › should sanitize int with illegal value

    expect(received).toContain(expected) // indexOf

    Expected substring: "must not be smaller than -1"
    Received string:    "Value (NaN) must not be smaller than 0"

       8 |       // eslint-disable-next-line @typescript-eslint/no-explicit-any
       9 |     } catch (error: any) {
    > 10 |       expect(error.message).toContain("must not be smaller than -1");
         |                             ^
      11 |     }
      12 |   });
      13 |

      at Object.toContain (src/vectorstores/tests/hanavector.test.ts:10:29)

  ● Sanity check tests › should sanitize int with negative values

    Value (-1) must not be smaller than 0

      243 |     const value = parseInt(inputInt.toString(), 10);
      244 |     if (Number.isNaN(value) || value < lowerBound) {
    > 245 |       throw new Error(
          |             ^
      246 |         `Value (${value}) must not be smaller than ${lowerBound}`
      247 |       );
      248 |     }

      at Function.sanitizeInt (src/vectorstores/hanavector.ts:245:13)
      at Object.sanitizeInt (src/vectorstores/tests/hanavector.test.ts:20:19)

  ● Sanity check tests › should sanitize int with illegal negative value

    expect(received).toContain(expected) // indexOf

    Expected substring: "must not be smaller than -1"
    Received string:    "Value (-2) must not be smaller than 0"

      27 |       // eslint-disable-next-line @typescript-eslint/no-explicit-any
      28 |     } catch (error: any) {
    > 29 |       expect(error.message).toContain("must not be smaller than -1");
         |                             ^
      30 |     }
      31 |   });
      32 |

      at Object.toContain (src/vectorstores/tests/hanavector.test.ts:29:29)


Test Suites: 1 failed, 3 skipped, 63 passed, 64 of 67 total
Tests:       3 failed, 7 skipped, 223 passed, 233 total
Snapshots:   6 passed, 6 total
Time:        22.836 s
Ran all test suites.

[cinqisap]

@jacoblee93 Hi Jacob, I've adjusted the unit tests for current sanitization methods.

[cinqisap]

Looks nice! One small comment. I'm also not familiar with SAP Hana syntax but would just ask that you scan for any injection-type attacks in the code.

Hi Jacob, I am currently "human-scanning" the source code. It looks well protected against SQL injections since we have input sanitization functions and parameterized the SQL statement. I am also trying to provide the tool-scan result if possible.