Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[catalog-server] Full text search across elements #1363

Open
justinfagnani opened this issue Oct 12, 2022 · 0 comments
Open

[catalog-server] Full text search across elements #1363

justinfagnani opened this issue Oct 12, 2022 · 0 comments
Assignees
Labels
New Site Issues for the rewrite of the site and catalog.

Comments

@justinfagnani
Copy link
Collaborator

We need a way to do full-text search across all elements.

GCP doesn't have a full-text search service, for some reason. We could use a service like Algolia or Elasticsearch or implement a minimal full-text search ourselves.

For a search service we need to figure out how to keep the search index live as the database changes, how to integrate with structured search, and how to mix full-text result relevance with other rankings like discussed in #1361. We could try to update the index as we import new packages, or run a scheduled job that updates the index X times per day and live with some lag.

For a custom search implementation we need to do some of the standard full-text search steps ourselves, but keeping the search index in sync with data updates in a bit easier, and can even be done within transactions. The steps include:

  1. Tokenizing and stemming search-contributing text fields and storing those in either the document to be searched, or a separate search table.
  2. Tokenizing and stemming the user query
  3. Performing an array-contains-any query with the user terms against the search field
@justinfagnani justinfagnani self-assigned this Oct 12, 2022
@justinfagnani justinfagnani added the New Site Issues for the rewrite of the site and catalog. label Oct 12, 2022
@justinfagnani justinfagnani moved this to ⏱ In Progress in Lit Project Board Oct 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
New Site Issues for the rewrite of the site and catalog.
Projects
None yet
Development

No branches or pull requests

1 participant