How to perform Wildcard, Fuzzy, Proximity or Fielded Searches in Pyserini?

[stefantrapp]

Hi, when trying out the LuceneSearcher class on a simple example index like

{"id": "doc1", "contents": "contents of doc one."}
{"id": "doc2", "contents": "contents of document two."}
{"id": "doc3", "contents": "here's some text in document three."}

I couldn't find out how to use the full power of the Lucene Query Parsers in Pyserini. For example

hits = searcher.search("doc*") does only return the result

1 doc1 0.53460

i.e. it's not performing Wildcard Search as I expected. (How) Is it possible to do Wildcard Search, Fuzzy Search, Proximity Search and so on, or a Fielded Search like for example

searcher.search("id:doc2")

Thanks.

[lintool]

Hi @stefantrapp - check out https://github.com/castorini/pyserini/blob/master/docs/usage-querybuilder.md

The entire Lucene query API should be available.

[stefantrapp]

Thanks, Jimmy, for your quick response and the link. I succeded to do a "Fielded Search" like this tq = querybuilder.get_term_query('doc2', 'id') hits = searcher.search(tq) but unfortunately I'm still struggling with Wildcards or Proximity. Maybe the example you linked can be elaborated a bit more on this? Sorry for following up. Thanks and kind regards, Stefan

…

________________________________ Von: Jimmy Lin ***@***.***> Gesendet: Samstag, 22. Oktober 2022 15:04:41 An: castorini/pyserini Cc: Trapp, Stefan; Mention Betreff: [Extern] Re: [castorini/pyserini] How to perform Wildcard, Fuzzy, Proximity or Fielded Searches in Pyserini? (Discussion #1315) Hi @stefantrapp<https://github.com/stefantrapp> - check out https://github.com/castorini/pyserini/blob/master/docs/usage-querybuilder.md The entire Lucene query API should be available. — Reply to this email directly, view it on GitHub<#1315 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMBTYXFJX72GCJRRJMGOKU3WEPQ6TANCNFSM6AAAAAARL2DOWI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[HAKSOAT]

Hi @stefantrapp

There's a PR in the works for this: #1330

When merged, you'll be able to do something like:

query = querybuilder.get_standard_query('doc*')
hits = searcher.search(query)

or work with nested logic like:

query = querybuilder.get_complex_phrase_query('"contents doc*"~3')
hits = searcher.search(query)

Does this address your use case?

[stefantrapp]

Hi Habeeb, Thanks for your email. That looks promising, yes. Thanks and best regards, Stefan

…

________________________________ Von: Habeeb Shopeju ***@***.***> Gesendet: Sonntag, 30. Oktober 2022 22:54:01 An: castorini/pyserini Cc: Trapp, Stefan; Mention Betreff: [Extern] Re: [castorini/pyserini] How to perform Wildcard, Fuzzy, Proximity or Fielded Searches in Pyserini? (Discussion #1315) Hi @stefantrapp<https://github.com/stefantrapp> There's a PR in the works for this: #1330<#1330> When merged, you'll be able to do something like: query = querybuilder.get_standard_query('doc*') hits = searcher.search(query) or work with nested logic like: query = querybuilder.get_complex_phrase_query('"contents doc*"~3') hits = searcher.search(query) Does this address your use case? — Reply to this email directly, view it on GitHub<#1315 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMBTYXDVUJYP5UM7LWRODM3WF3N6TANCNFSM6AAAAAARL2DOWI>. You are receiving this because you were mentioned.Message ID: ***@***.***>