Smart search without indexing?

[krileon]

Now that the previous search component is no longer provided with Joomla 4 this poses a big problem for our extensions. We have privacy controls, conditions, and various other access restrictions on content that's outside the scope of Joomlas ACL. These conditions can be based off the viewing users profile data and more. So the indexers limited ACL isn't good enough for searching our extensions content.

Is there not a way to have smart searching work with our content without needing to index it? I understand this is slower, but there isn't much of a choice here as we need the more advanced permissions.

Is the only way to do this to index the content then use onFinderResult to filter it further? This is a bit of a sloppy way to do it through since removing rows could just end up with an empty result set.

Has anyone had any luck with finding a solution? I've been digging through the smart search component and the above event usage was the best I could find.

[brianteeman]

No reason not to keep using the old search

[krileon]

That works fine if they migrated from Joomla 3, but the extension doesn't exist on clean Joomla 4 installs. Is there somewhere I can send people to download the old search extension for Joomla 4 for clean installs?

[brianteeman]

I assume it will be added to the JED in the core extensions part but don't know when.

In the meantime there is the release section https://github.com/joomla-extensions/search

[krileon]

Ok, thank you. Very much appreciated. 🙂

[Fedik]

Finder in general is a very good thing.

Is the only way to do this to index the content then use onFinderResult to filter it further? This is a bit of a sloppy way to do it through since removing rows could just end up with an empty result set.

Where did you found "onFinderResult" event? I cannot find :)

In general, yes, you can add extra data to Indexing result with onPrepareFinderContent event, example extra Taxonomy.
Then try filter by it

joomla-cms/components/com_finder/src/Model/SearchModel.php

Lines 396 to 398 in bfaf931

	// Get the dynamic taxonomy filters.
	$options['filters'] = $request->get('t', $params->get('t', array()), 'array');

However some onFinerPrepareResult event would be a good idea, @Hackwar what do you think? So the plugin can hookup to add extra filter.

[infograf768]

@Fedik

joomla-cms/plugins/system/highlight/highlight.php

Line 121 in bfaf931

public function onFinderResult($item, $query)

joomla-cms/components/com_finder/src/View/Search/HtmlView.php

Line 176 in bfaf931

$app->triggerEvent('onFinderResult', array(&$result, &$this->query));

[Fedik]

Thanks for a link, did not seen that exists.

[Fedik]

added to docs

[Hackwar]

The onFinderResult event is to prepare the content further, for example by adding updated content to it. It isn't really suitable for filtering content further. I would be really interested what access restrictions you need which can't be accomplished by the possibilities of Joomla. Have you looked at the taxonomies to allow further filtering?

[krileon]

Unless we can join other tables and filter off those joins it will be impossible for the finder to work with our content. The smart search just doesn't work with some content types in my opinion. It's great for cached efficient searching of static content with simple access controls, but that's where that ends. I think I can make some use cases work with Taxonomy and will have to be enough for my customers I guess.

I would've preferred to see the old search functionality rolled into the smart search so we had options here instead of 2 search components. Basically for smart search to support live searching as well. Additionally I don't know why I would want to store my content twice either when I prefer to just write an efficient query.

[bembelimen]

There is no restriction in further development of the smart search. So can you deliver some real world use cases where the smart search is not enough and probably it can be fixed/enhanced?

[Hackwar]

Again, can you please provide the situation where Smart Search is not working out for you? I've now been working for several years with the smart search that is now part of J4 and basically never found a situation where smart search wasn't a good fit.

I tried merging com_search and smart search. I wrote a com_search plugin that did what smart search is doing and I tried to integrate com_search into smart search, but in short, they are completely incompatible to each other. Not that you can't search everything in smart search, but there is no way to join those 2 systems in a usefull way. com_search has no real idea of filtering or relevance-ordering of results.

Why would you use the indexing and not use an efficient query? Because efficient queries are not easy and even efficient queries do add up over time. A pagebuilder, com_content, a forum, a shop, a calendar, maybe a sizeable gallery and all of a sudden each request to com_search means 20 very complex queries to your DB and an easy way to DoS a site.

[krileon]

Again, can you please provide the situation where Smart Search is not working out for you? I've now been working for several years with the smart search that is now part of J4 and basically never found a situation where smart search wasn't a good fit.

We have custom privacy controls and custom content display conditions that go far beyond basic ACL. These can change at any time. Specifically privacy controls. A field could be public and now it's private or it's only visible to a specific set of users based off their friend status. The smart search would just be re-indexing non-stop assuming it could even work with such criteria. The documentation isn't clear enough for me to say whether our criteria can be supported or not with it.

[dformdotdk]

I know this is an older thread, but I was wondering @krileon if you managed to find a good solution. I'm in a similar situation, where I need to filter the search results based on custom user fields values / custom content fields values.

So far, my only viable solution is a custom query and completely disregard smart search.

[krileon]

I opted to allow for indexing everything (user can select what fields to index) then on onFinderResult I'm outputting a parameter value that the admin provides for the search results display. So at least potential private data can't be displayed unless the admin explicitly configures for it to display, but can be potentially searched.

[dformdotdk]

I see. My case may be a bit different. I want to control the display of the entire article in the search results.

My problem with onFinderResult is that, while I can control the item and with some logic determine if the item should be displayed to the user, I have no control over the pagination. Say I have 10 items per page, but I unset 5 items in the onFinderResult event, then only 5 results would be displayed on the page. Some pages may not even have an item to display. And that's a dealbreaker.

In a perfect world, we would have an event to hook into, to manipulate the entire search result object.

[krileon]

Yeah, there's no way around that from what I could find. Maybe what you could do is instead of unset the results replace them with dummy text then use CSS to blur them with a hover text in the center saying "Private" so they'll know content is there, but they don't have the permissions yet.

A PR to add an event to the finder list query function that gives the query object would be all you'd need as that'd let you add to the query.

[Hackwar]

I'm sorry, but that would be out of scope for Smart Search/Finder. I'm actually pretty sure that you could fit into the "confines" of Joomla here with viewlevels, etc. If not, I'd advise you to write your own search. Unfortunately there are limits to what we can provide in core.

[krileon]

The problem we're having basically is the access checks for smart search results are from the viewing user to the results view access level and that's it. It doesn't work when you need to check beyond view access level like individual privacy controls on custom fields or conditions on custom fields. If we had some sort of generic way to add additional access checks that'd be fantastic, but I honestly don't have any idea as to how to best implement that right now.

For example lets say I have a field called Address. That field has privacy controls (via another plugin) that limits only those that are friends (via another plugin) see it. So a way to index that would probably be something like access_custom: "friends" then during the query I would need to be able to extend the sql to account for checking if the resulting object and the viewer are friends.

I think the simplest approach for Joomla to support this without any real maintenance is just be more liberal with events throughout Joomla. So an event for list queries for example would allow for easily adjusting the query to meet custom needs.