Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding feature descriptions #55

Open
wants to merge 1 commit into
base: feature-descriptions
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs-site/content/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,10 @@ let config = {
['/0.19.0/guide/features/typo-tolerance', 'Typo Tolerance'],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I run this on dev, I see this error:

No matching page found for sidebar item "/0.19.0/guide/features/curation"

I don't see a file called curation.md in the repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't added it yet. Will be added in future PR. Do you want me to add a placeholder file?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, in that case let's remove the line from the side-bar configuration, so the nav renders in the meantime.

['/0.19.0/guide/features/faceting', 'Faceting'],
['/0.19.0/guide/features/filtering', 'Filtering'],
['/0.19.0/guide/features/federated-search', 'Federated Search'],
['/0.19.0/guide/features/multi-tenant-indices', 'Multi-tenant Indices'],
['/0.19.0/guide/features/synonyms', 'Synonyms'],
['/0.19.0/guide/features/clustering', 'Raft Based Clustering'],
],
},
],
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs-site/content/.vuepress/public/images/typesense-filter.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
186 changes: 186 additions & 0 deletions docs-site/content/0.19.0/guide/features/federated.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
# Federated Search
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's call this file federated-search.md so the url has the word search


Federated or multi search is a way to search for documents in multiple collections as part of a single search query. You can also use multi-search to send multiple search queries to the same collections, essentially giving you a way to batch search queries in a single HTTP request. Federated search can help reduce network latencies. It can also be used to present similar content from other collections, that might encourage users to browse more content across your application. For example, your application might have different collections for `Nike` and `Adidas`. Now, if a user is looking for a shoe from a specific branch and they might not know what other brands are available, the search query can perform a search on both the collections and return relevant results from both the collections.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to present similar content from other collections

to present results for the same search query from multiple collections...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, your application might have different collections for Nike and Adidas. Now, if a user is looking for a shoe from a specific branch and they might not know what other brands are available, the search query can perform a search on both the collections and return relevant results from both the collections.

This example is a little confusing. Now that we have the BHP example below, let's remove this one.


For example, if you search for `Canon` on https://www.bhphotovideo.com/, you would see that there are multiple results shown including products, suggestions and help resources:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

products, suggestions and help resources...

each of which could be indexed in separate collections, but the search term Canon is sent to all three indices and results From multiple collections are shown side-by-side


![bhp federated example](~@images/bhp-federated.png)

Typesense supports searching across multiple collections in a single HTTP request. Let's create a search query for shoes:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use make an example though would mimic the BHP example from above, searching across 3 collections?


<Tabs :tabs="['JavaScript','PHP','Python','Ruby']">
<template v-slot:JavaScript>

```javascript
let searchRequests = {
'searches': [
{
'collection': 'products',
'q': 'shoe',
'filter_by': 'price:=[50..120]'
},
{
'collection': 'brands',
'q': 'Nike'
}
]
}

// Search parameters that are common to all searches go here
let commonSearchParams = {
'query_by': 'name',
}

client.multiSearch.perform(searchRequests, commonSearchParams)
```
</template>

<template v-slot:PHP>

```php
$searchRequests = [
'searches' => [
[
'collection' => 'products',
'q' => 'shoe',
'filter_by' => 'price:=[50..120]'
],
[
'collection' => 'brands',
'q' => 'Nike'
]
]
];

// Search parameters that are common to all searches go here
$commonSearchParams = [
'query_by' => 'name',
];

$client->multiSearch->perform($searchRequests, $commonSearchParams);
```
</template>
<template v-slot:Python>

```python
search_requests = {
'searches': [
{
'collection': 'products',
'q': 'shoe',
'filter_by': 'price:=[50..120]'
},
{
'collection': 'brands',
'q': 'Nike'
}
]
}

# Search parameters that are common to all searches go here
common_search_params = {
'query_by': 'name',
}

client.multi_search.perform(search_requests, common_search_params)
```
</template>
<template v-slot:Ruby>

```ruby
search_requests = {
'searches': [
{
'collection': 'products',
'q': 'shoe',
'filter_by': 'price:=[50..120]'
},
{
'collection': 'brands',
'q': 'Nike'
}
]
}

# Search parameters that are common to all searches go here
common_search_params = {
'query_by': 'name',
}

client.multi_search.perform(search_requests, common_search_params)
```
</template>
</Tabs>

Sample response:

```json
{
"results": [
{
"facet_counts": [],
"found": 1,
"hits": [
{
"document": {
"name": "Blue shoe",
"brand": "Adidas",
"id": "126",
"price": 50
},
"highlights": [
{
"field": "name",
"matched_tokens": [
"shoe"
],
"snippet": "Blue <mark>shoe</mark>"
}
],
"text_match": 130816
}
],
"out_of": 10,
"page": 1,
"request_params": {
"per_page": 10,
"q": "shoe"
},
"search_time_ms": 1
},
{
"facet_counts": [],
"found": 1,
"hits": [
{
"document": {
"name": "Nike shoes",
"brand": "Nike",
"id": "391",
"price": 60
},
"highlights": [
{
"field": "name",
"matched_tokens": [
"Nike"
],
"snippet": "<mark>Nike</mark>shoes"
}
],
"text_match": 144112
}
],
"out_of": 5,
"page": 1,
"request_params": {
"per_page": 10,
"q": "Nike"
},
"search_time_ms": 1
},
]
}
```

In the above example, the user is searching for a `Nike` shoe, but the `multiSerch` query returns results from the `Adidas` collection as well. You can control the number of maximum search requests using the `limit_multi_searches` parameter. By default, there is no limit. You can find more details on the argument [here](../../0.19.0/api/documents.html#federated-multi-search).
170 changes: 170 additions & 0 deletions docs-site/content/0.19.0/guide/features/multi-tenant-indices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Scoped API Keys

Typesense is designed with security and fine-grained access control in mind. To perform any action with Typesense, you need API keys. Typesense also allows access control on API keys. You can define capabilities as to what a user can or cannot do. You can also restrict access to a specific document or collection. In the case of a multi-tenant environment, you can scope API keys to a particular subset. This is helpful when you have indexed data from multiple tenants in your Typesense server and want to restrict users to only access their subset of data.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of a multi-tenant environment, you can scope API keys to a particular subset

... to a particular subset of records.

This is helpful when you have indexed data from multiple tenants

tenants -> users


Typesense allows you to create API keys that have pre-defined filters embedded in them. So, whenever you run a search query with these API keys, those filters are automatically applied and cannot be overridden. You can then provide those search API keys to users and they would only be able to access the data that is allowed by the set filter. To create scoped API keys, you just need a parent key.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To create scoped API keys, you just need a parent key

... a parent key that only has documents:search permissions on it.


Let's take example of a [company collection](../../api/collections.html#create-a-collection), that has the following documents:

```shell
{"company_id":124,"company_name":"Stark Industries","country":"USA","id":"0","num_employees":3355}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you update these records to contain one more field called accessible_by_user_id and then use that field as the embedded filter in the example? That's the most common way people use multi-tenant indices.

For eg:

{"accessible_by_user_id": [1, 2], "company_id":124,"company_name":"Stark Industries","country":"USA","id":"0","num_employees":3355}
...

Then it's easier to explain with a real-world example:

Let's say we now want to restrict users to only be able to access records that they have access to. We can create a scoped API key with an embedded accessible_by_user_id: user_id filter, based on the logged-in user's user_id. Let's see how...

or something along those lines

{"company_id":125,"company_name":"Wayne Enterprises","country":"USA","id":"1","num_employees":4538}
{"company_id":126,"company_name":"Daily Planet","country":"USA","id":"2","num_employees":2232}
{"company_id":127,"company_name":"New Stark Industries","country":"USA","id":"3","num_employees":7945}
```

Now, let's create a scoped API key that will restrict access to documents that have the `company_id` value as 124.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this, could you also add a code snippet to indicate how to create a parent key with just search permissions, so this article is self-contained?


<Tabs :tabs="['JavaScript','PHP','Python','Ruby', 'Shell']">
<template v-slot:JavaScript>

```javascript
keyWithSearchPermissions = 'RN23GFr1s6jQ9kgSNg2O7fYcAUXU7127'
client.keys().generateScopedSearchKey(keyWithSearchPermissions, {'filter_by': 'company_id:124', 'expires_at': 1611590465})
```
</template>

<template v-slot:PHP>

```php
$keyWithSearchPermissions = 'RN23GFr1s6jQ9kgSNg2O7fYcAUXU7127';
$client->keys()->generateScopedSearchKey($keyWithSearchPermissions, ['filter_by' => 'company_id:124', 'expires_at' => 1611590465]);
```
</template>
<template v-slot:Python>

```python
key_with_search_permissions = 'RN23GFr1s6jQ9kgSNg2O7fYcAUXU7127'
client.keys().generate_scoped_search_key(key_with_search_permissions, {"filter_by": "company_id:124", "expires_at": 1611590465})
```
</template>
<template v-slot:Ruby>

```ruby
key_with_search_permissions = 'RN23GFr1s6jQ9kgSNg2O7fYcAUXU7127'
client.keys().generate_scoped_search_key(key_with_search_permissions, {'filter_by': 'company_id:124', 'expires_at': 1611590465})
```
</template>
<template v-slot:Shell>

```bash
KEY_WITH_SEARCH_PERMISSIONS="RN23GFr1s6jQ9kgSNg2O7fYcAUXU7127"
EMBEDDED_SEARCH_PARAMETERS_JSON='{"filter_by":"company_id:124","expires_at":1611590465}'

digest=$(echo -n $EMBEDDED_SEARCH_PARAMETERS_JSON | openssl dgst -sha256 -hmac $KEY_WITH_SEARCH_PERMISSIONS -binary | base64)

scoped_api_key=$(echo -n "${digest}${KEY_WITH_SEARCH_PERMISSIONS:0:4}${EMBEDDED_SEARCH_PARAMETERS_JSON}" | base64)

echo $scoped_api_key
```

</template>
</Tabs>

Sample response:

```json
"RDhxa2VKTnBQVkxaVlFIOS9JWDZ2bDdtMU5HL3laa0pab2pTeEUzbFBhZz1STjIzeyJmaWx0ZXJfYnkiOiJjb21wYW55X2lkOjEyNCIsImV4cGlyZXNfYXQiOjE2MTE1OTA0NjV9"
```

The `expires_at` parameter sets the expiration date for the API key and must be less that the expiration of parent API key. Let's perform a search using the scoped API key:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New paragraph for "Let's perform a search using the scoped API key"


<Tabs :tabs="['JavaScript','PHP','Python','Ruby']">
<template v-slot:JavaScript>

```javascript
let searchParameters = {
'q' : 'Stark',
'query_by' : 'company_name',
'sort_by' : 'num_employees:desc'
}

client.collections('companies')
.documents()
.search(searchParameters)
.then(function (searchResults) {
console.log(searchResults)
})
```
</template>

<template v-slot:PHP>

```php
$$searchParameters = [
'q' => 'Stark',
'query_by' => 'company_name',
'sort_by' => 'num_employees:desc'
]

$client->collections['companies']->documents->search($searchParameters)
```
</template>
<template v-slot:Python>

```python
search_parameters = {
'q' : 'Stark',
'query_by' : 'company_name',
'sort_by' : 'num_employees:desc'
}

client.collections['companies'].documents.search(search_parameters)
```
</template>
<template v-slot:Ruby>

```ruby
search_parameters = {
'q' => 'Stark',
'query_by' => 'company_name',
'sort_by' => 'num_employees:desc'
}

client.collections['companies'].documents.search(search_parameters)
```
</template>
</Tabs>


Response:

```json
{
"facet_counts": [],
"found": 1,
"hits": [
{
"document": {
"company_id": 124,
"company_name": "Stark Industries",
"country": "USA",
"id": "0",
"num_employees": 3355
},
"highlights": [
{
"field": "company_name",
"matched_tokens": [
"Stark"
],
"snippet": "<mark>Stark</mark> Industries"
}
],
"text_match": 130816
}
],
"out_of": 4,
"page": 1,
"request_params": {
"collection_name": "companies",
"per_page": 10,
"q": "stark"
},
"search_time_ms": 0
}
```

As you see in the response, the document with `company_id` set to 124 is shown in the output. There is another document that has the `company_name` as "New Stark Industries", but it won't be shown in the result.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but it won't be shown in the result

... because it does not match the filter ... we embedded in the scoped API key.


You can find more details about scoped API keys [here](../../api/api-keys.html#generate-scoped-search-key).
5 changes: 5 additions & 0 deletions docs-site/content/0.19.0/guide/features/raft.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Raft Based Clustering

High availability is essential for production environments. Typesense uses the [Raft Consensus Algorithm](https://raft.github.io/) to create a highly available cluster with more than one Typesense servers. With Raft, you need to create a cluster of 3 nodes to tolerate single node failures. If you wish to handle 2-node failures, then you need a minimum of 5 nodes in the cluster. Note that adding more nodes will also increase write latencies.

More details on cluster operations can be found [here](../../api/cluster-operations.html).
Loading