perf(permissions)!: reduce queries in bulk permission assignment api TASK-828

[RuthShryock]

Checklist

1. If you've added code that should be tested, add tests
2. If you've changed APIs, update (or create!) the documentation
3. Ensure the tests pass
4. Make sure that your code lints and that you've followed our coding style
5. Write a title and, if necessary, a description of your work suitable for publishing in our release notes
6. Mention any related issues in this repository (as #ISSUE) and in other repositories (as kobotoolbox/other#ISSUE)
7. Open an issue in the docs if there are UI/UX changes

Description

Refactored assign_perm() and remove_perm() to support bulk assignment and removal of permissions instead of processing them individually

Notes

Some background

This was the original task: #3869 added more queries to the endpoint. We should review and either reduce the number of queries or prove why this is necessary. Of course we should also inspect average load times - the goal is to make it faster, not just reduce queries.
After some clarification, the issue is that we call assign_perm() and remove_perm() for each permission assignment, one by one. This is costly and we want to reduce the number of queries and speed-up the endpoint so we should rewrite assign_perm() and remove_perm() to support bulk assignment.

Testing

Test cases have been added but if you would like to test manually:

1. Pick a test in asset_permission_assignment.py such as test_partial_permission_grants_implied_view_asset
2. Add self.assertNumQueries
3. Compare before and after the PR

[notion-workspace]

Reduce number of queries in bulk permission assignment api

[rgraber]

Mostly style comments and a thought about testing

[noliveleger]

if we are renaming assign_perm to assign_perms, we may need to rename perm param to perms.
Let's use string annotations too. i.e.:

assign_perms(self, user_obj: 'User', perms: Union[str, list[str]], deny: bool = False, etc...)

[noliveleger]

We do not create new permission yet. It is just added to the list.

[noliveleger]

I think we should use ignore_conflicts=True in case race conditions and two concurrent requests try to add same permissions at the same time.

[noliveleger]

Why is this condition needed?

[RuthShryock]

This was needed because a lot of code calling assign_perm() was expecting a single string in return. So I wanted to make sure that if we just assigned one permission, it would return one string as expected. But now, since we are keeping the original assign_perm() function and that will still be used when assigning a single permission, it is no longer needed.

[noliveleger]

This may call clean_up_table() (within _update_partial_permissions ) several times in a row.
It would be great to avoid that if the list of permissions contains conflicting permissions.

[noliveleger]

post_assign_asset_perm expects the (the receiver method in kpi/signals.py) codename to be a string.
You need to adapt the code to be sure it works. I think a test is missing there. We need to be sure that the signal is called and the enketo server URL is updated when anonymous is granted add_submissions.

[noliveleger]

Do we have a test that checks expected permission assignments are there when we use assign_perms() with many multiple permissions at a time.

I would like to be sure that assigning permissions with collections for example we still work the way we expect.

By the way, I would keepassign_perm() (singular) and it would call your new assign_perms behind the scene. We would avoid changing assign_perm everywhere.
We can use typing annotation to know which type each method is expecting.

[jdimeo]

We're very excited about this PR because what we want to do (use APIs to create shared questions in the library and then automatically grant users access based on their country) is blocked by the inability of Kobo to grant the permissions before our API gateway kills the request.

[noliveleger]

Unfortunately, there is something wrong with the PR. @RuthShryock
When assigning a permission to user on a collection, it does not propagate to children of the collection.

But if I use the shell, it does work. My first theory could be some methods cached with @cache_request which does not return an accurate state of the DB. I did not test it, but that the first explanation I have why it does work in shell but not through the API.

[noliveleger]

We're very excited about this PR because what we want to do (use APIs to create shared questions in the library and then automatically grant users access based on their country) is blocked by the inability of Kobo to grant the permissions before our API gateway kills the request.

Hello @jdimeo sorry to hear that, if you are admin of your server, you may achieve what you want with ProjectView instead.

[rgraudin]

Hello @jdimeo sorry to hear that, if you are admin of your server, you may achieve what you want with ProjectView instead.

Hi @noliveleger, I'm in John's org. We are definitely willing to use a workaround if there is one. I have tested creating a ProjectView but could not quite figure out how to make use of it. First of all, I can't figure out how to add projects to a project view in the console or using the API by declaring the project view as the parent. It would be helpful if we could share content this way. Of course, I'm not sure Project Views would help us get all the way there because we have content divided in Collections that are only allowed in the Library section, not the Projects section. It would be helpful if a project view could be used like a referenceable project 'group' so we can share content with the members of the ProjectView but that may not be implemented.

Appreciate any help or even somewhere to look documenting Project Views or other potentially helpful functionalities.