New auth0 role VIAL WB Limited is not displaying in the Reporters page

[ugotsoul]

discord convo

user affected: https://vial.calltheshots.us/admin/core/reporter/370/change/

[ugotsoul]

Related to: #612

[simonw]

I have a hunch that we don't update the roles we store against reporters often, if ever. We update user roles when they sign into VIAL but reporters works differently.

[simonw]

Code in question is here - it looks like it should be updating the roles if they have changed but I'm not 100% sure it does:

vial/vaccinate/api/utils.py

Lines 154 to 240 in 9b6914e

	@beeline.traced("jwt_auth")
	def _jwt_auth(
	required_permissions: Set[str],
	request: HttpRequest,
	update_metadata: bool,
	) -> Optional[JsonResponse]:
	# Use Bearer token in Authorization header
	authorization = request.META.get("HTTP_AUTHORIZATION") or ""
	if not authorization.startswith("Bearer "):
	return JsonResponse(
	{"error": "Authorization header must start with 'Bearer'"},
	status=403,
	)
	# Check JWT token is valid
	jwt_access_token = authorization.split("Bearer ")[1]
	check_permissions = True
	try:
	jwt_payload = decode_and_verify_jwt(
	jwt_access_token, settings.HELP_JWT_AUDIENCE
	)
	except Exception as e:
	try:
	# We _also_ try to decode as the VIAL audience, since the
	# /api/requestCall/debug endpoint passes in _our_ JWT, not
	# help's. Our JWT is an id token, not an access token,
	# which means it won't have permissions in it (see below)
	jwt_payload = decode_and_verify_jwt(
	jwt_access_token, settings.VIAL_JWT_AUDIENCE
	)
	check_permissions = False
	except Exception:
	return JsonResponse(
	{"error": "Could not decode JWT", "details": str(e)}, status=403
	)
	# We have an _access_ token, not an _id_ token. This means that
	# it has authorization information, but no authentication
	# information -- just an id, and a statement that the user is
	# authenticated to hit this API. https://auth0.com/docs/tokens
	# describes this in more detail.
	jwt_auth0_role_names = ", ".join(
	sorted(jwt_payload.get("https://help.vaccinateca.com/roles", []))
	)
	name: Optional[str] = None
	email: Optional[str] = None
	external_id = "auth0:{}".format(jwt_payload["sub"])
	with transaction.atomic():
	# Get full metadata if the user doesn't exist; also take a lock on the user.
	reporter = (
	Reporter.objects.select_for_update().filter(external_id=external_id).first()
	)
	# We may want to update the email address and name; we do this
	# sparingly, since it's a round-trip to the Auth0 endpoint, which
	# is somewhat slow. Again, we must do this because we're getting
	# an access token, not an id token.
	if not reporter or update_metadata:
	with beeline.tracer(name="get user_info"):
	user_info_response = requests.get(
	"https://vaccinateca.us.auth0.com/userinfo",
	headers={"Authorization": "Bearer {}".format(jwt_access_token)},
	timeout=5,
	)
	beeline.add_context({"status": user_info_response.status_code})
	# If this fails, we don't fail the request; they still
	# had a valid access token, auth0 is just being slow
	# telling us their bio.
	if user_info_response.status_code == 200:
	user_info = user_info_response.json()
	name = user_info["name"]
	if user_info["email_verified"]:
	email = user_info["email"]
	jwt_auth0_role_names = ", ".join(
	sorted(user_info["https://help.vaccinateca.com/roles"])
	)
	external_id = "auth0:{}".format(jwt_payload["sub"])
	defaults = {"auth0_role_names": jwt_auth0_role_names}
	if name is not None:
	defaults["name"] = name
	if email is not None:
	defaults["email"] = email
	reporter = Reporter.objects.update_or_create(
	external_id=external_id,
	defaults=defaults,
	)[0]

[ugotsoul]

Can I get access to auth0 to test this on staging? 👀

[simonw]

#access-requests or Jesse should be able to help with that.

[simonw]

I think the ideal fix here - for both users and reporters - would be if auth0 could somehow inform VIAL any time a user is added or removed from a group using the auth0 interface.

I was hoping auth0 would have an obvious mechanism for doing this - maybe a webhooks API that can talk to VIAL, or some kind of customization hook that lets us react in real-time to auth0 configuration changes.

Auth0 offer rules and hooks and most recently actions - I would hope that one of these could do this for us, but I've not managed to figure out if it's possible with any of them yet: https://auth0.com/blog/introducing-auth0-actions/

[ugotsoul]

I didn't find an action that mapped to a supported event for rules, hooks or actions -- most of these fall in the authentication pipeline between when a user logs in to when a auth token is issued.

I did find events that map to a role being added / removed:

Event for assigning a role: https://manage.auth0.com/dashboard/us/vaccinateca/logs/90020210605001053900430196618579152991451933976477827074

Event for removing a role: https://manage.auth0.com/dashboard/us/vaccinateca/logs/90020210605001207488430196686485725205025274365420240898

There is a log streaming service available that we can connect to a custom webhook that VIAL can listen to for these type of events, and do its business logic etc. https://auth0.com/docs/monitor-auth0/streams

[ugotsoul]

Also, there only (thankfully!) 3 users who fall under the WB Trainee role right now: https://manage.auth0.com/dashboard/us/vaccinateca/roles/rol_wxjXHDoqafCt74FM/users

[ugotsoul]

In the log stream we can filter out API operation events so we are not inundated with logs: b
sapi Success API Operation
fapi - Operation on API failed

both the role events fall under these codes.

[ugotsoul]

@simonw We are only allowed one log stream under our current auth0 plan, can I delete this paused webhook: https://manage.auth0.com/dashboard/us/vaccinateca/log-streams/lst_0000000000000810/settings

[simonw]

@ugotsoul yes let's delete or re-use that one.

[simonw]

Awesome work figuring this out! It looks like this is the exact mechanism we need.

[ugotsoul]

Great! The one thing I missed is the log filters are not available in streams yet - the feature is in beta but not appearing in any stream types. :/

[ugotsoul]

Hmm it looks like it can be enabled in the dashboard but I'm not seeing it? https://auth0.com/changelog#3ks0rPxzumRAmCfIjcFQYW

[ugotsoul]

Hmm it looks like it can be enabled in the dashboard but I'm not seeing it? https://auth0.com/changelog#3ks0rPxzumRAmCfIjcFQYW

I opened a support ticket for this: https://support.auth0.com/tickets/00490599

[ugotsoul]

Hmm it looks like it can be enabled in the dashboard but I'm not seeing it? https://auth0.com/changelog#3ks0rPxzumRAmCfIjcFQYW

I opened a support ticket for this: https://support.auth0.com/tickets/00490599

Yay, this is resolved - auth0 had a safari bug that did not display a popup to opt-in to this beta feature. I used chrome and it worked fine.

[ugotsoul]

I've been testing an auth0 Webhook for the log stream locally with ngrok and mocking POST requests with postman, and I found out that the event only returns the role_id which is alphanumeric string auth0 uses to identify roles, and not the name we associate with roles. There is a way to get a role by id in the management api with this endpoint: https://auth0.com/docs/api/management/v2#!/Roles/get_roles_by_id

[ugotsoul]

Here's the process of using the auth0 sdk to access the Management API: https://github.com/auth0/auth0-python#management-sdk-usage

[ugotsoul]

Alternatively, we can use http/s to make calls to the management API: https://auth0.com/docs/tokens/management-api-access-tokens/get-management-api-access-tokens-for-production

Note the Management API is separate entity from the OAuth authentication api, but does use JWTs: https://auth0.com/docs/api/management/v2

My thought process for how this would work:

• Enable machine-to-machine applications for VIAL (local/development first) here, we seem to have one setup but not enabled: https://manage.auth0.com/dashboard/us/vaccinateca/apis/601b586afe2865003fd38872/authorized-clients

• Add logic to get auth token for Management v2 api (see above link)

• Get the name of the role from this endpoint: https://auth0.com/docs/api/management/v2#!/Roles/get_roles_by_id

• Update user groups based on the event in the webhook (adding or removing a role) OR logout the user?

  • The latter could make updating roles on the Reporter easier, as the field which it stores auth0 roles is string, which we overwrite the list of roles we get from the user record after logging in.

  @simonw let me know your thoughts 🙏🏾

[ugotsoul]

Alternatively, we can use http/s to make calls to the management API: https://auth0.com/docs/tokens/management-api-access-tokens/get-management-api-access-tokens-for-production

Note the Management API is separate entity from the OAuth authentication api, but does use JWTs: https://auth0.com/docs/api/management/v2

My thought process for how this would work:

• Enable machine-to-machine applications for VIAL (local/development first) here, we seem to have one setup but not enabled: https://manage.auth0.com/dashboard/us/vaccinateca/apis/601b586afe2865003fd38872/authorized-clients

• Add logic to get auth token for Management v2 api (see above link)

• Get the name of the role from this endpoint: https://auth0.com/docs/api/management/v2#!/Roles/get_roles_by_id

• Update user groups based on the event in the webhook (adding or removing a role) OR logout the user?

  • The latter could make updating roles on the Reporter easier, as the field which it stores auth0 roles is string, which we overwrite the list of roles we get from the user record after logging in.

  @simonw let me know your thoughts 🙏🏾

Ohh another way we could do this since we know the users roles changed is to simply force the user to logout. Logging back in will update the user's groups.