Extend roadmap: permissions automation

[outofambit]

My team is working on automated bluetooth testing in WPT, which has no way to automate managing device permissions. Instead of implementing something chrome specific, we’d like to explore device access/permission automation in BiDi. We’ll follow up with a design approach in an issue but wanted to start alignment conversations now.

• BiDi extensions: Meta: WebDriver BiDi extensions #506
• Manage Bluetooth device permissions: Permissions API Extension WebBluetoothCG/web-bluetooth#613

[outofambit]

cc @lolaodelola @mzgoddard @boazsender

[christian-bromann]

Hey @outofambit I am not quite sure but would fall this under this specification: https://w3c.github.io/permissions ?

[OrKoN]

I think permissions were discussed in #403 and looks like the agreement want that we want to do something in WebDriver BiDi. I think we should discuss it again.

[mathiasbynens]

This could become the first ever WebDriver BiDi (rather than WebDriver Classic) extension, as in, another spec defines its own commands/events. Exciting! cc @thiagowfx

[jugglinmike]

Folks looking to take this on might be interested in the 2017 collaboration which informed the design of the WebDriver Classic Extension Command. While some details may be worth replicating, I imagine that thread will be just as useful in determining what to discard (due to the bi-directional paradigm shift, BiDi's richer abstractions for browsing contexts, and six years of refinements to the Permissions specification).

The editorial history of that section may also be instructive--it was removed in 2021 for lack of implementation interest and reinstated a few months later.

This could become the first ever WebDriver BiDi (rather than WebDriver Classic) extension, as in, another spec defines its own commands/events. Exciting! cc @thiagowfx

If my experience with WebDriver is any indication, we can also expect it to surface latent issues in the specification's extension mechanism.

[css-meeting-bot]

The Browser Testing and Tools Working Group just discussed Adding Permissions Automation to the Roadmap.

The full IRC log of that discussion

<jgraham> topic: Adding Permissions Automation to the Roadmap
<jgraham> github: https://github.com//pull/523
<jgraham> RRSAgent: make minutes
<RRSAgent> I have made the request to generate https://www.w3.org/2023/09/14-webdriver-minutes.html jgraham
<orkon> mattreynolds (IRC): we have a webbluetooth api that has a chooser api for its permission UI
<orkon> mattreynolds (IRC): we developed a webdriver api for testing but we need to communicate back the list of devices that comes from the chooser
<orkon> mattreynolds (IRC): we are unable to bring the API into the webdriver classic so we are looking into an extension for the webbluetooth
<orkon> * extension for bidi for the webbluetooth
<orkon> mattreynolds (IRC): we are developing a proposal for the webdriver bidi extension. We eventually want to use for WPT
<jgraham> q?
<orkon> mattreynolds (IRC): is there anything more to know about webdriver bidi for us?
<shs96c> q+
<orkon> mattreynolds (IRC): we have a pull request that adds permissions to the roadmap
<jgraham> ack shs96c
<orkon> q+
<jgraham> q+
<orkon> shs96c: an extension sounds good
<sadym> q+
<orkon> mattreynolds (IRC): what is the best way to reach out?
<jfernandez> q+
<jgraham> ack next
<orkon> shs96c: this wg, e.g., via matrix
<jgraham> orkon: I'm familiar with device access API. This is probably the first extension for BiDi. You should be able to reach out to me to bring questions to the wg
<jgraham> ack next
<orkon> jgraham: I agree that it sounds like an extension. If I understand the model, it sounds right to have the event based model where the backend sends the choices and the client makes a choice based on the event.
<orkon> jgraham: this is exactly the kind of flow I would imagine in BiDi.
<orkon> mattreynolds (IRC): another kind of API is to bind/unbind to enable interception
<orkon> jgraham: you need to subscribe to existing events
<jgraham> ack next
<mzgoddard> q+
<orkon> sadym (IRC): the question is not only about the device choice automation but also about other extensions. Should regular permissions be part of the bidi API?
<orkon> jgraham: webdriver has a spec for permissions
<orkon> jgraham: there is certainly a precedent for permissions being an extension
<shs96c> q+
<orkon> jgraham: but it might be that general permissions could in the core spec. It depends?
<orkon> jgraham: e.g., device choice has different workflow compared to the yes/no permissions
<orkon> jgraham: we want to reformulate classic on top of bidi, so we would need an equivalent in bidi
<jgraham> ack jfernandez
<orkon> jfernandez: maybe I have not understood the use case completely. But it is very similar to the use case I tried to define for the protocol handlers API. SPC dialog also requires permissions. So in order to implement WPT they defined an extension command for webdriver which puts the feature into a specific mode that allows the feature to ignore the prompt.
<orkon> jfernandez: it seems that there are several features that require bypassing the workflow. Another option for you would be another extension command for webdriver to bypass prompts. If that is the case, do we want a lot of extension commands for the same purpose?
<orkon> mattreynolds (IRC): I would that agree that for apis that do not have chooser dialog it is a good way
<orkon> mattreynolds (IRC): and the bluetooth api dialog has a dynamic list and we need to tell which item to select
<jgraham> ack next
<orkon> mzgoddard (IRC): we talk about it as permissions but that is a chooser prompt that can get a list of many devices and they don't show up all at once. And bluetooth needs to wait for the wireless communication. The list does not always show up in the same order. And we cannot always say choose the first device and the order is not deterministic. And the second thing I would like to express: there are three other APIS, USB, Serial, ? that also has
<orkon> the list of devices that updates based on the filter.
<jgraham> s/?/WebHID/
<orkon> mattreynolds (IRC): technically there is also a find access dialog
<jgraham> s/find access/font access/
<jgraham> ack next
<orkon> shs96c: while those things should live in their own specifications. One thing we can have a non-normative section on how to perform common patterns so that there is a consistency between related specs
<jgraham> q?

[outofambit]

hi @OrKoN! I made a small update to the phrasing in the roadmap item to emphasize the "chooser" aspect of the bluetooth permission automation since that is somewhat unique compared to other permissions APIs. Based on the conversation today, do you think we can merge this PR in?

[thiagowfx]

LGTM but please wait for review from another project member as well

[OrKoN]

My understanding based on the discussion was that we want the specification for the Bluetooth specific part to be part of the Bluetooth spec and perhaps only define a general pattern for such interactions in the WebDriver BiDi spec. Perhaps we should add a more general item to the roadmap about supporting/clarifying how extensions could be defined and documenting the more general device access pattern (similar to the DeviceAccess in CDP)? For example, the existing permissions extension in WebDriver https://w3c.github.io/permissions/#automation

Also, should we file an issue in https://github.com/WebBluetoothCG/web-bluetooth/issues/ for specifying the Bluetooth specific extension and an issue in this repo for providing guidelines for extensions? We could link those issues in the roadmap. WDYT?

P.S. One of the issues about extensions we can probably link to is #506

[outofambit]

Hey @OrKoN, thanks for clarifying that. I dug in some more and what you're saying makes sense! I appreciate the extra links and resources. I took a quick pass at revising this PR, and will revise it again once I've gotten an issue filed in https://github.com/WebBluetoothCG/web-bluetooth/ next week. I'll ping you again once that's done to get another review. Thanks!

[thiagowfx]

I'm going to do the implementation in the chromium mapper side of whatever comes out of this spec, so please keep me in the loop as well, thank you Evelyn :-)

[OrKoN]

this line should describe the scenario, for example, something along the lines of "as a spec author I want to extend WebDriver BiDi functionality, therefore I define a WebDriver BiDi extension in the spec following WebDriver BiDi extension guidelines". The current text should become an action item in the list below.

@jgraham @whimboo do you think it's a good item for the roadmap? with the proposed changes it looks good to me. We can probably exclude the device access pattern as I am not sure it is a cross-browser pattern.

[outofambit]

@OrKoN Made this change, thank you for clarifying the right framing here. I also modified the bullet point to link to the issue in https://github.com/WebBluetoothCG/web-bluetooth/, does that seem right to you?

[OrKoN]

This looks fine to me, please wait for the feedback from other vendors.

[thiagowfx]

cc @jgraham @whimboo PTAL

[thiagowfx]

Friendly ping^ (#523 (review))

[whimboo]

Are you planning to have a long list of all the different permissions APIs listed here in the future, or should we have just a generic item for specifically the Permissions. I would find that more helpful given that there will be much more other extensions in the future like WebAuthn (which we potentially should support as well as for classic).

[OrKoN]

I think we will not extend the roadmap with permissions and web bluetooth since those are defined in external specifications and automation algorithms can be defined there. This already happened for Permissions and WebBluetooth:

• https://www.w3.org/TR/permissions/#automation-webdriver-bidi
• https://webbluetoothcg.github.io/web-bluetooth/#automated-testing