Is it important for capture control to keep captured tab from knowing it is captured?

[youennf]

getDisplayMedia is trying to ensure that a captured surface does not know it is captured, at least for tab capture.
The question is whether this API should keep this contract or whether we do not care about this constraint.

[youennf]

FWIW, if a captured tab visibility is hidden and it receives user gesture, then it would know that it is being captured.
But that the same time, it is unclear whether a captured tab visibility should hidden or visible.
This makes me wonder whether mediacapture-screenshare should say a word about this.

[eladalon1983]

That's a great question. At the moment, Meet is using Captured Surface Control via origin trial in Chrome, and so far, I have not heard of either (1) sites that fail to work correctly, nor (2) sites that abuse the knowledge that they are captured. So I think we could keep this as a non-blocking issue open and allow ourselves to make a more informed decision based on data that will flow in after the API gains adoption. Wdyt?

[jan-ivar]

Yes, I think it's important to keep the captured tab unaware. This keeps users in charge.

[eladalon1983]

Without real-world experience, it's hard to weigh the benefits and disadvantages.

• On the one hand, we have theoretical, potential abuse by apps that discover that they are captured.
  • First, this is completely theoretical at this stage.
  • Second, this risk is completely mitigated if the user never scrolls or zooms the captured tab.
• On the other hand, we have the risk that apps would function incorrectly if they think they are foregrounded when they are not.
  • But then again, it's not clear if this too might be purely academic.

I see no great urgency to specify this behavior one way or the other. We can let user agents experiment here and report back their findings.

[youennf]

I had a quick try and AIUI, Chrome is making sure that a tab thinks it is visible (which is sort of true) when captured.
This would also be consistent with unsuspending/ not-throttling the captured tab.
And this also helps ensuring that the surface control API is not leaking that a tab is being captured.

This might be good input for the screen share main spec.

[eladalon1983]

We appear to have consensus about what is likely preferable. Can we also have consensus that the spec should not yet actively mandate this behavior at this point, given our relative ignorance on real-world conditions?

[jan-ivar]

This might be good input for the screen share main spec.

Thanks for testing! I like Chrome's behavior, and agree it seems worth capturing. Opened w3c/mediacapture-screen-share#307.

As to this issue, unless the proposal leaks today, I see no reason to keep it open.

If you're asking would I oppose a proposal that leaked this information? Probably, unless it's a really good one.

But right now I don't see anything here requiring consensus.

[eladalon1983]

@youennf, are you also in favor of closing this issue?

[eladalon1983]

Issue transferred; heads up to those discussion participants who might otherwise be looking for it elsewhere: @jan-ivar, @youennf

[jan-ivar]

In w3c/mediacapture-screen-share#307 (comment) I show how this API might leak tab capture.

I think we should add restrictions in this spec to ignore zoom and scroll inputs whenever the captured page's document.visibilityState is "hidden".

Note that this doesn't appear to be an issue in Chrome which keeps the captured page "visible" even while minimized (for tab capture).

[eladalon1983]

I think we should add restrictions in this spec to ignore zoom and scroll inputs whenever the captured page's document.visibilityState is "hidden".

This is redundant. We have resolved to mandate that the captured tab should be "visible" anyway.

For setZoomLevel(), and therefore for increaseZoomLevel() and friends, the first step is:

1. If this is not actively capturing, return a promise rejected with a DOMException object whose name attribute has the value InvalidStateError.

Similarly, the forward wheel event algorithm reads:

1. If controller is not actively capturing, abort these steps.

[jan-ivar]

This is redundant. We have resolved to mandate that the captured tab should be "visible" anyway.

Only if it's capturing frames at the moment. There might be implementations in the future that don't capture frames if the window is minimized for instance. There might also be implementations that try to implement this API to work with both tab and window capture of a browser window.

If your implementation never does this then it would easily satisfy this requirement, and this shouldn't be an imposition. nevertheless, I think it would be good to have in the spec to cover potential other implementations.

... actively capturing ...

Thanks for this link. Maybe we can make this change there to check the docment's visibility state, which it doesn't do currently.

[eladalon1983]

There might be implementations in the future that don't capture frames if the window is minimized for instance.

I don't think that'd make a difference. The way I see it:

1. All captured tabs are mandated to be "visible" by the Screen Capture spec, as per yesterday's interim meeting.
2. No action from CSC can take place on anything other than a captured tab.

The combination of (1) and (2) produces "no action from CSC can take place on anything other than a tab whose main document is "visible".

What am I missing?

There might also be implementations that try to implement this API to work with both tab and window capture of a browser window.

Captured windows don't have document.visibilityState. Only captured tabs have that.

If we extend this API to support window-capture in the future, and if we then choose to give special treatment to captured browser windows (as opposed to treating them as any native window), then we will have to introduce a few new concepts and connections, including the "window's active tab's main page's visibility state", at which point we can indeed add this requirement. But it does not seem helpful now. Or what am I missing?

[jan-ivar]

Let's pick this up once w3c/mediacapture-screen-share#307 has been closed with final language.

When capturing a browser window, the webpage that is visible in that window is also captured. To support #55 we would need to handle e.g. Chrome not generating frames when the window is minimized. In that case, "hidden" seems least revealing if frame delivery halts, and "visible" otherwise.

[eladalon1983]

To support #55 we would need to handle e.g. Chrome not generating frames when the window is minimized.

I believe Chrome does keep producing frames when capturing one of its own windows, even if that window is minimized.

But I don't think that matters either way.