Reword Web VM requirement in phase doc?

[binji]

We discussed this in the December 11 meeting.

Should we change the entry requirement for phase 4 to allow for two non-web implementations? @rossberg suggested replacing "Web VMs" with "production VMs"

[binji]

This topic came up again in the June CG meeting.

We didn't come to a conclusion about how/whether to change the VM requirement. I've summarized some of the arguments made for and against this change below:

• Leaving it vague ("production VM") allows the group to make the decision on a proposal-by-proposal basis
• There is not a similar requirement for toolchains (text just says "toolchain" without specifying it)
• Having it vague means that the definition of which VMs are allowed could change over time for the same proposal
• Requiring two Web VMs helps enforce a web-wide consensus
• Weakening requirement may mean web wasm diverges from non-web wasm
• This requirement is meant to enforce implementability of the proposal, not whether it can be deployed on the web
• Wasm will have optional features; web wasm won't have all of them
• Browsers can still stop a proposal from advancing in phase 5 (standardization by WG)

We should continue this discussion in another meeting, but for now we can continue discussion here.

[titzer]

I would propose the following:

Change the requirement to "must be implemented in 2 relevant production engines." The idea being that there may be proposals that are relevant to some engines and not others.

I would also propose the following for discussion:

Add "without objection from other production engines". The idea being, a kind of veto power that provides for the case where a production engine may be unable to ever implement a feature in a performant way with reasonable complexity. Although, to be fair, objections should probably have come up sooner in the process.

Thoughts?

[fgmccabe]

Who are you going to give veto power to?

On Sat, Jun 22, 2019 at 1:38 PM Ben L. Titzer ***@***.***> wrote: I would propose the following: Change the requirement to "must be implemented in 2 *relevant* production engines." The idea being that there may be proposals that are relevant to some engines and not others. I would also propose the following for discussion: Add "without objection from other production engines". The idea being, a kind of veto power that provides for the case where a production engine may be unable to ever implement a feature in a performant way without reasonable complexity. Although, to be fair, objections should probably have come up sooner in the process. Thoughts? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#350?email_source=notifications&email_token=AAQAXUHXH2VNRPSTF2G7VCDP32EV3A5CNFSM4GKJ65OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYKQ7EI#issuecomment-504696721>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAQAXUAV6IBYDZ4RJ6KY5V3P32EV3ANCNFSM4GKJ65OA> .

-- Francis McCabe SWE

[Horcrux7]

What are the relevant production engines today? V8, SM and JSC? Is Chakra still relevant?
Which market share makes a VM relevant? This term is just as vague.

Because new engines will not fall from the sky, one could list the intended engines also naming.

[fgmccabe]

Actually the market share concept may be the right criterion: “enough engines to cover 25% of the installed base of engines” Or something like that ... Sun, Jun 23, 2019 at 1:24 AM Volker Berlin <notifications@github.com> wrote:

What are the *relevant* production engines today? V8, SM and JSC? Is Chakra still relevant? Which market share makes a VM relevant? This term is just as vague. Because new engines will not fall from the sky, one could list the intended engines also naming. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#350?email_source=notifications&email_token=AAQAXUBEPJF2QRTMWT36MNTP34XM3A5CNFSM4GKJ65OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYKZJ3Q#issuecomment-504730862>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAQAXUFHULYMDTY7MHIBJW3P34XM3ANCNFSM4GKJ65OA> .

-- Francis McCabe SWE

[rossberg]

Actually the market share concept may be the right criterion: “enough
engines to cover 25% of the installed base of engines”

And how would we measure that, esp across different ecosystems?

Seriously, this discussion still misses the point. The 2-engines criterion in the process is a technical criterion to ensure feasibility of a feature in realistic implementations, not to ensure buy-in from vendors.There is no automatism to accept a proposal just because it provides all technical deliverables; the CG and WG still have to agree to advance it. That's what we have meetings, discussions, polls and voting for. If a vendor has concerns, it is supposed to raise them as part of the discussion, not by gaming technicalities. We should assume good intentions on all sides for resolving any disagreements; random technical quorums would be terrible measures for that.

And because it is a technical criterion, it also makes sense to keep it somewhat flexible, since the exact details of which implementations sufficiently demonstrate a proposal's feasibility might depend on its contents. In fact, some proposals will not be implemented in engines but in other tools (e.g., anything related to the text format or certain custom sections). It's up to the CG to decide on a case-by-case basis when exactly this criterion is considered satisfied -- as with other criteria (e.g., does the test suite for the new feature have enough coverage?). The wording cannot be more than a guideline, but should not overconstrain either.

[titzer]

The 2-engines criterion in the process is a technical criterion to ensure feasibility of a feature in realistic implementations, not to ensure buy-in from vendors.

I agree, and my suggestion of the additional word relevant was designed to introduce some nuance into the wording to help us decide which engines count for a given feature.

Now that I think it about it more, the second proposal about a kind of "veto" power should rather be considered a technical criteria, though I probably phrased it poorly. It shouldn't be considered "veto power". Indeed, if it is possible to demonstrate technical feasibility, then it should also be possible to demonstrate technical infeasibility--i.e. a negative signal.

[binji]

We discussed this at the Feb 2020 CG meeting. There seemed to be agreement that we should decide, somewhere relatively early in the proposal process, which Wasm VMs are relevant, and should be required to prove feasibility for phase 4.

I think making it an entry to phase 2 seems like a good place. The current requirements for that phase are:

• Full proposed English spec text available in a forked repo around which a reasonably high level of consensus exists.
• Updates to the formal notation, test suite, and reference interpreter are NOT yet required.

We could add something like (with proper-wordsmithing):

• The kinds of WebAssembly VMs required for phase 4 advancement has been determined (e.g. web, non-web, etc.).

Then modify phase 4 wording:

• Two or more Web VMs implement the feature.

• Two relevant WebAssembly VMs (as determined in phase 2) implement the feature.

Thoughts?

[ajklein]

This direction seems reasonable to me. One thing I wanted to put on the record, though I don't think it really needs to be in the document (since the CG should be able to make all kinds of changes to proposals before phase 4), is that the CG shouldn't feel bound by whatever was chosen at phase 2 if things change during the development of the proposal (it can be a long time between phase 2 and phase 4).

I want to avoid a process-based objection that says "well the CG said we only needed engines X and Y, so the fact that it's not in Z is immaterial" even if there are good technical arguments for why Z would in fact be useful.

[tschneidereit]

I agree that this is a good direction. I also agree with @ajklein on making sure we don't treat the identified set of relevant VMs as immutable after stage 2 has been entered.

One corner case: what if there are three different kinds of VMs that are relevant? In that case, it seems we should have at least one of each. This does pose the question of what the ontology of VM types should be, of course :) As a first approximation, there might be web, IoT, cloud, general-purpose embeddable. On some level, this seems a bit much, but I could see how all of these are meaningfully different when it comes to implementability of at least some proposals.

[RossTate]

To @tschneidereit's point, one thing that surprised me at the meeting was how often there is an implicitly assumed implementation strategy. For example, there's an expectation that all globals and even memory tables are accessed through an instance pointer, but that implementation strategy wouldn't make sense for engines expecting every module to have only one instance or to have just one module instance entirely (like embedded devices?), in which case you wouldn't need to have an instance pointer and would just directly access globals/tables in memory. So if y'all want WebAssembly to really be usable outside of browsers, I think y'all should find ways to encourage other types of engines to implement these designs as early as possible so that we might uncover where these assumptions are problematic still in time to fix them.

[lukewagner]

Having had a bit of time to think about this since the CG meeting, it feels unfortunate to, at least at this early stage in wasm's development, allow less than 2 Web VM implementations for any feature that is intended to be implemented in a browser. The reason being that currently a valuable property of wasm is its portable implementation on the Web and the 2-Web-VM requirement helps ensure that going forward.

Thus, instead of generalizing the "2 Web VM" requirement, I'd be inclined to instead strengthen the requirement, saying that 2 Web VMs are required plus potential others, as appropriate for the feature. An exception would be features that were not intended to be implemented natively in browsers (like WASI).

[rossberg]

@lukewagner, see my earlier comment, that would be abusing a technical criterion for non-technical purposes.

[lukewagner]

Technical feasibility is only one subgoal to be achieved by this phase process; the more general goal is a successful standard and that requires more than just technical feasibility.

[rossberg]

I don't think formal process rules and arbitrary quorums are the right tool to force adoption. Web engines will continue to wield plenty of practical power over the development of the standard; it would be a good signal for the wider ecosystem to slowly reduce at least some of their formal privileges.