Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delta review (to CR) of Web Neural Network API #771

Closed
1 task done
anssiko opened this issue Sep 9, 2022 · 10 comments
Closed
1 task done

Delta review (to CR) of Web Neural Network API #771

anssiko opened this issue Sep 9, 2022 · 10 comments
Assignees

Comments

@anssiko
Copy link

anssiko commented Sep 9, 2022

Hi TAG!

I'm requesting a delta TAG review of the Web Neural Network API.

More details about this review request

The Web Neural Network API (or WebNN API in short) is a specification for constructing and executing computational graphs of neural networks. It provides web applications with the ability to create, compile, and run machine learning networks on the web browsers. The WebNN API may be implemented in web browsers using the available native operating system machine learning APIs for the best performance and reliability of results.

Further details:

  • I have reviewed the TAG's API Design Principles
  • Relevant time constraints or deadlines: CR publication slated Q4 2022
  • The group where the work on this specification is currently being done: Web Machine Learning Working Group
  • The group where standardization of this work is intended to be done:
  • Major unresolved issues with or opposition to this specification: N/A
  • This work is being funded by: N/A

For the full review template, please unfold the above section ⤴️

The initial TAG review completed Oct 2021. This delta request focuses your attention on the following architectural changes and issues since the previous review:

  • Naming of the sync and async methods: createContext, build and compute. The WG has considered two API naming conventions (x() + xSync() or xAsync() + x()) but was unable to reach consensus and resolved to seek TAG recommendation. See Support asynchronous context creation webmachinelearning/webnn#272

  • Related to the naming issue, the WG decided to restrict the sync API to worker context only. This API complements the async API. The key use case for the sync API is to support Wasm code generators. The async API is the recommended path for mainstream use cases. We would like to hear the TAG perspective on this API split. We are aware that the worker-only sync API design is a rare exception on the web platform.

  • The WG resolved to drop support for WebGL and focused on WebGPU interoperability.

The CR publication is slated for Q4 2022 so your feedback is the most impactful if it arrives by the end Oct 2022 latest.

We'd prefer the TAG provide feedback as:

💬 leave review feedback as a comment in this issue and @-notify @anssiko

For context, these are the related issues in the WebNN GH repo:

@anssiko
Copy link
Author

anssiko commented Sep 26, 2022

@cynthia, the first bullet "naming of the sync and async methods" is the most time critical for the WG. This is to avoid breaking changes to the mainstream parts of the API as we approach Dev Trial.

To ensure you use your review time effectively, I want to clarify we are not asking for a full (re-)review of this spec, but a recommendation on this one question. The latter two bullets are mainly for your information and to provide supporting information and context. Thank you!

@torgo
Copy link
Member

torgo commented Oct 18, 2022

Hi @anssiko just to let you know we're working on getting you feedback here this week.

@anssiko
Copy link
Author

anssiko commented Oct 27, 2022

@torgo thank you for keeping us updated. The WG is slated to discuss this on 2022-11-03.

@littledan
Copy link

I'm wondering, what's the motivation for the sync API? I think async APIs should be friendly to WebAssembly in the near future, given that WebAssembly JavaScript Promise Integration reached Phase 3 in the Wasm CG.

@anssiko
Copy link
Author

anssiko commented Oct 27, 2022

@littledan this was also brought up by @domenic in webmachinelearning/webnn#272 (comment), more context there.

Do we have an estimate when this integration might ship in multiple engines?

@littledan
Copy link

littledan commented Oct 27, 2022

Sorry, I don't have any information about timelines here; maybe @fgmccabe does

@fgmccabe
Copy link

Probably the next engine to implement JSPI will be Firefox. However, that will likely not be before sometime next year; they have a lot of other things to work on.

@cynthia
Copy link
Member

cynthia commented Nov 30, 2022

Apologies this took so long, especially given the small change. Given the implementation support ot do this nicely is currently not ready, we discussed this at length and came to a conclusion (albeit somewhat distant from unanimous) that the following tradeoff would probably be acceptable (while not ideal).

  • x() + xSync() as conventions - we will add a design principle on this.
  • Assuming xSync() would block the main thread for longer than we would likely be comfortable with, keeping it limited to worker should at least prevent inappropriate use degraging user experience.

That said, we would like to see a migration path forward once WASM Promise integration lands on multiple implementations. Thank you for your patience, and we're excited to see this work move forward.

@cynthia cynthia closed this as completed Nov 30, 2022
@anssiko
Copy link
Author

anssiko commented Nov 30, 2022

Thank you @cynthia & TAG on behalf of the WG. Both the spec and the WIP implementation now follow this convention.

@anssiko
Copy link
Author

anssiko commented Jan 26, 2024

Hi again TAG!

NB: I'm piggypacking on this issue to retain context, but please let me know if I should file a new issue instead. On behalf of the WG I hope the TAG is happy to see these changes and look forward to your comments.

We're looking to publish a new CR Snapshot of the Web Neural Network API in Q1'24 and wanted to give you a heads up with the following high-level summary of changes for your information and review:

Since the initial Candidate Recommendation Snapshot the Working Group has gathered further implementation experience and added new operations and data types needed for well-known transformers to support generative AI use cases. In addition, the group has removed select features informed by this implementation experience: higher-level operations that can be expressed in terms of lower-level primitives in a performant manner, and support for synchronous execution. The group has also updated the specification to use modern authoring conventions to improve interoperability and precision of normative definitions and is developing a new feature, a backend-agnostic storage type, to improve performance and interoperability between the WebNN, WebGPU APIs and purpose-built hardware for ML.

You are probably happy to see we're removing support for synchronous execution per your guidance (removal discussed in webmachinelearning/webnn#531, we expect to land this change ahead this publication) and moving toward JSPI that is coming finally.

Edit: fix link to webmachinelearning/webnn#531

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants