Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

heads up re: proposal to add Float16Array to JavaScript #373

Closed
bakkot opened this issue Apr 4, 2023 · 7 comments
Closed

heads up re: proposal to add Float16Array to JavaScript #373

bakkot opened this issue Apr 4, 2023 · 7 comments
Labels
v2

Comments

@bakkot
Copy link

bakkot commented Apr 4, 2023

I'm on TC39, and I'm working on advancing a proposal for Float16Array in JavaScript, which would hold IEEE binary16 floats (also called float16, fp16, f16, half precision floats, etc). I note that this spec has support for float16 values, and there is an outstanding issue in the spec about using ArrayBuffers containing float16s, which is currently impractical due to the lack of Float16Array in JS (the issue links to a thread in which this comment says "It would be good to have the Float16Array support").

The proposal is currently at stage 2, which means TC39 has not yet committed to add it to JS. But since it seems directly relevant here, I wanted to make participants here aware that it may be coming in the hopefully not-to-distant future.
@dontcallmedom dontcallmedom mentioned this issue Apr 4, 2023
Closed
@fdwr
Copy link
Collaborator

fdwr commented Apr 4, 2023

since it seems directly relevant here, I wanted to make participants here aware

Thanks @bakkot - looking forward to Float16Array over passing raw bits via Uint16Array.

@wchao1115
Copy link
Collaborator

@bakkot Please also consider adding comparable typed arrays for float8 and bfloat16 at the same time. These two new data types have gained support from major GPU hardware vendors in recent years. More ML models will be taking advantage of these in the near future. If you are adding float16, these two types should probably be added also.

This was referenced May 4, 2023
Closed
Merged
@anssiko
Copy link
Member

anssiko commented May 9, 2023

I'll assign a v2 label to this issue to signal the WG is supportive of TC39's Float16Array proposal. Once the proposal has matured adequately we should look into adding support in WebNN API.

This half-precision float is particularly useful and demonstrably improves inference performance of e.g. latent text-to-image diffusion models (Stable Diffusion et al.) and super-resolution models (discussed in #127). This is worth prototyping already, results to be shared in this issue to keep interested folks informed.

@anssiko anssiko added the v2 label May 9, 2023
anssiko added a commit that referenced this issue May 9, 2023
@anssiko
Add float16 to MLOperandType and ArrayBufferView compatibility table
f4b66dd 
- Note TC39 Float16Array proposal is WIP
- Update issue links

Related #373

(Bikeshed syntax not supported inside tables, thus plain HTML.)
@anssiko anssiko mentioned this issue May 9, 2023
Merged
@bakkot
Copy link
Author

bakkot commented May 16, 2023

Update: the proposal to add Float16Array to JavaScript today reached stage 3, meaning the design is finished, the committee is in favor, and engines can start implementing and shipping it.

anssiko added a commit that referenced this issue May 17, 2023
@anssiko
Add float16 to MLOperandType and ArrayBufferView compatibility table
dd8942f 
- Note TC39 Float16Array proposal is WIP
- Update issue links

Related #373

(Bikeshed syntax not supported inside tables, thus plain HTML.)
@anssiko
Copy link
Member

anssiko commented May 17, 2023

@bakkot thanks for letting us know and congrats! We're in process of updating the WebNN API spec accordingly in #386.

@anssiko
Copy link
Member

anssiko commented Jun 6, 2023

@bakkot I'm happy to let you know Float16Array just landed into the WebNN API spec as an official reference. Much thanks for specifying this feature important to WebNN API! Please feel free to close this issue if you're satisfied with our resolution.

Also, please let us know if there are substantive developments for Float16Array or other new kinds of TypedArrays or ECMA features that may intersect with this WG's interests. We may get in touch with you in the near future for some proposals.

@bakkot
Copy link
Author

bakkot commented Jun 6, 2023

Good to hear! I'll go ahead and close this issue.

The main remaining thing needed for Float16Array to be viable is for engines to actually implement it. You can follow along at this issue. I don't have the experience to do so myself, so it'll be up to the authors of the various engines to prioritize that work.

If anyone on this project has experience working with the implementation of TypedArrays in mainstream JS engines and wants to contribute, the issue linked above has pointers to the relevant issues on the the bug trackers for the big three engines.

@bakkot bakkot closed this as completed Jun 6, 2023
zolkis pushed a commit to zolkis/webnn that referenced this issue Jun 8, 2023
@anssiko @zolkis
Add float16 to MLOperandType and ArrayBufferView compatibility table
0625578 
- Note TC39 Float16Array proposal is WIP
- Update issue links

Related webmachinelearning#373

(Bikeshed syntax not supported inside tables, thus plain HTML.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
v2
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@anssiko @bakkot @fdwr @wchao1115