Canvas 3D: GL power, web-style
56 Comments Published by vladimir November 26th, 2007 in Canvas 3D, MozillaUpdated Add-On installation site, now hosted on addons.mozilla.org. Compatible with post-3.0b5 Firefox nightlies.
About a year and a half ago, I demoed an early prototype of an OpenGL 3D context for the canvas element at XTech. There was a lot of interest, but due to time constraints I wasn’t able to spend as much time working on it as I would have liked. In addition, Gecko 1.9/Firefox 3 wasn’t in a state where the extension could run against an unpatched build. Over the past few months I’ve been able to do some much-needed code cleanup and put together a preview of what an OpenGL context for canvas could look like.
There are two contexts provided by the extension. “moz-gles11″ follows the OpenGL ES 1.1 spec very closely, providing an almost identical API and feature set, and “moz-glweb20″ follows OpenGL ES 2.0 closely. Both are implemented directly on top of desktop OpenGL, so you must have support for at least OpenGL 1.5 on the desktop for the “moz-gles11″ context and OpenGL 2.0 for the “moz-glweb20″ context.
OpenGL was chosen as the base API for a number of reasons. It is a standardized, proven cross-platform API. 3D is a complex subject, and providing a fully-featured API will allow developers to create libraries and toolkits to abstract away the complexity in ways that are specific to their application, without being tied into a specific approach. Using a standard API also allows us to leverage the vast developer base out there who have OpenGL knowledge. OpenGL ES is also the standard for 3D on mobile devices, ensuring that any 3D content will be able to migrate to mobile devices as the web becomes more and more featureful on such devices.
The difficulty with using OpenGL is that the API is not very approachable for web developers; it’s a huge jump going from the 2D canvas API to a full 3D API. My hope is that those developers who are familiar with OpenGL will be able to create helper libraries to make 3D content and manipulation more accessible, much like there are tools such as photo montage scripts that authors can use without having to understand the implementation details. Scene graphs and other frameworks would also provide great fodder for the performance improvement work going on as part of ES4.
I hope to propose the new API to an appropriate standards group after people have had an opportunity to play with the features and provide feedback on the initial implementation. As the core of the API is already a standard, a spec would only need to be written for the specific GL-Web integration points.
Installation
You must be running Firefox 3 Beta 1 (or a later nightly, though compatibility with the nightlies is not guaranteed). Install the extension here (Windows, MacOS X x86, and Linux x86 only). Then, to allow web content to access the 3D canvas contexts, go into the Add-ons Manager and select Options under the Canvas 3D Extension, and make sure that “Enable 3D canvas in web content” is checked. Because the extension exposes a new set of native system APIs, only enable it on sites that you trust as there could be security concerns that have not yet been addressed. In the future, enabling 3D canvas contexts will be a site-specific preference (like page zoom is now), so that you’ll be able to set it on a per-site basis instead of globally. The 3D contexts are always available to extensions; combined with the drawWindow() API, extensions could use web pages as textures for interesting UI experiments.
You must also have a full implementation of desktop OpenGL 1.5 (for the GL ES 1.1 context) and, optionally, OpenGL 2.0 (for the 2.0 context) available. (A version of the canvas 3d extension will be available soon that supports software rendering using the Mesa GL library.) If support is not available, an error message will be printed in the JavaScript Console.
Documentation
Not a whole lot, at the moment. The interface files for the GL ES 1.1 context and for the GL ES 2.0 context should give you an idea of which functions are available; refer to the documentation for the equivalent OpenGL function on opengl.org. A few web-specific additions exist, such as “texImage2DHTML” for loading texture data from an HTML image element.
Examples
I’ve written a few examples:
Simple — just the basics; shows how to create a moz-gles11 context and to use glClearColor/glClear to clear the viewport to a specific color.
Simple 2.0 — just the basics again, but this time with moz-glweb20. Uses a simple fragment and vertex shader, and shows one possible way to include shaders in web pages (avoiding the need to specify long multiline string constants in javascript, which JS doesn’t handle all that well).
kmzview — Based on an earlier example by Philip Taylor, shows how to use the moz-gles11 context in conjuntion with E4X and the jar protocol handler to load Google Earth/Sketchup format kmz files (such as those available from the Google 3D Warehouse) and parse a (very limited) COLLADA model file and display it. This example uses the moz-gles11 context. It’s a bit of a hack, and could be very much improved.
modelview — Uses moz-glweb20 for displaying simple models that are specified as javascript objects, consisting of an array of vertices, faces, and normals. It uses the excellent Sylvester matrix math library, as well as a couple of tools that I’ve written that may be generally useful, such as glUtils.js and OrbitManipulator.js. I’ve also written a quick-and-dirty hack for converting OFF format files to js structures for faster loading.
Discussion
There is a 3D Canvas forums on the Mozilla Labs discussion site. I’ll be watching this forum closely, and I encourage people to discuss their experiments and experiences with this canvas 3d implementation there. This is also the best place for general bugs, though eventually Bugzilla will be the right place for these.
Technical implementation notes
The current implementation is directly tied to your platform native OpenGL implementation. If there are bugs or other issues, these will appear in the canvas as well. In addition, for the ‘moz-gles11′ context, some parameter checking is performed; for ‘moz-glweb20′, currently very little is performed. In the future, checks will be added to ensure that the number of available elements in bound attribute arrays are sufficient to cover the consumption of each shader and the number of elements being rendered.
The current implementation requires pbuffers; all rendering is done to a pbuffer. When swapBuffers() is called, the contents of the pbuffer are placed into the canvas buffer. This is the most straightforward way to obtain cross-platform integration with web content; that is, a canvas element that’s being drawn to using OpenGL will have the correct transparency blending and z-axis positioning occur with the rest of the HTML content on the page. The copy will often be the performance bottleneck; in future versions, OpenGL rendering will potentially happen directly to a native OpenGL view, removing that performance hit.
There is also very little control over the type of pixel format chosen by the canvas; in particular, there is no control over antialiasing or anisotropic filtering. In the future, these will be options provided by the Canvas 3D extension itself, and will apply to all 3D contexts.
Technical notes on the “moz-glweb20″ context
GL ES 2.0 removes a number of convenience/compatibility aspects of OpenGL 2.0. In particular, all aspects of the fixed function pipeline are completely removed. For example, desktop OpenGL 2.0 provides a gl_ModelViewMatrix uniform that corresponds to the current MODELVIEW matrix. GL ES 2.0 provides no matrix stacks nor is the gl_ModelViewMatrix available.
Because the moz-glweb20 context is built on top of desktop OpenGL, a vertex program attempting to use gl_ModelViewMatrix will compile fine; but, because the matrix manipulation methods are not exposed, no matrix except the default can be used. Instead, the developer should use a custom matrix attribute to pass the required matrix to the shader. For matrix math and management, I’ve found the Sylvester matrix math library to be excellent. With a few additional methods, such as those found in glUtils.js in the ModelView example, Sylvester integrates well with the moz-glweb20 context.
56 Comments to “Canvas 3D: GL power, web-style”
- 1 Pingback on Nov 27th, 2007 at 5:23 am
- 2 Pingback on Nov 27th, 2007 at 5:32 am
- 3 Pingback on Nov 27th, 2007 at 7:34 am
- 4 Pingback on Nov 27th, 2007 at 10:03 am
- 5 Trackback on Nov 27th, 2007 at 10:15 am
- 6 Pingback on Nov 27th, 2007 at 2:10 pm
- 7 Pingback on Nov 28th, 2007 at 10:15 am
- 8 Pingback on Dec 2nd, 2007 at 7:57 am
- 9 Pingback on Dec 2nd, 2007 at 2:20 pm
- 10 Trackback on Dec 3rd, 2007 at 5:04 am
- 11 Pingback on Dec 6th, 2007 at 5:05 am
- 12 Pingback on Dec 30th, 2007 at 6:11 pm
- 13 Pingback on Jan 22nd, 2008 at 3:41 pm
- 14 Pingback on Feb 1st, 2008 at 12:38 pm
- 15 Pingback on Feb 7th, 2008 at 12:52 pm
- 16 Pingback on Feb 17th, 2008 at 3:21 pm
- 17 Pingback on Feb 19th, 2008 at 8:25 am
- 18 Pingback on Apr 25th, 2008 at 2:45 am
- 19 Pingback on May 1st, 2008 at 9:55 am
- 20 Pingback on Jul 16th, 2008 at 5:02 pm
- 21 Pingback on Sep 12th, 2008 at 8:09 am
- 22 Pingback on Dec 12th, 2008 at 10:03 am
- 23 Pingback on Jan 21st, 2009 at 11:09 am
- 24 Pingback on Feb 6th, 2009 at 12:58 pm
- 25 Pingback on Mar 26th, 2009 at 4:06 pm
- 26 Pingback on Sep 21st, 2009 at 1:31 pm
This is pretty cool. I’m going to try a little fractal renderer or something.
Imagine GPGPU in web apps :D
I hope that Firefox will support Canvas 3D GL natively in the future.
I’ve tested you extension successfuly on Mac OS X 10.5 and Firefox 3b1.
The only examples not working are the “kmzview” ones.
http://asztal.net/demos/canvas3d/mandel.html
I’m not sure what sort of graphics card it might require to run this :)
Awesome. When would the extension become an integral part of Firefox?
> In the future, enabling 3D canvas contexts will be a site-specific preference (like page zoom is now), so that you’ll be able to set it on a per-site basis instead of globally.
Why isn’t it safe to enable 3D canvas for all content? and are there a reasonable chance that it will be enabled for all content at some point?
with al examples:
uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMHTMLCanvasElement.getContext]” nsresult: “0×80004005 (NS_ERROR_FAILURE)” location: “JS frame :: http://people.mozilla.com/~vladimir/canvas3d/examples/simple/simple20.html :: runTest :: line 60″ data: no] Line 0
using
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b2pre) Gecko/2007112505 Minefield/3.0b2pre
Extension seems to be installed successful and “enable 3D …” is checked
any hints?
just to be sure I installed a fresh copy of Beta 1 the extension (and enabled 3D)
but still get the error
Error: uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMHTMLCanvasElement.getContext]” nsresult: “0×80004005 (NS_ERROR_FAILURE)” location: “JS frame :: http://people.mozilla.com/~vladimir/canvas3d/examples/simple/simple20.html :: runTest :: line 60″ data: no]
OpenGL Vendor : ATI Technologies Inc.
OpenGL Render : ATI Mobility Radeon X1400 x86/SSE2
OpenGL Version : 2.0.6068 WinXP Release (200)
OpenGL Extensions: GL_ARB_multitexture GL_EXT_texture_env_add GL_EXT_compiled_vertex_array GL_S3_s3tc GL_ARB_depth_texture GL_ARB_fragment_program GL_ARB_fragment_program_shadow GL_ARB_fragment_shader GL_ARB_multisample GL_ARB_occlusion_query GL_ARB_point_parameters GL_ARB_point_sprite GL_ARB_shader_objects GL_ARB_shading_language_100 GL_ARB_shadow GL_ARB_shadow_ambient GL_ARB_texture_border_clamp GL_ARB_texture_compression GL_ARB_texture_cube_map GL_ARB_texture_env_add GL_ARB_texture_env_combine GL_ARB_texture_env_crossbar GL_ARB_texture_env_dot3 GL_ARB_texture_float GL_ARB_texture_mirrored_repeat GL_ARB_texture_rectangle GL_ARB_transpose_matrix GL_ARB_vertex_blend GL_ARB_vertex_buffer_object GL_ARB_vertex_program GL_ARB_vertex_shader GL_ARB_window_pos GL_ARB_draw_buffers GL_ATI_draw_buffers GL_ATI_element_array GL_ATI_envmap_bumpmap GL_ATI_fragment_shader GL_ATI_map_object_buffer GL_ATI_separate_stencil GL_ATI_shader_texture_lod GL_ATI_texture_compression_3dc GL_ATI_texture_env_combine3 GL_ATI_texture_float GL_ATI_texture_mirror_once GL_ATI_vertex_array_object GL_ATI_vertex_attrib_array_object GL_ATI_vertex_streams GL_ATIX_texture_env_combine3 GL_ATIX_texture_env_route GL_ATIX_vertex_shader_output_point_size GL_EXT_abgr GL_EXT_bgra GL_EXT_blend_color GL_EXT_blend_func_separate GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_clip_volume_hint GL_EXT_draw_range_elements GL_EXT_fog_coord GL_EXT_framebuffer_object GL_EXT_multi_draw_arrays GL_EXT_packed_pixels GL_EXT_point_parameters GL_EXT_rescale_normal GL_EXT_secondary_color GL_EXT_separate_specular_color GL_EXT_shadow_funcs GL_EXT_stencil_wrap GL_EXT_texgen_reflection GL_EXT_texture3D GL_EXT_texture_compression_s3tc GL_EXT_texture_cube_map GL_EXT_texture_edge_clamp GL_EXT_texture_env_combine GL_EXT_texture_env_dot3 GL_EXT_texture_filter_anisotropic GL_EXT_texture_lod_bias GL_EXT_texture_mirror_clamp GL_EXT_texture_object GL_EXT_texture_rectangle GL_EXT_vertex_array GL_EXT_vertex_shader GL_HP_occlusion_test GL_NV_blend_square GL_NV_occlusion_query GL_NV_texgen_reflection GL_SGI_color_matrix GL_SGIS_generate_mipmap GL_SGIS_multitexture GL_SGIS_texture_border_clamp GL_SGIS_texture_edge_clamp GL_SGIS_texture_lod GL_SUN_multi_draw_arrays GL_WIN_swap_hint WGL_EXT_extensions_string WGL_EXT_swap_control
What’s wrong here?
thank you
Which library should we install under linux (ubuntu), in order to have a “a full implementation of desktop OpenGL 1.5″ ?
I would rather have firefox support animated mouse cursors like IE. See URL.
Only the shifting background color example and the SF City Hall work for me. The rest gives blank canvasses and/or a FAIL message.
For the record, here is Opera’s approach [1]. It’s not bound to OpenGL or Direct3D, making the port to a platform not having them ( Wii, DS, TV sets, mobile phones, … ) a lot easier.
[1] http://my.opera.com/timjoh/blog/2007/11/13/taking-the-canvas-to-another-dimension
Sounds interesting but after installing and restarting on FF3b1 I just get blank squares and FAILED message when clicking on any of the samples. you included.
Any ideas? Thanks.
Any chance of a common API for simple 3D being worked out with the opera people? It would be really nice if for once a browser extension that’s under development would not require yet another toolkit to get it working cross-browser. Javascript toolkits are bloated enough as it is, we need convergence between browsers, not divergence.
IMHO, an opengl context is a good idea, but it’s less useful than a simple 3D context that does the basics and doesn’t require a wrapper library to make it workable. In practice people are going to use this stuff not for advanced simulations or game programming but for simple static models (because javascript isn’t fast enough), so the simpler the API the better.
Could you make the extension register for the [email protected] AppID instead of (or in addition to) the Firefox one (using Gecko version numbers with that one)? This would enable all Gecko-1.9-based toolkit apps to install the extension and not just Firefox, even any XULRunner apps that might actually want the 3D functionality even more than “just” the web-based use cases ;-)
Doesn’t work on my VISTA and WinXP installations. I enabled the 3D option after installing it. But I kept getting a Javascript error which indicates that the context is not available. Both my installations have OpenGL support (I checked for opengl32.dll in c:\windows\system32).
This sounds cool, but security could be an issue. If it’s not already the case, it should be disabled by default and turned on for specific sites only. There have been a few bugs in nVidia’s OpenGL implementation on Linux that can be exploited to gain root privileges. This type of plugin would create a whole new attack vector for malicious web coders.
Joeri Sebrechts wrote: “In practice people are going to use this stuff not for advanced simulations or game programming but for simple static models (because javascript isn’t fast enough), so the simpler the API the better.”
The API for static models should be “drawModel(xml)” to render a COLLADA/X3D file which has been exported from a 3d modelling program (3ds Max, Blender, SketchUp, etc). But COLLADA and X3D are pretty complex, and too poorly defined for web browsers that need interoperable handling of every edge case and error condition. And there’s at least those two standard formats, plus various others that people might prefer for various reasons. And web browser release cycles are pretty slow – it’s a pain if you have to wait a year until the next release adds the small features you require.
So I currently believe it’s more realistic for browser developers to provide a simpler, lower-level, more powerful, more interoperable cross-browser API that doesn’t have anything like a drawModel function, and then other people can implement scripting libraries to easily load and render static models.
p01 wrote: “[Opera's approach is] not bound to OpenGL or Direct3D, making the port to a platform not having them ( Wii, DS, TV sets, mobile phones, … ) a lot easier.”
The biggest unresolved question in my head now is that programmable shaders are the way of the future (and of the present), and are necessary if we want people to produce modern content and not be stuck with VRML-era primary-coloured cylinders; and shaders would have to be written in GLSL (because there’s no suitable alternative), which is easy for implementors using OpenGL/GLES and hard for anyone else. It’s hopefully not impossible, e.g. NVIDIA’s Cg Toolkit can convert GLSL into HLSL automatically, but it seems a significant pain. Everything else should be relatively straightforward.
@Philip
I really wish browser developers would stop pushing all the “simplification” to the toolkits. This not only leads to massive duplication of effort, but it’s headache for commercial application developers, because if you “roll your own” you end up having to invest a lot of effort, but if you use a 3rd party toolkit, your feature timeline is tied to theirs, and since every toolkit implements a different simplified api, you can’t switch toolkits when you’re unhappy with your current one because they aren’t on the ball as much as the toolkit your competitor uses.
I’m not saying that the low-level api’s shouldn’t be there, because they absolutely should, but I’m saying that for “the rest of us”, there needs to be a simple api as well (but only if it’s cross-browser).
“This type of plugin would create a whole new attack vector for malicious web coders.” — Which is another argument for defining a non-OpenGL API instead, so that the engine can block attackers from sending down arbitrary OpenGL.
Joeri: That sounds reasonable; but on the other hand, if the browser developers do all the implementation of nice easy-to-use APIs, then your feature timeline is tied to all of the browsers you want to support, and you can’t switch toolkits because there’s nothing at all to switch to (except proprietary or non-web platforms), and development of the API will be significantly delayed by the standardisation process. I’m not sure where the best tradeoff is.
The 2D canvas API looks a lot like existing 2D graphics libraries (Quartz 2D, Cairo) – I’m not aware of anything similarly universal for 3D other than OpenGL/DirectX, so it’s not obvious what a higher-level API would look like. I assume it would not be at all easy to design, so perhaps it’s best to let people play with a lower-level API for a while and then see what is successful and try to standardise something based on that.
John Stracke: An API based on OpenGL doesn’t allow arbitrary OpenGL. The browser would check array bounds and pointer lifetimes and whatever, to make things relatively safe – a non-OpenGL API would have the same problems, since it would still have to check the validity of the JS calls before passing data to the same unsafe OpenGL implementation.
Given the bugs found in every implementation of the 2D canvas API, any 3D canvas API should be similarly buggy. Since OpenGL rarely writes anything to memory, I guess the most common bugs would be crashes (when unallocated memory is passed to the GL implementation) and very-hard-to-exploit privacy violations (when browser memory is misinterpreted as data for a GL command) and not anything more severe.
There are certainly security concerns, but I’m not convinced a non-OpenGL API would be any better in that respect.
Awesome! ;)
Btw, kmzview didn`t work =(
I can’t tell you how much I’d like to see a 3D canvas in Firefox. We’re currently using Java3D applets for the Ogoglio platform (http://ogoglio.com/), which is extremely painful.
What can I do to help?
I just want to note that one should know very well the goals to achieve with such a project. While it may be great to inlcude simple mesh data into html pages, like the basic meshes in the examples, I do not see the target audience yet. As soon as you start to implement more complex applications, you come back to feature sets as already provided by x3d, which is already a standard for this (collada is made for something different afaik). And at least imho, it is a shame that x3d is still not supported in html-browsers. So what you target her is a bit like what Adobe put into their pdf format embedding u3d. It is nice, but very limited.
Wow, I just had the idea for a JavaScript binding for OpenGL, and here it is. I must be reading the collective unconscious.
This program seems pretty cool. I wonder if one would be able to use it for more complicated projects like 3D architectural renderings. Does anyone have an example of that with this program?
The plugin appears to be broken on firefox 3 beta 4 and beta 5
I installed and enabled it in its options on the latest Firefox 3 nightly, but when I open the samples, it behaves as I would expect if it wasn’t installed at all.
Perhaps a OpenGL|ES 1.0 could target a wider audience, as Intel video cards (~50% market share) only has OpenGL 1.4. As OpenGL|ES 1.0 is based on OpenGL 1.3, this can done I guess.
Also see the JavaME JSR 239 approach for OpenGL|ES. It uses interfaces to get OpenGL 1.0, 1.1, etc, from a static GLES class. This allow application to choose best path path for application, following platform.
Doesn’t work on my VISTA and WinXP installations. I enabled the 3D option after installing it. But I kept getting a Javascript error which indicates that the context is not available. Both my installations have OpenGL support (I checked for opengl32.dll in c:\windows\system32).
http://asztal.net/demos/canvas3d/mandel.html
is working fine on my maschine ^^ even with 90000x Speed
WinXPx64
Intel Q6600@2,8GHz
GeForce8800GTX
4x1GB DDR2 RAM