Validation
Check this document for XHTML 1.0 Strict conformance and CSS 2.1 conformance and see all these nasty errors.
Debugging
I would be very grateful if you could let me know about any language errors still remaining in this site. Thanks!
In the interview for The AROS Show blog I mentioned that I'm working on a NetSurf browser port for AROS. Krzysztof Śmiechowicz after hearing that advised me to take a look at Origyn Web Browser instead. It's based on a powerful WebKit engine used for example in Safari and Google Chrome browsers. OWB has few dependencies and can use SDL library for rendering graphics, which means relatively little work is needed to get some results. What's also important, OWB fulfills most of requirements of the Port an Open Source Browser To AROS bounty, so I decided to combine business with pleasure and in the middle of December 2008 I was assigned to the bounty.
I dislike writing code without seeing results of my work in reasonable amount of time. Therefore I started with basic OWB port that could be reworked step by step into the final form. OWB is written in C++, so the first thing I needed was a C++ cross-compiler. Fortunately one is already available in AROS thanks to the work of Markus Weiss and Nick Andrews. SDL and some basic libraries like libjpg, libpng, libz etc have been already ported to AROS, the only missing ones were OpenSSL and libcurl. Rob Norris, who was working on the browser bounty before, ported them to AROS, so I used his ports and integrated them into AROS build system to make using them easier. OWB is using cmake-based build system, I'm quite familiar with cmake, so adapting OWB build system to AROS wasn't a problem and in the end it took me only few days to build the basic version of OWB for AROS.
Then I encountered some problems. First, produced binary with debugging information was big. No, big is not a right word, it was huge: over 300 MB. Building or loading such binary file in debugger took several minutes. Second, it was crashing. These problems combined with each other resulted in the following days spent on trying to locate the source of the problem. And the problem was ssize_t type having size of 4 bytes on x86-64 AROS. Irony that I encountered this problem before while porting Python, spent few days locating the problem as well, and decided to apply a quick fix instead of doing it right.
After fixing that problem I could finally enjoy a somewhat faded Google search page on AROS:
 OWB (almost) working |
It appeared that the AROS SDL port was responsible for the strange look of the Google site above, also I had to add some missing SDL structure definitions to get OWB compiled, so I decided that I may as well create a new port of SDL. AROS SDL port is based on some SDL version from year 2002 and it was never updated with later upstream versions, so it's nothing strange that it didn't work the right way.
Having basic OWB working I could proceed with some better font support. The basic one is a single built-in generic font, but there are other options available like using Freetype library or Pango. There's a Freetype library port to AROS, so I decided to proceed with rendering fonts with Freetype. By luck Krzysztof Śmiechowicz was just finishing updating AROS Freetype at that time, so I could use the freshly updated port in OWB. It worked really good, just take a look at the screens below:
 OWB with Freetype font rendering |
 The most important AROS site |
 Google on AROS |
I really didn't expect it would be so easy to get a working web browser on AROS. The only problem was some minor font rendering glitches (try to find some on screenshots), but I decided to take care of them later. After all it was just a beginning. One of the bounty requirements states that the final product should be a Zune-based browser, so SDL-based one was not an option. It was time for Zunification.
The bounty says that "Ideally the engine is turned into an own .mcc (like the HTMLview.mcc) or .library at bounty release, so that it can be used in other applications". I felt that it was the right time to try doing it, as later the browser code would become too complex to modularize it. After first tries it became apparent that C++ compiler doesn't like the code produced by AROS genmodule tool, so I ended up creating a static library exporting a C interface to C++ WebKit classes, then I could simply link it to the Zune custom class that I was going to write. It worked fine, so I could finally start filling the body of my Zune class with code while building the browser user interface at the same time.
At first I rewrote the Expose event handling code to use SDL_Surface created in my mcc and draw it on the screen during MUIM_Draw. Amazingly the WebKit's WebView class interface is smart enough to update the screen contents incrementally, so the browser doesn't need to redraw the whole page only because for example some link should be highlighted. I also considered rewriting the whole drawing code to use graphics.library directly, but it doesn't support alpha channel, so I guess drawing with SDL is the only viable options for now.
I had to draw with SDL, but of course I could get rid of the rest of SDL code. I replaced SDL events with intuition.library events provided by the Zune event handler registered in the custom class. Code from AmigaOS4 OWB port written by Jörg Strohmayer saved me a lot of work in this stage. Seeing results of my efforts was very motivating, so I created a basic browser interface very quickly. I must say that writing MUI code again after all these years was a very pleasant experience. Inspired by the people from #AROS channel requesting the support for tabs decided to give it a try. Initially I used a little modified Register class to check how difficult is it to add tabs support in my OWB port. It worked very well, but Register class tabs can't be dynamically added or removed, also they are limited to displaying only text, so I decided to implement my own Zune tab class.
I didn't expect that it would take almost a week to design and implement such a class, During that time I found and fixed few Zune bugs and learned a lot about its internals and limitations. Fruits of my labor allowed arbitrary objects inside tabs and fully supported addition and removal of the tabs during runtime. I also added ability to automatically add auxiliary object to the active tab that could serve for example as a close button. Updating my OWB port to use new Zune class for tabs took only few hours and then I could finally enjoy a real multi-tab browser:
 Zune-based OWB browser |
New OpenSSL 0.9.8j version was released a week ago, so I decided to create a new port based on this version, hoping that it would magically resolve some problems I had with SSL. Of course it didn't, so I went to my good friend the GNU Debugger to get some help. It appeared that my OpenSSL port didn't work correctly for two main reasons. First reason was that it used arosc.library read() and write() functions operating on file descriptors to read and write data to network sockets. Second reason was incorrect set of defines for x86-64 AROS target in OpenSSL configuration file. After fixing these problems tests included with OpenSSL suite reported no more problems and recompiled libcurl was finally able to use URLs starting from https. Well, it wasn't exactly that simple, libcurl appeared to be affected by the arosc.library curse as well, but I got some experience during the fight with OpenSSL, so I was able to quickly find the cause of remaining problems.
I had OpenSSL working and I was able to log in to my Google account from OWB, but I noticed that a lot of links and some JavaScript-based menus in various Google apps are not working. After looking at them closely I discovered that they have one thing in common - they were opening new browser windows. Indeed I couldn't remember implementing such functionality before, so I added the missing parts and then everything started working correctly. Take a look at the screenshots:
 Google Maps |
 Google Mail |
 Google Docs |
Like I said, everything started working correctly, but with strange temporary hangs that were present from the beginning of my work on OWB. They were getting on my nerves and I felt it's time to investigate them properly. Day of debugging left me in connect() call in AROSTCP stack, so I started investigating network connections made from the hosted AROS and discovered many of them left in a strange state. OWB was not closing network connections at all and that was the cause of this strange behavior. Then on the curl library page I noticed some bug fixes related to closing and reusing sockets, so I decided to port the most recent curl version to AROS. It worked much better than the old one, loading pages was faster and connections were finally managed properly. However still sometimes browser was freezing the entire system for a moment. Debugger showed that it was spending that time allocating memory. Remembering the problem Krzysztof Śmiechowicz recently had with Eternal Lands port for AROS, I applied his memory allocator fix and then at last I could experience smooth browsing without any hangs.
One of the bounty requirements is "Small Aros Zune based browser that uses datatypes". I've been thinking about that datatypes usage part and wondering what could it mean. The only sane conclusion I could came to was to use datatypes for decoding images, that's what browsers on AmigaOS were doing as far as I remember. So I started investigating how could I integrate datatypes usage into OWB. OWB decodes imags with objects of classes derived from ImageDecoder class, each class is using a specialized library for decoding given image file format. Writing a new class doing that with datatypes was not complicated, however there were several other problems:
Unfortunately my new class inherited these restrictions, it decodes images only when all image data are available, also it uses temporary files to pass data to datatypes. I'm currently thinking about adding DTST_MEMORY support in datatypes.library to make it more straightforward. To test my class I downloaded some IFF images, created html with img tags and displayed it in OWB:
 Displaying IFF images with datatypes |
With no support for progressive decoding images directly from memory there's not much point in switching all image decoding to datatypes, it would make loading images much slower. I guess the only need for datatype-based image decoder will be to decode some rare image formats that couldn't be handled otherwise.
For the next few days I was looking for a way to allow multiple applications to use my custom Zune browser class simultaneously. First let me explain why it's not easy. I bet most of you know that every AROS program using arosc.library has its own arosc.library context with various internal data like opened file descriptors, memory allocator pool used in malloc() call and some other. This context is initialized during program startup and freed on program exit. It's done that way to prevent memory leaks that would be otherwise caused by not closing file descriptors or not freeing allocated chunks of memory. The context is process-specific, so arosc.library file descriptors from one process won't work in other processes, also memory allocated in one process can't be freed in some other process. Using browser class in multiple applications running simultaneously or even executed one after another would cause mixing memory owned by different processes in one MCC, it would inevitably lead to crashes caused by using memory that has been already deallocated because the process owning it exited or deallocating memory allocated by one process in another.
There are several ways to resolve this problem. For example I could override memory allocation routines, so they would use a separate memory pool belonging to the browser MCC. Other solution is to identify all places in OWB where memory from different processes could be mixed and somehow rework them to avoid it. Yet another solution is to create a separate process in the browser MCC and do all operations from within this process. I decided to try the last one, it was most promising one because of preventing possible reentrancy problems. But still it required creating an additional layer of functions for communication between MCC methods and WebView object residing within MCC process. When testing this solution I noticed that draw operations in the browser window are hanging because the window layer is locked during MUIM_Draw method, so I had to extend the communication layer with drawing operations sent from the MCC process to the browser program process. I ended up with bloated communication layer and a deadlock caused by the browser program process waiting for MCC process to finish drawing while MCC process was waiting for the browser program process to draw something in the window. Few hours of trying to fix this problem lead to more deadlocks and crashes. I didn't want to waste more time by working on things not actually required by the bounty, it was time to stop and admit failure.
The bounty says "Ideally the engine is turned into an own .mcc (like the HTMLview.mcc) or .library at bounty release". For now it looks like this ideal situation won't happen, I reworked the browser MCC into a statically linked private one to avoid mentioned problems and decided to keep it like that until I have another brilliant idea.
Below is the list of various features implemented in the meantime.
Some people said that default OWB scrollbars were ugly, so I made them a little more pretty. I could use native Zune scrollbars, but there were some problems with them, also using native scrollbars in WebKit is discouraged, let me cite the relevant fragments from webkit-dev mailing list: The current plan for refactoring is to eliminate PlatformScrollbar and make Scrollbar into a class that handles scrolling using a non-native cross-platform implementation, ...we would strongly encourage ports not to use native scrollbars. We will even be abandoning the native PlatformScrollbar implementation on the Mac.
 Scrollbars before the change |
 Scrollbars after the change |
I implemented alert and confirm requesters with MUI_Request(). Note that requester windows in Zune are not modal, so they are not blocking the main browser window. Hard to say if it's a bug or a feature.
 Confirm requester |
 Alert requester |
Selecting options from select tag having size 1 was possible only by changing the active item with arrow keys, so I added a popup list window to make it more convenient.
 Selecting items in select tag |
I implemented OWB context menu with standard Zune context menus. It has different options on different www page elements. Not all menu actions are implemented, though.
 Context menu for images |
I also implemented file requester for choosing files in input form elements with type file. I used a standard asl.library requester. File upload was initially not working because of a bug in arosc.library, but I managed to fix it and now it's working correctly.
 Uploading a file |
There is no download manager in OWB, on the other hand downloading files is a vital part of web browsing, so I wanted to implement one. I found sources of WebKit download manager for gtk and was going to adapt them to my needs, however Fabien Coeurjoly, who is working on OWB port for MorphOS did it first and kindly shared his code. I refactored it to add support for DownloadDelegate objects, created a simple GUI and here it is:
 Download manager |
A simple text search window:
 Search window |
Another added feature is text clipboard support, I based it on AmigaOS4 code written by Jörg Strohmayer.
To make the browser look consistent with the default AROS theme I decided to use icons from Gorilla set. Now the browser looks like that:
 OWB for AROS with Gorilla icon set |
OWB has a lot of internal settings, so I made a window for setting most of them. I also added options for enabling datatypes usage.
 Preferences window |
Preferences are loaded and saved with Zune MUIM_Application_Load and MUIM_Application_Save methods. They were missing in Zune, so I spent some time implementing them and adding missing MUIM_Import / MUIM_Export is some classes.
Next thing I wanted to implement was the proper character encoding conversion between the browser engine and AROS. Initially I thought that adding codesets.library conversion calls here and there will be enough to solve the problem. How naive of me! But let's start from the beginning.
I started with creating a test html file titled "żółw" (it means turtle in Polish language) and wanted to get the title displayed properly in OWB. I switched my hosted AROS to Polish locales, language, keyboard map and ISO-8859-2 fonts. Then I noticed that changing fonts had no effect at all. It appeared that font preferences utility was simply not working on x86-64 AROS because FontPrefs structure on this architecture was incompatible with preferences loaded from disk due to increased pointer size and different structure alignment. Initially I solved this problem by changing the format of the prefs file to match the structure, but Georg Steger suggested that all AROS flavors should use the same format of the font preferences file, and it's the font prefs utility fault that it can't read this format. I reverted my changes and proceeded accordingly to Georg's suggestions and then I had finally working font prefs and could write Polish characters on x86-64 AROS.
AROS was handling Polish characters correctly, codesets.library encoding conversion was working, but still OWB displayed my test page title as "???w". Debugging showed that OWB simply returned the title string like that, somehow it didn't know how to properly convert "żółw" from UTF-16 internal representation to UTF-8. Apparently the current OWB generic i18n implementation is simplified to the point where it can handle only ASCII set correctly. Looking at its limitations I decided to use ICU library instead.
Some magic tricks were needed to cross-compile ICU, so I simply built it on AROS. It was really easy thanks to all my previous work on AROS self-compilation and development environment. But when I issued "make check" command to make sure everything is working properly I noticed that many library tests failed. Few hours of debugging lead me to arosc.library strcmp() implementation which appeared to be buggy. It compared characters with codes larger than 127 incorrectly. Later I noticed the same problem in strncmp(). After fixing these functions ICU library started working fine. Unfortunately it caused OWB binary size to increase by few megabytes, but I think that advantages of using ICU are fully compensating this.
Take a look at the screnshots:
 Żółw finally displayed correctly |
 Example Polish site with ISO-8895-2 encoding |
 Turtle goes to Russia |
 Example Russian site with Windows-1251 encoding |
During my work on the browser bounty I ported the following software to AROS:
| Package name and version | Work done | Commited |
|---|---|---|
| OpenSSL 0.9.8j | Created a new port based on 0.9.8g port done by Rob Norris. | yes |
| Curl 7.9.12 | Created a new port based on 7.17.1 port done by Rob Norris. | yes |
| Fontconfig 2.6.0 | Created a new port. | yes |
| ICU 4.0.1 | Created a new port based on 3.8 port done by Rob Norris. | yes |
| SDL 1.2.11 | Created a new port. | yes |
| OWB (SVN trunk) | Created a new port partially based on AmigaOS4 port done by Jörg Strohmayer. | yes |
Besides that, I wrote a Zune-based browser using the OWB port as web browser engine and fixed numerous bugs in arosc.library and Zune.
Patches containing AROS-related changes for all ported software along with sources of the Zune-based browser are already commited to the AROS SVN repository. Building of all ported packages except ICU library is now possible from within the AROS build system.
The browser engine is kept separated from the Zune-based browser. The browser is using WebView Zune private custom class which exports the browser engine interface in a Zune-friendly way. Later this class can be changed into a public class shared among all applications using the web browser engine.
Here you can download the latest binary build of OWB for i386 AROS:
Click here to download OWB for AROS
Unpack the archive and run owb. Note that you need a recent AROS build to use OWB. Builds starting from 19.02.2008 should be fine. OWB will crash on earlier builds due to a bug in arosc.library. If your AROS build is quite recent and you don't want to reinstall, you can try copying arosc.library and muimaster.library from the new build to your AROS installation - but I can't guarantee it will help.
Here you can see some screenshots of the build above running on i386 AROS sent by Nikolaos Tomatsidis (on real hardware) and kas1e (on VMware):
 OWB running on i386 AROS (real hardware) |
 OWB running on i386 AROS (VMware) |
OWB for AROS has its own bug tracker on Sand-Labs, please register to report bugs and send feature requests:
