Open Bug 703995 Opened 13 years ago Updated 2 years ago

Loading pictures from cache is much slower compared to Internet Explorer or Chromium

Categories

(Core :: Graphics: ImageLib, enhancement)

Product:
Core
Component:
Graphics: ImageLib
Version:
8 Branch
Platform:
x86_64
Windows 7
Type:
enhancement

Tracking

()

People

(Reporter: pbcmueller, Unassigned)

References

(
URL
)

Details

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0) Gecko/20100101 Firefox/9.0
Build ID: 20111116091359

Steps to reproduce:

When loading my page the second time from cache, Firefox 8.0 needs around 43 Seconds on my machine (Windows 7 64 Bit, Intel Core i7, 8 GB RAM, FireBug reports for all request a "304 Not Modified") to load. Chromium (14.0.797.0 (Entwickler-Build 89612)) needs under 2 seconds, Internet Explorer 9 around 13 seconds.

The page I'm developing is a html and javascript only based - i.e. no flash - ePaper: http://www.patrickmueller.li/epaper_ml1110_2/Blaettern_V11_39.html


Actual results:

Firefox 8.0 is much slower then the competition.

I once even did notice, that with a very fast internet connection (> 50 Mbit/s), loading from the server was even faster then loading from the cache. I will verify that in a week when I have access again to the fast internet.


Expected results:

Firefox should load the page much faster.
URL: http://www.patrickmueller.li/epaper_m...
Severity: normal → enhancement
Summary: Loading pictures from cache is much slower compared to Internet Explorere or Chromium → Loading pictures from cache is much slower compared to Internet Explorer or Chromium
Patrick, how are you measuring the times in comment 0?  I see this page load in under 2-3 seconds over here....

Please make sure to test _without_ Firebug installed, since it has a fairly significant impact on pageload times.
Hi Boris

After the first (visible) page/picture is loaded (what only takes a few seconds), the remaining 83 pages are dynamically loaded in the background - I do measure the time until all 84 pages are loaded. The loading algorithm is "smart": if you jump to a page that is not yet preloaded, that page is loaded next and it continues preloading the pages following your current page - this works also if your flipping the pages in the opposite direction. This shall make the user experience as snappy as possible. Depending on your screen resolution it loads a total of some 7 to 15 MBytes for the whole magazine/ePaper.

Yes, I do measure the loading time with Firebug. I will add an in-page indicator this weekend for measuring the pageload times without firebug to check whether firebug has any influence regarding cache.
Patrick, thanks.  Given an in-page indicator, it'll also be a lot simpler to see where the time is going even if Firebug is not the problem...
Hi Boris 

I did add some speed/timing information on my page and also did upload the plain/non-minified javascript (its that one: http://www.patrickmueller.li/epaper_ml1110_2/Blaettern_V11_39.js). I tried again and loading from cache is now +/- as fast as IE 9 - but Chromium and Safari are still at least 6 times faster!
Another interesting effect I did discover: I did upload the pictures to Amazon S3 to check whether it might be faster: http://www.patrickmueller.li/epaper_ml1110_2/Blaettern_V11_39_AS3.html
Loading from cache is slower no in Firefox - not so with Chromium, that is as fast as before.
Patrick, sorry for the lag.  Holiday weekend here...

I just tried loading http://www.patrickmueller.li/epaper_ml1110_2/Blaettern_V11_39.html and I see the various background image requests, but no in-page indicator of load completion...

Honza, Patrick, want to take a look at this?
Hi Boris

It's fixed - I did play too much around - sorry. 

One more thing that I did only notice in FF: I open my page and afterward some tabs and browse a bit inside them. If I switch back to the ePaper the picture/page is white and it takes a few seconds until the pic/page is shown again.
btw. with "It's fixed" I did mean the broken links. The issues itself is still present.
Patrick, I'm finally getting back to this.  Sorry for the lag....  Thank you for those in-page indicators!

I can definitely reproduce this.  After the first load, if I put the cursor in the url bar in Firefox (a current nightly) and hit enter, the load takes about 16s for all the 84 images.  Chrome takes about 2 seconds. 

The 2 second thing (and the <7ms load times I see on a per-image basis) is only possible if they're not sending any sort of revalidation requests at all of if the images are being loaded in parallel: ping time from me to the server the images are on is 100+ms, so it should take at least 8s to load all the images if each one has to do a conditional GET and get a 304.

Is your code doing the requests in parallel, or is it waiting for one image to fire onload before starting the next request?  Do you see Chrome actually hitting the network at all for these images?
Status: UNCONFIRMED → NEW
Ever confirmed: true
OK, looking at an HTTP log for us, it looks like the next load does in fact start once the previous one completes.  Also, we do load all the images from cache without having to hit the network for the image loads themselves.

The one thing I see us hitting the network for are URIs of this form: <http://www.patrickmueller.li/epaper_ml1110_2/Bilder/Seite_0061.txt.gz?getid=true>.  These do get cached, but have to be revalidated every time; as the log says:

  Validating based on RFC 2616 section 13.9 (query-url w/o explicit expiration-time)

These URIs do come back with 304 responses.

Patrick, is the script doing those in series as well (and gating the image loads on those)?  If it is, that would explain the observed behavior in Firefox, though not in Chrome (unless they're violating the relevant part of the HTTP RFC, of course).
Hi Boris

Sorry for my late replay too: I just came back today from vacation.

Yes I do load everything one after another: Actually I do first load the picture of the page and afterwards the containing text, that is compressed with gzip (.gz) (to make the page searchable): that's that URI <http://www.patrickmueller.li/epaper_ml1110_2/Bilder/Seite_0061.txt.gz?getid=true>. Afterwards the next pic and text are loaded.
Yes it looks like Chrome also loads the "Seite_0061.txt.gz" from cache - even if I didn't add an expiration date till now for it. But now I did add but Firefox still doesn't loads it from cache - do I do something wrong?
I did change my code now so also the http://www.patrickmueller.li/epaper_ml1110_2/Bilder/Seite_0061.txt.gz can be loaded from cache - what now also works with Firefox - but Firefox still takes around 5,6 seconds and Chrome around 1,5 seconds with my configuration...
> But now I did add but Firefox still doesn't loads it from cache - do I do something wrong?

Is the server still in this configuration?  Comment 12 suggests no, making it hard to answer that question...

Going to look into the time situation with the new setup.
With Aurora and Nightly debug I can see a hang of UI for a time (about 5 seconds).  That is also interesting to investigate on.  I can see most of the time spent in Name                                            	Address    	Self 	Children 	Total 	
 mozilla::imagelib::nsJPEGDecoder::WriteInternal	0x5d1d1254 	0    	289      	289   	

1 function, Total: 0 self samples, 289 child samples

But there seems to be lot more other calls.

I'm looking into this too.
About the hang, CodeAnalyst tells me the significant amount of time is spent at ycc_rgb_convert_argb function.  Probably worth of reporting a different bug.  I can see the hang just the first time we load the page in a browser session.  Any other attempts by focusing the address bar and pressing enter don't exhibit that behavior and load times are in some 454ms, for me faster then Chrome 16 (it loads the page in some 540ms).  The first load is however about 3700ms in Aurora.
As I recall, Chrome decodes images on paint while we do it eagerly.  There are existing bugs on that.  So if it takes a few seconds to decode all the images, that would explain the difference, esp. if decoding blocks onload for the image.
(In reply to Patrick Müller from comment #12)
> I did change my code now so also the
> http://www.patrickmueller.li/epaper_ml1110_2/Bilder/Seite_0061.txt.gz can be
> loaded from cache - what now also works with Firefox - but Firefox still
> takes around 5,6 seconds and Chrome around 1,5 seconds with my
> configuration...

My understanding is that previously Chrome would cache your images but Firefox wouldn't, but then you changed your test so that Firefox would also cache the images.

I am *very* curious about what change you made to cause Firefox to cache your images. (Rather, I am interested why Chrome would cache them in situations that Firefox wouldn't.)
Both Chrome and Firefox were caching the images.

Chrome was also caching URIs of the form http://www.patrickmueller.li/epaper_ml1110_2/Bilder/Seite_0061.txt.gz?getid=true that were sent without an explicit Expires and loading them directly from the cache without a conditional GET.  We were caching them too, but revalidating the cache, which requires a server round-trip.  Once we got the 304 response we loaded them from the cache.  As far as I can tell, Chrome is just violating a MUST-level requirement from the HTTP RFC here.  See bug 468594 and nsHttpChannel::MustValidateBasedOnQueryUrl.

The page has since been changed to send an Expires header for those URIs, so now we are reading them directly from cache.
Currently we decode all images associated with <img> elements in the current tab, even if those <img> elements are hidden or not in the document at all.  Those decodes may or may not block the document's load event, so depending on what exactly you're measuring, you may see that decode time in your measurements.

We also currently attempt to synchronously decode all images under a certain size.  When we're getting images off the network we normally don't have the entire image downloaded when decoding begins, but when we get them from the cache, we can decode clear through.  This may be the cause of UI jank you're seeing.

There are bugs on file for all of this, of course.
> Those decodes may or may not block the document's load event

I think for purposes of this script what matters is whether they block the image element's load event.

Is there something I can do locally to disable the eager decode just to see what the effect is on the numbers?
(In reply to Boris Zbarsky (:bz) from comment #20)
> Is there something I can do locally to disable the eager decode just to see
> what the effect is on the numbers?

Hack http://mxr.mozilla.org/mozilla-central/source/content/base/src/nsImageLoadingContent.cpp#350 to not requestDecode if the content node is not in the document *and* nuke (Un)LockEnumerator at http://mxr.mozilla.org/mozilla-central/source/content/base/src/nsDocument.cpp#8291.  That should ensure that nodes that are not in the document never get decodes requested.  It will also discard images on the current tab, but that doesn't matter for your perf test.

Note that decodes do not block the image's load event, but they are triggered immediately before firing the load event, and if the decode happens synchronously ...
The images here all look to be about 75 KB, and the cutoff for switching to async decoding is a little under 150 KB, so I expect that the sync decoding off the cache is what's killing us here.
One other thing you can do, if you want to test comment 22, is twiddle the image.mem.max_bytes_for_sync_decode pref down to say, 1 byte.
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #23)
> One other thing you can do, if you want to test comment 22, is twiddle the
> image.mem.max_bytes_for_sync_decode pref down to say, 1 byte.

Doesn't help, even after restart.  Still seeing 1300ms of self-time in ycc_rgb_convert_argb.
Well the decoding is all still going to happen eventually, but it may let the load event fire earlier.
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #25)
> Well the decoding is all still going to happen eventually, but it may let
> the load event fire earlier.

And still the same long loading times... :(
Just tell me if I should setup up a testpage like before to show chrome caching without explicit Expires. 

(In reply to Boris Zbarsky (:bz) from comment #18)
> Both Chrome and Firefox were caching the images.
> 
> Chrome was also caching URIs of the form
> http://www.patrickmueller.li/epaper_ml1110_2/Bilder/Seite_0061.txt.
> gz?getid=true that were sent without an explicit Expires and loading them
> directly from the cache without a conditional GET.  We were caching them
> too, but revalidating the cache, which requires a server round-trip.  Once
> we got the 304 response we loaded them from the cache.  As far as I can
> tell, Chrome is just violating a MUST-level requirement from the HTTP RFC
> here.  See bug 468594 and nsHttpChannel::MustValidateBasedOnQueryUrl.
> 
> The page has since been changed to send an Expires header for those URIs, so
> now we are reading them directly from cache.
(In reply to Boris Zbarsky (:bz) from comment #18)
> we got the 304 response we loaded them from the cache.  As far as I can
> tell, Chrome is just violating a MUST-level requirement from the HTTP RFC
> here.  See bug 468594 and nsHttpChannel::MustValidateBasedOnQueryUrl.

I filed bug 718797. It seems like there is room for improvement in this area.

> The page has since been changed to send an Expires header for those URIs, so
> now we are reading them directly from cache.

(In reply to Patrick Müller from comment #27)
> Just tell me if I should setup up a testpage like before to show chrome
> caching without explicit Expires. 

First, thank you very much for your help here! Like I said, I filed bug 718797 about us possibly being too eager to revalidate in these situations. For bug 718797, it would actually be great to have the one without the Expires header. But, for this bug, it would be better to have the one with the Expires header. So, if it isn't too much work, having both available would be great.
Hi Brian
It's my pleasure to help - and makes me proud that a nobody like me gets so much attention - thanks (btw. just offtopic how do you guys like my epaper - dos it feel and behave like one would expect it (double click on a pic zooms in, left top is the search field)?).

I did add the testcase without the Expires header for *.txt and *.gz files - *.jpg still have an Expires header like before - if you like to have no expires header for the *.jpg files too just tell me.

You find it here: http://www.patrickmueller.li/epaper_ml1110_nocache/Blaettern_V11_39.html
You can also access all testcases with links and short description on my top URL:
http://www.patrickmueller.li
The issue with caching and query string has been fixed.. comment 19 may or may not have been fixed, I'm not sure.. pushing over to imglib to see
Component: Networking: Cache → ImageLib
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.