How we wrote our own Service Worker
As we continue the process of
reinvigorating Tumblr’s frontend web development, we’re always on the lookout for modern web technologies, especially ones that make our mobile
site feel faster and more native. You could have guessed that we are making the mobile dashboard
into a progressive app when we
open-sourced our webpack plugin to make web app manifests
back in August. And you would’ve been right. But to make a high quality progressive web app, you
need more than just a web app manifest—you also need a service worker.
What is a service worker?
A service worker is a helper script that a page registers with the browser. After it is registered
(some people like to also call it “installed”), the browser periodically checks the script for
changes. If any part of the script contents changes, the browser reinstalls the updated script.
Service workers are most commonly used to intercept browser fetches and do various things with
them.
https://serviceworke.rs
has a lot of great ideas about what you can do with service workers, with code examples. We
decided to use our service worker to cache some JS, CSS, and font assets when it is installed, and
to respond with those assets when the browser fetches any of them.
Using a service worker to precache assets
You might be wondering “why would you want to pre-cache assets when the service worker is
installed? Isn’t that the same thing that the browser cache does?” While the browser cache does
cache assets after they’re requested, our service worker can cache assets before they’re
requested. This greatly speeds up parts of the page that we load in asynchronously, like the notes
popover, or blogs that you tap into from the mobile dashboard.
While there are open-source projects that generate service workers to pre-cache your assets (like,
for example,
sw-precache), we chose to build our own service worker. When I started this project, I didn’t have any idea
what service workers were, and I wanted to learn all about them. And what better way to learn
about service workers than building one?
How our service worker is built
Because the service worker needs to know about all of the JS, CSS, and font assets in order to
pre-cache them, we build a piece of the service worker during our build phase. This part of the
service worker changes whenever our assets are updated. During the build step, we take a list of
all of the assets that are output, filter them down into just the ones we want to pre-cache, and
write them out to an array in a JS file that we call sw.js.
That service worker file importScripts()’s a separate file that contains all of our service worker
functionality. All of the service worker functionality is built separately and
written in TypeScript, but the file that contains all of our assets is plain JavaScript.
We decided to serve our service worker directly from our node.js app. Our other assets are served
using
CDNs. Because our CDN servers are often geographically closer to our users, our assets load faster
from there than they do from our app. Using CDNs also keeps simple, asset-transfer traffic away
from our app, which gives us space us to do more complicated things (like rendering your dashboard
with React).
To keep asset traffic that reaches our app to a minimum, we tell our CDNs not to check back for
updates to our assets for a long time. This is sometimes referred to as caching with a long TTL
(time to live). As we know,
cache-invalidation
is a tough computer science problem, so we generate unique filenames based on the asset contents
each time we build our assets. That way, when we request the new asset, we know that we’re going
to get it because we use the new file name.
Because the browser wants to check back in with the service worker script to see if there are any
changes, caching it in our CDNs is not a good fit. We would have to figure out how to do cache
invalidation for that file, but none of the other assets. By serving that file directly from our
node.js application, we get some additional asset-transfer traffic to our application but we think
it’s worth it because it avoids all of the issues with caching.
How does it pre-cache assets?
When the service worker is installed, it compares the asset list in sw.js to the list of assets
that it has in its cache. If an asset is in the cache, but not listed in sw.js, the asset gets
deleted from the cache. If an asset is in sw.js, but not in the service worker cache, we download
and cache it. If an asset is in sw.js and in the cache, it hasn’t changed, so we don’t need to do
anything.
// in sw.js
self.ASSETS = [
‘main.js’,
'notes-popover.js’,
'favorit.woff’
];
// in service-worker.ts
self.addEventListener('install’, install);
const install = event
=> event.waitUntil(
caches.open('tumblr-service-worker-cache’)
.then(cache
=> {
const currentAssetList
= self.ASSETS;
const oldAssets
=/* Instead of writing our own array diffing, we use lodash’s */;
const newAssets
=/* differenceBy() to figure out which assets are old and new */;
return Promise.all([
…oldAssets.map(oldAsset => cache.delete(oldAsset)), cache.addAll(newAssets)]);
});
);
We launched 🚀
Earlier this month, we launched the service worker to all users of our mobile web dashboard. Our
performance instrumentation initially found a small performance regression, but we fixed it. Now
our mobile web dashboard load time is about the same as before, but asynchronous bundles on the
page load much faster.
We fixed the performance regression by improving performance of the service worker cache.
Initially, we naively opened the service worker cache for every request. But now we only open the
cache once, when the service worker starts running. Once the cache is opened, we attach listeners
for fetch requests, and those closures capture the open cache in their scope.
// before
self.addEventListener('fetch’, handleFetch);
const handleFetch = event
=>
event.respondWith(
caches.open('tumblr-service-worker-cache’)
.then(cache
=> cache.match(request)
.then(cacheMatch
=> cacheMatch
?
Promise.resolve(cacheMatch)
: fetch(event.request)
)
)
);
// now
caches.open('tumblr-service-worker-cache’)
.then(cache
=>
self.addEventListener('fetch’, handleFetch(cache));
const handleFetch
= openCache => event
=>
event.respondWith(
openCache.match(request)
.then(cacheMatch
=> cacheMatch
?
Promise.resolve(cacheMatch)
: fetch(event.request)
)
);
Future plans
We have lots of future plans to make the service worker even better than it is now. In addition to
pre-emptive caching, we would also like to do reactive caching, like the browser cache does. Every
time an asset is requested that we do not already have in our cache, we could cache it. That will
help keep the service worker cache fresh between installations.
We would also like to try building an API cache in our service worker, so that users can view some
stale content while they’re waiting for new content to load. We could also leverage this cache if
we built a service-worker-based offline mode. If you have any interest in service workers or ideas
about how Tumblr could use them in the future,
we would love to have you on our team.
- Paul / @blistering-pree
