In the ten years that Tumblr’s been around, a lot has changed in web technology. We’ve kept up, of course, but it’s always been a process of addition, layering one new technology on top of another. And what we were working with—a custom framework built on top of Backbone, messily entangled with a PHP backend and its associated templates—was becoming unmanageable. Our piecemeal conversions to new technologies meant we had thousands of ways posts were rendered (only a moderate exaggeration). And each of those had to be updated individually to support new features or design changes.

It was time to step back, survey the world of web technology, and clean house in a big way. That we could finally test some of the new tech we’ve been itching to use was just a little bonus.

We started by laying out our goals:

• A web client codebase fully separated from the PHP codebase that gets its data from the API in the same way our mobile apps do
• A development environment that’s as painless as possible
• Dramatically improved performance
• Isomorphic rendering
• Robust testing tools
• Built on a framework with a healthy and active community, with some critical mass of adoption

With those goals in mind, we spent the beginning of the year on research - figuring out what kinds of things people were building web apps with these days, tooling around with them ourselves, and trying to assess if they would be right for Tumblr. We landed, eventually, on React, with a Node server (running Express) to make isomorphism as easy as possible. On top of that, we’re using Cosmos for developing components, React Router for routing, and TypeScript to make our lives better in general. (My colleague Paul already wrote about what went into our decision to use TypeScript here.)

As if writing an entirely new stack wasn’t enough, we realized along the way that this was our perfect chance to start deploying containerized applications with Kubernetes, a first for Tumblr. We had never previously deployed a node application to production here, and didn’t have the infrastructure for it, so it was a perfect green field on which to build another new and exciting thing. There’ll be more to come later on Kubernetes.

So where are we now? Well, we’ve launched one page powered by this new app - image pages, like this - with more to come very soon. 

Though it may seem simple, there’s a whole new technological world between you clicking that link and seeing that page. There’s a ton more exciting stuff happening now and still to happen in the future, and we’re looking forward to sharing it here. Wanna get in on the action yourself? Come work with us: https://www.tumblr.com/jobs.

- Robbie Dawson / @idiot

TL;DR

Flux helped bring the complexity of Data Lasso down, replacing messy event bus structure. React helped make the UI more manageable and reduce code duplication. More below on our experience.

Keep reading

We just published v1.1.0 of the tumblr.js API client. We didn’t make too much of a fuss when we released a bigger update in May, but here’s a quick run-down of the bigger updates you may have missed if you haven’t looked at the JS client in a while:

• Method names on the API are named more consistently. For example, blogInfo and blogPosts and blogFollowers rather than blogInfo and posts and followers.
• Customizable API baseUrl. We use this internally when we’re testing new API features during development, and it’s super convenient.
• data64 support, which is handy for those times when you have a base64-encoded image just lying around and you want to post it to Tumblr.
• Support for Promise objects. It’s way more convenient, if you ask me. Regular callbacks are still supported too.
• Linting! We’ve been using eslint internally for a while, so we decided to go for it here too. We’re linting in addition to running mocha tests on pull requests.

Check it out on GitHub and/or npm and star it, if you feel so inclined.

tumblr.js REPL

When we were updating the API client, we were pleasantly suprised to discover a REPL in the codebase. If you don’t know, that’s basically a command-line console that you can use to make API requests and examine the responses. We dusted it off and decided to give it its own repository. It’s also on npm.

If you’re interested in exploring the Tumblr API, but don’t have a particular project in mind yet, it’s a great way to get your feet wet. Try it out!

If you’re an avid user of Tumblr on mobile web, then you might’ve noticed some improvements we made. Bigger font sizes and higher contrast text? Your screen reader actually reads what you hope it would? You’ve guessed it, we’re making Tumblr ✨accessible✨.

Why?

Since we’re rewriting the web, we wanted to make sure we did so with accessibility in mind. I could give you a long description why, but plenty of articles explain better than I can. Put simply: the web should be made useable for everyone.

We began with using the accessibility auditing tool in Google Lighthouse to check the improvements that could be made. Initially, our score wasn’t that great: 62. If you factored in areas that need to be manually checked then our score would have been abysmal. However, we’ve made great strides since then and are on our way to achieving that coveted 💯

We had inaccessible menus and poorly described elements, among other things. Using a tool like VoiceOver or TalkBalk you can see what experiencing Tumblr on mobile web with a screen reader was like. Here’s a gif showing what the mobile web experience on Tumblr was like prior to the changes.

What we did

Some of the more noticeable improvements we made were introducing design changes to increase readability and making improvements following WAI-ARIA guidelines. We’ll walk through a few other changes we made using React.

Visual order on the page follows DOM order

One of the larger changes we made was to revamp modals and popovers (e.g., the post activity screen). Originally we used React Portals but it isn’t always the most friendly for accessibility. Ideally you want to have elements appear in logical DOM order and Portals provides a way to circumvent that. So, no more Portals!

The user’s focus is directed to new content added to the page

Next step was to provide a way to manage focus. We want to a) direct focus to the modal when it’s opened and b) return focus to the element that opened the fullscreen modal. Using React’s lifecycle methods and refs, this is simple enough to implement. In your modal component:

public targetEl: HTMLElement; // The element used to open the modal
public buttonEl: HTMLElement;

public componentDidMount() {
  // We add an event listener to get the element that opened the modal
  document.addEventListener(‘focus’, this.setOriginalTargetEl, true);
  // We set focus to some element inside your modal
  this.buttonEl.focus();
}

public componentWillUnmount() {
  // Return focus to the element that opened the modal
  if (this.targetEl) {
    this.targetEl.focus();
  }
}

public setOriginalTargetEl = event => {
  // Only set it once to get the initial target
  if (!this.targetEl) {
    this.targetEl = event.relatedTarget;
    document.removeEventListener('focus’, this.setOriginalTargetEl, true);
  }
};

public render() {
  return (
    <div>
      <button ref={(el) => this.buttonEl = el}>
        Back
      </button>
      <div>Your content</div>
    </div>
  );
}

This can make navigation a lot easier.

Tada!

Of course, we’re still fine-tuning different elements of the site since accessibility is more than just a number. A lot of these changes will be even more noticeable when the new Tumblr dashboard comes to your desktop. There’s still more to come, so keep your eyes open!

Think there’s a way to make Tumblr more accessible? Hit us up at tumblr.com/jobs and come work with us!

    - Nora Mohamed / @nomo

A Breath of Life: Welcome the Brand New Mobile Web Dashboard

We rebuilt another piece of the site with React

Background

A few months ago my colleague Robbie wrote a post about how the Core Web team is introducing a new web stack built on Docker, Node, and React. It is a beautiful vision, but we had only launched one tiny page on that infrastructure. This spring, we used that new stack to rebuild the mobile web dashboard from the ground up. And then we launched it.

Why we did it

We decided on building the mobile dashboard next for a few reasons. First, the dashboard is one of the most important parts of Tumblr—for many people it is their primary window into the content here. Because of that, most things that we develop inevitably touch the dashboard. It was time to raise the stakes.

The desktop dashboard and the mobile apps have been moving forward at a blistering pace, but have you seen the mobile web dashboard? It was stuck in 2015. Nobody ever made features for it because it was a completely separate codebase. We avoided that issue with the rewrite. Since the new version we’re building is responsive, our new mobile web dashboard will eventually also power the desktop web dashboard experience. This’ll make it much easier to maintain feature parity between the desktop and mobile web experiences.

Finally, we wanted to make something a little bit more complex than the image page, and the mobile web dashboard seemed like a good next step. It allowed us to start making authenticated API calls and figuring out how to pass cookies through our new server to and from our API; to think about laying the groundwork for a totally responsive dashboard, so we can eventually launch this for the desktop; to make a testing ground for progressive web app features like web app manifests; to start rendering posts that are backed by the new post format; and to figure out how to slowly roll out the new stack to a consistent, small set of users. We also had a pretty bad performance bug and learned a whole lot about profiling Node. (We plan on writing more in-depth posts about some of these topics soon.)

It is good

And the rewrite turned out really well. Even with more features like pull-to-refresh, a new activity popover, and modern audio and video players, we sped up our DOM content loaded time by 35%! The new page has a lot of modern web standards in it too like srcsets, flexbox, asynchronous bundle loading, and a web app manifest. You can install the mobile web dashboard as a standalone app now. It was also way faster and simpler to write using React than it was on our old PHP/Backbone stack.

We’re really proud of the new mobile web dashboard. We look forward to bringing even more new features to it in the near future, and launching even more pages on our new web infrastructure.

If you think this is cool too, come work with us!

– Paul Rehkugler (@blistering-pree)

Making a progressive web app with webpack just got a little bit easier

Today we are releasing webpack-web-app-manifest-plugin, which generates an app manifest that shows up in your assets manifest.

I heard you like manifests

Turns out, there are a lot of web things called “manifests”. When talking about web app manifests and assets manifests, sometimes it’s hard to keep track. Buckle up, because we made a webpack plugin that deals with both of these types of manifests.

Web app manifests are JSON files that allow your application to specify the way it should be treated when installed as an application on a mobile device. You may want to specify what the application name and icon should be. Maybe you want to tell the browser to tint some of its UI elements to match the color scheme of your page, or even hide the browser chrome entirely. You can do all of that with a web app manifest.

Assets manifests are JSON files that contain paths to assets that are generated by webpack. They’re generated by plugins such as assets-webpack-plugin. If you add hashes to the end of your filenames to allow cache busting, assets manifests can be very useful. For example, we use our assets manifest to add JavaScript and CSS files to our <script> and <link> tags.

So I put a manifest in your manifest

While we were building our web app manifest, we wanted to be able to add a hash to the file path and <link> to it. So we needed to add it to our assets manifest. Unfortunately, we were unable to find any existing open-source plugins that output the file in the correct way to add it to the app manifest. So, we built webpack-web-app-manifest-plugin.

By default, webpack-web-app-manifest-plugin assumes that you will name your icon files in the format manifest/icon_[square dimension].(png|jpeg|jpg). If you name them using that scheme, you can use this plugin just like this:

// in your webpack config
import AppManifestPlugin from ‘webpack-web-app-manifest-plugin’;

…

plugins: [
  new AppManifestPlugin({
    content: {
      name:'Tumblr’,
      short_name:'Tumblr’,
      background_color:’#36465d’,
    },
    destination: ’/manifest’,
  }),
],

…

// in your page template
const manifest = // however you usually access your asset manifest in code
const appManifestPath = manifest['app-manifest’].json;

<link rel=“manifest” href={appManifestPath} />

If you named your icons with some other naming scheme, you can still add them to the web app manifest, it’s just a little more work. That process is detailed in the README.

Please use it

We’re really proud of the work we’ve done to make web app manifests compatible with asset manifests, which is why we’ve decided to open source it and publish it on npm. Please use it.

If this plugin doesn’t meet your needs, we welcome pull requests. And if you have a passion for progressive web applications, webpack, and open source, join our team!

- Paul Rehkugler (@blistering-pree)

How we wrote our own Service Worker

As we continue the process of reinvigorating Tumblr’s frontend web development, we’re always on the lookout for modern web technologies, especially ones that make our mobile site feel faster and more native. You could have guessed that we are making the mobile dashboard into a progressive app when we open-sourced our webpack plugin to make web app manifests back in August. And you would’ve been right. But to make a high quality progressive web app, you need more than just a web app manifest—you also need a service worker.

What is a service worker?

A service worker is a helper script that a page registers with the browser. After it is registered (some people like to also call it “installed”), the browser periodically checks the script for changes. If any part of the script contents changes, the browser reinstalls the updated script.

Service workers are most commonly used to intercept browser fetches and do various things with them. https://serviceworke.rs has a lot of great ideas about what you can do with service workers, with code examples. We decided to use our service worker to cache some JS, CSS, and font assets when it is installed, and to respond with those assets when the browser fetches any of them.

Using a service worker to precache assets

You might be wondering “why would you want to pre-cache assets when the service worker is installed? Isn’t that the same thing that the browser cache does?” While the browser cache does cache assets after they’re requested, our service worker can cache assets before they’re requested. This greatly speeds up parts of the page that we load in asynchronously, like the notes popover, or blogs that you tap into from the mobile dashboard.

While there are open-source projects that generate service workers to pre-cache your assets (like, for example, sw-precache), we chose to build our own service worker. When I started this project, I didn’t have any idea what service workers were, and I wanted to learn all about them. And what better way to learn about service workers than building one?

How our service worker is built

Because the service worker needs to know about all of the JS, CSS, and font assets in order to pre-cache them, we build a piece of the service worker during our build phase. This part of the service worker changes whenever our assets are updated. During the build step, we take a list of all of the assets that are output, filter them down into just the ones we want to pre-cache, and write them out to an array in a JS file that we call sw.js.

That service worker file importScripts()’s a separate file that contains all of our service worker functionality. All of the service worker functionality is built separately and written in TypeScript, but the file that contains all of our assets is plain JavaScript.

We decided to serve our service worker directly from our node.js app. Our other assets are served using CDNs. Because our CDN servers are often geographically closer to our users, our assets load faster from there than they do from our app. Using CDNs also keeps simple, asset-transfer traffic away from our app, which gives us space us to do more complicated things (like rendering your dashboard with React).

To keep asset traffic that reaches our app to a minimum, we tell our CDNs not to check back for updates to our assets for a long time. This is sometimes referred to as caching with a long TTL (time to live). As we know, cache-invalidation is a tough computer science problem, so we generate unique filenames based on the asset contents each time we build our assets. That way, when we request the new asset, we know that we’re going to get it because we use the new file name.

Because the browser wants to check back in with the service worker script to see if there are any changes, caching it in our CDNs is not a good fit. We would have to figure out how to do cache invalidation for that file, but none of the other assets. By serving that file directly from our node.js application, we get some additional asset-transfer traffic to our application but we think it’s worth it because it avoids all of the issues with caching.

How does it pre-cache assets?

When the service worker is installed, it compares the asset list in sw.js to the list of assets that it has in its cache. If an asset is in the cache, but not listed in sw.js, the asset gets deleted from the cache. If an asset is in sw.js, but not in the service worker cache, we download and cache it. If an asset is in sw.js and in the cache, it hasn’t changed, so we don’t need to do anything.

// in sw.js

self.ASSETS = [

  ‘main.js’,

  ‘notes-popover.js’,

  'favorit.woff’

];

// in service-worker.ts

self.addEventListener('install’, install);

const install = event => event.waitUntil(

  caches.open('tumblr-service-worker-cache’)

    .then(cache => {

      const currentAssetList = self.ASSETS;

      const oldAssets =/* Instead of writing our own array diffing, we use lodash’s */;

      const newAssets =/* differenceBy() to figure out which assets are old and new */;

      return Promise.all([ …oldAssets.map(oldAsset => cache.delete(oldAsset)), cache.addAll(newAssets)]);

  });

);

We launched 🚀

Earlier this month, we launched the service worker to all users of our mobile web dashboard. Our performance instrumentation initially found a small performance regression, but we fixed it. Now our mobile web dashboard load time is about the same as before, but asynchronous bundles on the page load much faster.

We fixed the performance regression by improving performance of the service worker cache. Initially, we naively opened the service worker cache for every request. But now we only open the cache once, when the service worker starts running. Once the cache is opened, we attach listeners for fetch requests, and those closures capture the open cache in their scope.

// before

self.addEventListener('fetch’, handleFetch);

const handleFetch = event =>

  event.respondWith(

    caches.open('tumblr-service-worker-cache’)

      .then(cache => cache.match(request)

      .then(cacheMatch => cacheMatch

        ? Promise.resolve(cacheMatch)

        : fetch(event.request)

      )

    )

  );

// now

caches.open('tumblr-service-worker-cache’)

  .then(cache =>

    self.addEventListener('fetch’, handleFetch(cache));

const handleFetch = openCache => event =>

  event.respondWith(

    openCache.match(request)

      .then(cacheMatch => cacheMatch

        ? Promise.resolve(cacheMatch)

        : fetch(event.request)

      )

  );

Future plans

We have lots of future plans to make the service worker even better than it is now. In addition to pre-emptive caching, we would also like to do reactive caching, like the browser cache does. Every time an asset is requested that we do not already have in our cache, we could cache it. That will help keep the service worker cache fresh between installations.

We would also like to try building an API cache in our service worker, so that users can view some stale content while they’re waiting for new content to load. We could also leverage this cache if we built a service-worker-based offline mode. If you have any interest in service workers or ideas about how Tumblr could use them in the future, we would love to have you on our team.

- Paul / @blistering-pree