Steps to reproduce:

When accessing web applications that utilize a Service Worker (SW) for resource caching, a "NetworkError when attempting to fetch resource" error occurs exclusively in Firefox's Responsive Design Mode. This error appears after the SW is installed and attempts to fetch resources via caches.match. Notably, the issue is not present in the Firefox mobile application or when using Firefox desktop outside of Responsive Design Mode. The problem has been observed on multiple sites, including https://app.clickup.com/login and https://www.voodoodreams.com/en/, which work as expected in other browsers' responsive modes and in Firefox under normal browsing conditions.

• 0pen Firefox and enable Responsive Design Mode (Tools > Web Developer > Responsive Design Mode).
• Navigate to https://app.clickup.com/login or https://www.voodoodreams.com/en/.
• verify Service worker is installed correctly.
• refresh the website
• Observe assets are failing (CSS, JS or images) with the following kind of errors:
  Failed to load ‘https://app-cdn.clickup.com/assets/images/loading/wave.gif’. A ServiceWorker passed a promise to FetchEvent.respondWith() that rejected with ‘Error: Response not Ok (fetchAndCacheOnce): request for https://app-cdn.clickup.com/assets/images/loading/wave.gif?ngsw-domain=app.clickup.com returned response 504 Gateway Timeout’.
• debugging the Service worker i could see the error is on the caches.match getting "NetworkError when attempting to fetch resource."
• Disable Responsive Design Mode and reload the page to see that it loads correctly.

Actual results:

assets and script files don't load correctly when the Service worker is installed

Expected results:

assets and script files all loaded correctly through the Service worker

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Networking' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Some notes: the service worker on both websites does not install immediately. Not sure if we need to wait or interact with the page a bit, but it takes some time until it shows up.

I managed to reproduce the issue, but only when RDM is passing a custom user agent string, eg

Mozilla/5.0 (Linux; Android 8; Pixel 3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.120 Mobile Safari/537.36

I don't get a fully blank page in my case, but a few resources fail to load with the error mentioned in the bug Description.
I tried to use the same settings on Chrome (UA, screen size etc...) and everything seems to load fine.

Hi Tom,

Did you already encounter websites where service worker requests only fail in RDM with Firefox, but work fine in other browsers RDM modes, or in Firefox Mobile? Maybe you already have a bug filed for a similar issue?

Given that it doesn't fail when setting an empty user agent in RDM, I was wondering if that could relates to that particular bit of RDM.

We end up triggering this particular code:
https://searchfox.org/mozilla-central/rev/b73676a106c1655030bb876fd5e0a6825aee6044/netwerk/protocol/http/HttpBaseChannel.cpp#545-552
which adds a User-agent header to all network requests.

But I couldn't really understand how this could cause the various issues we see in the console:

Blocage d’une requête multiorigine (Cross-Origin Request) : la politique « Same Origin » ne permet pas de consulter la ressource distante située sur https://app-cdn.clickup.com/fr-FR/runtime.fd15602909f7f6ee.js. Raison : échec de la requête CORS. Code d’état : (null).

Impossible de charger «  ». Un service worker a intercepté la requête et a rencontré une erreur inattendue.   ngsw-worker-entry.js:1473:13

HTTP 504 when loading https://app-cdn.clickup.com/fr-FR/styles.f75be9f801c70619.css from the Service Worker

Pardon my delay here. No, I haven't run into this yet, and am not aware of any other bugs filed on it.

The severity field is not set for this bug.
:nchevobbe, could you have a look please?

For more information, please visit BugBot documentation.

we'll discuss it in our next triage meeting

clearing the ni so it appears in our dashboard

(In reply to Alexandre Poirot [:ochameau] from comment #4)

Given that it doesn't fail when setting an empty user agent in RDM, I was wondering if that could relates to that particular bit of RDM.

We end up triggering this particular code:
https://searchfox.org/mozilla-central/rev/b73676a106c1655030bb876fd5e0a6825aee6044/netwerk/protocol/http/HttpBaseChannel.cpp#545-552
which adds a User-agent header to all network requests.

But I couldn't really understand how this could cause the various issues we see in the console:

Blocage d’une requête multiorigine (Cross-Origin Request) : la politique « Same Origin » ne permet pas de consulter la ressource distante située sur https://app-cdn.clickup.com/fr-FR/runtime.fd15602909f7f6ee.js. Raison : échec de la requête CORS. Code d’état : (null).

Impossible de charger «  ». Un service worker a intercepté la requête et a rencontré une erreur inattendue.   ngsw-worker-entry.js:1473:13

HTTP 504 when loading https://app-cdn.clickup.com/fr-FR/styles.f75be9f801c70619.css from the Service Worker

Valentin: do you know if setting a custom user agent for a request at https://searchfox.org/mozilla-central/rev/b73676a106c1655030bb876fd5e0a6825aee6044/netwerk/protocol/http/HttpBaseChannel.cpp#545-552 could cause issues in general with requests coming from service workers? Otherwise it might be something coming from the server?

I don't think it "should" cause issues, but for some reason it does.
I tried to track down the flow of headers into the service worker and found this part of the code:

https://searchfox.org/mozilla-central/rev/7a8904165618818f73ab7fc692ace4a57ecd38c9/netwerk/protocol/http/nsHttpChannel.cpp#10365-10378

// Some APIs, like fetch(), allow content to set non-standard headers.
// Normally these APIs are responsible for copying these headers across
// redirects.  In the e10s parent-side intercept case, though, we currently
// "hide" the internal redirect to the InterceptedHttpChannel.  So the
// fetch() API does not have the opportunity to move headers over.
// Therefore, we do it automatically here.
//
// Once child-side interception is removed and the internal redirect no
// longer needs to be "hidden", then this header copying code can be
// removed.
nsCOMPtr<nsIHttpHeaderVisitor> visitor =
    new CopyNonDefaultHeaderVisitor(intercepted);
rv = VisitNonDefaultRequestHeaders(visitor);
NS_ENSURE_SUCCESS(rv, rv);

I find removing this code block seems to make the bug go away 🙂
Andrew, I think child-side interception has been removed from the tree, right?

(In reply to Valentin Gosu [:valentin] (he/him) from comment #10)

Andrew, I think child-side interception has been removed from the tree, right?

Re-capping https://phabricator.services.mozilla.com/D210478#7233068 and https://phabricator.services.mozilla.com/D210478#7234156 child-side intercept code was removed, but this is explicitly parent intercept logic; more context there.

Some additional investigation about the redirection hasn't turned up anything particularly useful. Although I did rediscover bug 1704877 that has https://phabricator.services.mozilla.com/D112675 about making the intercepted channel have its own id, which does raise the general meta issue around interception where:

• We extracted InterceptedHttpChannel out of nsHttpChannel for complexity reasons; it allows decoupling interception logic from the core http channel logic.
• Telling the content process about the redirect would seem to necessarily involve more IPC and potentially a new HttpChannelChild being set up, with the redirect from the non-intercepted HttpChannelChild to the yes-intercepted HttpChannelChild, I think?

Note that I did some additional digging which I'll make in the next comment because it's somewhat of a separate thought.

So the 504 gateway error is synthetic, being generated by the ServiceWorker for a fetch that throws:

      async safeFetch(req) {
        try {
          return await this.scope.fetch(req);
        } catch (err) {
          this.debugger.log(err, `Driver.fetch(${ req.url })`);
          return this.adapter.newResponse(null, {
            status: 504,
            statusText: 'Gateway Timeout'
          });
        }
      }

I had some trouble setting breakpoints/logpoints via the debugger via the prettified source (although it seemed to figure out how to map them back if I reloaded? but they just didn't work) so I hooked the synthetic response generation via:

savedResponse = Response;
Response = function(...args) {
console.trace();
console.log("resp", ...args);
return new savedResponse(...args);}

and then I set a breakpoint in that code I'd added by clicking on the "debugger eval code:2.9" in the console to get the source view and set a breakpoint on the console.trace.

The request that is throwing looks like:

​​bodyUsed: false
​​cache: "reload"
​​credentials: "same-origin"
​​destination: "font"
​​headers: Headers
​​integrity: ""
​​method: "GET"
​​mode: "cors"
​​redirect: "follow"
​​referrer: "https://fonts.googleapis.com/"
​​referrerPolicy: "strict-origin-when-cross-origin"
​​signal: AbortSignal
​​url: "https://fonts.gstatic.com/s/roboto/v30/KFOlCnqEu92Fr1MmSU5fBBc4.woff2"

Manually calling [...req.headers.entries()] gets us:

Array(3) [ (2) […], (2) […], (2) […] ]
​0: Array [ "accept", "application/font-woff2;q=1.0,application/font-woff;q=0.9,*/*;q=0.8" ]
​1: Array [ "accept-language", "en-US,en;q=0.5" ]
​2: Array [ "user-agent", "Mozilla/5.0 (Linux; Android 11; SAMSUNG SM-G973U) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/14.2 Chrome/87.0.4280.141 Mobile Safari/537.36" ]
​length: 3

Note that there is logic that intentionally propagates the headers from the fetch event that was received:

      newRequestWithMetadata(url, options) {
        return this.adapter.newRequest(url, {
          headers: options.headers
        });
      }

The browser will help explain what's going on if we run the equivalent fetch from an uncontrolled voodoodreams page (use ctrl-shift-refresh to bypass serviceworker interception). Run:

frh = new Headers() 
frh.append("user-agent", "Mozilla/5.0 (Linux; Android 11; SAMSUNG SM-G973U) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/14.2 Chrome/87.0.4280.141 Mobile Safari/537.36")
fr = new Request("https://fonts.gstatic.com/s/roboto/v30/KFOlCnqEu92Fr1MmSU5fBBc4.woff2", { mode: "cors", headers: frh})
z = fetch(fr) 

We get errors:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://fonts.gstatic.com/s/roboto/v30/KFOlCnqEu92Fr1MmSU5fBBc4.woff2. (Reason: header ‘user-agent’ is not allowed according to header ‘Access-Control-Allow-Headers’ from CORS preflight response).

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://fonts.gstatic.com/s/roboto/v30/KFOlCnqEu92Fr1MmSU5fBBc4.woff2. (Reason: CORS request did not succeed). Status code: (null).

Uncaught (in promise) TypeError: NetworkError when attempting to fetch resource.

So the problem here, as I understand it is a somewhat emergent behavior from:

• We are surfacing the synthetic user-agent header to the intercepted channel.
• That header is then propagated through a "cors" request.
• The server does not allow-list that header.
• The fetch throws.
• The ServiceWorker turns it into a 504 response.

Presumably we should avoid propagating the synthetic header to the intercepted channel. Maybe the visitor could be made smart enough to understand that the user-agent override is synthetic and should not match the eFilterSkipDefault filter at (https://searchfox.org/mozilla-central/rev/f60bb10a5fe6936f9e9f9e8a90d52c18a0ffd818/netwerk/protocol/http/HttpBaseChannel.cpp#2041-2042)?:

return mRequestHead.VisitHeaders(visitor,
                                 nsHttpHeaderArray::eFilterSkipDefault);

Thank you for the detailed analysis, Andrew. That made it really easy to figure out that the problem here is the code setting the userAgentOverride doesn't use the correct flag

https://searchfox.org/mozilla-central/rev/b476ffaef761ff85c012e2d93050cf444ff7be34/netwerk/protocol/http/HttpBaseChannel.cpp#538,552

HttpBaseChannel::SetDocshellUserAgentOverride() {
...
  nsresult rv = SetRequestHeader("User-Agent"_ns, utf8CustomUserAgent, false);

Like we do here:

https://searchfox.org/mozilla-central/rev/b476ffaef761ff85c012e2d93050cf444ff7be34/netwerk/protocol/http/nsHttpChannel.cpp#6229-6234

if (!LoadIsUserAgentHeaderModified()) {
  rv = mRequestHead.SetHeader(
      nsHttp::User_Agent,
      gHttpHandler->UserAgent(nsContentUtils::ShouldResistFingerprinting(
          this, RFPTarget::HttpUserAgent)),
      false, nsHttpHeaderArray::eVarietyRequestEnforceDefault);

Passing eVarietyRequestEnforceDefault should resolve this issue. I'll update the patch ASAP.

https://hg.mozilla.org/mozilla-central/rev/2c96a82a6b09