This bug was initially created as a clone of bug 1655584 and tracks the popups not rendering correctly issue for XWayland.

This doesn't seem to happen with EGL. Given we are enabling it on nightly soon, that will probably be the path we take to fix this.

While I was trying to reproduce a different bug, I discovered that this bug also occurs with the OpenGL Compositor. So this doesn't seem to be a WebRender issue at all.

(In reply to Viktor Jägersküpper from comment #2)

While I was trying to reproduce a different bug, I discovered that this bug also occurs with the OpenGL Compositor. So this doesn't seem to be a WebRender issue at all.

Just to be sure, can you confirm this doesn't happen with the EGL backend? I.e. MOZ_X11_EGL=1 env var?

Short answer: I couldn't reproduce this bug with the EGL backend (neither with WebRender nor with the OpenGL Compositor).

Long answer:
Using the OpenGL compositor I found a regression window for this bug with mozregression. Using WebRender for this doesn't make sense because it wasn't enabled for these popups all the time (only since bug 1574746). The result was:

Last good revision: 05331fb8f5338974b913217bc67815df25a32e86 (2018-11-12)
First bad revision: 24e87b02707bee36e1e98eb37c94fbaf3834e898 (2018-11-13)
Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=05331fb8f5338974b913217bc67815df25a32e86&tochange=24e87b02707bee36e1e98eb37c94fbaf3834e898

I couldn't narrow it down further with mozregression because the inbound/autoland builds from that time don't exist anymore (they are only stored for one year if I am not mistaken). But there was a Nightly build in between those two revisions and I found that it was "good", so I created the following shorter pushlog by hand:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=f6df375b86987b2772067a61873ebfe3a98c353a&tochange=24e87b02707bee36e1e98eb37c94fbaf3834e898

If anyone could look into that list and point to the probable regressing change, that would be great. I still have a Debian 9 based build environment available and could build two revisions to test if that change is indeed the "bad" one. Doing a proper bisection with Mercurial would probably take several weeks for me because of my slow hardware and I don't know how much time I can spend on this in the next weeks. But I would rather be certain that this bug is really fixed by using EGL than just closing it as "WORKSFORME", although this bug should of course not be a blocker for WebRender on Linux from my point of view if EGL is enabled together with WebRender.

For completeness here are my testing results for various cases:
Nightly build 2018-11-12 (only tested with OpenGL compositor):
X11 + GLX: good
XWayland + GLX: good

Nightly build 2018-11-13 (only tested with OpenGL compositor):
X11 + GLX: bad (only for a few milliseconds) [1]
XWayland + GLX: bad

Latest Nightly build:
X11 + GLX: good (OpenGL), good (WebRender)
X11 + EGL: good (OpenGL), good (WebRender)
XWayland + GLX: bad (OpenGL), bad (WebRender)
XWayland + EGL: good (OpenGL) [2], good (WebRender)

[1] It looks like this bug is only visible for a few milliseconds, it was hard to see. Perhaps addressed by bug 1655584, I didn't check that.
[2]: Sometimes (very hard to reproduce) the popups are not displayed correctly, but that's a different bug. I am not sure if it's worth reporting it because the OpenGL compositor will never be used in an official release for Linux as far as I know. There is however bug 1587060, but I wasn't able to reproduce this different bug with the Wayland backend because I observed two kind of crashes while I was testing that case, which I should report first.

I also observed that using XWayland makes Firefox a bit slower in comparison to the conventional X11/X.org-Server with some noticeable annoyances concerning the menu navigation. I don't know if that's expected because of the underlying graphics software architecture, so let me know if I should report this. Maybe I only notice it because of my slow CPU.

• MOZ_ENABLE_WAYLAND (=Mesa): Popups on Wayland have their own problems: bug 1633989
• MOZ_X11_EGL/Xwayland (=Mesa): bug 1650246 became much better and was only reproducible with llvmpipe last time I tested it.
• this bug: GLX/XWayland (=Mesa): I didn't test GLX/Xwayland much because EGL is already used by many users interested in hardware video decoding, fast WebGL, partial present. Also, GLX has some perf problem (bug 1672414) that I haven't encountered with EGL when it still used GLX vsync. GLX vsync can be used until EGL vsync implementation is completed (bug 1640779). It was disabled for EGL/Mesa without apparent technical reason while narrowing down an EGL/X11/proprietary Nvidia bug: bug 1669275 is about reenabling GLX vsync/framerate detection for EGL/Mesa.

Thanks for the list! I am mostly testing with default graphics settings except for WebRender. That's why I was "lucky" finding this one even for the OpenGL compositor because I dared to try the Wayland backend again (with WebRender) and after falling back to OpenGL compositor (bug 1549099) I got this bug (and then the crashes).

I just checked if [1] in comment 4 is addressed by bug 1655584, but I got bad news: It is not fixed as I wrote above, I can still reproduce it with both the OpenGL compositor and WebRender, although that's really hard. I can reproduce it more easily if I use mozregression however, don't know why. Once I even got a black box that didn't disappear until I moved the mouse. So my table for this bug (using the latest Nightly build) is now:

X11 + GLX: bad (OpenGL), bad (WebRender) (almost always only for a few milliseconds in both cases)
X11 + EGL: good (OpenGL), good (WebRender)
XWayland + GLX: bad (OpenGL), bad (WebRender)
XWayland + EGL: good (OpenGL), good (WebRender)

This means that this bug is not only for XWayland and not only for WebRender and I should really bisect with Mercurial if noone can point to a change in the pushlog above.

As per https://bugzilla.mozilla.org/show_bug.cgi?id=1656211#c37, With Firefox 85 build 20201118215158 and EGL the top left corner transparency issue doesn't happen, but this issue still happens as before: https://youtu.be/jghvl0R31LQ
Window protocol shows as XWayland and i'm using MOZ_X11_EGL

For some reason the animation of the tracker details popup is much choppier when i'm not recording a screencast, but it's still visible in the previous comment.

(In reply to Lyubomir from comment #8)

For some reason the animation of the tracker details popup is much choppier when i'm not recording a screencast, but it's still visible in the previous comment.

This is likely because of bug 1677892 - can you try again with gfx.webrender.max-partial-present-rects:0?

I didn't notice big difference when i changed that preference. However the difference with what can be seen on my phone and via screencast is bigger - https://www.youtube.com/watch?v=kDJgjXtvNnQ (it'll upload in a minute or two).

Sorry for bothering here, but am i supposed to be running on a Gnome Wayland session with MOZ_X11_EGL=1? Or this is only for X.org and i should instead use MOZ_ENABLE_WAYLAND=1? I just found out the glorious Arch Wiki article (https://wiki.archlinux.org/index.php/firefox#Hardware_video_acceleration)

(In reply to Lyubomir from comment #11)

Sorry for bothering here, but am i supposed to be running on a Gnome Wayland session with MOZ_X11_EGL=1? Or this is only for X.org and i should instead use MOZ_ENABLE_WAYLAND=1? I just found out the glorious Arch Wiki article (https://wiki.archlinux.org/index.php/firefox#Hardware_video_acceleration)

In a Wayland session you can do both - in both cases the EGL backend will be used and video acceleration should work. MOZ_X11_EGL=1 will run on Xwayland though, so it's likely less smooth. It can be useful though if you run into Wayland specific issues :)

I finished the bisection with Mercurial, this is the first bad revision:
changeset: 445998:f82c7330018b
user: Hiroyuki Ikezoe <[email protected]>
date: Tue Nov 13 10:23:20 2018 +0000
summary: Bug 1504929 - Further optimizations for RestyleManager::AddLayerChangesForAnimations.. r=birtles,sotaro

See here: https://hg.mozilla.org/mozilla-central/rev/f82c7330018b3ad8cf263e1b7fa7eef311d90870

Although this is fixed for me by using MOZ_X11_EGL=1, it would be great to hear if there is anything that can/should be fixed nevertheless.

Is it worth asking for Hiroyuki's help?

After bug 1710855 I can no longer reproduce this bug with regular toolbar popups (e.g. library), but still with extension popups (e.g. HTTPS Everywhere). I also checked the current release (90), and I couldn't reproduce this bug with extension popups using the default settings (Software-WebRender, no EGL).

Can the fix from bug 1710855 be applied for extension popups, too?

(In reply to Viktor Jägersküpper from comment #15)

Can the fix from bug 1710855 be applied for extension popups, too?

The "fix" was enforcing software WebRender. :-(