Found while fuzzing m-c 20221118-dfe42e743b9c (--enable-address-sanitizer --enable-fuzzing)

To reproduce via Grizzly Replay:

$ pip install fuzzfetch grizzly-framework
$ python -m fuzzfetch -a --fuzzing -n firefox
$ python -m grizzly.replay ./firefox/firefox testcase.html

==16851==ERROR: AddressSanitizer: stack-overflow on address 0x7ffcb8b11f08 (pc 0x561eea64a6cb bp 0x7ffcb8b12730 sp 0x7ffcb8b11f00 T0)
    #0 0x561eea64a6cb in StackTrace /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:53:45
    #1 0x561eea64a6cb in BufferedStackTrace /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:113:26
    #2 0x561eea64a6cb in malloc /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #3 0x561eea68dec5 in moz_xmalloc /gecko/memory/mozalloc/mozalloc.cpp:52:15
    #4 0x7fe3e340930b in operator new /builds/worker/workspace/obj-build/dist/include/mozilla/cxxalloc.h:33:10
    #5 0x7fe3e340930b in NS_NewPlainTextSerializer(nsIContentSerializer**) /gecko/dom/serializers/nsPlainTextSerializer.cpp:84:38
    #6 0x7fe3e46588a9 in CreatePlainTextSerializer(nsID const&, void**) /gecko/layout/build/nsLayoutModule.cpp:167:1
    #7 0x7fe3daee224f in mozilla::xpcom::CreateInstanceImpl(mozilla::xpcom::ModuleID, nsID const&, void**) /builds/worker/workspace/obj-build/xpcom/components/StaticComponents.cpp:11820:27
    #8 0x7fe3daf1d7bb in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsID const&, void**) /gecko/xpcom/components/nsComponentManager.cpp:840:19
    #9 0x7fe3daf24ea6 in CallCreateInstance /gecko/xpcom/components/nsComponentManagerUtils.cpp:128:43
    #10 0x7fe3daf24ea6 in nsCreateInstanceByContractID::operator()(nsID const&, void**) const /gecko/xpcom/components/nsComponentManagerUtils.cpp:169:21
    #11 0x7fe3dad8a4b1 in nsCOMPtr_base::assign_from_helper(nsCOMPtr_helper const&, nsID const&) /gecko/xpcom/base/nsCOMPtr.cpp:109:7
    #12 0x7fe3e33fbf80 in operator= /builds/worker/workspace/obj-build/dist/include/nsCOMPtr.h:787:5
    #13 0x7fe3e33fbf80 in nsDocumentEncoder::EncodeToStringWithMaxLength(unsigned int, nsTSubstring<char16_t>&) /gecko/dom/serializers/nsDocumentEncoder.cpp:1384:17
    #14 0x7fe3de2ea24a in mozilla::dom::Selection::ToStringWithFormat(nsTSubstring<char16_t> const&, unsigned int, int, nsTSubstring<char16_t>&, mozilla::ErrorResult&) /gecko/dom/base/Selection.cpp:377:17
    #15 0x7fe3de2e9c8e in mozilla::dom::Selection::Stringify(nsTSubstring<char16_t>&, mozilla::dom::Selection::FlushFrames) /gecko/dom/base/Selection.cpp:339:3
    #16 0x7fe3e3dbe40b in mozilla::AccessibleCaretManager::StringifiedSelection() const /gecko/layout/base/AccessibleCaretManager.cpp:837:16
    #17 0x7fe3e3dc118c in mozilla::AccessibleCaretManager::DispatchCaretStateChangedEvent(mozilla::dom::CaretChangedReason, nsPoint const*) /gecko/layout/base/AccessibleCaretManager.cpp:1477:31
    #18 0x7fe3e3db88ff in mozilla::AccessibleCaretManager::UpdateCaretsForSelectionMode(mozilla::EnumSet<mozilla::AccessibleCaretManager::UpdateCaretsHint, unsigned char> const&) /gecko/layout/base/AccessibleCaretManager.cpp:376:5
    #19 0x7fe3e3db771a in mozilla::AccessibleCaretManager::UpdateCarets(mozilla::EnumSet<mozilla::AccessibleCaretManager::UpdateCaretsHint, unsigned char> const&) /gecko/layout/base/AccessibleCaretManager.cpp:210:7
    #20 0x7fe3e3dbda03 in mozilla::AccessibleCaretManager::OnScrollPositionChanged() /gecko/layout/base/AccessibleCaretManager.cpp:763:7
    #21 0x7fe3e77d38ee in nsDocShell::NotifyScrollObservers() /gecko/docshell/base/nsDocShell.cpp:2452:12
    #22 0x7fe3e77d3bf6 in non-virtual thunk to nsDocShell::NotifyScrollObservers() /gecko/docshell/base/nsDocShell.cpp
    #23 0x7fe3e40a5dae in mozilla::ScrollFrameHelper::ScrollToImpl(nsPoint, nsRect const&, mozilla::ScrollOrigin, mozilla::ScrollTriggeredByScript) /gecko/layout/generic/nsGfxScrollFrame.cpp:3450:15
    #24 0x7fe3e40a6d39 in mozilla::ScrollFrameHelper::CompleteAsyncScroll(nsRect const&, mozilla::UniquePtr<mozilla::ScrollSnapTargetIds, mozilla::DefaultDelete<mozilla::ScrollSnapTargetIds>>, mozilla::ScrollOrigin) /gecko/layout/generic/nsGfxScrollFrame.cpp:2503:3
    #25 0x7fe3e409e4ec in mozilla::ScrollFrameHelper::ScrollToWithOrigin(nsPoint, nsRect const*, mozilla::ScrollFrameHelper::ScrollOperationParams&&) /gecko/layout/generic/nsGfxScrollFrame.cpp:2661:5
    #26 0x7fe3e40c6ba4 in mozilla::ScrollFrameHelper::CurPosAttributeChanged(nsIContent*, bool) /gecko/layout/generic/nsGfxScrollFrame.cpp:6004:5
    #27 0x7fe3e4432bda in nsScrollbarFrame::AttributeChanged(int, nsAtom*, int) /gecko/layout/xul/nsScrollbarFrame.cpp:105:15
    #28 0x7fe3e3e04fca in mozilla::RestyleManager::AttributeChanged(mozilla::dom::Element*, int, nsAtom*, int, nsAttrValue const*) /gecko/layout/base/RestyleManager.cpp:3493:19
    #29 0x7fe3e3e04afe in mozilla::PresShell::AttributeChanged(mozilla::dom::Element*, int, nsAtom*, int, nsAttrValue const*) /gecko/layout/base/PresShell.cpp:4553:37
    #30 0x7fe3de2782c8 in operator() /gecko/dom/base/MutationObservers.cpp:146:3
    #31 0x7fe3de2782c8 in Notify<(IsRemoval)0, (ShouldAssert)1, (lambda at /builds/worker/checkouts/gecko/dom/base/MutationObservers.cpp:146:3), (lambda at /builds/worker/checkouts/gecko/dom/base/MutationObservers.cpp:146:3)> /gecko/dom/base/MutationObservers.cpp:97:5
    #32 0x7fe3de2782c8 in mozilla::dom::MutationObservers::NotifyAttributeChanged(mozilla::dom::Element*, int, nsAtom*, int, nsAttrValue const*) /gecko/dom/base/MutationObservers.cpp:148:3
    #33 0x7fe3de185522 in mozilla::dom::Element::SetAttrAndNotify(int, nsAtom*, nsAtom*, nsAttrValue const*, nsAttrValue&, nsIPrincipal*, unsigned char, bool, bool, bool, mozilla::dom::Document*, mozAutoDocUpdate const&) /gecko/dom/base/Element.cpp:2662:5
    #34 0x7fe3de17e564 in mozilla::dom::Element::SetAttr(int, nsAtom*, nsAtom*, nsTSubstring<char16_t> const&, nsIPrincipal*, bool) /gecko/dom/base/Element.cpp:2502:10
    #35 0x7fe3e3f2a793 in SetAttr /builds/worker/workspace/obj-build/dist/include/mozilla/dom/Element.h:955:12
    #36 0x7fe3e3f2a793 in SetAttr /builds/worker/workspace/obj-build/dist/include/mozilla/dom/Element.h:951:12
    #37 0x7fe3e3f2a793 in nsSetAttrRunnable::Run() /gecko/layout/base/nsLayoutUtils.cpp:7912:20
    #38 0x7fe3dddd896a in nsContentUtils::RemoveScriptBlocker() /gecko/dom/base/nsContentUtils.cpp:5918:17
    #39 0x7fe3e40cb3ae in ~nsAutoScriptBlocker /builds/worker/workspace/obj-build/dist/include/nsContentUtils.h:3541:28
    #40 0x7fe3e40cb3ae in mozilla::ScrollFrameHelper::ReflowFinished() /gecko/layout/generic/nsGfxScrollFrame.cpp:6901:1
    #41 0x7fe3e3e00455 in mozilla::PresShell::HandlePostedReflowCallbacks(bool) /gecko/layout/base/PresShell.cpp:4175:21
    #42 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #43 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #44 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #45 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #46 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #47 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #48 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #49 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #50 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #51 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #52 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #53 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #54 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #55 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #56 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #57 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #58 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #59 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #60 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #61 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #62 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #63 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #64 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #65 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #66 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #67 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #68 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #69 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #70 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #71 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #72 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #73 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #74 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #75 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #76 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #77 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #78 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #79 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #80 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #81 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #82 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #83 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #84 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #85 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #86 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #87 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #88 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #89 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #90 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #91 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #92 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #93 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #94 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #95 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #96 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    #97 0x7fe3e3e01b2a in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) /gecko/layout/base/PresShell.cpp:4413:11
    #98 0x7fe3e3df21e7 in mozilla::PresShell::DidDoReflow(bool) /gecko/layout/base/PresShell.cpp:9456:3
    #99 0x7fe3e3e2a0dc in mozilla::PresShell::ProcessReflowCommands(bool) /gecko/layout/base/PresShell.cpp:9854:7
    #100 0x7fe3e3e01b2a in DoFlushLayout /gecko/layout/base/PresShell.cpp:9904:10
    ...

Unable to reproduce bug 1801374 using build mozilla-central 20221118094451-dfe42e743b9c. Without a baseline, bugmon is unable to analyze this bug.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.

I cannot seem to reproduce this with on Linux (Which it has to be since it's an asan build) mozregression -B asan-debug --launch dfe42e743b9c, even with multiple refreshes.

Since bugmon is unable to reproduce either, going to close it as WORKSFORME, and monitor for any new instance.

The testcase reproduces for me but requires the accessibility.force_disabled pref. I've uploaded a new version of the testcase that includes this pref. To reproduce the issue using this testcase, please use the following command:

$ python -m grizzly.replay ./firefox/firefox testcase.zip

(TIL that accessibility.force_disabled=-1 will force-enable a11y... That is rather confusing)

Verified bug as reproducible on mozilla-central 20221129214749-e46a721b2af4.
Unable to bisect testcase (Testcase reproduces on start build!):

Start: 89800efd9e5cfcf0146767961a63d5c4e2a86e2c (20211201050507)
End: dfe42e743b9cf77ffab15b03a99db902811a3121 (20221118094451)
BuildFlags: BuildFlags(asan=True, tsan=False, debug=False, fuzzing=True, coverage=False, valgrind=False, no_opt=False, fuzzilli=False, nyx=False)

Testcase crashes using the initial build (mozilla-central 20221118094451-dfe42e743b9c) but not with tip (mozilla-central 20221230213139-0254637cfb2f.)

The bug appears to have been fixed in the following build range:

Start: 9b5c52e4d5ce3d83895213c0f5ffcdce5c46d220 (20221229085942)
End: d3dd3b74e57bf40acc0373cf07e5ca4713cea70e (20221229201843)
Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=9b5c52e4d5ce3d83895213c0f5ffcdce5c46d220&tochange=d3dd3b74e57bf40acc0373cf07e5ca4713cea70e

tsmith, can you confirm that the above bisection range is responsible for fixing this issue?
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.

I cannot reproduce the issue with the attached test case. The fix range provided by bugmon tracks with when the fuzzers stopped reporting the issue.