Testcase found while fuzzing mozilla-central rev 9790289bfed7 (built with: --enable-address-sanitizer --enable-fuzzing).

Testcase can be reproduced using the following commands:

$ pip install fuzzfetch grizzly-framework
$ python -m fuzzfetch --build 9790289bfed7 --asan --fuzzing -n firefox
$ python -m grizzly.replay ./firefox/firefox testcase.zip

Assertion failure: !mImpl->IsConnected(), at /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293

    =================================================================
    ==248851==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x7fad074267e3 bp 0x7faceca4cce0 sp 0x7faceca4ccd0 T35)
    ==248851==The signal is caused by a WRITE memory access.
    ==248851==Hint: address points to the zero page.
        #0 0x7fad074267e3 in mozilla::Mirror<nsMainThreadPtrHandle<nsIPrincipal> >::~Mirror() /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293:5
        #1 0x7fad07a0613a in mozilla::DecodedStream::~DecodedStream() /dom/media/mediasink/DecodedStream.cpp:471:1
        #2 0x7fad07a0649d in mozilla::DecodedStream::~DecodedStream() /dom/media/mediasink/DecodedStream.cpp:469:33
        #3 0x7fad07a123b9 in Release /dom/media/mediasink/MediaSink.h:40:3
        #4 0x7fad07a123b9 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:50:40
        #5 0x7fad07a123b9 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:381:36
        #6 0x7fad07a123b9 in ~RefPtr /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:81:7
        #7 0x7fad07a123b9 in mozilla::VideoSink::~VideoSink() /dom/media/mediasink/VideoSink.cpp:92:1
        #8 0x7fad07a1251d in mozilla::VideoSink::~VideoSink() /dom/media/mediasink/VideoSink.cpp:88:25
        #9 0x7fad0742b777 in Release /dom/media/mediasink/MediaSink.h:40:3
        #10 0x7fad0742b777 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:50:40
        #11 0x7fad0742b777 in Release /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:381:36
        #12 0x7fad0742b777 in assign_assuming_AddRef /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:69:7
        #13 0x7fad0742b777 in RefPtr<mozilla::MediaSink>& RefPtr<mozilla::MediaSink>::operator=<mozilla::MediaSink>(already_AddRefed<mozilla::MediaSink>&&) /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:206:5
        #14 0x7fad0743748e in mozilla::MediaDecoderStateMachine::ResumeMediaSink() /dom/media/MediaDecoderStateMachine.cpp:3901:14
        #15 0x7fad0760108d in applyImpl<mozilla::MediaDecoderStateMachine, void (mozilla::MediaDecoderStateMachine::*)()> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1147:12
        #16 0x7fad0760108d in apply<mozilla::MediaDecoderStateMachine, void (mozilla::MediaDecoderStateMachine::*)()> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1153:12
        #17 0x7fad0760108d in mozilla::detail::RunnableMethodImpl<mozilla::MediaDecoderStateMachine*, void (mozilla::MediaDecoderStateMachine::*)(), true, (mozilla::RunnableKind)0>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1200:13
        #18 0x7fad013926a0 in mozilla::AutoTaskDispatcher::TaskGroupRunnable::Run() /builds/worker/workspace/obj-build/dist/include/mozilla/TaskDispatcher.h:227:35
        #19 0x7fad013a1cbd in mozilla::TaskQueue::Runner::Run() /xpcom/threads/TaskQueue.cpp:208:20
        #20 0x7fad013c9b3f in nsThreadPool::Run() /xpcom/threads/nsThreadPool.cpp:303:14
        #21 0x7fad013bbbfb in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1142:16
        #22 0x7fad013c63dc in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:467:10
        #23 0x7fad0286a0d5 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:300:20
        #24 0x7fad026eb371 in RunInternal /ipc/chromium/src/base/message_loop.cc:331:10
        #25 0x7fad026eb371 in RunHandler /ipc/chromium/src/base/message_loop.cc:324:3
        #26 0x7fad026eb371 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:306:3
        #27 0x7fad013b5488 in nsThread::ThreadFunc(void*) /xpcom/threads/nsThread.cpp:390:10
        #28 0x7fad1f431cce in _pt_root /nsprpub/pr/src/pthreads/ptthread.c:201:5
        #29 0x7fad23548608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477:8
        #30 0x7fad23110292 in clone /build/glibc-eX1tMB/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
    
    AddressSanitizer can not provide additional info.
    SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/obj-build/dist/include/mozilla/StateMirroring.h:293:5 in mozilla::Mirror<nsMainThreadPtrHandle<nsIPrincipal> >::~Mirror()
    Thread T35 (MediaDe~hine #1) created by T0 (Web Content) here:
        #0 0x5575e7eb10cc in pthread_create /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cpp:205:3
        #1 0x7fad1f421d34 in _PR_CreateThread /nsprpub/pr/src/pthreads/ptthread.c:458:14
        #2 0x7fad1f41320e in PR_CreateThread /nsprpub/pr/src/pthreads/ptthread.c:533:12
        #3 0x7fad013b7e9a in nsThread::Init(nsTSubstring<char> const&) /xpcom/threads/nsThread.cpp:602:18
        #4 0x7fad013c3dff in nsThreadManager::NewNamedThread(nsTSubstring<char> const&, unsigned int, nsIThread**) /xpcom/threads/nsThreadManager.cpp:576:12
        #5 0x7fad013cf0d1 in NS_NewNamedThread(nsTSubstring<char> const&, nsIThread**, already_AddRefed<nsIRunnable>, unsigned int) /xpcom/threads/nsThreadUtils.cpp:163:57
        #6 0x7fad013c8740 in NS_NewNamedThread /xpcom/threads/nsThreadUtils.cpp:155:10
        #7 0x7fad013c8740 in nsThreadPool::PutEvent(already_AddRefed<nsIRunnable>, unsigned int) /xpcom/threads/nsThreadPool.cpp:118:17
        #8 0x7fad013ca9bf in nsThreadPool::Dispatch(already_AddRefed<nsIRunnable>, unsigned int) /xpcom/threads/nsThreadPool.cpp:354:5
        #9 0x7fad0139f93b in mozilla::TaskQueue::DispatchLocked(nsCOMPtr<nsIRunnable>&, unsigned int, mozilla::AbstractThread::DispatchReason) /xpcom/threads/TaskQueue.cpp:68:26
        #10 0x7fad013d3043 in mozilla::TaskQueue::Dispatch(already_AddRefed<nsIRunnable>, mozilla::AbstractThread::DispatchReason) /builds/worker/workspace/obj-build/dist/include/mozilla/TaskQueue.h:87:14
        #11 0x7fad0139215a in mozilla::AutoTaskDispatcher::DispatchTaskGroup(mozilla::UniquePtr<mozilla::AutoTaskDispatcher::PerThreadTaskGroup, mozilla::DefaultDelete<mozilla::AutoTaskDispatcher::PerThreadTaskGroup> >) /builds/worker/workspace/obj-build/dist/include/mozilla/TaskDispatcher.h:275:20
        #12 0x7fad013915a6 in mozilla::AutoTaskDispatcher::~AutoTaskDispatcher() /builds/worker/workspace/obj-build/dist/include/mozilla/TaskDispatcher.h:121:7
        #13 0x7fad013936d9 in reset /builds/worker/workspace/obj-build/dist/include/mozilla/Maybe.h:639:19
        #14 0x7fad013936d9 in mozilla::XPCOMThreadWrapper::MaybeFireTailDispatcher() /xpcom/threads/AbstractThread.cpp:195:23
        #15 0x7fad0138f46c in AfterProcessNextEvent /xpcom/threads/AbstractThread.cpp:133:5
        #16 0x7fad0138f46c in non-virtual thunk to mozilla::XPCOMThreadWrapper::AfterProcessNextEvent(nsIThreadInternal*, bool) /xpcom/threads/AbstractThread.cpp
        #17 0x7fad013bb726 in nsThread::ProcessNextEvent(bool, bool*) /xpcom/threads/nsThread.cpp:1172:3
        #18 0x7fad013c63dc in NS_ProcessNextEvent(nsIThread*, bool) /xpcom/threads/nsThreadUtils.cpp:467:10
        #19 0x7fad02868a24 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /ipc/glue/MessagePump.cpp:107:5
        #20 0x7fad026eb371 in RunInternal /ipc/chromium/src/base/message_loop.cc:331:10
        #21 0x7fad026eb371 in RunHandler /ipc/chromium/src/base/message_loop.cc:324:3
        #22 0x7fad026eb371 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:306:3
        #23 0x7fad0911f417 in nsBaseAppShell::Run() /widget/nsBaseAppShell.cpp:137:27
        #24 0x7fad0d95419f in XRE_RunAppShell() /toolkit/xre/nsEmbedFunctions.cpp:917:20
        #25 0x7fad026eb371 in RunInternal /ipc/chromium/src/base/message_loop.cc:331:10
        #26 0x7fad026eb371 in RunHandler /ipc/chromium/src/base/message_loop.cc:324:3
        #27 0x7fad026eb371 in MessageLoop::Run() /ipc/chromium/src/base/message_loop.cc:306:3
        #28 0x7fad0d95336e in XRE_InitChildProcess(int, char**, XREChildData const*) /toolkit/xre/nsEmbedFunctions.cpp:749:34
        #29 0x5575e7efa60d in content_process_main(mozilla::Bootstrap*, int, char**) /browser/app/../../ipc/contentproc/plugin-container.cpp:57:28
        #30 0x5575e7efaa3d in main /browser/app/nsBrowserApp.cpp:327:18
        #31 0x7fad230150b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
    
    ==248851==ABORTING

Bugmon Analysis
Verified bug as reproducible on mozilla-central 20211004215121-cc37b1400a58.
The bug appears to have been introduced in the following build range:

Start: 1130661c79c222fb1acd29b7ec5dc5202cdd0d2d (20201211213049)
End: a3add3f43cbcbfdf053d901689882975479842eb (20201211175457)
Pushlog: https://hg.mozilla.org/mozilla-unified/pushloghtml?fromchange=1130661c79c222fb1acd29b7ec5dc5202cdd0d2d&tochange=a3add3f43cbcbfdf053d901689882975479842eb

The Mirror value in DecodedStream[1] is supposed to be disconnected at Stop()[2] or Shutdown()[3]. From the stack it looks like the object is VideoSink::mAudioSink, which should be disconnected in VideoSink::Stop() or VideoSink::Shutdown().

Also, I couldn't see any related changes in the mentioned commits that possibly causes it.

Jason, when try to run the replay command locally on my Mac I got the following error. Do you know how to fix it? Thanks a lot!

[2021-10-11 01:29:37] Starting Grizzly Replay
[2021-10-11 01:29:37] Ignoring: log-limit, timeout
[2021-10-11 01:29:37] Using time limit: 15s, timeout: 30s
[2021-10-11 01:29:37] Repeat: 1, Minimum crashes: 1, Relaunch 1
[2021-10-11 01:29:42] Running test (1/1)...
[2021-10-11 01:29:46] Test case was not served
[2021-10-11 01:29:46] Delayed startup failure detected
[2021-10-11 01:29:46] Result: Different signature: Hit MOZ_CRASH(Attempting to connect to non-local address!) at /builds/worker/checkouts/gecko/netwerk/base/nsSocketTransport2.cpp:1236 (8b52d4d6:2aeaeb0f)
[2021-10-11 01:29:46] Failed to reproduce results
[2021-10-11 01:29:46] Shutting down...
[2021-10-11 01:29:46] Done.

[1] https://searchfox.org/mozilla-central/source/dom/media/mediasink/DecodedStream.h#113
[2] https://searchfox.org/mozilla-central/source/dom/media/mediasink/DecodedStream.cpp#623
[3] https://searchfox.org/mozilla-central/source/dom/media/mediasink/DecodedStream.cpp#640

John, that error typically indicates that you are using a prefs file that does not have the network.proxy.autoconfig_url set. The attached testcase however, includes a pref file with this set so I'm not entirely sure why you're seeing that error. Just out of curiosity, are you seeing this with a local build or one from Fuzzfetch?

ResumeMediaSink is triggered asynchrously, and it assumes that we have already stopped the media sink in the SuspendMedia call

However, resumming the sink is an asychrous task, it's possible that we start the sink before running that task. If so, we would destroy the started sink without properly calling its stop method.

BTW I wonder if it's possible to let grizzly.replay to print the debug log while it's running the test case?

:alwu, since this bug contains a bisection range, could you fill (if possible) the regressed_by field?
For more information, please visit auto_nag documentation.

:alwu, currently there is no way to enable grizzly.replay to report stdout/stderr while the testcase is running. The only other option available is --dump which will print the contents of stderr/stdout to the console once the browser is closed.

I found that --dump isn't defined in my grizzly.replay, but I can use -l to define the log path and works well.

This situation happens when we try to resume the media sink which is already running.

The resume calls are called from HTMLMediaElement [1][2] and media element doesn't know the internal status of the decoder. When [2] is called after [1], then it triggers resuming twice in MediaDecoderStateMachine.

Therefore, we should check mIsMediaSinkSuspended first to see if we really need to perform the suspend/resume on the media sink.

[1] https://searchfox.org/mozilla-central/rev/483cfde5a54f6c1cd94c6295564993aeb4f10980/dom/html/HTMLMediaElement.cpp#6391
[2] https://searchfox.org/mozilla-central/rev/483cfde5a54f6c1cd94c6295564993aeb4f10980/dom/html/HTMLMediaElement.cpp#4344

(In reply to Alastor Wu [:alwu] from comment #9)

I found that --dump isn't defined in my grizzly.replay, but I can use -l to define the log path and works well.

Ah, my apologies. That argument actually belongs to ffpuppet which is what grizzly uses to drive firefox.

https://hg.mozilla.org/mozilla-central/rev/33082d9250aa
https://hg.mozilla.org/mozilla-central/rev/ac232c892ea5

Bugmon Analysis
Verified bug as fixed on rev mozilla-central 20211101215926-08eb1047d841.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.