Found while fuzzing m-c 20240224-dba8ff89abb9 (--enable-undefined-sanitizer="shift" --enable-fuzzing)

Since this was found by manually enabling the shift UBSan check bugmon is not available. Please ni? if you would like a Pernosco session.

src/modules/fdlibm/src/e_powf.cpp:249:9: runtime error: left shift of negative value -12
    #0 0x7fca24645303 in fdlibm_powf src/modules/fdlibm/src/e_powf.cpp:249:9
    #1 0x7fca192302c6 in mozilla::dom::WebAudioUtils::ConvertDecibelsToLinear(float) src/objdir-ff-ubsan/dist/include/WebAudioUtils.h:53:10
    #2 0x7fca192302c6 in mozilla::AudibilityMonitor::AudibilityMonitor(unsigned int, float) src/dom/media/AudibilityMonitor.h:23:7
    #3 0x7fca192302c6 in mozilla::AudioSink::AudioSink(mozilla::AbstractThread*, mozilla::MediaQueue<mozilla::AudioData>&, mozilla::AudioInfo const&, bool) src/dom/media/mediasink/AudioSink.cpp:47:7
    #4 0x7fca18b022fe in mozilla::MediaDecoderStateMachine::CreateAudioSink()::$_0::operator()() const src/dom/media/MediaDecoderStateMachine.cpp:3546:40
    #5 0x7fca18b022fe in std::_Function_handler<mozilla::UniquePtr<mozilla::AudioSink, mozilla::DefaultDelete<mozilla::AudioSink>> (), mozilla::MediaDecoderStateMachine::CreateAudioSink()::$_0>::_M_invoke(std::_Any_data const&) /home/twsmith/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_function.h:282:9
    #6 0x7fca1923f9e5 in std::function<mozilla::UniquePtr<mozilla::AudioSink, mozilla::DefaultDelete<mozilla::AudioSink>> ()>::operator()() const /home/twsmith/.mozbuild/sysroot-x86_64-linux-gnu/usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_function.h:687:14
    #7 0x7fca1923f9e5 in mozilla::AudioSinkWrapper::SyncCreateAudioSink(mozilla::media::TimeUnit const&) src/dom/media/mediasink/AudioSinkWrapper.cpp:479:36
    #8 0x7fca19240390 in mozilla::AudioSinkWrapper::Start(mozilla::media::TimeUnit const&, mozilla::MediaInfo const&) src/dom/media/mediasink/AudioSinkWrapper.cpp:321:10
    #9 0x7fca19256821 in mozilla::VideoSink::Start(mozilla::media::TimeUnit const&, mozilla::MediaInfo const&) src/dom/media/mediasink/VideoSink.cpp:233:29
    #10 0x7fca1891cbea in mozilla::MediaDecoderStateMachine::StartMediaSink() src/dom/media/MediaDecoderStateMachine.cpp:4135:29
    #11 0x7fca188f8f89 in mozilla::MediaDecoderStateMachine::MaybeStartPlayback() src/dom/media/MediaDecoderStateMachine.cpp:3727:3
    #12 0x7fca18ae3e95 in mozilla::MediaDecoderStateMachine::CompletedState::Step() src/dom/media/MediaDecoderStateMachine.cpp:2747:16
    #13 0x7fca18ae5771 in mozilla::MediaDecoderStateMachine::CompletedState::Enter() src/dom/media/MediaDecoderStateMachine.cpp:2730:5
    #14 0x7fca18ae37b9 in decltype(ReturnTypeHelper(&mozilla::MediaDecoderStateMachine::CompletedState::Enter)) mozilla::MediaDecoderStateMachine::StateObject::CallEnterMemberFunction<mozilla::MediaDecoderStateMachine::CompletedState>(mozilla::MediaDecoderStateMachine::CompletedState*, std::tuple<>&, std::integer_sequence<unsigned long>) src/dom/media/MediaDecoderStateMachine.cpp:258:16
    #15 0x7fca188f6b8b in decltype(ReturnTypeHelper(&mozilla::MediaDecoderStateMachine::CompletedState::Enter)) mozilla::MediaDecoderStateMachine::StateObject::SetState<mozilla::MediaDecoderStateMachine::CompletedState>() src/dom/media/MediaDecoderStateMachine.cpp:301:12
    #16 0x7fca188fc1de in mozilla::MediaDecoderStateMachine::DecodingState::HandleEndOfAudio() src/dom/media/MediaDecoderStateMachine.cpp:3165:5
    #17 0x7fca18b15d4a in mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1::operator()(mozilla::MediaResult const&) const src/dom/media/MediaDecoderStateMachine.cpp
    #18 0x7fca18b15d4a in std::enable_if<TakesArgument<void (mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1::*)(mozilla::MediaResult const&) const>::value, mozilla::detail::MethodTrait<void (mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1::*)(mozilla::MediaResult const&) const>::ReturnType>::type mozilla::MozPromise<RefPtr<mozilla::AudioData>, mozilla::MediaResult, true>::InvokeMethod<mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1, void (mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1::*)(mozilla::MediaResult const&) const, mozilla::MediaResult>(mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1*, void (mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1::*)(mozilla::MediaResult const&) const, mozilla::MediaResult&&) src/objdir-ff-ubsan/dist/include/mozilla/MozPromise.h:654:12
    #19 0x7fca18b15d4a in std::enable_if<!false, void>::type mozilla::MozPromise<RefPtr<mozilla::AudioData>, mozilla::MediaResult, true>::InvokeCallbackMethod<false, mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1, void (mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1::*)(mozilla::MediaResult const&) const, mozilla::MediaResult, RefPtr<mozilla::MozPromise<RefPtr<mozilla::AudioData>, mozilla::MediaResult, true>::Private>>(mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1*, void (mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1::*)(mozilla::MediaResult const&) const, mozilla::MediaResult&&, RefPtr<mozilla::MozPromise<RefPtr<mozilla::AudioData>, mozilla::MediaResult, true>::Private>&&) src/objdir-ff-ubsan/dist/include/mozilla/MozPromise.h:685:5
    #20 0x7fca18b15d4a in mozilla::MozPromise<RefPtr<mozilla::AudioData>, mozilla::MediaResult, true>::ThenValue<mozilla::MediaDecoderStateMachine::RequestAudioData()::$_0, mozilla::MediaDecoderStateMachine::RequestAudioData()::$_1>::DoResolveOrRejectInternal(mozilla::MozPromise<RefPtr<mozilla::AudioData>, mozilla::MediaResult, true>::ResolveOrRejectValue&) src/objdir-ff-ubsan/dist/include/mozilla/MozPromise.h:874:9
    #21 0x7fca1885f7a6 in mozilla::MozPromise<RefPtr<mozilla::AudioData>, mozilla::MediaResult, true>::ThenValueBase::ResolveOrRejectRunnable::Run() src/objdir-ff-ubsan/dist/include/mozilla/MozPromise.h:490:21
    #22 0x7fca10005096 in mozilla::AutoTaskDispatcher::TaskGroupRunnable::Run() src/objdir-ff-ubsan/dist/include/mozilla/TaskDispatcher.h:230:35
    #23 0x7fca0ffe5c69 in mozilla::TaskQueue::Runner::Run() src/xpcom/threads/TaskQueue.cpp:257:20
    #24 0x7fca100438b6 in nsThreadPool::Run() src/xpcom/threads/nsThreadPool.cpp:341:14
    #25 0x7fca10033033 in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1193:16
    #26 0x7fca1003f289 in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:480:10
    #27 0x7fca11ec33a1 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:300:20
    #28 0x7fca11cc5674 in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:370:10
    #29 0x7fca11cc5674 in MessageLoop::RunHandler() src/ipc/chromium/src/base/message_loop.cc:363:3
    #30 0x7fca11cc5674 in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:345:3
    #31 0x7fca1002874f in nsThread::ThreadFunc(void*) src/xpcom/threads/nsThread.cpp:370:10
    #32 0x7fca3cae2ee3 in _pt_root src/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #33 0x55f5de12304a in asan_thread_start(void*) /builds/worker/fetches/llvm-project/compiler-rt/lib/asan/asan_interceptors.cpp:225:31
    #34 0x7fca3d73c608 in start_thread /build/glibc-wuryBv/glibc-2.31/nptl/pthread_create.c:477:8
    #35 0x7fca3d2e7352 in __clone /build/glibc-wuryBv/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

webaudio related.

Karl knows more about fdlib.

https://treeherder.mozilla.org/jobs?repo=try&revision=f6210feca4366ceec13063cd75c4d585445e911f&selectedTaskRun=EH6udhCzR7SLbPZMHBCXLw.0 indicates that a similar bug exists in the double version, fdlibm_pow().

https://pernos.co/debug/PtvIZsiScEjBk5TjMTYDaw/index.html

https://pernos.co/debug/NOeV3flTHzWtdbThOjMVcA/index.html doesn't have all of the key function optimized away.

I'll submit a patch upstream. The code is assuming that a signed left shift is equivalent to an unsigned left shift of the two's complement representation.

https://github.com/freebsd/freebsd-src/pull/1137

I'm guessing this is unlikely to cause much trouble IRL, but if a fix is important to get a clean build, then let me know and i'll put together a local patch.