This issue is hit frequently by fuzzers, please prioritize it appropriately.

Assertion failure: !aWM.IsOrthogonalTo(wm), at src/layout/generic/nsIFrameInlines.h:157

#0 0x7f5319f7c6d1 in nsIFrame::SynthesizeBaselineBOffsetFromContentBox(mozilla::WritingMode, mozilla::BaselineSharingGroup) const src/layout/generic/nsIFrameInlines.h:157:3
#1 0x7f531a03da56 in GetRowBaseline src/layout/tables/nsTableRowFrame.cpp:402:29
#2 0x7f531a03da56 in nsTableFrame::GetNaturalBaselineBOffset(mozilla::WritingMode, mozilla::BaselineSharingGroup, int*) const::$_1::operator()(nsTableRowGroupFrame*, nsTableRowFrame*) const src/layout/tables/nsTableFrame.cpp:3632:41
#3 0x7f531a03d845 in nsTableFrame::GetNaturalBaselineBOffset(mozilla::WritingMode, mozilla::BaselineSharingGroup, int*) const src/layout/tables/nsTableFrame.cpp:3639:22
#4 0x7f531a03d573 in nsTableFrame::GetLogicalBaseline(mozilla::WritingMode) const src/layout/tables/nsTableFrame.cpp:3612:8
#5 0x7f531a051a9f in nsTableWrapperFrame::GetLogicalBaseline(mozilla::WritingMode) const src/layout/tables/nsTableWrapperFrame.cpp:45:15
#6 0x7f5319e23f0b in nsLayoutUtils::GetFirstLinePosition(mozilla::WritingMode, nsIFrame const*, nsLayoutUtils::LinePosition*) src/layout/base/nsLayoutUtils.cpp:5691:36
#7 0x7f5319e23bb6 in nsLayoutUtils::GetFirstLinePosition(mozilla::WritingMode, nsIFrame const*, nsLayoutUtils::LinePosition*) src/layout/base/nsLayoutUtils.cpp:5737:11
#8 0x7f5319e23a56 in nsLayoutUtils::GetFirstLineBaseline(mozilla::WritingMode, nsIFrame const*, int*) src/layout/base/nsLayoutUtils.cpp:5656:8
#9 0x7f531a025f9c in nsTableCellFrame::GetCellBaseline() const src/layout/tables/nsTableCellFrame.cpp:681:7
#10 0x7f531a046e84 in nsTableRowFrame::ReflowChildren(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsTableFrame&, nsReflowStatus&) src/layout/tables/nsTableRowFrame.cpp:861:51
#11 0x7f531a047ecb in nsTableRowFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableRowFrame.cpp:1019:3
#12 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#13 0x7f531a04b605 in nsTableRowGroupFrame::ReflowChildren(nsPresContext*, mozilla::ReflowOutput&, mozilla::TableRowGroupReflowInput&, nsReflowStatus&, bool*) src/layout/tables/nsTableRowGroupFrame.cpp:408:7
#14 0x7f531a04f42b in nsTableRowGroupFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableRowGroupFrame.cpp:1389:3
#15 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#16 0x7f531a037895 in nsTableFrame::ReflowChildren(mozilla::TableReflowInput&, nsReflowStatus&, nsIFrame*&, nsOverflowAreas&) src/layout/tables/nsTableFrame.cpp:2989:7
#17 0x7f531a035ab7 in nsTableFrame::ReflowTable(mozilla::ReflowOutput&, mozilla::ReflowInput const&, int, nsIFrame*&, nsReflowStatus&) src/layout/tables/nsTableFrame.cpp:2052:3
#18 0x7f531a034dd6 in nsTableFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableFrame.cpp:1838:5
#19 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#20 0x7f531a055938 in nsTableWrapperFrame::OuterDoReflowChild(nsPresContext*, nsIFrame*, mozilla::ReflowInput const&, mozilla::ReflowOutput&, nsReflowStatus&) src/layout/tables/nsTableWrapperFrame.cpp:780:3
#21 0x7f531a056af7 in nsTableWrapperFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableWrapperFrame.cpp:933:3
#22 0x7f5319e9377a in nsBlockReflowContext::ReflowBlock(mozilla::LogicalRect const&, bool, nsCollapsingMargin&, int, bool, nsLineBox*, mozilla::ReflowInput&, nsReflowStatus&, mozilla::BlockReflowInput&) src/layout/generic/nsBlockReflowContext.cpp:294:11
#23 0x7f5319e8f5e9 in nsBlockFrame::ReflowBlockFrame(mozilla::BlockReflowInput&, nsLineList_iterator, bool*) src/layout/generic/nsBlockFrame.cpp:3839:11
#24 0x7f5319e8d346 in nsBlockFrame::ReflowLine(mozilla::BlockReflowInput&, nsLineList_iterator, bool*) src/layout/generic/nsBlockFrame.cpp:3175:5
#25 0x7f5319e880c3 in nsBlockFrame::ReflowDirtyLines(mozilla::BlockReflowInput&) src/layout/generic/nsBlockFrame.cpp:2712:7
#26 0x7f5319e8425b in nsBlockFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsBlockFrame.cpp:1372:3
#27 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#28 0x7f531a026f63 in nsTableCellFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableCellFrame.cpp:910:3
#29 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#30 0x7f531a046d9e in nsTableRowFrame::ReflowChildren(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsTableFrame&, nsReflowStatus&) src/layout/tables/nsTableRowFrame.cpp:817:9
#31 0x7f531a047ecb in nsTableRowFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableRowFrame.cpp:1019:3
#32 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#33 0x7f531a04b605 in nsTableRowGroupFrame::ReflowChildren(nsPresContext*, mozilla::ReflowOutput&, mozilla::TableRowGroupReflowInput&, nsReflowStatus&, bool*) src/layout/tables/nsTableRowGroupFrame.cpp:408:7
#34 0x7f531a04f42b in nsTableRowGroupFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableRowGroupFrame.cpp:1389:3
#35 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#36 0x7f531a037895 in nsTableFrame::ReflowChildren(mozilla::TableReflowInput&, nsReflowStatus&, nsIFrame*&, nsOverflowAreas&) src/layout/tables/nsTableFrame.cpp:2989:7
#37 0x7f531a035ab7 in nsTableFrame::ReflowTable(mozilla::ReflowOutput&, mozilla::ReflowInput const&, int, nsIFrame*&, nsReflowStatus&) src/layout/tables/nsTableFrame.cpp:2052:3
#38 0x7f531a034dd6 in nsTableFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableFrame.cpp:1838:5
#39 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#40 0x7f531a055938 in nsTableWrapperFrame::OuterDoReflowChild(nsPresContext*, nsIFrame*, mozilla::ReflowInput const&, mozilla::ReflowOutput&, nsReflowStatus&) src/layout/tables/nsTableWrapperFrame.cpp:780:3
#41 0x7f531a056af7 in nsTableWrapperFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/tables/nsTableWrapperFrame.cpp:933:3
#42 0x7f5319e9377a in nsBlockReflowContext::ReflowBlock(mozilla::LogicalRect const&, bool, nsCollapsingMargin&, int, bool, nsLineBox*, mozilla::ReflowInput&, nsReflowStatus&, mozilla::BlockReflowInput&) src/layout/generic/nsBlockReflowContext.cpp:294:11
#43 0x7f5319e8f5e9 in nsBlockFrame::ReflowBlockFrame(mozilla::BlockReflowInput&, nsLineList_iterator, bool*) src/layout/generic/nsBlockFrame.cpp:3839:11
#44 0x7f5319e8d346 in nsBlockFrame::ReflowLine(mozilla::BlockReflowInput&, nsLineList_iterator, bool*) src/layout/generic/nsBlockFrame.cpp:3175:5
#45 0x7f5319e880c3 in nsBlockFrame::ReflowDirtyLines(mozilla::BlockReflowInput&) src/layout/generic/nsBlockFrame.cpp:2712:7
#46 0x7f5319e8425b in nsBlockFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsBlockFrame.cpp:1372:3
#47 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#48 0x7f5319eab3a6 in nsCanvasFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsCanvasFrame.cpp:757:5
#49 0x7f5319eac010 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, mozilla::WritingMode const&, mozilla::LogicalPoint const&, nsSize const&, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1081:14
#50 0x7f5319eee178 in nsHTMLScrollFrame::ReflowScrolledFrame(mozilla::ScrollReflowInput*, bool, bool, mozilla::ReflowOutput*) src/layout/generic/nsGfxScrollFrame.cpp:756:3
#51 0x7f5319eeeaa5 in nsHTMLScrollFrame::ReflowContents(mozilla::ScrollReflowInput*, mozilla::ReflowOutput const&) src/layout/generic/nsGfxScrollFrame.cpp:880:3
#52 0x7f5319ef2917 in nsHTMLScrollFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/nsGfxScrollFrame.cpp:1278:3
#53 0x7f5319e79b38 in nsContainerFrame::ReflowChild(nsIFrame*, nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, int, int, nsIFrame::ReflowChildFlags, nsReflowStatus&, nsOverflowContinuationTracker*) src/layout/generic/nsContainerFrame.cpp:1121:14
#54 0x7f5319e795da in mozilla::ViewportFrame::Reflow(nsPresContext*, mozilla::ReflowOutput&, mozilla::ReflowInput const&, nsReflowStatus&) src/layout/generic/ViewportFrame.cpp:339:7
#55 0x7f5319d85c74 in mozilla::PresShell::DoReflow(nsIFrame*, bool, mozilla::OverflowChangedTracker*) src/layout/base/PresShell.cpp:9640:11
#56 0x7f5319d8f36e in mozilla::PresShell::ProcessReflowCommands(bool) src/layout/base/PresShell.cpp:9813:24
#57 0x7f5319d8ea6d in mozilla::PresShell::DoFlushPendingNotifications(mozilla::ChangesToFlush) src/layout/base/PresShell.cpp:4225:11
#58 0x7f5319d57fb1 in FlushPendingNotifications /builds/worker/workspace/obj-build/dist/include/mozilla/PresShell.h:1404:5
#59 0x7f5319d57fb1 in nsRefreshDriver::Tick(mozilla::layers::BaseTransactionId<mozilla::VsyncIdType>, mozilla::TimeStamp) src/layout/base/nsRefreshDriver.cpp:2243:20
#60 0x7f5319d5fd91 in TickDriver src/layout/base/nsRefreshDriver.cpp:374:13
#61 0x7f5319d5fd91 in mozilla::RefreshDriverTimer::TickRefreshDrivers(mozilla::layers::BaseTransactionId<mozilla::VsyncIdType>, mozilla::TimeStamp, nsTArray<RefPtr<nsRefreshDriver> >&) src/layout/base/nsRefreshDriver.cpp:353:7
#62 0x7f5319d5fc7c in mozilla::RefreshDriverTimer::Tick(mozilla::layers::BaseTransactionId<mozilla::VsyncIdType>, mozilla::TimeStamp) src/layout/base/nsRefreshDriver.cpp:368:5
#63 0x7f5319d5f228 in RunRefreshDrivers src/layout/base/nsRefreshDriver.cpp:829:5
#64 0x7f5319d5f228 in mozilla::VsyncRefreshDriverTimer::RefreshDriverVsyncObserver::TickRefreshDriver(mozilla::layers::BaseTransactionId<mozilla::VsyncIdType>, mozilla::TimeStamp) src/layout/base/nsRefreshDriver.cpp:747:16
#65 0x7f5319d5eb30 in mozilla::VsyncRefreshDriverTimer::RefreshDriverVsyncObserver::NotifyParentProcessVsync() src/layout/base/nsRefreshDriver.cpp:649:7
#66 0x7f5319d5e5a9 in mozilla::VsyncRefreshDriverTimer::RefreshDriverVsyncObserver::NotifyVsync(mozilla::VsyncEvent const&) src/layout/base/nsRefreshDriver.cpp:570:9
#67 0x7f531a11d667 in mozilla::layout::VsyncChild::RecvNotify(mozilla::VsyncEvent const&) src/layout/ipc/VsyncChild.cpp:55:16
#68 0x7f53163d0305 in mozilla::layout::PVsyncChild::OnMessageReceived(IPC::Message const&) /builds/worker/workspace/obj-build/ipc/ipdl/PVsyncChild.cpp:187:54
#69 0x7f5316181b4d in mozilla::ipc::PBackgroundChild::OnMessageReceived(IPC::Message const&) /builds/worker/workspace/obj-build/ipc/ipdl/PBackgroundChild.cpp:6268:32
#70 0x7f5315e3ebde in mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) src/ipc/glue/MessageChannel.cpp:2150:25
#71 0x7f5315e3b39f in mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) src/ipc/glue/MessageChannel.cpp:2074:9
#72 0x7f5315e3c7a6 in mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) src/ipc/glue/MessageChannel.cpp:1922:3
#73 0x7f5315e3d3cb in mozilla::ipc::MessageChannel::MessageTask::Run() src/ipc/glue/MessageChannel.cpp:1953:13
#74 0x7f53155361df in mozilla::RunnableTask::Run() src/xpcom/threads/TaskController.cpp:245:16
#75 0x7f531553485a in mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:515:26
#76 0x7f5315533904 in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) src/xpcom/threads/TaskController.cpp:374:15
#77 0x7f5315533ab7 in mozilla::TaskController::ProcessPendingMTTask(bool) src/xpcom/threads/TaskController.cpp:171:36
#78 0x7f5315539706 in operator() src/xpcom/threads/TaskController.cpp:85:37
#79 0x7f5315539706 in mozilla::detail::RunnableFunction<mozilla::TaskController::InitializeInternal()::$_3>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:577:5
#80 0x7f531554a957 in nsThread::ProcessNextEvent(bool, bool*) src/xpcom/threads/nsThread.cpp:1197:14
#81 0x7f531555069a in NS_ProcessNextEvent(nsIThread*, bool) src/xpcom/threads/nsThreadUtils.cpp:513:10
#82 0x7f5315e444c6 in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:87:21
#83 0x7f5315db63b3 in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:334:10
#84 0x7f5315db62cd in RunHandler src/ipc/chromium/src/base/message_loop.cc:327:3
#85 0x7f5315db62cd in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:309:3
#86 0x7f5319ab0308 in nsBaseAppShell::Run() src/widget/nsBaseAppShell.cpp:137:27
#87 0x7f531b2abaa3 in XRE_RunAppShell() src/toolkit/xre/nsEmbedFunctions.cpp:913:20
#88 0x7f5315e45289 in mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) src/ipc/glue/MessagePump.cpp:237:9
#89 0x7f5315db63b3 in MessageLoop::RunInternal() src/ipc/chromium/src/base/message_loop.cc:334:10
#90 0x7f5315db62cd in RunHandler src/ipc/chromium/src/base/message_loop.cc:327:3
#91 0x7f5315db62cd in MessageLoop::Run() src/ipc/chromium/src/base/message_loop.cc:309:3
#92 0x7f531b2ab688 in XRE_InitChildProcess(int, char**, XREChildData const*) src/toolkit/xre/nsEmbedFunctions.cpp:744:34
#93 0x564462dd6647 in content_process_main src/browser/app/../../ipc/contentproc/plugin-container.cpp:56:28
#94 0x564462dd6647 in main src/browser/app/nsBrowserApp.cpp:304:18

A Pernosco session is available here: https://pernos.co/debug/KczS-zvs7xcyQD9gbocwGQ/index.html

Bugmon Analysis:
Verified bug as reproducible on mozilla-central 20201027155819-6df4ac02675f.
The bug appears to have been introduced in the following build range:

Start: a77ca5bd6b7c49099c672e103771ca2bf077cc40 (20200514211114)
End: 5a0a960a8d555795d4d1db432090fec007850716 (20200515093304)
Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=a77ca5bd6b7c49099c672e103771ca2bf077cc40&tochange=5a0a960a8d555795d4d1db432090fec007850716

Running a quick mozregression locally between 20200514 and 20200515 yields bug 312971, which unprefixed :read-write. If changing the pseudo to :-moz-read-write, the testcase can trigger assertion in earlier builds, too.

The fuzzers are hitting this frequently, marking as fuzzblocker.

The severity field for this bug is relatively low, S3. However, the bug has 3 duplicates.
:dholbert, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

This bug prevents fuzzing from making progress; however, it has low severity. It is important for fuzz blocker bugs to be addressed in a timely manner (see here why?).
:dholbert, could you increase the severity?

For more information, please visit auto_nag documentation.

Here's the frametree at the moment of the assertion failure, from the pernosco trace.

At that point, in nsIFrame::SynthesizeBaselineBOffsetFromContentBox, this is (const nsTableCellFrame *) 0x55d22b36e280

From the frame tree dump, it looks like everything is v-lrl-ltr up until this point:

                      TableCell(menu)(0)@55d22b36dd80 parent=55d22b36dcb8 (x=0, y=0, w=0, h=0) [content=55d22b2637e0] [cs=55d22b33dc28:-moz-table-cell] <
                        Block(menu)(0)@55d22b36de48 parent=55d22b36dd80 (x=0, y=0, w=0, h=0) wm=v-lr-ltr logical-size=((0 x 0)) [content=55d22b2637e0] [cs=55d22b2dcb58:-moz-cell-content] <

Note that the menu/ins elements and the contenteditable attribute are irrelevant here -- this same issue reproduces with a simpler testcase, essentially just with some nested tables with a disagreeing writing-mode.

Going up the backtrace (in comment 0), it looks like things become unreasonable in nsTableFrame::GetNaturalBaselineBOffset, where we defer to the rows. In a case where the writing mode is orthogonal, it might not make sense to do that. We probably want to do some simpler fallback at that point or earlier...

(In reply to Daniel Holbert [:dholbert] from comment #10)

Note that the menu/ins elements and the contenteditable attribute are irrelevant here -- this same issue reproduces with a simpler testcase, essentially just with some nested tables with a disagreeing writing-mode.

Here's a testcase to illustrate this^. This triggers the same assertion failure.

Hm.. So the asserting call to nsIFrame::SynthesizeBaselineBOffsetFromBorderBox is from nsTableRowFrame::GetRowBaseline, which, by the time it makes the call, recognizes that there is no baseline in the row, and tries to make one up by using the content box edge. The conversion of border/padding contributions all seem correct.
The assert has been there for a while, but I think bug 1631941, specifically D71884, fixed it as mentioned above.

Note, the table still looks broken - this is a known issue - bug 1785333 filed for tracking.

So... It seems like removing the assert will do?

(In reply to David Shin[:dshin] from comment #13)

So... It seems like removing the assert will do?

I think the question here is, does it make sense that we're asking a table-row to synthesize a baseline here (the nsTableRowFrame::GetRowBaseline call that you mentioned), with respect to a writing-mode whose baseline-positioning is actually orthogonal (i.e. not related to) the layout of the table-row?

This seems analogous to a scenario where e.g. we have an inline-block with several lines of text, and we placed the inline-block into an orthogonal flow, and we (hypothetically) determined its baseline by asking its first line to synthesize one for us. (We don't actually do this; I'm just presenting it as an analogy.) This would not make sense, since baseline alignment is happening in the wrong axis for any one line-of-text in this orthogonal-to-its-surroundings inline-block to have anything meaningful to tell us.

So: probably we should be bailing out (i.e. falling back to a "synthesize a baseline based on our border-box" behavior) higher up, in one of the nsTableFrame methods.

Hmmm. Thought it'd be "symmetric" to have the table's baseline be the first in flow, but what you said makes sense - I suppose the baseline of the line in the cell no longer matters if it's orthogonal.
Further, Flexbox has the precedent on this.

I think https://drafts.csswg.org/css-align-3/#baseline-export is one fairly-authoritative section on baseline alignment.

It does say...

The first/last baseline set of a table box is the first/last baseline set of its first/last row.

... but I think that section is implicitly about the "baseline set" for the table's own writing mode. And I think it has no "baseline set" in the other axis. So for the testcase here, this further-down spec text would be the relevant thing to follow:

If a box that participates in baseline alignment has no baseline set, then its alignment baseline is synthesized according to the rules of the formatting context in which it participates.

Obviously at this point in the code (in e.g. nsTableFrame baseline methods) we don't have direct knowledge of what "the rules of the formatting context in which [we're participating]" should be; but I suspect just using the border-box edge is probably reasonable for now. (Though it'd be worth testing interop and e.g. seeing if that seems to do the right thing for display:inline-table in an inline context, as well as for a table inside of a flex container with "align-items: baseline", with margin/border/padding.)

(In reply to David Shin[:dshin] from comment #13)

Hm.. So the asserting call to nsIFrame::SynthesizeBaselineBOffsetFromBorderBox is [...]

Correction: it looks like the asserting method here is really SynthesizeBaselineBOffsetFromContentBox (not BorderBox).

SynthesizeBaselineBOffsetFromBorderBox is actually fine to call in an orthogonal writing-mode; it handles that properly here:
https://searchfox.org/mozilla-central/rev/bce4b0be48762f15a1ad2ab52b6bdd42f5b0f0e1/layout/generic/nsIFrameInlines.h#147-151

You likely want to add a call to that method from one of the table functions here.

Probably we want to be handling this at the level of nsTableWrapperFrame (the outermost table box).

And it looks like that already has a IsContainLayout() fallback codepath that maybe would be appropriate to share here? i.e. maybe we can just add || aWritingMode.IsOrthogonalTo(GetWritingMode())) to this condition:
https://searchfox.org/mozilla-central/rev/bce4b0be48762f15a1ad2ab52b6bdd42f5b0f0e1/layout/tables/nsTableWrapperFrame.cpp#29-35

nscoord nsTableWrapperFrame::GetLogicalBaseline(
    WritingMode aWritingMode) const {
  if (StyleDisplay()->IsContainLayout()) {
    // We have no baseline. Fall back to the inherited impl which is
    // appropriate for this situation.
    return nsContainerFrame::GetLogicalBaseline(aWritingMode);
  }

https://hg.mozilla.org/mozilla-central/rev/4dcf1336bf2b

Verified bug as fixed on rev mozilla-central 20221117214129-f85cd6ba1889.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.