Skip to content

arangodb 3.12.5 fails to start on GNU/Linux (Assertion '__b != memory_order_release' failed.) #21811

Open
@sergiodj

Description

@sergiodj

Hi,

I'm currently testing arangodb v3.12.5 on Wolfi, and I'm seeing a crash when trying to start the arangod service.

Here's how I'm building the package:

  - name: Configure the build
    runs: |
      cmake \
        -DSTATIC_EXECUTABLES=OFF \
        -DOpenMP_C_FLAGS="-fopenmp" \
        -DOpenMP_CXX_FLAGS="-fopenmp" \
        -DOpenMP_C_LIB_NAMES="omp" \
        -DOpenMP_CXX_LIB_NAMES="omp" \
        -DOpenMP_omp_LIBRARY="/usr/lib/libomp.so" \
        -DOpenMP_ROOT="/usr/lib" \
        -DCMAKE_C_COMPILER=clang-19 \
        -DCMAKE_CXX_COMPILER=clang++ \
        -DCMAKE_C_COMPILER_AR="/usr/bin/llvm-ar" \
        -DCMAKE_CXX_COMPILER_AR="/usr/bin/llvm-ar" \
        -DCMAKE_C_COMPILER_RANLIB="/usr/bin/llvm-ranlib" \
        -DCMAKE_CXX_COMPILER_RANLIB="/usr/bin/llvm-ranlib" \
        -DCMAKE_EXE_LINKER_FLAGS="-Wl,--build-id=sha1 -fno-stack-protector -fuse-ld=lld -ljitterentropy" \
        -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=lld" \
        -DUSE_LIBUNWIND=true -DUSE_PRECOMPILED_V8=false \
        -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
        -DCMAKE_C_FLAGS="-fPIC" \
        -DCMAKE_CXX_FLAGS="-fPIC" \
        -DSNAPPY_INCLUDE_DIR=/usr/include \
        -DJEMALLOC_INCLUDE=/usr/include \
        -DZLIB_INCLUDE_DIR=/usr/include \
        -DUNWIND_INCLUDE_DIR=/usr/include \
        -DOPENSSL_ROOT_DIR=/usr/lib \
        --preset community

  - name: Build
    runs: |
      cmake \
        --build \
        --preset community \
        --target arangodb \
        -- -j$(nproc) VERBOSE=1

Here's the command I'm using to start it:

sudo -u arangodb arangod -c /etc/arangodb3/arangod.conf --default-language-check=false --server.endpoint tcp://127.0.0.1:8529 '--server.authentication=false' /tmp/database-dir

I'm seeing the following crash:

2025-06-19T18:44:26.615695Z [655-1] INFO [cf3f4] {general} ArangoDB (version 3.12.5 [linux]) is ready for business. Have fun!  
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/13/../../../gcc/x86_64-pc-linux-gnu/13/include/c++/bits/atomic_base.h:834: __pointer_type std::__atomic_base<arangodb::containers::ThreadOwnedList<arangodb::async_registry::Promise>::Node *>::load(memory_order) c
onst [_ITp = arangodb::containers::ThreadOwnedList<arangodb::async_registry::Promise>::Node *]: Assertion '__b != memory_order_release' failed.                                                                                                               
2025-06-19T18:44:27.413154Z [655-25] FATAL [a7902] {crash} 💥 ArangoDB 3.12.5 [linux], build-id e8ddafe21edac8e2495bd2def3ec6555571269ee, thread 25 [arangod] caught unexpected signal 6 (SIGABRT, sub type unknown) from pid 655 in state "in wait" in direct
ory "/home/build": signal handler invoked - image base address: 0x00005623d1d02000 - CPU context: rip: 0x00007f5776ca002c, rsp: 0x00007f56cd7d8ba0, efl: 0x0000000000000246, rbp: 0x00007f56ed2eb010, rsi: 0x0000000000000363, rdi: 0x000000000000028f, rax: 0
x0000000000000000, rbx: 0x0000000000000363, rcx: 0x00007f5776ca002c, rdx: 0x0000000000000006, r8: 0x0000000000000000, r9: 0x0000000000000000, r10: 0x0000000000000000, r11: 0x0000000000000246, r12: 0x00007f56ed2eb040, r13: 0x00007f56ed2eb000, r14: 0x00000
00000000006, r15: 0x0000000000000000      
2025-06-19T18:44:27.413191Z [655-25] INFO [c962b] {crash} Backtrace of thread 25 [arangod]                                     
2025-06-19T18:44:27.414836Z [655-25] INFO [308c3] {crash} frame  1 [+0x0000000006c1635f] (anonymous namespace)::crashHandlerSignalHandler(int, siginfo_t*, void*) [clone .llvm.13269059078790395198] (+0x6f)                                                  
2025-06-19T18:44:27.415126Z [655-25] INFO [308c3] {crash} frame  2 [+0x00002933a4f48ae0] __sigaction (+0xe0)
2025-06-19T18:44:27.415331Z [655-25] INFO [308c3] {crash} frame  3 [+0x00002933a4f9e02c] __pthread_kill_implementation (+0x10c) 
2025-06-19T18:44:27.415506Z [655-25] INFO [308c3] {crash} frame  4 [+0x00002933a4f489a6] gsignal (+0x16)
2025-06-19T18:44:27.415729Z [655-25] INFO [308c3] {crash} frame  5 [+0x00002933a4f31345] abort (+0x26)
2025-06-19T18:44:27.415963Z [655-25] INFO [308c3] {crash} frame  6 [+0x00002933a51ab26c] std::__glibcxx_assert_fail(char const*, int, char const*, char const*) (+0x61)
2025-06-19T18:44:27.417053Z [655-25] INFO [308c3] {crash} frame  7 [+0x00000000042a68ec] std::thread::_State_impl<std::thread::_Invoker<std::tuple<arangodb::async_registry::Feature::PromiseCleanupThread::PromiseCleanupThread(unsigned long)::{lambda(std::
stop_token)#1}, std::stop_token> > >::_M_run() (+0x3fc)
2025-06-19T18:44:27.417442Z [655-25] INFO [308c3] {crash} frame  8 [+0x00002933a51eeb94] execute_native_thread_routine (+0x14)
2025-06-19T18:44:27.417632Z [655-25] INFO [308c3] {crash} frame  9 [+0x00002933a4f9c3ed] start_thread (+0x2ad)
2025-06-19T18:44:27.417780Z [655-25] INFO [308c3] {crash} frame 10 [+0x00002933a5019dfc] __clone3 (+0x2c)
2025-06-19T18:44:27.417859Z [655-25] INFO [ded81] {crash} available physical memory: 53022552064, rss usage: 352305152, vsz usage: 4173643776, threads: 120
Aborted

This wasn't happening with v3.12.4.4.

I'm happy to provide more details if needed. I'm also starting to debug this further on my side, so I'll keep this issue updated with my findings.

Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions