Skip to content

[Bug] deadlock happens when async threads unreserve() and the main thread apply memory #781

Description

@fzhedu

Component Selection

  • Core Engine (Expression eval, Memory, Vector)
  • Connectors / File Formats (Hive, Parquet, etc.)
  • API / Bindings (Python, etc.)
  • Build
  • Other

Describe the Bug

two threads hold two locks and deadlock happens.

Reproduction Steps

no

Bolt Version / Commit ID

main

System Configuration

- **OS**: (e.g. Ubuntu 22.04, CentOS 7)
- **Compiler**: (e.g. GCC 11, Clang 14)
- **Build Type**: (Debug / Release / RelWithDebInfo)
- **CPU Arch**: (e.g. x86_64 AVX2, ARM64)
- **Framework**: (e.g. Spark 3.3, PrestoDB)

Logs / Stack Trace

Thread 250 (LWP 103 "Executor task l"):
#0  0x00007fbaf7a2e12b in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fbaf7a344da in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fba8adf7705 in facebook::velox::memory::sparksql::VeloxMemoryManager::shrink(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#3  0x00007fba8ade9640 in facebook::velox::memory::sparksql::SpillTrigger::spillTree(std::weak_ptr<facebook::velox::memory::sparksql::TreeMemoryTarget>, long, facebook::velox::memory::sparksql::SpillerPhase const&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#4  0x00007fba8ade8a8e in facebook::velox::memory::sparksql::SpillTrigger::spillTree(std::weak_ptr<facebook::velox::memory::sparksql::TreeMemoryTarget>, long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#5  0x00007fba8ade8ff8 in facebook::velox::memory::sparksql::SpillTrigger::spillTree(std::weak_ptr<facebook::velox::memory::sparksql::TreeMemoryTarget>, long, facebook::velox::memory::sparksql::SpillerPhase const&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#6  0x00007fba8ade8a8e in facebook::velox::memory::sparksql::SpillTrigger::spillTree(std::weak_ptr<facebook::velox::memory::sparksql::TreeMemoryTarget>, long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#7  0x00007fba8ade8ff8 in facebook::velox::memory::sparksql::SpillTrigger::spillTree(std::weak_ptr<facebook::velox::memory::sparksql::TreeMemoryTarget>, long, facebook::velox::memory::sparksql::SpillerPhase const&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#8  0x00007fba8ade8a8e in facebook::velox::memory::sparksql::SpillTrigger::spillTree(std::weak_ptr<facebook::velox::memory::sparksql::TreeMemoryTarget>, long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#9  0x00007fba8ade9c66 in facebook::velox::memory::sparksql::ConsumerTargetBridge::spill(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#10 0x00007fba8ade216f in facebook::velox::memory::sparksql::TaskMemoryManager::acquireExecutionMemory(long, std::weak_ptr<facebook::velox::memory::sparksql::MemoryConsumer>) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#11 0x00007fba8ade6600 in facebook::velox::memory::sparksql::ConsumerTargetBridge::acquireMemory(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#12 0x00007fba8adeb2fa in non-virtual thunk to facebook::velox::memory::sparksql::ConsumerTargetBridge::borrow(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#13 0x00007fba8adeb15c in facebook::velox::memory::sparksql::TreeMemoryTargetNode::borrow0(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#14 0x00007fba8adeb825 in facebook::velox::memory::sparksql::TreeMemoryTargetNode::borrow(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#15 0x00007fba8adeb15c in facebook::velox::memory::sparksql::TreeMemoryTargetNode::borrow0(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#16 0x00007fba8adeb825 in facebook::velox::memory::sparksql::TreeMemoryTargetNode::borrow(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#17 0x00007fba8ade294c in facebook::velox::memory::sparksql::OverAcquireMemoryTarget::borrow(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#18 0x00007fba8ae05cc7 in facebook::velox::memory::sparksql::ManagedAllocationListener::reserve(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#19 0x00007fba8adf71e6 in facebook::velox::memory::sparksql::ListenableArbitrator::growPoolLocked(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#20 0x00007fba8adf8c1f in facebook::velox::memory::sparksql::ListenableArbitrator::growCapacity(facebook::velox::memory::MemoryPool*, std::vector<std::shared_ptr<facebook::velox::memory::MemoryPool>, std::allocator<std::shared_ptr<facebook::velox::memory::MemoryPool> > > const&, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#21 0x00007fba8ad9eded in facebook::velox::memory::MemoryManager::growPool(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#22 0x00007fba8adb8e7a in facebook::velox::memory::MemoryPoolForGluten::maybeReserveIncrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#23 0x00007fba8adb8e2f in facebook::velox::memory::MemoryPoolForGluten::maybeReserveIncrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#24 0x00007fba8adb8e2f in facebook::velox::memory::MemoryPoolForGluten::maybeReserveIncrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#25 0x00007fba8adb8e2f in facebook::velox::memory::MemoryPoolForGluten::maybeReserveIncrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#26 0x00007fba8adb9496 in facebook::velox::memory::MemoryPoolForGluten::maybeReserveThreadSafe(unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#27 0x00007fba8adba3ab in facebook::velox::memory::MemoryPoolForGluten::maybeReserve(unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#28 0x00007fba81d221e9 in gluten::VeloxRowBasedSortShuffleWriter::split(std::shared_ptr<gluten::ColumnarBatch>, long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#29 0x00007fba81d2438b in gluten::SparkShuffleWriter::addInput(std::shared_ptr<facebook::velox::RowVector>) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#30 0x00007fba8931191d in facebook::velox::exec::Driver::runInternal(std::shared_ptr<facebook::velox::exec::Driver>&, std::shared_ptr<facebook::velox::exec::BlockingState>&, std::shared_ptr<facebook::velox::RowVector>&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#31 0x00007fba89312f76 in facebook::velox::exec::Driver::next(std::shared_ptr<facebook::velox::exec::BlockingState>&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#32 0x00007fba893ccc25 in facebook::velox::exec::Task::next(folly::SemiFuture<folly::Unit>*) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#33 0x00007fba81b9db26 in gluten::WholeStageResultIterator::next() () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#34 0x00007fba81e2a119 in Java_io_glutenproject_vectorized_ColumnarBatchOutIterator_nativeHasNext () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#35 0x00007fbaeb16c0d3 in ?? ()



Thread 51 (LWP 332 "AsyncLoadThr23"):
#0  0x00007fbaf7a2e12b in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fbaf7a344da in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fba8adeb3c3 in facebook::velox::memory::sparksql::TreeMemoryTargetNode::repay(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#3  0x00007fba8adeb464 in facebook::velox::memory::sparksql::TreeMemoryTargetNode::repay(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#4  0x00007fba8ade2606 in facebook::velox::memory::sparksql::OverAcquireMemoryTarget::repay(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#5  0x00007fba8ae05d27 in facebook::velox::memory::sparksql::ManagedAllocationListener::unreserve(long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#6  0x00007fba8adf7301 in facebook::velox::memory::sparksql::ListenableArbitrator::growPoolLocked(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#7  0x00007fba8adf8c1f in facebook::velox::memory::sparksql::ListenableArbitrator::growCapacity(facebook::velox::memory::MemoryPool*, std::vector<std::shared_ptr<facebook::velox::memory::MemoryPool>, std::allocator<std::shared_ptr<facebook::velox::memory::MemoryPool> > > const&, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#8  0x00007fba8ad9eded in facebook::velox::memory::MemoryManager::growPool(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#9  0x00007fba8adb291f in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#10 0x00007fba8adb28af in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#11 0x00007fba8adb28af in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#12 0x00007fba8adb28af in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#13 0x00007fba8adb317a in facebook::velox::memory::MemoryPoolImpl::reserveThreadSafe(unsigned long, bool) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#14 0x00007fba8ada5aff in facebook::velox::memory::MemoryAllocator::allocateNonContiguous(unsigned long, facebook::velox::memory::Allocation&, std::function<void (unsigned long, bool)>, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#15 0x00007fba8adb64b5 in facebook::velox::memory::MemoryPoolImpl::allocateNonContiguous(unsigned long, facebook::velox::memory::Allocation&, unsigned long) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#16 0x00007fba8742f84e in facebook::velox::dwio::common::DirectCoalescedLoad::loadData(bool) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#17 0x00007fba8ac08b74 in facebook::velox::cache::CoalescedLoad::loadOrFuture(folly::SemiFuture<bool>*) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#18 0x00007fba8742f133 in facebook::velox::dwio::common::DirectBufferedInput::readRegions(std::vector<facebook::velox::dwio::common::LoadRequest*, std::allocator<facebook::velox::dwio::common::LoadRequest*> > const&, bool, std::vector<int, std::allocator<int> > const&)::{lambda()#1}::operator()() const () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#19 0x00007fba81ec479a in folly::ThreadPoolExecutor::runTask(std::shared_ptr<folly::ThreadPoolExecutor::Thread> const&, folly::ThreadPoolExecutor::Task&&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#20 0x00007fba81eb7d8b in void folly::detail::function::FunctionTraits<void ()>::callBig<folly::IOThreadPoolExecutor::add(folly::Function<void ()>, std::chrono::duration<long, std::ratio<1l, 1000l> >, folly::Function<void ()>)::{lambda()#1}>(folly::detail::function::Data&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#21 0x00007fba81eeb0eb in folly::EventBase::FunctionLoopCallback::runLoopCallback() () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#22 0x00007fba81ee3dd6 in folly::EventBase::runLoopCallbacks(boost::intrusive::list<folly::EventBase::LoopCallback, boost::intrusive::constant_time_size<false> >&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#23 0x00007fba81ee3ef9 in folly::EventBase::runLoopCallbacks() () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#24 0x00007fba81ee5f32 in folly::EventBase::loopMain(int, bool) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#25 0x00007fba81ee6900 in folly::EventBase::loopBody(int, bool) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#26 0x00007fba81ee6976 in folly::EventBase::loop() () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#27 0x00007fba81ee9496 in folly::EventBase::loopForever() () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#28 0x00007fba81eb8f61 in folly::IOThreadPoolExecutor::threadRun(std::shared_ptr<folly::ThreadPoolExecutor::Thread>) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#29 0x00007fba81ec53fa in void folly::detail::function::FunctionTraits<void ()>::callSmall<std::_Bind<void (folly::ThreadPoolExecutor::*(folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)> >(folly::detail::function::Data&) () from /data01/yodel/datacache/usercache/liumingzhu.904/appcache/application-1784530213999-9726088/gluten-fabc2dad-c7f6-4d83-bdee-36d643e393cc/jni/dc18128c-3b71-4edf-ab84-7d93387d5116/gluten-5334291764254043070/libvelox_backend.amd64.so
#30 0x00007fbaf77824a3 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#31 0x00007fbaf7a311f5 in ?? () from /lib/x86_64-linux-gnu/libc.so.6

Expected Behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions