Opened 14 months ago
Last modified 14 months ago
#67928 new defect
apache-arrow: segmentation fault, potentially related to jemalloc
Reported by: | essandess (Steve Smith) | Owned by: | |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.8.1 |
Keywords: | Cc: | mascguy (Christopher Nielsen) | |
Port: | apache-arrow jemalloc |
Description
When compiled with https://github.com/apache/arrow, the py*-pyarrow
packages create a segfault on import at a jemalloc
call. Both arm64
and x86_64
architectures are affected.
The issue disappears using the cmake
directive -DARROW_JEMALLOC:BOOL=OFF
.
lldb trace
/opt/local/bin/lldb-mp-16 (lldb) command script import pyarrow PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump: 0. Program arguments: /opt/local/libexec/llvm-16/bin/lldb 1. HandleCommand(command = "command script import pyarrow") Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 libLLVM.dylib 0x0000000113a91c6c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56 1 libLLVM.dylib 0x0000000113a90c38 llvm::sys::RunSignalHandlers() + 112 2 libLLVM.dylib 0x0000000113a922f8 SignalHandler(int) + 344 3 libsystem_platform.dylib 0x000000018d386a24 _sigtramp + 56 4 libjemalloc.2.dylib 0x000000010f5940e0 je_free_default + 1104 5 libprotobuf.3.21.12.0.dylib 0x000000010f2471f0 google::protobuf::internal::ArenaStringPtr::Destroy() + 56 6 libprotobuf.3.21.12.0.dylib 0x000000010f2d0000 google::protobuf::FileDescriptorProto::SharedDtor() + 144 7 libprotobuf.3.21.12.0.dylib 0x000000010f2d00e4 google::protobuf::FileDescriptorProto::~FileDescriptorProto() + 48 8 libprotobuf.3.21.12.0.dylib 0x000000010f2ea634 google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) + 168 9 libprotobuf.3.21.12.0.dylib 0x000000010f29e878 google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) + 40 10 libprotobuf.3.21.12.0.dylib 0x000000010f3056a8 google::protobuf::(anonymous namespace)::AddDescriptors(google::protobuf::internal::DescriptorTable const*) + 136 11 libprotobuf.3.21.12.0.dylib 0x000000010f3056dc google::protobuf::internal::AddDescriptorsRunner::AddDescriptorsRunner(google::protobuf::internal::DescriptorTable const*) + 24 12 dyld 0x000000018d01c1d8 invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 168 13 dyld 0x000000018d05de94 invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 340 14 dyld 0x000000018d0511a4 invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 528 15 dyld 0x000000018cffc2d8 dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 296 16 dyld 0x000000018d0501cc dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 192 17 dyld 0x000000018d05d958 dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 516 18 dyld 0x000000018d01885c dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 448 19 dyld 0x000000018d018c10 dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 220 20 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 21 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 22 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 23 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 24 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 25 dyld 0x000000018d01c264 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_1::operator()() const + 112 26 dyld 0x000000018d018d90 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 304 27 dyld 0x000000018d036d58 dyld4::APIs::dlopen_from(char const*, int, void*) + 1440 28 Python 0x000000010378ae64 _imp_create_dynamic + 616 29 Python 0x00000001036ce5d0 cfunction_vectorcall_FASTCALL + 80 30 Python 0x0000000103760d94 _PyEval_EvalFrameDefault + 50396 31 Python 0x0000000103763424 _PyEval_Vector + 116 32 Python 0x000000010368859c object_vacall + 224 33 Python 0x0000000103688458 PyObject_CallMethodObjArgs + 92 34 Python 0x00000001037873a8 PyImport_ImportModuleLevelObject + 1224 35 Python 0x000000010375b8f0 _PyEval_EvalFrameDefault + 28728 36 Python 0x0000000103753d74 PyEval_EvalCode + 168 37 Python 0x0000000103750174 builtin_exec + 332 38 Python 0x00000001036ce6cc cfunction_vectorcall_FASTCALL_KEYWORDS + 76 39 Python 0x0000000103760d94 _PyEval_EvalFrameDefault + 50396 40 Python 0x0000000103763424 _PyEval_Vector + 116 41 Python 0x000000010368859c object_vacall + 224 42 Python 0x0000000103688458 PyObject_CallMethodObjArgs + 92 43 Python 0x00000001037873a8 PyImport_ImportModuleLevelObject + 1224 44 Python 0x000000010375b8f0 _PyEval_EvalFrameDefault + 28728 45 Python 0x0000000103753d74 PyEval_EvalCode + 168 46 Python 0x00000001037a5d38 run_eval_code_obj + 84 47 Python 0x00000001037a5c9c run_mod + 112 48 Python 0x00000001037a8084 PyRun_StringFlags + 112 49 liblldb.16.0.6.dylib 0x000000010469effc lldb_private::python::runStringMultiLine(llvm::Twine const&, lldb_private::python::PythonDictionary const&, lldb_private::python::PythonDictionary const&) + 120 50 liblldb.16.0.6.dylib 0x00000001046a5838 lldb_private::ScriptInterpreterPythonImpl::ExecuteMultipleLines(char const*, lldb_private::ExecuteScriptOptions const&) + 960 51 liblldb.16.0.6.dylib 0x00000001046ab314 lldb_private::ScriptInterpreterPythonImpl::LoadScriptingModule(char const*, lldb_private::LoadScriptOptions const&, lldb_private::Status&, std::__1::shared_ptr<lldb_private::StructuredData::Object>*, lldb_private::FileSpec) + 2092 52 liblldb.16.0.6.dylib 0x0000000104777158 CommandObjectCommandsScriptImport::DoExecute(lldb_private::Args&, lldb_private::CommandReturnObject&) + 328 53 liblldb.16.0.6.dylib 0x00000001043469c0 lldb_private::CommandObjectParsed::Execute(char const*, lldb_private::CommandReturnObject&) + 484 54 liblldb.16.0.6.dylib 0x000000010433d2e8 lldb_private::CommandInterpreter::HandleCommand(char const*, lldb_private::LazyBool, lldb_private::CommandReturnObject&) + 2136 55 liblldb.16.0.6.dylib 0x00000001043409a0 lldb_private::CommandInterpreter::IOHandlerInputComplete(lldb_private::IOHandler&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) + 852 56 liblldb.16.0.6.dylib 0x00000001042731a0 lldb_private::IOHandlerEditline::Run() + 304 57 liblldb.16.0.6.dylib 0x0000000104256968 lldb_private::Debugger::RunIOHandlers() + 140 58 liblldb.16.0.6.dylib 0x0000000104341c14 lldb_private::CommandInterpreter::RunCommandInterpreter(lldb_private::CommandInterpreterRunOptions&) + 156 59 liblldb.16.0.6.dylib 0x00000001040a0614 lldb::SBDebugger::RunCommandInterpreter(bool, bool) + 124 60 lldb 0x0000000102ff48e0 Driver::MainLoop() + 2784 61 lldb 0x0000000102ff541c main + 2168 62 dyld 0x000000018cffff28 start + 2236 LLDB diagnostics will be written to /var/folders/2s/r9z6gsps7gn5wd89c0sqs8k80000gq/T/diagnostics-5f4976 Please include the directory content when filing a bug report Segmentation fault: 11
Upstream issue link:
Downstream issue links
Change History (5)
comment:1 Changed 14 months ago by jmroot (Joshua Root)
comment:2 Changed 14 months ago by essandess (Steve Smith)
Perhaps, but the issue disappears without jemalloc
.
comment:3 Changed 14 months ago by jmroot (Joshua Root)
Maybe it disappears, or maybe it just isn't detected. Could go either way; I'm just cautioning against drawing conclusions without sufficient evidence.
comment:4 Changed 14 months ago by mascguy (Christopher Nielsen)
Cc: | mascguy added |
---|---|
Port: | apache-arrow added |
Summary: | jemalloc Segmentation Fault with apache-arrow → apache-arrow: segmentation fault, potentially related to jemalloc |
Steve, now that jemalloc
is no longer in play for apache-arrow
(as of the following PR), should this be closed?
https://github.com/macports/macports-ports/pull/19664
Or would you prefer to keep it open, until/unless upstream discussion helps to clear this up...?
comment:5 Changed 14 months ago by essandess (Steve Smith)
This is still an issue, as upstream Apache-arrow
says that jemalloc
is preferred: https://github.com/apache/arrow/issues/37010#issuecomment-1668601173
The latest merged MacPorts PR that removes {{jemalloc}}} is just a stop-gap so that all the downstream stuff like py-pandas
that will attempt to limport pyarrow
if it’s installed won’t crash.
It's not at all clear from the backtrace that jemalloc is at fault here. It could just as easily be protobuf freeing an invalid pointer.