#48557 closed defect (fixed)
py27-pandas crashes with blosc
Reported by: | dershow | Owned by: | stromnov (Andrey Stromnov) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.3.3 |
Keywords: | Cc: | mamoll (Mark Moll), petrrr | |
Port: | py-pandas |
Description
I am trying to create an HDF5 file, and if I use blosc compression it segmentation faults every time when I try to close the file. Here is an example:
$ python Python 2.7.10 (default, May 26 2015, 17:31:25) [GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pandas as pd >>> a=pd.DataFrame([1,2]) >>> store=pd.HDFStore("test.h5",complib='blosc') >>> store['test']=a >>> store.close() Segmentation fault: 11
I listed this as a py27-pandas port problem, because that is where I am seeing it. But, I suppose it could be, pandas, python, hdf5, pytables, or blosc related. If I use no compression, or zlib, the store closes without an error.
Here is the thread that crashed:
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_platform.dylib 0x00007fff978a9b50 _platform_strcmp + 80 1 libblosc.1.dylib 0x000000011275dc78 blosc_compname_to_compcode + 21 2 libblosc.1.dylib 0x000000011275e0ca blosc_compress_ctx + 75 3 utilsextension.so 0x00000001124f0da8 blosc_filter + 504 4 libhdf5.10.dylib 0x00000001126dc084 H5Z_pipeline + 956 5 libhdf5.10.dylib 0x0000000112555413 H5D__chunk_flush_entry + 355 6 libhdf5.10.dylib 0x00000001125536da H5D__chunk_flush + 113 7 libhdf5.10.dylib 0x0000000112564aed H5D__flush_real + 192 8 libhdf5.10.dylib 0x000000011256458f H5D_close + 208 9 libhdf5.10.dylib 0x00000001125e0478 H5I_dec_ref + 132 10 libhdf5.10.dylib 0x00000001125e0331 H5I_dec_app_ref + 35 11 libhdf5.10.dylib 0x00000001125e05ed H5I_dec_app_ref_always_close + 35 12 libhdf5.10.dylib 0x000000011254db28 H5Dclose + 110 13 hdf5extension.so 0x00000001128838a9 __pyx_pw_6tables_13hdf5extension_4Leaf_9_g_close + 57 14 org.python.python 0x000000010f58a3a8 PyEval_EvalFrameEx + 15944 15 org.python.python 0x000000010f586102 PyEval_EvalCodeEx + 1666 16 org.python.python 0x000000010f58da86 fast_function + 118 17 org.python.python 0x000000010f589c4f PyEval_EvalFrameEx + 14063 18 org.python.python 0x000000010f586102 PyEval_EvalCodeEx + 1666 19 org.python.python 0x000000010f58da86 fast_function + 118 20 org.python.python 0x000000010f589c4f PyEval_EvalFrameEx + 14063 21 org.python.python 0x000000010f586102 PyEval_EvalCodeEx + 1666 22 org.python.python 0x000000010f58da86 fast_function + 118 23 org.python.python 0x000000010f589c4f PyEval_EvalFrameEx + 14063 24 org.python.python 0x000000010f58db66 fast_function + 342 25 org.python.python 0x000000010f589c4f PyEval_EvalFrameEx + 14063 26 org.python.python 0x000000010f58db66 fast_function + 342 27 org.python.python 0x000000010f589c4f PyEval_EvalFrameEx + 14063 28 org.python.python 0x000000010f586102 PyEval_EvalCodeEx + 1666 29 org.python.python 0x000000010f58da86 fast_function + 118 30 org.python.python 0x000000010f589c4f PyEval_EvalFrameEx + 14063 31 org.python.python 0x000000010f58db66 fast_function + 342 32 org.python.python 0x000000010f589c4f PyEval_EvalFrameEx + 14063 33 org.python.python 0x000000010f586102 PyEval_EvalCodeEx + 1666 34 org.python.python 0x000000010f585a76 PyEval_EvalCode + 54 35 org.python.python 0x000000010f5af3d6 PyRun_InteractiveOneFlags + 406 36 org.python.python 0x000000010f5aee8e PyRun_InteractiveLoopFlags + 206 37 org.python.python 0x000000010f5aed38 PyRun_AnyFileExFlags + 136 38 org.python.python 0x000000010f5c515c Py_Main + 3084 39 libdyld.dylib 0x00007fff928b55c9 start + 1
Change History (8)
comment:1 Changed 9 years ago by mamoll (Mark Moll)
comment:2 Changed 9 years ago by mf2k (Frank Schima)
Cc: | stromnov@… removed |
---|---|
Owner: | changed from macports-tickets@… to stromnov@… |
Port: | py-pandas added; py27-pandas removed |
Summary: | Pandas crashes with blosc → py27-pandas crashes with blosc |
The port is still "py-pandas" so that searches can find it. The description, as I have modified it, should specify the python version that failed. Although based on mmoll's comment, this does not seem to be specific to python 2.7.
comment:3 Changed 9 years ago by stromnov (Andrey Stromnov)
1) "pyx_pw_6tables_13hdf5extension_4Leaf_9_g_close" - so it happens inside of py-tables
2) py-tables port depends on external blosc library (directly or via hdf5)
Note: py-tables also contains specific version (even specially downgraded) of blosc library
3) Since version 1.7.0 blosc library no longer supports HDF5 filter (excerpt from blosc 1.7.0 ChangeLog: The HDF5 filter has been removed from c-blosc and moved into its own repo at: https://github.com/Blosc/hdf5)
comment:4 Changed 9 years ago by dershow
I am not sure if this is the same bug or a different one. But, there is a known bug in pytables 3.2 that has been fixed in 3.2.1. I have created a separate ticket for it: https://trac.macports.org/ticket/48566 Perhaps that upgrade will help with this, and even if not, it will help with another crash that keeps happening with pandas.
comment:5 Changed 9 years ago by stromnov (Andrey Stromnov)
Example from pytables documentation (leads to crash in py-tables 3.2 and 3.2.1):
import numpy from tables import * fileh = open_file('test5.h5', mode='w') atom = Float32Atom() filters = Filters(complevel=1, complib='blosc', fletcher32=True) arr = fileh.create_earray(fileh.root, 'earray', atom, (0,2), "A growable array", filters=filters) # Append several rows in only one call arr.append(numpy.array([[1., 2.], [2., 3.], [3., 4.]], dtype=numpy.float32)) # Print information on that enlargeable array print("Result Array:") print(repr(arr)) fileh.close()
comment:6 Changed 9 years ago by stromnov (Andrey Stromnov)
Explanation: https://github.com/PyTables/PyTables/issues/451
comment:7 Changed 9 years ago by mamoll (Mark Moll)
Resolution: | → fixed |
---|---|
Status: | new → closed |
Fixed in r139337.
I can reproduce the problem with py34-pandas, but have no idea what the problem might be.