PDAL @2.3.0_2: "zsh: trace trap" on M1

[ATL-Flaneur (Andreas Yankopolus)]

Running a PDAL pipeline to create a TIFF gives:

% pdal pipeline surface_star.json
zsh: trace trap  pdal pipeline surface_star.json

The same pipeline runs fine with PDAL on Linux.

[ATL-Flaneur (Andreas Yankopolus)]

The problem seems triggered by bounds in the writers.gdal block or adding one to a filters.crop operation. I can create pipelines that perform a variety of processing tasks, but once I try to crop/bound, then the error gets triggered.

[Veence (Vincent)]

Thanks for looking into this. I’m (once more) quite swamped these days, but I shall have a look and I’ll keep you in touch

[ATL-Flaneur (Andreas Yankopolus)]

I tried HomeBrew at the recommendation of a colleague when this M1 MacBook Pro arrived, and while I quickly went back to MacPorts, HomeBrew did produce a pdal executable that ran these filters just fine.

I'm happy to help with troubleshooting—pdal is a utility that I use daily and it would be nice to run it on this faster machine.

[Veence (Vincent)]

There might be a patch already published by the Homebrew team. Do you still have it installed, or did you wipe it all out?

[ATL-Flaneur (Andreas Yankopolus)]

It's no longer on my system—I removed everything before installing MacPorts.

[Veence (Vincent)]

Darn. Okay, I’m going to have a look at the Homebrew web interface. Thanks for the input

[ATL-Flaneur (Andreas Yankopolus)]

Here it is: ​https://formulae.brew.sh/formula/pdal

[Veence (Vincent)]

I don’t see anything special in the recipe. :/ Could you hand me a copy of the dataset that causes the crash? I might try to reproduce the bug and get to the bottom of it.

[ATL-Flaneur (Andreas Yankopolus)]

I've created an example using the ​rural Iowa dataset from ​Applied Imagery. Expanding the archive will create las files in ./Iowa_Rural_LiDAR/Iowa Rural LiDAR Point Cloud.

I'm uploading an attachment iowa_rural_surface.json designed to run from directory in which you expanded the archive with pdal pipeline iowa_rural_surface.json. It creates ./iowa_rural_surface.tiff on Linux but exits with zsh: trace trap pdal pipeline iowa_rural_surface.json on my M1 MBP.

[ATL-Flaneur (Andreas Yankopolus)]

PDAL pipeline to reproduce the error.

[ATL-Flaneur (Andreas Yankopolus)]

The bounds command in the writers clause triggers the error. Remove it and the pipeline runs fine.

"bounds": "([-91.25, -91.18], [43.27, 43.32])",

[Veence (Vincent)]

Thanks. I’ll check that over the weekend

[Veence (Vincent)]

Oops. Did you compile it or installed the binary? When compiling it I get an error:

/opt/local/include/ogr_srs_api.h:455:39: error: type alias redefinition with different types ('struct OGRSpatialReferenceHS *' vs 'void *')
typedef struct OGRSpatialReferenceHS *OGRSpatialReferenceH;
                                      ^
/opt/local/var/macports/build/_Users_vincent_Macports_gis_pdal/PDAL/work/PDAL-2.3.0-src/pdal/Geometry.hpp:44:7: note: previous definition is here
using OGRSpatialReferenceH = void *;

[ATL-Flaneur (Andreas Yankopolus)]

I installed the pdal binary. I initially installed MacPorts from source but have not been specifying install types since then.

I just cleared out my pdal install and and did sudo port -s -v install pdal. MacPorts built & installed just fine. The crash due to the bounds statement remains.

[ATL-Flaneur (Andreas Yankopolus)]

Anything I can do to help out with this?

[Veence (Vincent)]

I have updated the port to the new 2.4.0RC1 version (I masqueraded it under the name 2.3.99.1 for obvious reasons). Let me know if it helps.

[ATL-Flaneur (Andreas Yankopolus)]

I'm sadly getting the same exact error 😕

[Veence (Vincent)]

Drat. I will have a look tomorrow, pinkie promise.

[Veence (Vincent)]

The code inside PDAL is a mess. Cheesy programming at its best. Excerpt from the code that causes the crash:

void Bounds::parse(const std::string& s, std::string::size_type& pos)
{
    try
    {
        BOX3D box3d;
        box3d.parse(s, pos);
        set(box3d);
    }
    catch (const BOX3D::error&)
    {
        try
        {
            pos = 0;
            BOX2D box2d;
            box2d.parse(s, pos);
            set(box2d);
        }
        catch (const BOX2D::error& err)
        […]

So, in other words: I have to decode a line which contains coordinates. Let's first suppose they’re 3D and decode them as such. If that assumption throws an exception, then let's decode the coordinates as 2D. Try … catch blocks should NEVER be used that way. They should be used to deal with things that cannot be predicted at compile time, such that an out of memory error, or a file that disappears in the middle of a write operation. But here, it is easy to know what kind of box the user has given as parameter.

In your iowa_rural_surface.json file, change the line

 "bounds": "([-91.25, -91.18], [43.27, 43.32])",

into

"bounds": "([-91.25, -91.18], [43.27, 43.32], [0, 0])",

et voilà! Shazam! It now works like a charm.

I’m not sure what causes the crash, really. I think the difference of behaviour between MacOS and Linux boils down to how libunwind handles traps (but that’s just a wild guess). Maybe on Linux/GCC the trap is caught by the try…catch block as if it were a software exception, and the execution carries on with the BOX2D decoding which should run fine. No such luck with MacOS/Clang, for which such traps obviously don’t enter into the scope of try…catch blocks.

In any case, I would say that the MacOS executable is right to crash. The problem cannot be fixed by a patch, as it is by nature a flaw in the design of the code. I’ll escalade it upstream. In the meantime, the workaround of using 3D boxes anywhere should work.

[ATL-Flaneur (Andreas Yankopolus)]

Impressive work! Adding the [0,0] coordinates to my JSON file results in clean processing. I'll post this thread to the PDAL mailing list.