Opened 11 years ago

Closed 11 years ago

#41124 closed defect (fixed)

graphviz uses a fuzzy check whether alloca(3) uses malloc(3), which fails in some cases

Reported by: neverpanic (Clemens Lang) Owned by: ryandesign (Ryan Carsten Schmidt)
Priority: Normal Milestone:
Component: ports Version: 2.2.1
Keywords: Cc:
Port: graphviz

Description (last modified by neverpanic (Clemens Lang))

Graphviz uses a check in lib/vmalloc/features/vmalloc that is supposed to find out whether alloca(3) is internally using malloc(3). To do this, this check provides a custom implementation of malloc(3) and assumes that if this implementation is going to be called it was called by alloca(3).

However, this assumption doesn't hold in all cases, e.g. it breaks when having DYLD_INSERT_LIBRARIES set, as is the case for trace mode, because the Mac OS loader dyld internally calls malloc(3), causing the check to fail.

The check should instead use a global volatile variable to check whether the current call to malloc(3) happened while trying to use alloca(3). The overriden malloc(3) should behave like the standard one in all other cases.

The attached patch fixes this. Additional checks for dlsym(3) and dlfcn.h might be required. Can you take this upstream? I assume the same applies to graphviz-devel, too.

Attachments (1)

patch-lib__vmalloc__features__vmalloc-dyld-uses-malloc.diff (628 bytes) - added by neverpanic (Clemens Lang) 11 years ago.
Patch against lib/vmalloc/features/vmalloc

Download all attachments as: .zip

Change History (4)

Changed 11 years ago by neverpanic (Clemens Lang)

Patch against lib/vmalloc/features/vmalloc

comment:1 Changed 11 years ago by neverpanic (Clemens Lang)

Description: modified (diff)

comment:2 in reply to:  description Changed 11 years ago by ryandesign (Ryan Carsten Schmidt)

Replying to cal@…:

Can you take this upstream?

Done: http://www.graphviz.org/mantisbt/view.php?id=2389

comment:3 Changed 11 years ago by ryandesign (Ryan Carsten Schmidt)

Resolution: fixed
Status: newclosed

Upstream has applied your patch.

Note: See TracTickets for help on using tickets.