radare2 @0.9.9_0 build failure

[mopihopi]

radare2 @0.9.9_0 fails to build on OS X 10.10.5 XCode 7.1 x86_64

:info:build /usr/bin/clang -shared -o io_windbg.dylib io_windbg.o -I../../include -Wall -DWORDSIZE=64 -fPIC -dynamiclib -L.. -DCORELIB -pipe -Os -arch x86_64 -MD   -fPIC -g -Wall -D__UNIX__=1 -I/opt/local/include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_radare2/radare2/work/radare2-0.9.9/libr/../shlr/gdb/include/ -I/opt/local/include -I/opt/local/include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_radare2/radare2/work/radare2-0.9.9/libr/../shlr/wind \
:info:build             -L../../socket -lr_socket -L../../util -lr_util -L.. -lr_io -L/opt/local/lib -lssl -lcrypto -L../../util -lr_util -L.. -lr_io -L../../util -lr_util -L.. -lr_io -L../../util -lr_util -L.. -lr_io -L../../util -lr_util -L.. -lr_io -dynamiclib -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_radare2/radare2/work/radare2-0.9.9/libr/../shlr/gdb/lib/libgdbr.a -L/opt/local/lib -lssl -lcrypto /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_radare2/radare2/work/radare2-0.9.9/libr/../shlr/wind/libr_wind.a
:info:build Undefined symbols for architecture x86_64:
:info:build   "_r_run_free", referenced from:
:info:build       ___open in io_debug.o
:info:build   "_r_run_new", referenced from:
:info:build       ___open in io_debug.o
:info:build   "_r_run_parsefile", referenced from:
:info:build       ___open in io_debug.o
:info:build   "_r_run_parseline", referenced from:
:info:build       ___open in io_debug.o
:info:build   "_r_run_start", referenced from:
:info:build       ___open in io_debug.o
:info:build   "_r_sys_fork", referenced from:
:info:build       ___open in io_debug.o
:info:build ld: symbol(s) not found for architecture x86_64

[g5pw (Aljaž Srebrnič)]

Do you have the old radare2 installed? If so, could you try removing it manually (with port uninstall radare2) and reinstalling it from scratch?

[g5pw (Aljaž Srebrnič)]

Ok, I see some inconsistencies in your build log that may indicate a bug in the build system. It looks like it uses gcc as compiler for some parts of the software, and clang for others.

[g5pw (Aljaž Srebrnič)]

Do the changes in r142258 fix the problem?

[mopihopi]

upgrade failure with r142258

[mopihopi]

port clean radare2; port upgrade radare2 still fails with r142258 (log attached). @0.9.6_0+ewf+openssl was previously installed.

I tried uninstalling it and then port install radare2, but encountered a different error:

$ sudo port install radare2
--->  Computing dependencies for radare2
--->  Fetching archive for radare2
--->  Attempting to fetch radare2-0.9.9_1+openssl.darwin_14.x86_64.tbz2 from http://packages.macports.org/radare2
--->  Attempting to fetch radare2-0.9.9_1+openssl.darwin_14.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/radare2
--->  Attempting to fetch radare2-0.9.9_1+openssl.darwin_14.x86_64.tbz2 from http://nue.de.packages.macports.org/macports/packages/radare2
--->  Fetching distfiles for radare2
--->  Verifying checksums for radare2
--->  Extracting radare2
--->  Applying patches to radare2
--->  Configuring radare2
--->  Building radare2
--->  Staging radare2 into destroot
--->  Installing radare2 @0.9.9_1+openssl
--->  Activating radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  Found 5 broken file(s), matching files to ports    
--->  Found 1 broken port(s), determining rebuild order
--->  Rebuilding in order
     radare2 @0.9.9 +openssl
--->  Computing dependencies for radare2
--->  Cleaning radare2
--->  Scanning binaries for linking errors
--->  Found 5 broken file(s), matching files to ports    
--->  Found 1 broken port(s), determining rebuild order
--->  Rebuilding in order
     radare2 @0.9.9 +openssl
--->  Computing dependencies for radare2
--->  Fetching distfiles for radare2
--->  Verifying checksums for radare2
--->  Extracting radare2
--->  Applying patches to radare2
--->  Configuring radare2
--->  Building radare2
--->  Staging radare2 into destroot
--->  Deactivating radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Uninstalling radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Computing dependencies for radare2
--->  Installing radare2 @0.9.9_1+openssl
--->  Activating radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  Found 5 broken file(s), matching files to ports    
--->  Found 1 broken port(s), determining rebuild order
--->  Rebuilding in order
     radare2 @0.9.9 +openssl
--->  Computing dependencies for radare2
--->  Fetching distfiles for radare2
--->  Verifying checksums for radare2
--->  Extracting radare2
--->  Applying patches to radare2
--->  Configuring radare2
--->  Building radare2
--->  Staging radare2 into destroot
--->  Deactivating radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Uninstalling radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Computing dependencies for radare2
--->  Installing radare2 @0.9.9_1+openssl
--->  Activating radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  Found 5 broken file(s), matching files to ports    
Error: Port radare2 is still broken after rebuilding it more than 3 times.
Error: Please run port -d -y rev-upgrade and use the output to report a bug.
Port radare2 still broken after rebuilding 3 time(s)
    while executing
"error "Port $portname still broken after rebuilding [expr {$broken_port_counts($portname) - 1}] time(s)""
    (procedure "revupgrade_scanandrebuild" line 280)
    invoked from within
"revupgrade_scanandrebuild broken_port_counts $opts"
    (procedure "macports::revupgrade" line 5)
    invoked from within
"macports::revupgrade $opts"
    (procedure "action_revupgrade" line 2)
    invoked from within
"action_revupgrade $action $portlist $opts"
    (procedure "action_target" line 96)
    invoked from within
"$action_proc $action $portlist [array get global_options]"
    (procedure "process_cmd" line 103)
    invoked from within
"process_cmd $remaining_args"
    invoked from within
"if { [llength $remaining_args] > 0 } {

    # If there are remaining arguments, process those as a command
    set exit_status [process_cmd $remaining..."
    (file "/opt/local/bin/port" line 5268)
$ 

[mopihopi]

port -d -y rev-upgrade

[mopihopi]

I see that there is now a new binary package available for Yosemite and that one seems to work.

As for building from source, I found that it is successful if trace mode (-t) is used. Apparently it is messed up by a program from another package.

$ sudo port -s -t install radare2
--->  Computing dependencies for radare2
--->  Fetching distfiles for radare2
--->  Verifying checksums for radare2
--->  Extracting radare2
--->  Applying patches to radare2
--->  Configuring radare2
Warning: The following existing file was hidden from the build system by trace mode:
  /opt/local/bin/pkg-config
--->  Building radare2
Warning: The following existing files were hidden from the build system by trace mode:
  /opt/local/bin/ar
  /opt/local/bin/ranlib
  /opt/local/bin/xz
  /var
--->  Staging radare2 into destroot
Warning: The following existing file was hidden from the build system by trace mode:
  /opt/local/bin/xz
--->  Installing radare2 @0.9.9_1+openssl
--->  Activating radare2 @0.9.9_1+openssl
--->  Cleaning radare2
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  No broken files found.
$

pkg-config and xz are required by ImageMagick and a number of other packages. cctools (which provides ar and ranlib) is required by gcc. I'm not sure why the radare2 build would be messed up by those.

[g5pw (Aljaž Srebrnič)]

That's odd. I'll try to look into it, but I have already updated to El Capitan on my machine :/

EDIT: just a thought: it's possible it opportunistically (incorrectly) links against libewf, do you have it installed?

[mopihopi]

Replying to g5pw@…:

That's odd. I'll try to look into it, but I have already updated to El Capitan on my machine :/

EDIT: just a thought: it's possible it opportunistically (incorrectly) links against libewf, do you have it installed?

I did have libewf installed, probably because radare2 @0.9.6 had a dependency on it with the default variants. I don't have any other ports that need it. But I just tried uninstalling radare2 and libewf, and then re-installing radare2 from source (sudo port -s install radare2), and I get the same rev-upgrade issue as before.

Based on the trace mode output, it looks like the issue is due to having pkgconfig, xz, and/or cctools installed, because it worked when trace mode hid the files from those packages. Are you able to reproduce the issue on El Capitan with those packages installed? In case it matters, the variants I have installed are pkgconfig @0.28_0, xz @5.2.2_0+universal, and cctools @870_0+llvm37.

[g5pw (Aljaž Srebrnič)]

Just a thought: Are you building it as universal binary?

[mopihopi]

I am not building radare2 as universal. x86_64 only.

[g5pw (Aljaž Srebrnič)]

Then the only difference we have is that your xz is built as universal binary.

[mopihopi]

I tried deactivating pkgconfig, xz, and cctools. That required -f (force deactivate) because I have other ports I need that require these, but I just wanted to deactivate them temporarily.

Deactivating xz and cctools had no effect (same rev-upgrade issue as before). With pkgconfig deactivated it was successful.

The radare2 Portfile uses the configure argument --with-syscapstone. With this option, pkg-config --libs capstone is used to pick up the system (MacPorts) capstone. When pkgconfig and capstone from MacPorts are installed, this outputs -L/opt/local/lib -lcapstone. However this does not work (see below). capstone is listed as a dependency but pkgconfig is not; without pkgconfig installed, it appears that --with-syscapstone is ignored and it builds its own copy of capstone and links it statically, which does work. So apparently in the builds that work this option was ignored.

With pkgconfig (broken):

...
DEBUG: Executing command line:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_radare2/radare2/work/radare2-0.9.9" && ./configure --prefix=/opt/local --with-sysmagic --with-syscapstone --with-syszip --with-compiler=clang --with-openssl 
...
checking build system type... x86_64-unknown-darwin
checking host system type... x86_64-unknown-darwin
checking target system type... x86_64-unknown-darwin
checking for working directories... current
using prefix '/opt/local'
checking for c compiler... /usr/bin/clang
checking for dynamic library... required
checking host endianness... little
checking for libmagic ... yes
Using PKGCONFIG: pkg-config
checking pkg-config flags for capstone... yes
checking for libz ... yes
checking for libzip ... yes
checking for libssl ... yes
Using PKGCONFIG: pkg-config
checking pkg-config flags for openssl... yes
checking for liblua5.1 ... no
creating ./config-user.mk
...
clang -dynamiclib -o libr_asm.dylib ... -arch x86_64 -L/opt/local/lib -lcapstone
...
$ radare2
dyld: Library not loaded: libcapstone.3.dylib
  Referenced from: /opt/local/lib/libr_asm.dylib
  Reason: image not found
Trace/BPT trap: 5
$ otool -L /opt/local/lib/libr_asm.dylib 
/opt/local/lib/libr_asm.dylib:
	/opt/local/lib/libr_asm.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_util.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_parse.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_db.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_syscall.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_lang.dylib (compatibility version 0.0.0, current version 0.0.0)
	libcapstone.3.dylib (compatibility version 3.0.0, current version 3.0.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
$ DYLD_LIBRARY_PATH=/opt/local/lib radare2
Usage: r2 [-dDwntLqv] [-P patch] [-p prj] [-a arch] [-b bits] [-i file]
          [-s addr] [-B blocksize] [-c cmd] [-e k=v] file|pid|-|--|=
$ 

Note that libcapstone is linked dynamically, but it has no directory path in the otool -L output. Since it is not present in the directories that are searched by default, it is not found at run time. However it is found if DYLD_LIBRARY_PATH is set manually.

Without pkgconfig (works):

...
DEBUG: Executing command line:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_radare2/radare2/work/radare2-0.9.9" && ./configure --prefix=/opt/local --with-sysmagic --with-syscapstone --with-syszip --with-compiler=clang --with-openssl 
...
checking build system type... x86_64-unknown-darwin
checking host system type... x86_64-unknown-darwin
checking target system type... x86_64-unknown-darwin
checking for working directories... current
using prefix '/opt/local'
checking for c compiler... /usr/bin/clang
checking for dynamic library... required
checking host endianness... little
checking for libmagic ... yes
Using PKGCONFIG: pkg-config
checking pkg-config flags for capstone... no
checking for libz ... yes
checking for libzip ... yes
checking for libssl ... yes
Using PKGCONFIG: pkg-config
checking pkg-config flags for openssl... no
checking for liblua5.1 ... no
creating ./config-user.mk
...
if [ ! -d capstone ]; then \
                git clone https://www.github.com/aquynh/capstone.git ; \
        fi
Cloning into 'capstone'...
...
clang -dynamiclib -o libr_asm.dylib ... -arch x86_64 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_radare2/radare2/work/radare2-0.9.9/libr/../shlr/capstone/libcapstone.a
...
$ radare2
Usage: r2 [-dDwntLqv] [-P patch] [-p prj] [-a arch] [-b bits] [-i file]
          [-s addr] [-B blocksize] [-c cmd] [-e k=v] file|pid|-|--|=
$ otool -L /opt/local/lib/libr_asm.dylib 
/opt/local/lib/libr_asm.dylib:
	/opt/local/lib/libr_asm.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_util.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_parse.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_db.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_syscall.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libr_lang.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
$ 

Note that capstone is obtained from git, and built and linked statically (despite --with-syscapstone).

[g5pw (Aljaž Srebrnič)]

Thanks for the very detailed analysis :) yeah, it looks like the radare2 build system fails to link properly with system capstone using pkgconfig. I'll try to understand why.

[mopihopi]

I think that the MacPorts capstone shared library is not built correctly. The install name (ld -install_name) (LC_ID_DYLIB) is just libcapstone.3.dylib. It should either use rpath or be the full path (e.g. /opt/local/lib/libcapstone.3.dylib), so that programs linked against this shared library will be able to find it at runtime.

It appears that libr2.dylib has the same issue, although I don't know if there is anything in MacPorts that links against that.

[g5pw (Aljaž Srebrnič)]

Yep, this looks like a bug in the capstone port. But, on my machine I have

$ otool -L /opt/local/lib/libcapstone.dylib
/opt/local/lib/libcapstone.dylib:
	/opt/local/lib/libcapstone.3.dylib (compatibility version 3.0.0, current version 3.0.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1225.1.1)

What's the output on your end?

[g5pw (Aljaž Srebrnič)]

Ok, I'm sorry. It looks like I updated the capstone port in my tree, and that contains the fixes for the capstone library.

[mopihopi]

$ otool -L /opt/local/lib/libcapstone.dylib 
/opt/local/lib/libcapstone.dylib:
	libcapstone.3.dylib (compatibility version 3.0.0, current version 3.0.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

[g5pw (Aljaž Srebrnič)]

Yep, that's not right. I'll review and commit the fix to libcapstone and revbump radare2, ideally later today.

[g5pw (Aljaž Srebrnič)]

Capstone fixed in r142835, and radare2 rev-bumped in r142836. Try again in half an hour :)

[mopihopi]

Works great, thanks.