jemalloc @5.3.0_2: universal build fails for Monterey M1 arm64 when building x86_64 slice

[potterpg]

I seem to have a build failure on this port.

Universal build on a mac M1 device.

Adding log file to assist in debug.

[potterpg]

Log file of build...

[mascguy (Christopher Nielsen)]

Sergey, are you able to build this port Universal, for PPC/Rosetta?

[kencu (Ken)]

(I thought ...) I fixed this two months ago, as per 65213.

[44412d7dbfb8102f854a0353ce2645893e83dc17/macports-ports]

and then moved that into muniversal and out of the jemalloc portfile.

[c9a6e03fc195486bbc91148a44393ae9f333f109/macports-ports]

[a05eaf3b301f7cdddfe8cbc9f4ff69a140c3e92c/macports-ports]

Is it not working again?

[kencu (Ken)]

One thing -- I had no M1 machine to test on, so I did my work on a Monterey Intel system. It is conceivable that things work differently on an M1. I'll see if Monterey Intel is still fixed later on today.

[mascguy (Christopher Nielsen)]

This appears to potentially be a run-of-the-mill compilation error, not necessarily universal-specific. But perhaps I'm missing something...?

include/jemalloc/internal/rtree.h:118:3:
error: constant expression evaluates to -12 which cannot be narrowed to type 'unsigned int' [-Wc++11-narrowing]
        {RTREE_NSB, RTREE_NHIB + RTREE_NSB}
         ^~~~~~~~~

[kencu (Ken)]

Yes, this error this time seems quite different. During the Intel build on an M1, it seems to be this:

4362	:info:build In file included from src/jemalloc_cpp.cpp:10:
4363	:info:build In file included from include/jemalloc/internal/jemalloc_internal_includes.h:52:
4364	:info:build In file included from include/jemalloc/internal/arena_structs.h:4:
4365	:info:build In file included from include/jemalloc/internal/arena_stats.h:8:
4366	:info:build In file included from include/jemalloc/internal/pa.h:6:
4367	:info:build In file included from include/jemalloc/internal/ecache.h:5:
4368	:info:build In file included from include/jemalloc/internal/san.h:5:
4369	:info:build In file included from include/jemalloc/internal/emap.h:5:
4370	:info:build include/jemalloc/internal/rtree.h:118:3: error: constant expression evaluates to -12 which cannot be narrowed to type 'unsigned int' [-Wc++11-narrowing]

[kencu (Ken)]

builds fine as universal on Monterey Intel, didn't touch a thing in the Portfile:

 % port -v installed jemalloc
The following ports are currently installed:
  jemalloc @5.3.0_2+universal (active) requested_variants='+universal' platform='darwin 21' archs='arm64 x86_64' date='2022-08-22T17:31:05-0700'

so -- either clean and try again, or will have to find someone with an M1 to see if they can reproduce the build error and take it from there.

[Dave-Allured (Dave Allured)]

Configure did not correctly detect the LG_VADDR parameter in this case. ​Main.log, line 456:

:info:configure checking number of significant virtual address bits... 0

The bogus zero value corresponds with the indicated errors, "can not narrow expression to unsigned int". This should be a processor-specific value like 48 or 56. I'm not sure which.

There is a configure option to provide a fixed value, and skip auto detection. From jemalloc/INSTALL.md:

• --with-lg-vaddr=<lg-vaddr>

Specify the number of significant virtual address bits. By default, the configure script attempts to detect virtual address size on those platforms where it knows how, and picks a default otherwise. This option may be useful when cross-compiling.

There is an upstream issue with some useful discussion: ​https://github.com/jemalloc/jemalloc/issues/1997

[kencu (Ken)]

yep, that looks like it. There is a patch in your referenced issue that might prove useful.

[Dave-Allured (Dave Allured)]

All right. Jemalloc is currently in good support, so I would like to try to fix this upstream. Bit layout of the virtual address is highly specialized knowledge, so I would prefer to keep that upstream, rather than in a port file.

I am not at all familiar with MP universal builds. Does anyone have advice for best practice to pass processor selection to upstream's configure process, when cross compiling?

[potterpg]

OK, so add this somewhere:

--with-lg-page=48

where should I add it, so I can try it out on my M1 Mac? I'm guessing in the jemalloc-5.3.0-arm64 folder somewhere?

Tried manual

 ./configure --with-lg-page=48

complained with too small a hugepage so tried

./configure --with-lg-page=48 --with-lg-hugepage=48

which it seemed to accept. followed by a make, which just had many errors, so I guess missing many parameters.

Happy to try any suggestions to help, but clearly I need advice of what needs doing.

[kencu (Ken)]

you would have to add the right args to the merger universal args for the arm64 universal build only, if you prefer not to use the patch.

see the libvpx Portfile, for an example, amongst many others

​https://github.com/macports/macports-ports/blob/master/multimedia/libvpx/Portfile

I would do it, but no M1 to test it, so..l

[potterpg]

not a preference, I don't know how to use 'the' patch!

So, line 73

configure.args-append  --enable-shared

add something similar ?

configure.args-append --with-lg-page=48 --with-lg-hugepage=48

Maybe seperate lines?

or do you mean something else?

Let me know, and I can test it for you...

[potterpg]

In the Portfile I Tried:

platform darwin {
    if {${build_arch} eq "arm64"} {
        configure.args-append --with-lg-page=48
        configure.args-append --with-lg-hugepage=48
    }
}

The config.log file suggests it was used as planned:

It was created by configure, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure --prefix=/opt/local --with-jemalloc-prefix= --with-lg-page=48 --with-lg-hugepage=48 --host=aarch64-apple-darwin21.6.0

With further errors! ie

:info:build include/jemalloc/internal/bitmap.h:90:4: error: "Unsupported bitmap size"
:info:build #  error "Unsupported bitmap size"

and

:info:build include/jemalloc/internal/slab_data.h:9:18: error: use of undeclared identifier 'BITMAP_GROUPS_MAX'
:info:build         bitmap_t bitmap[BITMAP_GROUPS_MAX];

Which seemed odd errors for a seemingly working port on other architectures...

:info:build fatal error: too many errors emitted, stopping now [-ferror-limit=]
:info:build 29 warnings and 20 errors generated.

[kencu (Ken)]

look at the libvpx portfile I referenced above, and the muniversal Portgroup

you will need to manipulate

merger_configure_args(${my_arch})

to add the needed argument to the arm build.

or use the patch here instead if you prefer that, which also should work

​https://github.com/jemalloc/jemalloc/issues/1997#issuecomment-1041589117

[potterpg]

Thanks Ken,

Finally understood what that means, literally create a file with that as contents, and then install the port...

So, duplicated the patch-quantum.diff, changed the contents to the linked comment...

Still seem to have the reported errors above though...

[potterpg]

uninstalled it completely, then installed again...

This time it seemed to be fine?

Yes, confirmed, the lib is universal...

So, don't know what the previous errors were about.

[kencu (Ken)]

In b96e6fc66897b29c45391b8cf70dd191f9d78167/macports-ports (master):

jemalloc: fix universal build on arm64

if rosetta is installed the test for address bits fails
when running the x86_64 configure.

closes: #65671
closes: #65748