rust: Update to 1.74.0

[herbygillot (Herby Gillot)]

I know this is annoying since you just got an update out, but just FYI: ​https://newreleases.io/github/rust-lang/rust?version=1.72.0

Is there a way we can make the process of updating Rust in MacPorts easier, or make it possible so that other maintainers can help do so?

[mascguy (Christopher Nielsen)]

Rust and Cargo aren't even building across-the-board yet for the recent update.

So that needs to be addressed first, before updating again.

[mascguy (Christopher Nielsen)]

Replying to mascguy:

Rust and Cargo aren't even building across-the-board yet for the recent update.

So that needs to be addressed first, before updating again.

Going forward, we need validate that rust, cargo, cargo-c, etc, build across-the-board, before updating. That can be done with -devel ports, which I'll take care of adding over the next few days.

These are way too critical to remain broken for weeks, even if it's only for older macOS releases.

[mascguy (Christopher Nielsen)]

Replying to mascguy:

Going forward, we need validate that rust, cargo, cargo-c, etc, build across-the-board, before updating. That can be done with -devel ports, which I'll take care of adding over the next few days.

I'm going to see if I can fix the current build issues for rust and cargo, before adding the -devel ports. Stay tuned...

[erikbs]

Any update on the issue?

I have tried a few things, including passing different values to configure.compiler as part of the port update command, without success. I also tried isolating the bug by compiling only the file where things fail, so that I could set RUST_MIN_STACK=104857600 as suggested in comment:ticket:68015:1, but I am not familiar enough with Rust to achieve what I wanted. I would guess that the Rust developers would also be more likely to investigate the issue if they had a minimum working example to look at, especially if it is possible to tweak it so that compilation fails on supported platforms too.

Finally I tried compiling 1.72 instead of 1.71, updating the other crate versions as needed. Given that the bug suddenly appeared and only on some Mac versions, I hoped that it could somehow disappear just as quickly as it came. It took some time and, unfortunately, did not help. A part of me wants to do 1.73, but my expectations are too low since 1.72 also failed.

Might be outdated info (2015), but I read that RUST_MIN_STACK ​does not control the stack size of main. There is a link in that post to the source code on Git, but the file no longer exists and I have not been able to find out where the code went. Apparently the source code controls (or at least controlled in 2015) the stack size of the main program, so I would think that a patch could be a possible solution if only we find out where that stack size constant is set. Also I wonder what kind of stack overflow it is that we see. Is it the program that overflows the max stack size Rust imposes on it or is it that the stack size in the Rust source code exceeds the value of the operating system? I also wonder why this happens on e.g. 10.9 (where I am) and 10.10, while both older and newer versions do not have this issue.

No matter what I do with RUST_MIN_STACK, the otool reports a stack size of 0, which apparently means that none was provided:

$ otool -lV /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_rust/rust/work/rustc-1.71.1-src/build/bootstrap/debug/bootstrap | grep stack
 stacksize 0

Running lldb I get an EXC_BAD_ACCESS stop with code 2, which is KERN_PROTECTION_FAILURE, i.e. stack buffer overflow, the same as comment:ticket:68015:2. The backtrace, being 23 frames long, reveals that there is no infinite recursion. In frame 0 the value of the RSP register is 0x00007fff5fb80238, while in frame 23 it is 0x00007fff5fbff8d8. The difference is 521 888, but, if I have understood things correctly, since RSP holds the return address, this number only tells us the approximate stack size. The instruction that fails is 0x1000cc8ee: movq $0x0, (%rsp). From this I would guess that the stack size is 512 kiB? And that the instruction at 0x1000cc8ee writes to an address that causes stack buffer overflow with a stack size of 512 kiB? However, according to the Rust docs, ​the default stack size on “Tier-1 platforms” is 2 MiB. x86_64-apple-darwin is a “Tier-1 platform”, but the notes say 10.12+. Maybe it somehow is still set to 512 kiB, I have no idea, but with that in mind I wanted to try to set the stack size to 2 MiB explicitly during linking (​helpful resource). Just to have it mentioned, ulimit -s reports a stack size limit of 8192 (8 MiB).

Just to make sure that I am on the right track, I compiled the following simple C++ program (stack.cpp):

int main()
{
  char buf[1024*1024];
  return buf[777];
}

with clang++ -o stack stack.cpp -Wl,-stack_size,0x200000 (stack size 2 MiB) and clang++ -o stack stack.cpp -Wl,-stack_size,0x100000 (stack size 1 MiB). As expected, it runs normally in the first case and SEGFAULTs in the second case (with LLDB giving me EXC_BAD_ACCESS/KERN_PROTECTION_FALIURE).

When running man ld, it reports a default stack size of 8 MiB:

     -stack_size size
                 Specifies the maximum stack size for the main thread in a
                 program.  Without this option a program has a 8MB stack.  The
                 argument size is a hexadecimal number with an optional lead-
                 ing 0x. The size should be a multiple of the architecture's
                 page size (4KB or 16KB).

Thus I have no idea why or how the Rust bootstrap binary would end up with a stack size of 512 kiB – maybe I am wrong about it being that after all.

Back to setting stack size explicitly. I edited /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_rust/rust/work/rustc-1.71.1-src/src/bootstrap/bootstrap.py, adding -C link-args=-Wl,-stack_size,0x200000 (stack size 2 MB) to the linker flags hoping to adjust it up or down until I hit a suitable value. I did so by editing line 890 (in the function build_bootstrap) so that it reads

env["RUSTFLAGS"] += " -Wrust_2018_idioms -Wunused_lifetimes -C link-args=-Wl,-stack_size,0x200000"

Unfortunately, the linker flag is passed to the linker during all stages of the linking. I therefore get the following error:

ld: -stack_size option can only be used when linking a main executable

I have spent a few minutes trying to find out if it possible to pass a linker flag to the final linking only, but that does not seem to be possible.

[erikbs]

I had to try one last thing: editing the linker, which is /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_rust/rust/work/compwrap/ld/opt/local/bin/clang-mp-17. This is a bash script that inserts Macports Legacy Support flags and forwards its input to /opt/local/bin/clang-mp-17. I modified it so that unless the command contained dynamiclib, it would set the stack size to 16 MiB:

#!/bin/bash
if [[ "${@}" == *"dynamiclib"* ]]; then
   exec /opt/local/bin/clang-mp-17    ${MACPORTS_LEGACY_SUPPORT_CPPFLAGS} "${@}"
else
   exec /opt/local/bin/clang-mp-17    ${MACPORTS_LEGACY_SUPPORT_CPPFLAGS} "${@}" -Wl,-stack_size,0x1000000
fi

Now otool reports a stack size of 16 MiB:

$ otool -lV /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_rust/rust/work/rustc-1.71.1-src/build/bootstrap/debug/bootstrap | grep stack
 stacksize 16777216

Of course I also had to increase the ulimit -s value for this to work – I chose 32 MiB. Still the bootstrap binary SEGFAULTs and LLDB still reports EXC_BAD_ACCESS (code 2) with a RSP difference of about 512 kiB between the top and bottom frame, just as it did before. I also tried setting the stack size to 2 MiB, but the result was the same.

I give up, at least for now.

[raimue (Rainer Müller)]

Current version is rust 1.74.1:

• ​https://blog.rust-lang.org/2023/11/16/Rust-1.74.0.html
• ​https://github.com/rust-lang/rust/releases/tag/1.74.1

[halostatue (Austin Ziegler)]

@schamshula and I have been trying to get 1.74.1 working on #68364. I understand that older versions of macOS are still failing to build rust, but there are several common tools that cannot be updated because they are now depending on a newer version of Rust and 1.75 is due to be released on the 28th of December.

[raimue (Rainer Müller)]

Then let's move the discussion to #68364 and mark this as a duplicate.