mame: update to latest release (0.226 or later)

[mascguy (Christopher Nielsen)]

It looks like the current version available via MacPorts is 0.206 (nearly two years old), while the latest release is 0.226 (released 10/28/2020). And version 0.227 may be released before the end of 2020.

Would it be possible to update this port?

[ryandesign (Ryan Carsten Schmidt)]

When I updated the port to 0.206 in December 2019, it was the newest version that I was able to build. I also tried building 0.207, 0.208, 0.210 and 0.216 and they all failed to build.

[ryandesign (Ryan Carsten Schmidt)]

0.226 fails the same way that 0.207 and later fail:

Undefined symbols for architecture x86_64:
  "pstring_t<putf_traits<1ul, char> >::operator=(pstring_t<putf_traits<1ul, char> >&&)", referenced from:
      plib::pfmt& plib::pfmt::format_element<char const*>(char32_t, char const*&&) in liboptional.a(netlist.o)
      plib::pfmt& plib::pfmt::format_element<unsigned int const&>(char32_t, unsigned int const&&&) in liboptional.a(netlist.o)
      plib::pfmt& plib::pfmt::format_element<unsigned long const&>(char32_t, unsigned long const&&&) in liboptional.a(netlist.o)
      plib::pfmt& plib::pfmt::format_element<char const*&>(char32_t, char const*&&&) in liboptional.a(netlist.o)
      plib::pfmt& plib::pfmt::format_element<double const&>(char32_t, double const&&&) in liboptional.a(netlist.o)
      std::__1::vector<std::__1::pair<pstring_t<putf_traits<1ul, char> >, netlist::factory::element_t*>, std::__1::allocator<std::__1::pair<pstring_t<putf_traits<1ul, char> >, netlist::factory::element_t*> > >::insert(std::__1::__wrap_iter<std::__1::pair<pstring_t<putf_traits<1ul, char> >, netlist::factory::element_t*> const*>, std::__1::pair<pstring_t<putf_traits<1ul, char> >, netlist::factory::element_t*>&&) in libnetlist.a(nl_setup.o)
      netlist::nlparse_t::register_param(pstring_t<putf_traits<1ul, char> > const&, pstring_t<putf_traits<1ul, char> > const&) in libnetlist.a(nl_setup.o)
      ...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[ryandesign (Ryan Carsten Schmidt)]

Here is the bug report: ​https://github.com/mamedev/mame/issues/6004

[ryandesign (Ryan Carsten Schmidt)]

The developers of mame have repeated closed my bug report saying it's not their problem. If you want mame updated in MacPorts, please help us figure out why it doesn't build and how to fix it.

[mascguy (Christopher Nielsen)]

Sounds good Ryan. I'll take a look!

[mascguy (Christopher Nielsen)]

So far, I've been able to successfully build and run Mame 0.226, via MacPorts, on MacOS 10.13, 10.14, and 10.15.

The only local change necessary, was updating the portfile to specify version 0.226 (plus matching checksums). Otherwise, a standard 'sudo port -N install mame' was all that was required.

The only minor snafu I encountered, occurred at runtime: Mame was unable to find the requisite 'bgfx' files. As a workaround, I extracted the 'bgfx' tree from the release tarball, and copied those to ~/.mame/bgfx. There are other missing files too, such as those for language translations, plugins, etc... as well as the man pages. (Net-Net, there are some minor port tweaks needed, to ensure a complete Mame installation.)

In any case, my Xcode versions are:

MacOS Version	Xcode Version
10.13.6	10.1
10.14.6	11.3.1
10.15.6	12.0

As for MacPorts, everything is up-to-date with the latest ports, as of 10:00am EST today.

Just curious... which release(s) of MacOS have you tried to build against?

[kencu (Ken)]

I made up my own mame port a couple of years ago and copied in all the other parts to make it work.

I haven't updated it in a while, but you might use some or all of what I added to bring the current Portfile into shape.

<​https://github.com/kencu/myports/blob/master/emulators/mame/Portfile>

[mascguy (Christopher Nielsen)]

Thanks Ken, that's exactly what I was looking for!

As for the builds: In the interest of completeness, I tried building Mame on MacOS 10.12 this morning. And unfortunately it failed, with a link error very similar to Ryan's:

:info:build Undefined symbols for architecture x86_64:
:info:build   "pstring_t<putf_traits<1ul, char> >::pstring_t(pstring_t<putf_traits<1ul, char> > const&)", referenced from:
:info:build       netlist::factory::truthtable_create(netlist::tt_desc&, netlist::factory::properties&&) in libnetlist.a(nlid_truthtable.o)
:info:build ld: symbol(s) not found for architecture x86_64

However, I'm presently using a custom portfile, with Ken's key changes merged in. So I may need to go back to the original, to eliminate any variables.

Regardless, I haven't given up yet, as this link error seems a bit suspicious. Hopefully it's simply a matter of ensuring the correct libraries are being referenced, during the link phase.

Comments/thoughts are welcome. Otherwise I'll keep digging.

[kencu (Ken)]

I am not sure what is causing the link error exactly, but if it builds on 10.13 but not on 10.12, the first thing I'd think of would be either the c++ standard (c++17?) needs to be set in the cxxflags, or the clang compiler itself on 10.12 is too old, and we need to blacklist that clang and force the build to clang-9.0 (or newer).

If neither of those two simple fixes does the trick, then it is more complicated -- Google is your friend there, as usually someone else has already fixed the issue, whatever it is. If Google gives you nothing, then you need to dig in.

[kencu (Ken)]

I don't speak deep c++, but that missing symbol would appear to be defined here in this c++ header:

<​https://git.redump.net/mame/tree/src/lib/netlist/plib/pstring.h>

if that header is actually being included in the build, then -- not sure -- maybe the function is somehow being optomized right out of code by the compiler?

[kencu (Ken)]

Oh -- I see you're on it in the upstream bug report.

[ryandesign (Ryan Carsten Schmidt)]

I didn't see the comments in this ticket until now. I was about to happily commit the update, along with blacklisting clang < 1000, but then I see what you've already said above, that it doesn't work:

$ mame64
Error opening translation file English
Unable to open effect file bgfx/effects/gui_opaque.json
Unable to open effect file bgfx/effects/gui_blend.json
Unable to open effect file bgfx/effects/gui_multiply.json
Unable to open effect file bgfx/effects/gui_add.json
Unable to open effect file bgfx/effects/screen_opaque.json
Unable to open effect file bgfx/effects/screen_blend.json
Unable to open effect file bgfx/effects/screen_multiply.json
Unable to open effect file bgfx/effects/screen_add.json
Ignoring MAME exception: BGFX: Unable to load required shaders. Please check and reinstall the bgfx folder

Fatal error: BGFX: Unable to load required shaders. Please check and reinstall the bgfx folder

Segmentation fault: 11

[ryandesign (Ryan Carsten Schmidt)]

compiles but does not run

[kencu (Ken)]

it never did run without the additional parts. maybe your simple bump + my destrooting additions could make a functional current port.

[mascguy (Christopher Nielsen)]

Yeah, I can definitely confirm that you need those extra components.

There's just one caveat with Ken's portfile: Mame is installed to /opt/local/libexec/mame, necessitating that users add that directory to their path. Whereas ideally, the main Mame executable would be placed in /opt/local/bin, while the resources would be located somewhere like /opt/local/share/mame.

I believe that might be doable with a Mame config file we provide. There's also a makefile flag that needs to be set, to specify where that would be read from.

I'll spend a few minutes on that today, and report back.

[mascguy (Christopher Nielsen)]

On a different note, there were also some issues generating the documentation. I'll go through my notes from the past few days, and provide the details. Stay tuned.

[kencu (Ken)]

usually we make a symlink in destroot or post-destroot to ${prefix}/bin with our desired bin

I forget just now how it came to be that mame installed into libexec...

[mascguy (Christopher Nielsen)]

Replying to kencu:

usually we make a symlink in destroot or post-destroot to ${prefix}/bin with our desired bin

I forget just now how it came to be that mame installed into libexec...

My guess is that you were trying to avoid dealing with the stuff I mentioned in my earlier comment, re: a custom config file, etc.

[kencu (Ken)]

oh, I didn't set it up to install into libexec. It either did that by default or someone else set that up.

[mascguy (Christopher Nielsen)]

Okay, so as far as the translations go, I'm seeing failures when processing language 'Portuguese_Brazil':

language/Portuguese_Brazil/strings.po:1693:19: invalid control sequence
language/Portuguese_Brazil/strings.po:3024: end-of-line within string
language/Portuguese_Brazil/strings.po:3024: keyword "come" unknown
language/Portuguese_Brazil/strings.po:3024:5: syntax error
language/Portuguese_Brazil/strings.po:3024: keyword "a" unknown
language/Portuguese_Brazil/strings.po:3024: keyword "com" unknown
language/Portuguese_Brazil/strings.po:3024: keyword "algo" unknown
language/Portuguese_Brazil/strings.po:3024: keyword "mas" unknown
language/Portuguese_Brazil/strings.po:3024: keyword "o" unknown
language/Portuguese_Brazil/strings.po:3024: keyword "perde" unknown
language/Portuguese_Brazil/strings.po:3025: end-of-line within string
language/Portuguese_Brazil/strings.po:3031: end-of-line within string
language/Portuguese_Brazil/strings.po:3031: keyword "sem" unknown
language/Portuguese_Brazil/strings.po:3031:4: syntax error
language/Portuguese_Brazil/strings.po:3031: keyword "um" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "item" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "e" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "depois" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "voc" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "o" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "consegue" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "por" unknown
language/Portuguese_Brazil/strings.po:3031: keyword "exemplo" unknown
language/Portuguese_Brazil/strings.po:3032: end-of-line within string
msgmerge: found 24 fatal errors

I'm wondering if this is due to the version of the 'gettext' port? (It looks like the latest release is 0.21, dated July 2020. While ours is presently 0.19.8.1, from June 2016.)

The update is tracked by issue:58526.

[kencu (Ken)]

Replying to kencu:

oh, I didn't set it up to install into libexec. It either did that by default or someone else set that up.

you know, I think all these years later, maybe i did set that up...

[mascguy (Christopher Nielsen)]

As for the docs, I'd suggest we limit the build to just the man pages. Among other things, that eliminates the dependency on the various tex* ports, which are huge.

However, the man pages still depend on py-sphinx and the various sphinxcontrib Python libraries. And some of the latter aren't presently published via MacPorts, such as sphinxcontrib-blockdiag and sphinxcontrib-svg2pdfconverter. (Plus the various dependencies those both pull in.)

In the interest of getting Mame built locally, I installed the py-pip port, and then used that to install those two sphinxcontrib components.

But we'll need to publish the missing components via MacPorts, if we want to provide man pages.

Thoughts?

[kencu (Ken)]

in my local port, if "Brazilian Portuguese" doesn't build I'll just delete that directory post-extract.

But for the MacPorts port, really anything you might do to get it working will be an improvement from the broken state of affairs we've had the past few years.

[mascguy (Christopher Nielsen)]

Translation update: After looking into it, I'm reasonably confident that the errors are caused by a legitimate issue with the Portuguese_Brazil translation files.

I've updated the Mame issue with my latest findings.

[mascguy (Christopher Nielsen)]

Created a new Mame issue, covering the translation errors:

Mame 0.226: Translations: Minor Portuguese_Brazil Message Issues ​https://github.com/mamedev/mame/issues/7510

[mascguy (Christopher Nielsen)]

Created a new Mame pull request, covering the translation fixes:

Fixed Portuguese_Brazil translation errors noted in issue 7510 ​https://github.com/mamedev/mame/pull/7511

Changes accepted by the maintainers, and merged into the Mame master branch. Issue 7510 also closed.

[kencu (Ken)]

that's great, very well done. That's the right fix, no idea how it wasn't spotted before, to be honest.

We can add your patch to this mame build if you like, pending the next release.

[mascguy (Christopher Nielsen)]

Replying to kencu:

We can add your patch to this mame build if you like, pending the next release.

Sounds great Ken!

[kencu (Ken)]

This py-sphinxcontrib-applehelp looks like a reasonable template for new sphinx contrib ports, I think.

[mascguy (Christopher Nielsen)]

Update on Mame configuration, and the use of a soft link to the binary...

Based on some quick tests, it looks like Mame isn't soft-link aware: If it's run via link, it looks for resources relative to the link itself... and doesn't attempt to resolve said link to the actual location.

Not surprisingly, a hard link isn't any more effective. But I tested that too, just-in-case. LOL

I'm still reviewing the Mame docs, and source code, to see if I'm missing anything. But at the moment, it's not encouraging.

I'll keep digging, and see what code changes are necessary to rectify this...

[mascguy (Christopher Nielsen)]

Alternatively, we could use a simple launch wrapper script. That would also work, so long as all resource files are in child directories relative to the executable.

Thoughts?

[ryandesign (Ryan Carsten Schmidt)]

Yes, a wrapper script that cds to the right directory and launches the real executable is probably the way to go.

[ryandesign (Ryan Carsten Schmidt)]

If you're working on updating the port, #60986 is another suggested improvement you could incorporate, or leave for later.

I'm finding it frustrating to work on the port since it takes hours to compile, so if you want to work on it instead I appreciate it.

[mascguy (Christopher Nielsen)]

Replying to ryandesign:

If you're working on updating the port, #60986 is another suggested improvement you could incorporate, or leave for later.

Sounds good, I'll take a look.

[mascguy (Christopher Nielsen)]

I updated my Mame build to include the command-line tools, and they built successfully on MacOS 10.13. They all run without error as well, though I haven't tested them thoroughly.

When I have time, I'll also rebuild on 10.12, 10.14, and 10.15. Each iteration takes 50 minutes on my antique 2008-era MacPro, so I certainly appreciate the frustration. :-)

But assuming there aren't any build issues across MacOS releases, we should be able to include the tools without hesitation.

[kencu (Ken)]

I made up a local port for py-sphinxcontrib-svg2pdfconverter and it works fine. All the documentation builds with that. If it looks like it would pass the gauntlet, I might push that.

I agree wrapper script, that cd's to the libexec directory then launches. That seems to have changed since 0.206, as it used to work to have a symlink. Well, things change.

You're almost done!

[kencu (Ken)]

here it is: <​https://github.com/macports/macports-ports/commit/3808b1e877aeaf406eea20b4d9e8e723f4a00ee6>.

I don't do much python, but it looked pretty easy. I think there is a list of automatically enabled sphinx contribs that all get brought in as a group -- not sure yet, but maybe I should add this to the list, if it exists. For now, I just spec'd it separately.

That was the only additional sphinx component that I needed to build all the documentation -- I think :> I didn't see an error about the blockdiag component you mentioned. If we need that, should be simple enough to add too.

[mascguy (Christopher Nielsen)]

Replying to kencu:

That was the only additional sphinx component that I needed to build all the documentation -- I think :> I didn't see an error about the blockdiag component you mentioned. If we need that, should be simple enough to add too.

I uninstalled sphinxcontrib-blockdiag, and now I'm not seeing any doc build errors either. LOL

Indeed, I even tried building every doc variant, with 'latexpdf' being the sole failure. And that's only because the additional tex* ports, specified in your portfile, aren't presently installed.

Ah well, thanks for the catch!

[mascguy (Christopher Nielsen)]

Build testing has been completed for Mame tools, and everything looks fine across MacOS 10.12 through 10.15.

[mascguy (Christopher Nielsen)]

Created patch for translation fix:

​https://github.com/mascguy/macports-ports/blob/mascguy-mame-upgrade-0226/emulators/mame/files/mame-patch-0226-language-portuguese_brazil.diff

[mascguy (Christopher Nielsen)]

Currently working on portfile changes.

[kencu (Ken)]

don't forget to have your patches end with *.diff or *.patch

[mascguy (Christopher Nielsen)]

Replying to kencu:

don't forget to have your patches end with *.diff or *.patch

Done. Also updated URL in earlier comment.

[mascguy (Christopher Nielsen)]

While the Portfile is still being refined, it's far enough along that it could use some feedback.

Take a look, and let me know what you folks think.

​https://github.com/mascguy/macports-ports/blob/mascguy-mame-upgrade-0226/emulators/mame/Portfile

[mascguy (Christopher Nielsen)]

Great news, the work is finally done!

Plenty of comments were added to the Portfile, particularly concerning anything that might prompt questions.

Also, no warnings or errors are flagged by 'port lint':

$ sudo port lint --nitpick mame
--->  Verifying Portfile for mame
--->  0 errors and 0 warnings found.

Please review when you folks have time, and let me know your thoughts.

[mascguy (Christopher Nielsen)]

Pull request created:

​https://github.com/macports/macports-ports/pull/9314

[kencu (Ken)]

I will run this install later today; thanks for doing it.

[mascguy (Christopher Nielsen)]

Thanks for the help guys!

[mascguy (Christopher Nielsen)]

Looking forward to hearing whether you folks are able to successfully install this port locally, as most of the CI jobs failed... without any apparent Mame-specific issues.

[mascguy (Christopher Nielsen)]

Added support for the Test phase, for completeness.

[mascguy (Christopher Nielsen)]

Mame 0.226 is now officially live, so this ticket can be closed.