Opened 8 years ago

Closed 7 years ago

Last modified 7 years ago

#53625 closed defect (fixed)

qscintilla: requires C++11, not using the right compiler

Reported by: mojca (Mojca Miklavec) Owned by: michaelld (Michael Dickens)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Port: qscintilla

Description

qscintilla no longer compiles without C++11:

../src/Document.cpp:22:10: fatal error: 'regex' file not found
#include <regex>
         ^

Attachments (1)

qscintilla.log.txt.bz2 (9.5 KB) - added by chrpepper 7 years ago.
10.6.8 build (error c++11 compiler option)

Download all attachments as: .zip

Change History (41)

comment:1 Changed 8 years ago by mojca (Mojca Miklavec)

Cc: MarcusCalhoun-Lopez added

comment:2 Changed 8 years ago by michaelld (Michael Dickens)

In 01168f73/macports-ports:

qscintilla: require c++11; rev-bump just in case; addresses ticket #53625 .

comment:3 Changed 8 years ago by michaelld (Michael Dickens)

Resolution: fixed
Status: newclosed

Thanks for the pointer!

comment:4 Changed 8 years ago by mojca (Mojca Miklavec)

Resolution: fixed
Status: closedreopened

This doesn't work yet because the port is not using the right compiler.

Neither the compiler nor the flags setting the stdlib are respected during the build.

Last edited 8 years ago by mojca (Mojca Miklavec) (previous) (diff)

comment:5 Changed 8 years ago by mojca (Mojca Miklavec)

Summary: qscintilla: requires C++11qscintilla: requires C++11, not using the right compiler

comment:6 Changed 8 years ago by michaelld (Michael Dickens)

In b9de20c6/macports-ports:

qscintilla: use qmake PortGroup instead of qt4, to "use the right compiler" and c++11 flags per ticket #53625 .

comment:7 Changed 8 years ago by michaelld (Michael Dickens)

OK. How about now?

comment:8 Changed 8 years ago by mojca (Mojca Miklavec)

Thanks, that looks better. Or at least the compiler is properly chosen. It fails to compile though:

In file included from ../src/Document.cpp:22:
In file included from /opt/local/include/gcc6/c++/regex:35:
/opt/local/include/gcc6/c++/bits/c++0x_warning.h:32:2: error: This file requires compiler and library support for the ISO C++ 2011 standard. This support must be
      enabled with the -std=c++11 or -std=gnu++11 compiler options.
#error This file requires compiler and library support \
 ^
../src/Document.cpp:2581:41: error: expected ';' at end of declaration list
        ByteIterator(const ByteIterator &other) NOEXCEPT {
                                               ^
                                               ;
../src/Document.cpp:2756:41: error: expected ';' at end of declaration list
        UTF8Iterator(const UTF8Iterator &other) NOEXCEPT {
                                               ^
                                               ;

Adding -std=c++11 helps to go over this error, but let me try to check the full build first. I guess that this flag should be added by the PortGroup, not by the port itself.

comment:9 Changed 8 years ago by mojca (Mojca Miklavec)

Adding -std=c++11 in the PortGroup helped. The compilation almost finished, except that the build is still not UsingTheRightCompiler for the final linking step.

As well as some weird "library not found" error.

/usr/bin/clang++ -headerpad_max_install_names -arch x86_64 -single_module -dynamiclib -Xarch_x86_64 -mmacosx-version-min=10.7 -o libqscintillaplugin.dylib qscintillaplugin.o moc_qscintillaplugin.o  -F/opt/local/libexec/qt4/Library/Frameworks -F/opt/local/libexec/qt4/lib  -L../Qt4Qt5 -lqscintilla2_qt4 -F/opt/local/libexec/qt4/Library/Frameworks -F/opt/local/libexec/qt4/lib -L/opt/local/libexec/qt4/lib -framework QtScript -F/opt/local/Library/Frameworks -framework QtCore -framework QtXml -framework QtGui -framework QtDesigner  
ld: library not found for -lqscintilla2_qt4

The build folder does contain Qt4Qt5/libqscintilla2_qt4.13.0.0.dylib, but probably a symlink from Qt4Qt5/libqscintilla2_qt4.dylib is missing.

The library name is also super strange and looks like "someone forgot to run install_name_tool":

> otool -L /opt/local/libexec/qt4/lib/libqscintilla2_qt4.13.0.0.dylib 
/opt/local/libexec/qt4/lib/libqscintilla2_qt4.13.0.0.dylib:
	libqscintilla2_qt4.13.0.0.dylib (compatibility version 13.0.0, current version 13.0.0)
	/opt/local/libexec/qt4/Library/Frameworks/QtGui.framework/Versions/4/QtGui (compatibility version 4.8.0, current version 4.8.7)
	/opt/local/libexec/qt4/Library/Frameworks/QtCore.framework/Versions/4/QtCore (compatibility version 4.8.0, current version 4.8.7)
	/opt/local/lib/libgcc/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.22.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
> otool -L /opt/local/libexec/qt4/share/plugins/designer/libqscintillaplugin.dylib
/opt/local/libexec/qt4/share/plugins/designer/libqscintillaplugin.dylib:
	libqscintillaplugin.dylib (compatibility version 0.0.0, current version 0.0.0)
	libqscintilla2_qt4.13.0.0.dylib (compatibility version 13.0.0, current version 13.0.0)
	/opt/local/libexec/qt4/Library/Frameworks/QtScript.framework/Versions/4/QtScript (compatibility version 4.8.0, current version 4.8.7)
	/opt/local/libexec/qt4/Library/Frameworks/QtCore.framework/Versions/4/QtCore (compatibility version 4.8.0, current version 4.8.7)
	/opt/local/libexec/qt4/Library/Frameworks/QtXml.framework/Versions/4/QtXml (compatibility version 4.8.0, current version 4.8.7)
	/opt/local/libexec/qt4/Library/Frameworks/QtGui.framework/Versions/4/QtGui (compatibility version 4.8.0, current version 4.8.7)
	/opt/local/libexec/qt4/Library/Frameworks/QtDesigner.framework/Versions/4/QtDesigner (compatibility version 4.8.0, current version 4.8.7)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
Last edited 8 years ago by mojca (Mojca Miklavec) (previous) (diff)

comment:10 Changed 8 years ago by mojca (Mojca Miklavec)

See also #53599 and a64f1147 in particular.

comment:11 Changed 8 years ago by mojca (Mojca Miklavec)

I managed to compile the library after applying the following patch:

  • _resources/port1.0/group/cxx11-1.1.tcl

    diff --git a/_resources/port1.0/group/cxx11-1.1.tcl b/_resources/port1.0/group/cxx11-1.1.tcl
    index b72a9e74f5..88fb1cb15c 100644
    a b if {${cxx_stdlib} eq "libstdc++" } { 
    6363        # for OS X Maverick and above, users must select libstdc++, so
    6464        #    assume they want default ABI compatibility
    6565        # see https://gcc.gnu.org/onlinedocs/gcc-5.2.0/libstdc++/manual/manual/using_dual_abi.html
    66         configure.cxxflags-append -D_GLIBCXX_USE_CXX11_ABI=0
     66        configure.cxxflags-append -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0
    6767    }
    6868} else {
    6969    # GCC compilers can not use libc++
  • _resources/port1.0/group/qmake-1.0.tcl

    diff --git a/_resources/port1.0/group/qmake-1.0.tcl b/_resources/port1.0/group/qmake-1.0.tcl
    index a1fb1028ca..387c991d1c 100644
    a b pre-configure { 
    4444                                    "QMAKE_CXX=${configure.cxx}" \
    4545                                    "QMAKE_OBJC=${configure.objc}" \
    4646                                    "QMAKE_CFLAGS=\"${configure.cflags} [get_canonical_archflags cc]\"" \
    47                                     "QMAKE_CXXFLAGS=\"${configure.cxxflags} [get_canonical_archflags cxx]\""
     47                                    "QMAKE_CXXFLAGS=\"${configure.cxxflags} [get_canonical_archflags cxx]\"" \
     48                                    "QMAKE_LINK_C=${configure.cc}" \
     49                                    "QMAKE_LINK_C_SHLIB=${configure.cc}" \
     50                                    "QMAKE_LINK=${configure.cxx}" \
     51                                    "QMAKE_LINK_SHLIB=${configure.cxx}"
     52
    4853    configure.universal_args-delete --disable-dependency-tracking
    4954
    5055    if {[variant_exists universal] && [variant_isset universal]} {

(Perhaps -std=c++11 should be added for all ports that include the cxx11 PortGroup?)

I believe that cxxflags for the linker are still missing above since I had to manually add -D_GLIBCXX_USE_CXX11_ABI=0 -stdlib=macports-libstdc++ when calling the linker for libqscintilla2_qt4, else the linker would fail (I added both flags just in case, maybe just fixing the stdlib would suffice). This wasn't necessary for the other library libqscintillaplugin, but I guess that's just because it doesn't use C++11?

The qmake5 PortGroup uses some pretty "complex" code to decide which linker flags to use and I wasn't sure which one to apply here, but QMAKE_LFLAGS should be set in one way or another.

I also had to do the following in order to finish compilation:

  • in Qt4Qt5:
    sudo -u macports install_name_tool -id /opt/local/libexec/qt4/lib/libqscintilla2_qt4.13.0.0.dylib libqscintilla2_qt4.13.0.0.dylib 
    sudo -u macports ln -s libqscintilla2_qt4.13.0.0.dylib libqscintilla2_qt4.dylib 
    
  • in designer-Qt4Qt5:
    sudo -u macports install_name_tool -id /opt/local/libexec/qt4/share/plugins/designer/libqscintillaplugin.dylib libqscintillaplugin.dylib 
    

Let me check whether Octave works after this trickery.

comment:12 Changed 8 years ago by mojca (Mojca Miklavec)

Octave compiles and works fine, the built-in editor doesn't start (once I try to start it, it hangs forever). But also keep in mind that I didn't use the right linker flags when compiling qscintilla.

Some errors/warnings shown:

octave-gui[91413:3803] unlockFocus called too many time.
objc[91413]: ALT HANDLERS: *** over 1000 alt handlers installed; this is probably a bug

comment:13 Changed 8 years ago by michaelld (Michael Dickens)

Interesting. On what OS version are you compiling Qsci? I'm using latest stock Sierra & Xcode, and Qsci compiles fine with or without -std=c++11 added. It is certainly true that <regex> is a C++11 file, so I'm guessing there's a warning in the log file somewhere about it.

What would be ideal is if the default CXXFLAGS were "-std=c++98" or whatever. Then if C++11 is desired, we swap that to "-std=c++11 -Wc++11-extension". Likewise for C++14 or C++17, assuming the updated -W is valid for those. Basically: Try to force the compiler to a certain standard & warn if not abiding by it.

Sort of like checking for whether the port can "use the right compiler".

comment:14 Changed 8 years ago by michaelld (Michael Dickens)

Back to your hacking: The Qsci build script has an install_name_tool tweak that's supposed to rename the libqsci library to be just "*.13.dylib". Not sure why it's not being executed for you. Maybe I'll give your qmake PG tweaks a whirl & see what comes of it. There has got to be a reasonable & cross-port way to coerce qmake into doing the right thing for both compilers and linkers & C++XX support for the vast majority of ports.

comment:15 Changed 8 years ago by mojca (Mojca Miklavec)

Compare the following:

> clang++-mp-4.0 -stdlib=libc++ test.cpp -o test
# OK
> clang++-mp-4.0 -stdlib=libstdc++ test.cpp -o test
test.cpp:1:10: fatal error: 'regex' file not found
#include <regex>
         ^~~~~~~
1 error generated.
> clang++-mp-4.0 -stdlib=macports-libstdc++ test.cpp -o test
In file included from test.cpp:1:
In file included from /opt/local/include/gcc6/c++/regex:35:
/opt/local/include/gcc6/c++/bits/c++0x_warning.h:32:2: error: This file requires compiler and library support for the ISO C++ 2011 standard. This support must be
      enabled with the -std=c++11 or -std=gnu++11 compiler options.
#error This file requires compiler and library support \
 ^
1 error generated.

Maybe Marcus just needs to improve his patch for macports-libstdc++.

What would be ideal is if the default CXXFLAGS were "-std=c++98" or whatever. Then if C++11 is desired, we swap that to "-std=c++11 -Wc++11-extension". Likewise for C++14 or C++17, assuming the updated -W is valid for those. Basically: Try to force the compiler to a certain standard & warn if not abiding by it.

I'm not particularly happy about this idea. Clang already works painlessly in most cases, C++98 soucres compile just fine under C++11 compiler, it's just the recent change with -stdlib=macports-libstdc++ that could be improved and use C++11 by default. Note that this macports-libstdc++ is pretty experimental.

If we force the C++98 flag, we are just calling for a huge number of broken port.

PS: playing on 10.7.

comment:16 Changed 8 years ago by michaelld (Michael Dickens)

1) So on my 10.7 box, macports-clang-4.0 is unstable & I have to specify using 3.9 instead. As an example from watching debug output, when cmake hits "checking for log" (as in the function 'log'), it hangs & free file storage ("hard drive" or SSD) quickly decreases until the system hangs. No idea why. I'd much prefer to not be bleeding edge for C++11 support. 3.9 seems like a stable release, yes?

2) Using "-stdlib=macports-libstdc++" should definitely use C++11 by default: Agreed! Note that C++11 is not a strict superset of C++03 (or C++98), so there might be some idiosyncratic programming that fails to build using this change.

3) On my 10.7 box, I couldn't get the necessary dependencies (re)installed to test out how Qsci works with the C++11 change. I'll try again this week. The issue that I found was the qt5-* won't use the right compiler; it always chooses clang-4.0 even if I specify 3.9, and then (1) happens. I haven't looked into that issue yet.

comment:17 Changed 8 years ago by mojca (Mojca Miklavec)

One important note. clang-x.y needs a special patch to support -stdlib=macport-libstdc++ which has only been enabled by default on clang-4.0.

> port info clang-3.9

clang-3.9 @3.9.1_3 (lang)
Variants:             [+]analyzer, assertions, debug, libstdcxx, universal

If you want to test all this with clang 3.9, you need to make sure that you install it with the +libstdcxx variant (and thus have to recompile it since the version compiled by the buildbot doesn't have that option enabled). If you then want to use clang 3.9 with the cxx11-1.1 PortGroup, you need to add macports-clang-3.9 to compiler.whitelist.

But I find it strange that clang 4.0 misbehaves. It works just fine for me.

comment:18 Changed 7 years ago by michaelld (Michael Dickens)

BTW> Since we last wrote here, clang-4.0 has been updated & now works on my 10.8 install.

Is this ticket's issue still relevant?

comment:19 Changed 7 years ago by chrpepper

Yes, at least for my installation... Just tried on 10.6.8 after installing clang-4.0; -stdlib=macports-libstdc++ is used automatically, error is "/opt/local/include/gcc6/c++/bits/c++0x_warning.h:32:2: error: This file requires compiler and library support for the ISO C++ 2011 standard. "

comment:20 Changed 7 years ago by michaelld (Michael Dickens)

OK. Can you do the following so that we an see what's going on?

sudo port clean qscintilla
sudo port install qscintilla

and, when that errors out, do:

cp $(port logfile qscintilla) ~/Desktop/qscintilla.log.txt
bzip2 ~/Desktop/qscintilla.log.txt

and then attach the compressed log file here.

I'll also see if I see this issue on my 10.8 box ... not the same as 10.6, but the compiler issue should be similar.

Changed 7 years ago by chrpepper

Attachment: qscintilla.log.txt.bz2 added

10.6.8 build (error c++11 compiler option)

comment:21 Changed 7 years ago by chrpepper

Sorry for the delay, did not watch this ticket the last days. Here is the log file, hope it helps.

comment:22 Changed 7 years ago by chrpepper

Cc: chrpepper added

comment:23 Changed 7 years ago by kencu (Ken)

FYI, I had no issue installing this on 10.6.8, without modifications to the portfile. I have the default compiler set to clang-3.9, as per the LibcxxOnOlderSystems instructions, and I'm using libc++.

$ port -v installed qscintilla
The following ports are currently installed:
  qscintilla @2.10_1 (active) platform='darwin 10' archs='x86_64' date='2017-03-01T23:19:28-0800'

comment:24 Changed 7 years ago by michaelld (Michael Dickens)

OK. I'm following the issue now (again). I've seen this with other ports that are using the clang-4.0 / macports-libstdc++ build. We have to add "-std=c++11" to the build somehow, just for these, and probably won't hurt for newer compilers either. I'll see what I can do.

comment:25 Changed 7 years ago by kencu (Ken)

I worked on this a bit regarding adding into the clang code directly <https://trac.macports.org/ticket/53634#comment:6>. It's possible, but it's a bit hard as there needs to be a test you can call in the clang code to see if macports-libstdc++ is the selected stdlib, and I don't know how to do that from that point in the code.

MUCH easier would be to add -std=c++11 in the cxx11 1.1 portgroup code, but that runs the risk of trashing some other specified standard in the port itself.

comment:26 Changed 7 years ago by michaelld (Michael Dickens)

It's almost trivial to add in the std=c++11 to CXXFLAGS in each port that uses the cxx11 1.1 PG. That avoids the trashing you mention. Maybe a little clumsy, but since different ports use different build systems (automake, cmake, bmake, etc), this is probably the right way to go for now instead of trying a blanket approach that is likely to cause headaches. In my opinion.

So at least for this port I'll just add in the -std argument, to the port itself as Mojca describes. That will at least fix this specific issue, I think.

comment:27 Changed 7 years ago by chrpepper

I've added a line "configure.cxxflags-append -std=c++11" to the Portfile. This lets it compile for my 10.6.8 box (it could cause problems for newer OS X versions), but at linking there are undefined symbols like "std::throw_out_of_range_fmt(char const*, ...)" , "std::throw_bad_function_call()" etc. So, some linker flag is needed, too...

comment:28 Changed 7 years ago by chrpepper

I am afraid I'll have to go the way Ken (comment 23) described, as this seems to be the cleanest solution but it will take a very long time to re-build all the ports. So I have some hope I can get qscintilla (and later octave) running without rebuilding all ports. Do I have a chance?

comment:29 Changed 7 years ago by kencu (Ken)

I would say so:

$ port -v installed octave
The following ports are currently installed:
  octave @4.2.1_1+accelerate+app+docs+fltk+gfortran+graphicsmagick+qt4+sound (active) platform='darwin 10' archs='x86_64' date='2017-04-25T11:21:43-0700'

at least that has been my experience.

comment:30 Changed 7 years ago by michaelld (Michael Dickens)

I'm almost there with a fix for libstdc++ users. As Mojca described above, it's multi-part: (1) fixing the qmake PG to make sure qmake has the right settings for linking (generically); (2) getting the right settings to qmake to do c++11 compiling; and (3) making sure the correct libstdc++ setting is provided for both compiling and linking, so that the correct C++ framework is used. Fun fun fun!

comment:33 Changed 7 years ago by michaelld (Michael Dickens)

In 94da98fbe2b661d134e69715b52cca0c012e4901/macports-ports:

qscintilla: set the correct stdlib for linking for older compilers.

Addresses ticket #53625 .

comment:34 Changed 7 years ago by michaelld (Michael Dickens)

Resolution: fixed
Status: reopenedclosed

OK. With these 3 changes in place, QSci should now build correctly when using libstdc++ or libc++. I haven't tried -using- it yet, but that can be a different ticket ;)

comment:35 Changed 7 years ago by chrpepper

awesome! Thank you very much for taking care of this ticket, Michael, and thanks to all the contributors! Now the last 10.6.8 users will be happy using QSci again. I've just had success in "port upgrade qscintilla" and will now upgrade the remaining ports. Great!

comment:36 Changed 7 years ago by chrpepper

Cc: chrpepper removed

comment:37 Changed 7 years ago by michaelld (Michael Dickens)

You're welcome! Good luck!

comment:38 Changed 7 years ago by cer54 (Carl Edward Rasmussen)

also compiles on my 10.7.5.

comment:39 Changed 7 years ago by chrpepper

And I could finally build the actual octave 4.2.0, it even seems to work. Thanks to you, Michael, I now have an up-to-date Macports on my old 10.6.8 MBP late 2006. Maintaining a port using C++ for different OS X versions must be lots of fun...

comment:40 Changed 7 years ago by michaelld (Michael Dickens)

Yay! I'm so glad this is working for folks!

Note: See TracTickets for help on using tickets.