root6 : Update to 6.04.02

[cjones051073 (Chris Jones)]

Updates to the latest release.

Due to a build issue in the tagged release, using a slightly new git tag. To signify this I've versioned it 6.04.02.99.

Updated the default gcc variant (used just for fortran) to gcc5

Added a new variant to enable c++ 14 support. Disabled by default.

Added clang 3.7 to the compiler fallback list.

Chris

[cjones051073 (Chris Jones)]

Please hold off committing this. I'm experiencing a few issues that needs resolving first....

[cjones051073 (Chris Jones)]

Updated Portfile now attached.

There is also a new patch file to work around the issue I found relating to PCH generation.

Update now ready again to be applied...

cheers Chris

[cjones051073 (Chris Jones)]

Sorry, please hold off again for a short while....

Just noticed upstream have asked me to test another patch, this time to fix c++14 support.

Will report back shortly...

Chris

[cjones051073 (Chris Jones)]

Updated Portfile again.

Patch file for PCH generation no longer needed as now included in the git tag.

Chris

[mojca (Mojca Miklavec)]

:info:build [ 49%] Generating etc/dictpch/allLinkDefs.h, etc/dictpch/allHeaders.h, etc/dictpch/allCppflags.txt
:info:build /path/to/root6/work/root-b43f80bccfd6b67be37708c4a5a9386b7c850d29/build/unix/makepchinput.py /path/to/root6/work/root-b43f80bccfd6b67be37708c4a5a9386b7c850d29 . etc/cling/Interpreter/DynamicExprInfo.h etc/cling/Interpreter/DynamicLookupRuntimeUniverse.h etc/cling/Interpreter/DynamicLookupLifetimeHandler.h etc/cling/Interpreter/Interpreter.h etc/cling/Interpreter/InvocationOptions.h etc/cling/Interpreter/RuntimeException.h etc/cling/Interpreter/RuntimeUniverse.h etc/cling/Interpreter/Value.h etc/cling/llvm/ADT/IntrusiveRefCntPtr.h etc/cling/llvm/ADT/StringRef.h etc/cling/llvm/ADT/SmallVector.h etc/cling/llvm/ADT/iterator_range.h etc/cling/llvm/Config/llvm-config.h etc/cling/llvm/Support/AlignOf.h etc/cling/llvm/Support/Allocator.h etc/cling/llvm/Support/Casting.h etc/cling/llvm/Support/Compiler.h etc/cling/llvm/Support/DataTypes.h etc/cling/llvm/Support/MathExtras.h etc/cling/llvm/Support/Memory.h etc/cling/llvm/Support/SwapByteOrder.h etc/cling/llvm/Support/type_traits.h -- -pipe -Os -arch\ x86_64 -stdlib=libc++\  -m64 -pipe -W -Wall -Woverloaded-virtual -fsigned-char -fno-common -Qunused-arguments -pthread -std=c++11 -Wno-deprecated-    declarations -Wc++11-narrowing -Wsign-compare -Wsometimes-uninitialized -Wconditional-uninitialized -Wheader-guard -Warray-bounds -Wcomment -Wtautological-compare -Wstrncat-size -Wloop-analysis -Wbool-conversion -stdlib=libc++ -DR__HAVE_CONFIG
:info:build   File "/path/to/root6/work/root-b43f80bccfd6b67be37708c4a5a9386b7c850d29/build/unix/makepchinput.py", line 420
:info:build     print "\nGenerating PCH for %s\n" %" ".join(modulesList)
:info:build                                     ^
:info:build SyntaxError: invalid syntax
:info:build make[2]: *** [etc/dictpch/allLinkDefs.h] Error 1
:info:build make[2]: Leaving directory `/path/to/root6/work/build'
:info:build make[1]: *** [CMakeFiles/onepcm.dir/all] Error 2
:info:build make[1]: *** Waiting for unfinished jobs....

I didn't test it yet, but here's what I would do:

build/unix/makepchinput.py

@@ -417 +417 @@
 #-------------------------------------------------------------------------------
 def printModulesMessageOnScreen(selModules):
    modulesList = sorted(list(selModules))
-   print "\nGenerating PCH for %s\n" %" ".join(modulesList)
+   print ("\nGenerating PCH for {}\n".format(" ".join(modulesList)))
 
 #-------------------------------------------------------------------------------
 def makePCHInput():

[mojca (Mojca Miklavec)]

Also, I'm a tiny bit worried about

#! /usr/bin/env python

in the same python file. This means that the execution of the script strongly depends on user settings for the choice of python.

[mojca (Mojca Miklavec)]

And from the same "category" of problems:

:info:build Traceback (most recent call last):
:info:build   File "/path/to/root6/work/root-b43f80bccfd6b67be37708c4a5a9386b7c850d29/build/unix/makepchinput.py", line 15, in <module>
:info:build     from sets import Set
:info:build ImportError: No module named 'sets'

See ​http://stackoverflow.com/questions/12028204/python-cant-import-set-from-sets-no-module-named-sets

It seems that the installation script has not been tested against Python 3, but Python 3 got picked up automatically for me because of port select:

> which python
/opt/local/bin/python
> python --version
Python 3.4.3

[mojca (Mojca Miklavec)]

Moreover:

etc/dictpch/makepch.py

@@ -25 +25 @@
    argv = sys.argv
    argc = len(argv)
    if argc < 2:
-      print "ERROR: too few arguments specified!"
+      print("ERROR: too few arguments specified!")
    pchFileName = argv[1]
    cxxflags = ""
    if argc > 2:
@@ -72 +72 @@
    if not os.path.exists(loc1):
       rootdir = rootsys
       if not os.path.exists(loc2):
-         print "ERROR: cannot find %s file here %s nor here %s" %(allheadersFilename, loc1, loc2)
+         print("ERROR: cannot find {} file here {} nor here {}".format(allheadersFilename, loc1, loc2))
          sys.exit(1)
    else:
       rootbuildFlag="-rootbuild"

and

:info:build Traceback (most recent call last):
:info:build   File "/path/to/root6/work/root-b43f80bccfd6b67be37708c4a5a9386b7c850d29/etc/dictpch/makepch.py", line 114, in <module>
:info:build     ret = makepch()
:info:build   File "/path/to/root6/work/root-b43f80bccfd6b67be37708c4a5a9386b7c850d29/etc/dictpch/makepch.py", line 68, in makepch
:info:build     if os.environ.has_key(rootSysEnvName):
:info:build AttributeError: '_Environ' object has no attribute 'has_key

See ​http://stackoverflow.com/questions/8427352/how-to-solve-attributeerror-environ-object-has-no-attribute-has-key

[cjones051073 (Chris Jones)]

Can you try port selecting a 2.X version and see if that works ?

If it does, then for me this is an upstream ROOT issue with python 3, that should be reported there... If you patch works for both python 2 and 3 ? You could suggest it to them. I know though from experience they don't really support python 3, support is kinda 'as is'....

Chris

[mojca (Mojca Miklavec)]

The patches above work for both Python 2 and 3. Another two patches (for has_key and sets) one would need a bit more careful checking. I'm not sure why I'm only experiencing these problems now. It's either because I sometimes trusted your changes (or someone else did the commit) and then fetched the binaries from the server, and a while back Python 3 wasn't the default on my machine.

I'll change the python to version 2.7 for now (for testing), but we certainly need to make the installation independent of user's default selection of Python. As a next step we might want to start submitting bug reports or patches for compatibility with Python 3. The two patches above should be compatible with both, so we could start there.

Main question: did something fundamental change in installation in the latest few version (or maybe in this update specifically) or were the same problems most likely already present earlier, just that nobody was bitten by them? If first, we might want to fix the problem with a higher priority (= before upgrading).

[cjones051073 (Chris Jones)]

Certainly something changed w.r.t. the PCH generation in the last release, which is why I had to do a little work to get this going. See

​https://sft.its.cern.ch/jira/browse/ROOT-7478

so yes, they could easily have changed something that broke python 3 support in this particular update.

Honestly, I haven't really tested the python 3 support for sometime, as I have learnt that the response from upstream has been a little 'lukewarm' shall we say in the past to bug reports in this area... So it could also have been broken for a while..

Chris

[mojca (Mojca Miklavec)]

I did port select --set python python27 and installation worked now.

(Note that I didn't explicitly enable any python support. It seems that python27 is enabled by default, but any-random-python is then used for building and installation.)

I would suggest to try to fix these problems before committing the update unless you have different preferences. If nothing else it would be nice to add some reinplace during post-patch to change #! /usr/bin/env python to a "deterministic value".

[cjones051073 (Chris Jones)]

Replying to mojca@…:

I did port select --set python python27 and installation worked now.

(Note that I didn't explicitly enable any python support. It seems that python27 is enabled by default, but any-random-python is then used for building and installation.)

I would suggest to try to fix these problems before committing the update unless you have different preferences. If nothing else it would be nice to add some reinplace during post-patch to change #! /usr/bin/env python to a "deterministic value".

Fine by me.... I'm pretty busy with other things right now though so will not have time to do it myself for a bit.

[mojca (Mojca Miklavec)]

This is an incomplete list of files that might need a change:

./build/unix/git_coreteam.py                  - #!/usr/bin/python
./build/unix/makepchinput.py                  - #! /usr/bin/env python
./etc/dictpch/makepch.py                      - #! /usr/bin/env python
./documentation/primer/macros/runall.py       - #! /usr/bin/env python
./documentation/primer/rootDocPreprocessor.py - #! /usr/bin/env python

[cjones051073 (Chris Jones)]

Just thinking out loud, but one idea might be to just force the use of the system python in all cases ?

So the first line above stays, and the others are changed to do the same ?

The downside of this is it means we then use different python versions on different OS X versions, but as this is just used during the build to run a few (fairly basic) scripts, it might be fine.

The alternative I think would be to explicitly use instead one of MP's python versions, 2.7 seems the obvious one, and then make ROOT always depend on this (even if the user then asks for a different python version to be the one the python bindings are built for).

Generally speaking, I think the scripts using '#! /usr/bin/env python' are in principle doing the right thing here, so I suspect we would have a hard job suggesting upstream do something else... So this might just be a patch we have to live with...

Chris

[cjones051073 (Chris Jones)]

Please take a look at the new updated I just attached.

It does two things.

Before configuring it patches all python scripts to force the use of the system python version. This is in fact what was happening if the user had not port selected python, so just enforces this if they have.

Then, in post-destroot if the user has a python variant active, the installed scripts are patched to force the use of that MacPorts python version.

I considered in the first part using MP's python 2.7, instead of the system one. But this would force a build time dependency on python 2.7, even if the user selects a different python variant, or turns it off completely. However, if you feel this is better I would be happy to do this instead (the python27 variant is enabled by default, so for most users will not change anything).

Chris

[mojca (Mojca Miklavec)]

r139082