gnuplot: upgrade to 5.0

[mojca (Mojca Miklavec)]

Gnuplot 5.0-RC1 is out, however there are a bunch of slight incompatibilities compared with version 4.6.

I would like to request some testing before the official release.

[mojca (Mojca Miklavec)]

                GNUPLOT VERSION 5.0.rc1 RELEASE NOTES
                =====================================

We are happy to announce a release candidate for gnuplot version 5.
Version 5 will be a major release with significant new capabilities
and enhancements compared to previous gnuplot versions.

MAJOR NEW FEATURES
==================

* The dot/dash pattern of a line can now be controlled independently
  from other properties using the keyword "dashtype".

* Text markup now supports bold and italic font settings in addition to
  the subscript, superscript, font size and other options previously
  provided by the "enhanced text" mode.  This mode is now the default.

* Command scripts may place in-line data in a named data block for
  repeated plotting.

* RGB colors can include an alpha-channel for transparency.
  E.g. SEETHRUBLU = (0xDD << 24) + (0x0 << 16) + (0x0 << 8) + (0xFF)
       set linetype N linecolor rgb SEETHRUBLU

* Bit shift operators << and >> (used in above example)

* Secondary axes (x2, y2) can be locked to the primary axis via a mapping
  function.  In the simplest case this guarantees that the primary and
  secondary axis ranges are identical.  In the general case it allows you
  to define a non-linear axis, something that previously was only possible
  for the special case of log scaling.

* The "import" command attaches a user-defined function name to a
  function provided by an extenal shared object (i.e. a plugin).

* Previous commands in the history list of an interactive session can be
  reexecuted by number.  For example "history !5" will reexecute the
  command numbered 5 in the list reported by "history".

* New plot styles "with parallelaxes", labeled contours.

* New coordinate system (Degrees, Minutes, Seconds) "set xdata geographic".

* Hypertext labels in the interactive terminals including web display
  using the HTML canvas or svg terminals.

Many other additions are described in the "New Features" section of the
documentation.


CHANGES
=======

Gnuplot development assigns very high priority to backward compatibility
with earlier versions.  For example any command script that worked in
version 4.0 is expected to continue to work for all version 4 releases
including the most recent one (4.6.5).  However changes introduced in
version 5 can affect the operation of some version 4 scripts.

A brief summary of potentially incompatible changes is given here.

* Earlier versions of gnuplot used the keyword "linetype" to mean both
  the color and the solid/dot/dash pattern of a line.  Version 5 has
  separate keywords "linecolor" and "dashtype".  You can assign any
  desired color and a dash pattern to any linetype.  The program now
  uses a default set of 8 linetypes, all solid.  You can change these
  or add new linetypes as you please, or include the desired color or
  dash pattern directly in a plot command.  You do not need to change
  the current terminal or terminal mode in order to used dashed lines.

* The handling of input data containing NaN, Inf, inconsistent number of
  data columns, or other unexpected content has changed. See documentation
  under "missing" for examples and figures.

* Time coordinates are stored internally as the number of seconds relative
  to the standard unix epoch 1-Jan-1970.  Earlier versions of gnuplot used
  a different epoch internally (1-Jan-2000). This change resolves
  inconsistencies introduced when time in seconds was generated externally.
  The epoch convention used by a particular gnuplot installation can be
  determined using the command `print strftime("%F",0)`.
  Time is now stored to at least millisecond precision.

* The "reverse" keyword (e.g. "set xrange [*:*] reverse") now affects only
  autoscaling.  It has no effect on explicit ranges.
  "set xrange [0:1] reverse" is _not_ the same as "set xrange [1:0]".

* Options to the "fit" command are now given by "set fit ..." rather than
  by setting environmental variables.  Fit can handle up to MAX_NUM_VAR
  independent variables (currently 12).  Variables other than the first
  two (x, y) have been dissociated from axis names.
  E.g. "set urange [U1:U2]" has no effect on fitting.  Use the command
  "set dummy ..." to assign names to fit variables 3 ... 12.

* The function `timecolumn(N,"timeformat")` now has 2 parameters.
  Because the required second parameter is not associated with a particular
  data axis, this allows using the `timecolumn` function to read time data
  for reasons other than specifying the x or y coordinate.

* The `call` command is implemented by providing a set of variables ARGC,
  ARG0, ..., ARG9. ARG0 holds the name of the script file being executed.
  ARG1 to ARG9 are string variables and thus may either be referenced directly
  or expanded as macros, e.g. @ARG1.  The older convention for referencing
  call parameters as tokens $0 ... $9 is deprecated.

* "unset xrange" (and other axis ranges) restores the default range.

* "unset terminal" restores the original terminal of the current session.



                 NOTES TO PACKAGERS AND TESTERS
                ===============================

Configuration options for interactive use
-----------------------------------------

The 5.0 source code supports three primary cross-platform output modes
in addition to several platform-specific modes.

1) Qt
   The qt terminal supports interactive display with menu-driven
   output to png, svg or pdf.  The final 5.0 release may also support
   scripted output to these same file formats but this is not present
   in rc1.  If either Qt4 or Qt5 is detected by the configure script,
   this will be the default terminal.  It is now the fastest and most
   full-featured interactive terminal option.
   To disable this terminal:
      ./configure --without-qt
   To force use of Qt4 even if Qt5 is present:
      ./configure --with-qt=qt4

2) Cairo/pango/wxWidgets
   This set of terminals includes
   - pngcairo, pdfcairo, epscairo, and cairolatex for output to a file
   - wxt for interactive display
   All of these will be built by default if the configuration script finds
   the required libcairo, libpango, libcairo, libwxgtk, and related
   support libraries
   To disable these terminals:
       ./configure --without-cairo
       ./configure --with-cairo --disable-wxt

3) X11 (the "classic" interactive interface)
   This used to be the preferred interactive interface, but the newer
   wxt and qt terminals offer nicer output and a wider range of features.


Options for output to files
---------------------------

Of course the terminals (output modes) present in previous gnuplot versions
are also still available. These include, among many more obscure options:
- png/jpeg/gif output via libgd
- PostScript
- Many flavors of TeX/LaTeX output, including TikZ and ConTeXt
- Bitmapped output to support many older devices (e.g. HP deskjet, epson,
  seiko printers, pbm bitmapped graphics files) is available if needed
  but is no longer configured in by default.
      ./configure --with-bitmap-terminals

Options for generating interactive plots for web display
--------------------------------------------------------

- Mouseable output for display on the web can be created using either
  the canvas terminal (HTML5 2D canvas element) or the svg terminal.
  Both allow zooming, toggling plot elements on/off, and user-scriptable
  hot keys.

Online demo plots
-----------------

Demo plots illustrating new and old features are online at

        http://gnuplot.sourceforge.net/demo_5.0/


                          OTHER NOTES
                ===============================

Installation
------------

You can download a source tarball for gnuplot version 5.0.rc1 from the
gnuplot development site on SourceForge.

        http://sourceforge.net/project/showfiles.php?group_id=2055

Installation instructions are available in the source itself; the short
version for linux/unix-like systems is to unpack the tarball and then
build it:
                cd gnuplot-5.0.rc1 ; ./configure ; make
test it:
                make check
install it:
                make install

Pay careful attention to the output of the ./configure script.
It may indicate that some output drivers have been omitted because the
necessary support libraries were not found.  In general you need to have
previously installed the "*-devel-*" versions of these libraries.


Known issues
------------

- Mac OSX ships with a terminal input library that appears to be GNU
libreadline, but isn't really.  The program tries to cope with this, but
you may get better results by configuring gnuplot to use either its own
built-in readline routines or the real GNU libreadline.

- The gnuplot build system is not very good at figuring out where to find
or install LaTeX-related files.  This can affect use of the new lua/tikz
and ConTeXt terminals.

- You can configure support for both wxt and qt into the same gnuplot
executable, but only one of these two output modes can be used in any
given gnuplot session.


Support
-------
Please report all bugs and installation problems to the bug tracker
on SourceForge:

        http://sourceforge.net/tracker/?group_id=2055&atid=102055

There is also an gnuplot discussion forum on usenet group

        comp.graphics.apps.gnuplot

[mkae (Marko Käning)]

Cc Me!

[mkae (Marko Käning)]

Hi Mojca, I just got it built fine on 10.9.2, but during destroy ended up with this:

--->  Staging gnuplot into destroot
Error: org.macports.destroot for port gnuplot returned: xinstall: Cannot stat: /opt/local/var/macports/distfiles/gnuplot/5.0.rc1/gnuplot.pdf, No such file or directory

[mojca (Mojca Miklavec)]

Thanks for noticing, just a stupid mistake:

Portfile

@@ -163 +163 @@
 
     set destdocdir ${destroot}${prefix}/share/doc/${name}
     xinstall -d -m 0755 ${destdocdir}
-    xinstall -m 0644 ${distpath}/${name}.pdf ${destdocdir}
+    xinstall -m 0644 ${workpath}/${distname}/docs/${name}.pdf ${destdocdir}
 
     # copy demo files
     file copy ${workpath}/${distname}/demo ${destdocdir}

[mojca (Mojca Miklavec)]

Gnuplot 5.0-RC1

[mkae (Marko Käning)]

OK, now it installs fine. I didn't do any real tests.

[dbevans (David B. Evans)]

A few lint problems

$ port lint --nitpick
--->  Verifying Portfile for gnuplot
Error: Line 8 repeats inclusion of PortGroup xcodeversion
Warning: Line 87 has trailing whitespace before newline
Warning: Dependency path:lib/pkgconfig/pango.pc:pango specified multiple times in depends_lib
--->  1 errors and 2 warnings found.

[dbevans (David B. Evans)]

Dependency issue:

gnuplot will optionally build with libcerf support if that port is installed. Probably should add this as a dependency or explicitly disable this feature (don't see an obvious configure option to do this though).

[mkae (Marko Käning)]

My goodness, how did you spot this?

[mojca (Mojca Miklavec)]

Replying to devans@…:

Error: Line 8 repeats inclusion of PortGroup xcodeversion

That's a bug in MacPorts, not in the port. MacPorts have problems with uppercase letters in the PortGroup name.

Warning: Line 87 has trailing whitespace before newline

OK, trivial, I'll fix this

Warning: Dependency path:lib/pkgconfig/pango.pc:pango specified multiple times in depends_lib

Would the following patch be OK? (But that's also the case in current port.)

Portfile

@@ -108 +108 @@
 
 variant wxwidgets description "Enable wxt terminal" {
     wxWidgets.use           wxWidgets-3.0
-    depends_lib-append      port:${wxWidgets.port} path:lib/pkgconfig/pango.pc:pango
+    depends_lib-append      port:${wxWidgets.port}
+    if {![variant_isset pangocairo]} {
+        depends_lib-append  path:lib/pkgconfig/pango.pc:pango
+    }
     configure.args-delete   --disable-wxwidgets
     configure.args-append   --with-wx=${wxWidgets.wxdir}
 }

[mojca (Mojca Miklavec)]

Replying to devans@…:

Dependency issue:

gnuplot will optionally build with libcerf support if that port is installed. Probably should add this as a dependency or explicitly disable this feature (don't see an obvious configure option to do this though).

Weird. Because I was the creating a port for libcerf and I was complaining to gnuplot developers that they need to add better support for searching for libcerf to configure.in. And I forgot to add dependency. The dependency is totally optional (I don't know what it's used for), it's small (the compressed binary package is 54K) and has no further dependencies on its own. So maybe I should just include it unconditionally.

[pixilla (Bradley Giesbrecht)]

I was not aware of #!diff :)

How about this:

math/gnuplot/Portfile

@@ -100 +100 @@
 }
 
 variant pangocairo description "Enable cairo-based terminals" {
-    depends_lib-append      path:lib/pkgconfig/pango.pc:pango
     configure.args-delete   --without-cairo
 }
 
@@ -111 +110 @@
 
 variant wxwidgets description "Enable wxt terminal" {
     wxWidgets.use           wxWidgets-3.0
-    depends_lib-append      port:${wxWidgets.port} path:lib/pkgconfig/pango.pc:pango
+    depends_lib-append      port:${wxWidgets.port}
     configure.args-delete   --disable-wxwidgets
     configure.args-append   --with-wx-config=${wxWidgets.wxdir}
 }
@@ -138 +137 @@
     default_variants-append +wxwidgets
 }
 
+if {[variant_isset pangocairo] || [variant_isset wxwidgets]} {
+    depends_lib-append      path:lib/pkgconfig/pango.pc:pango
+}
 # Building and installing TeX macro files for TikZ could/should be done by gnuplot's Makefile,
 # but without a patch that would require a dependency on texlive.
 # Apart from that the files would fly to the wrong place in TDS,

[dbevans (David B. Evans)]

Replying to mk@…:

My goodness, how did you spot this?

Since we have been using the buildbots, I usually try testing ports as follows

sudo port deactivate active
sudo port configure gnuplot

This emulates the way the buildbots work and helps spot missing dependencies. Doing this yielded a configure message regarding no libcerf and then I looked to see if it was available in MacPorts.

Note that I do this on a separate testing machine not a production one as this can be time consuming if you have a lot of ports installed. But it does help spot build failures due to missing dependencies before submission to the buildbots.

[dbevans (David B. Evans)]

Replying to mojca@…:

Replying to devans@…:

Dependency issue:

gnuplot will optionally build with libcerf support if that port is installed. Probably should add this as a dependency or explicitly disable this feature (don't see an obvious configure option to do this though).

Weird. Because I was the creating a port for libcerf and I was complaining to gnuplot developers that they need to add better support for searching for libcerf to configure.in. And I forgot to add dependency. The dependency is totally optional (I don't know what it's used for), it's small (the compressed binary package is 54K) and has no further dependencies on its own. So maybe I should just include it unconditionally.

Yes, I'd just add the dependency. My understanding is that it adds additional scripting commands based on the functionality of libcerf. One of the included demos, cerf.dem, illustrates this usage.

[mojca (Mojca Miklavec)]

Changes that were relevant to the current port applied in r120163 and r120166.

[dbevans (David B. Evans)]

The port has a vert extensive and well anotated set of demos which not only serve as a test of functionality but also a nice tutorial. After building the port, I was able to run them all successfullly but only after I overcame a few of my own naive assumptions.

First try. The all-inclusive demo, all.dem is installed in ${prefix}/share/doc/gnuplot/demo so I did this

cd ~
gnuplot ${prefix}/share/doc/gnuplot/demo/all.dem

This fails with file not found errors because the file references in the scripts are interpreted as relative to the CWD.

Second try

cd ${prefix}/share/doc/gnuplot/demo
gnuplot all.dem

This works better but fails again if the script issues any warning/error messages. gnuplot is trying to write these messages to a log file which by defalt is located in the CWD but the user usually doesn't have write permission to the MacPorts installation tree.

Third try, copy the demos to a writable directory in the user's home directory

mkdir ~/macports/gnuplots
cp -R ${prefix}/share/doc/gnuplot/demo ~/macports/gnuplots
cd ~/macports/gnuplots/demo
gnuplot all.dem

Success!! All demos work correctly.

I acknowlege again that this was all due to my naive ideas about how a demo might work but you might consider adding some notes to the Portfile advertizing the demos as a starting point for the naive user and how to install them locally so they work without hiccups. This would answer the question, "Ok, I installed the port -- what now?"

[mojca (Mojca Miklavec)]

Replying to devans@…:

This would answer the question, "Ok, I installed the port -- what now?"

I fully sympathize with that, but ... isn't this a problem with almost any given package (other than those with a nice gui)?

[dbevans (David B. Evans)]

Replying to mojca@…:

Replying to devans@…:

This would answer the question, "Ok, I installed the port -- what now?"

I fully sympathize with that, but ... isn't this a problem with almost any given package (other than those with a nice gui)?

Yes, but this is a particularly complex package and the demos are an easy way to come up to speed. However, there is nothing that immediately tells you that there are demos or how to run them. But I guess you could leave it up to the user to figure it out (as I did) and wait for the tickets to come in saying the demos are broken. :-) I'll leave it up to you.

[dbevans (David B. Evans)]

One more minor suggestion concerning the +qt variant. I note that the package supports both qt4 and qt5 so perhaps the current variant should be +qt4 anticipating the porting of qt5 some time in the future when you could add +qt5.

Other than what I've posted previously, it looks like everything is working. I did some more testing this afternoon and was able to output to aquaterm, a wxt term window and an x11 term window with no problems using the default variants. However, when I build with +qt in addition to the default variants and try to use a wxt term for output I get the following:

gnuplot> set term wxt
Terminal type set to 'wxt'
         The wxt terminal cannot be used in a qt session

Using

set term qt

works fine, so it may be that the +qt and +wxWidgets-3.0 variants should be made mutually exclusive.

Overall, a nice data visualization package.

[mojca (Mojca Miklavec)]

I just wanted to say that Qt5 needs testing ;) We spent o lot of time debugging it, but I'm not sure if the work was ever finished (I did not dive into programming, I was just a tester), but of course there is no way to do it inside macports unless you are willing to compile it yourself with external Qt5.

But indeed some extra testing for Qt4 (including printing which was broken in 4.x, at least to some point) would be helpful.

To answer your question: qt and wxt are not mutually exclusive. The problem is that (they claim that) one cannot switch between qt and wxt. If you have aquaterm installed, you can always switch to either of them. But once you switch to qt (and do something with it), you cannot switch to wxt any longer. Not sure if qt as default terminal prevents you from using wxt altogether.

The suggestion about qt4 makes sense. I'm just not sure whether it's better to introduce it now or when qt5 starts being supported in MacPorts.

[mojca (Mojca Miklavec)]

Replying to devans@…:

Yes, but this is a particularly complex package and the demos are an easy way to come up to speed. However, there is nothing that immediately tells you that there are demos or how to run them. But I guess you could leave it up to the user to figure it out (as I did) and wait for the tickets to come in saying the demos are broken. :-) I'll leave it up to you.

What exactly would you write in the notes?

I remember my first experience from windows. One runs the exe, uses File->Open and it opens at location where the program is installed, so one just needs to pick the folder demo and then all.dem (or any other file for that matter). But I don't know how to replicate that user experience on Mac lacking a standalone Gnuplot.app (which would require a tiny bit of Qt expertise to write a Qt gui with "built-in" "terminal" to resemble the behaviour in windows).

[cooljeanius (Eric Gallager)]

Cc Me!

[mojca (Mojca Miklavec)]

I added support for Qt5 in r121585. I suspect there's something wrong either with Gnuplot's configuration or with Qt5 installation, else I wouldn't need those extra flags.

[mojca (Mojca Miklavec)]

I committed RC3 to my personal folder in r128087.

Do we need gnuplot-devel perhaps? Here's a notice sent to the gnuplot mailing list (for me the nicest thing is "save to file" from wxt):

There is a source tarball on SourceForge for the 3rd (final) version 5.0 release candidate.

I do not expect much to change between -rc3  and the final 5.0 release,
but if your valuable testing uncovers a problem there is still time to fix it.

The user-visible changes since -rc2 are
=============================

- revised "fit" command
 + syntax:
       fit {<ranges>} <expression>
           '<datafile>' {datafile-modifiers}
           {{unitweights} | {y|xy|z}error | errors <var1>{,<var2>,...}}
           via '<parameter file>' | <var1>{,<var2>,...}
  + this is a CHANGE from -rc1 and -rc2
  + most version 4 "fit" syntax is detected and emulated even though
     the command options have changed. Also there is now an explicit
     command "set fit {v4|v5}" to force ambiguous commands to be
     interpreted as they were in version 4.

- completion of time/date format changes begun in -rc1 and -rc2
 + formats used for input and output are now distinct
 + fime format used to read input data or range/position commands
    is specified by 'set timefmt' or  timecolumn(column,"timeformat')
 + timecolumn(column) uses default from 'set timefmt' as in version 4
 + time format used to output axis tic labels is specified by
    set {xyz}tics {timedate|geographic|numeric} format "..."
 + new format modifier "t" distinguishes a time interval from a date

- wxt terminal has an Export-to-File widget on the toolbar

- various bug fixes (see NEWS and ChangeLog)

Known issues (no resolution expected before 5.0)
====================================

- the wxt terminal emits runtime warnings and errors if gnuplot is
  built using wxWidgets 3.0 rather than 2.8

[mojca (Mojca Miklavec)]

Upgrade to version 5.0 and make a gnuplot-4.6 subport

[mojca (Mojca Miklavec)]

I attached a diff with an upgrade to version 5.0. A subport for version 4.6.6 is also included. Please test and provide feedback.

[mojca (Mojca Miklavec)]

I'm also thinking about inclusion of the following patch to show help in HTML format rather than just in text mode:

• ​http://sourceforge.net/p/gnuplot/patches/707/

It seems very useful. I got it working with wxt and I still need to debug what's wrong with qt, but it looks nice.

[Schamschula (Marius Schamschula)]

Cc Me!

[Schamschula (Marius Schamschula)]

Sometimes need to scroll back to the bottom…

Missed that mojca had put up the Patchfile for gnuplot 5.0.0…

[mojca (Mojca Miklavec)]

Yes, I attached a patch to upgrade to 5.0 a few days ago and I'm waiting for some feedback.

On the other hand I'm thinking about trying to simply upgrade to 5.0 and wait if anyone will have any problems at all. If no users have problems switching to 5.0, maybe there's no need to bother about support for 4.6.

[maehne (Torsten Maehne)]

Cc Me!

[lrmlalo@…]

Cc Me!

[lrmlalo@…]

Replying to mojca@…:

Yes, I attached a patch to upgrade to 5.0 a few days ago and I'm waiting for some feedback.

On the other hand I'm thinking about trying to simply upgrade to 5.0 and wait if anyone will have any problems at all. If no users have problems switching to 5.0, maybe there's no need to bother about support for 4.6.

Is there an estimated time of release for version 5? I think 5.0.0 offers some very useful new possibilities I'd need; but I was to silly to use your provided patchfile

[mojca (Mojca Miklavec)]

r133179

Only if anyone starts complaining about the differences, I might reconsider adding back support for 4.6.