#57528 closed defect (fixed)
dvisvgm: parallel builds hang, during execution of 'touch -r'
Reported by: | dershow | Owned by: | mascguy (Christopher Nielsen) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | |
Keywords: | local-testing-needed | Cc: | basmac, MarcusCalhoun-Lopez (Marcus Calhoun-Lopez), michaelld (Michael Dickens), ballapete (Peter "Pete" Dyballa) |
Port: | dvisvgm |
Description
I have dvisvgm 2.3.4_4 installed and I'm trying to upgrade to 2.6.1_0. It gets as far as building, and then it just hangs. Yesterday, I let it run for 13 hours before I stopped it. I've cleaned and redone is several times, always with the same result that it just hangs while trying to make. Make takes 100% of one CPU core. I'm on a fairly recent macbook pro, so there is no reason that it should take a long time. I've attached the partial log, since the log file doesn't actually finish. I just have to ctrl-C the build.
Attachments (1)
Change History (39)
Changed 6 years ago by dershow
comment:1 Changed 6 years ago by dershow
comment:2 Changed 6 years ago by murrayE
Same thing here. (macOS Mojave 10.14.1, Xcode 10.1 with CLT installed.)
comment:4 Changed 6 years ago by dershow
It was suggested that I try forcing it to no do a parallel build, and that works as a work around. Very strange.
So this worked for me:
sudo port -u -v upgrade dvisvgm build.jobs=1
comment:6 Changed 6 years ago by mf2k (Frank Schima)
Must be a file locking issue. I'm thinking the best solution is to add this to the Portfile:
use_parallel_build no
comment:7 Changed 6 years ago by neilt (Neil Tiffin)
This worked for me also (macOS Mojave 10.14.1, Xcode 10.1)
comment:8 Changed 6 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Cc: | MarcusCalhoun-Lopez added |
---|
comment:9 Changed 6 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Owner: | set to MarcusCalhoun-Lopez |
---|---|
Resolution: | → fixed |
Status: | new → closed |
comment:10 Changed 4 years ago by mascguy (Christopher Nielsen)
Cc: | mascguy added |
---|
comment:11 Changed 4 years ago by mascguy (Christopher Nielsen)
Cc: | mascguy removed |
---|
comment:12 Changed 3 years ago by mascguy (Christopher Nielsen)
Cc: | mascguy added |
---|
comment:13 Changed 3 years ago by mascguy (Christopher Nielsen)
Just felt the pinch, related to the lengthy build time of this port [due to parallel builds being disabled]. And since the license precludes publishing binaries, everyone experiences it too.
Given that this appears to be occurring on touch
, has anyone tested with gtouch
(via port coreutils
)? Might be a simple fix...
comment:14 Changed 3 years ago by Christopher Nielsen <mascguy@…>
comment:15 Changed 3 years ago by mascguy (Christopher Nielsen)
Resolution: | fixed |
---|---|
Status: | closed → reopened |
comment:16 Changed 3 years ago by mascguy (Christopher Nielsen)
Cc: | mascguy removed |
---|---|
Owner: | changed from MarcusCalhoun-Lopez to mascguy |
Status: | reopened → assigned |
comment:17 Changed 3 years ago by mascguy (Christopher Nielsen)
Summary: | dvisvgm hangs during build → dvisvgm: parallel builds hang, during execution of 'touch -r' |
---|
comment:18 Changed 3 years ago by mascguy (Christopher Nielsen)
Cc: | michaelld added |
---|
comment:19 Changed 3 years ago by mascguy (Christopher Nielsen)
Folks, is anyone interested in testing the parallel build fix, by installing dvisvgm-devel
from source?
comment:20 Changed 3 years ago by mascguy (Christopher Nielsen)
Keywords: | local-testing-needed added |
---|
comment:21 Changed 3 years ago by Christopher Nielsen <mascguy@…>
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
comment:22 follow-up: 23 Changed 3 years ago by dershow
I'm trying to upgrade from 2.12_2 to 2.12_3 and again, I've let my computer compile overnight (13 hours). Make is using a CPU at 100% and it seems to be building. But, that's a really long time on a fast computer. So, I don't think that this is fixed.
comment:23 Changed 3 years ago by mascguy (Christopher Nielsen)
Replying to dershow:
I'm trying to upgrade from 2.12_2 to 2.12_3 and again, I've let my computer compile overnight (13 hours). Make is using a CPU at 100% and it seems to be building. But, that's a really long time on a fast computer. So, I don't think that this is fixed.
Given the hardware you're running on, I'd expect it to build within 10-ish minutes. So no need to let it run all night.
I'll take another look, and report back.
comment:24 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)
Resolution: | fixed |
---|---|
Status: | closed → reopened |
#64279 also reports that the problem still exists.
comment:25 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)
Cc: | ballapete added |
---|
comment:26 follow-up: 29 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)
This ticket was filed three years ago... did anybody in all that time bother to report the problem to the developers? If not, let's do so now.
comment:27 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
It is certainly a problem of timing in a parallel build. The port builds when I kill the job while it's stuck somewhere, and when I re-start, without cleaning, it finishes at once.
comment:28 follow-up: 31 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
I tried to build the port with gmake
. It works for configure
that it states checking whether /opt/local/bin/gmake sets $(MAKE)... yes
, but then make
is used for building:
Executing: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_dvisvgm/dvisvgm/work/dvisvgm-2.12" && /usr/bin/make -j12 -w all
comment:29 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)
comment:30 follow-up: 35 Changed 3 years ago by kencu (Ken)
perhaps adding .NOTPARALLEL
to the doc Makefile.am just above the touch will do the trick.
https://github.com/mgieseki/dvisvgm/blob/master/doc/Makefile.am
https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
Then you can actually PR a fix.
comment:31 follow-ups: 32 33 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)
Replying to ballapete:
I tried to build the port with
gmake
.
Good suggestion. Using gmake works for me. So I guess it's a bug in make that was fixed sometime between 3.81 and 4.3.
It works for
configure
that it stateschecking whether /opt/local/bin/gmake sets $(MAKE)... yes
, but thenmake
is used for building:
I guess you didn't set build.cmd ${prefix}/bin/gmake
.
comment:32 Changed 3 years ago by kencu (Ken)
Replying to ryandesign:
Replying to ballapete:
I tried to build the port with
gmake
.Good suggestion. Using gmake works for me. So I guess it's a bug in make that was fixed sometime between 3.81 and 4.3.
No, not necessarily.
Could be just a random timing difference between the different versions, as it is already intermittent with 3.81, according to Chris above.
comment:33 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
Replying to ryandesign:
Builds here too with gmake
!
comment:34 Changed 3 years ago by ballapete (Peter "Pete" Dyballa)
doc/Makefile.am
has this block:
dvisvgm.1: dvisvgm-man.xml $(CONF_DBLATEX_MAN) xmlto -m $(CONF_DBLATEX_MAN) $(addprefix --stringparam , $(DB_VARS)) man $< dvisvgm.pdf: dvisvgm-man.xml dvisvgm.sty $(CONF_DBLATEX_PDF) $(TWEAK_DBLATEX_PDF) $(TWEAK_DBLATEX_TEX) mv $< $<.tmp xsltproc -o $< $(TWEAK_DBLATEX_PDF) $<.tmp dblatex -bxetex --texstyle=./dvisvgm.sty -p $(CONF_DBLATEX_PDF) -r plugin:$(basename $(TWEAK_DBLATEX_TEX)) $< mv $<.tmp $< mv $(basename $<).pdf $@ dvi-pdf: dvisvgm.dvi dvipdfm $< rm -f $< dvisvgm.dvi: dvisvgm.1 groff -Tdvi -mandoc ./$< >$@ dvisvgm.html: dvisvgm-man.xml db2html.xsl xsltproc -o $@ db2html.xsl $< dvisvgm.md: dvisvgm-man.xml db2md.xsl xsltproc -o $@ db2md.xsl $< dvisvgm.epub: dvisvgm-article.xml a2x -darticle -fepub -L --icons --icons-dir=. $< mv $(basename $<).epub $@ .SECONDARY: dvisvgm-article.xml dvisvgm-man.xml dvisvgm.sty dvisvgm-man.xml: dvisvgm.txt $(TWEAK_DB_REFENTRY) asciidoc -a icons -a 'iconsdir=.' -a badges -a 'revnumber=@VERSION@' --unsafe -bdocbook -dmanpage -o $@.tmp $< xsltproc -o $@ $(TWEAK_DB_REFENTRY) $@.tmp rm $@.tmp dvisvgm-article.xml: dvisvgm.txt $(TWEAK_DB_ARTICLE) asciidoc -a icons -a 'iconsdir=.' -a badges -a 'revnumber=@VERSION@' --unsafe -bdocbook -darticle -o $@.tmp $< xsltproc -o $@ $(TWEAK_DB_ARTICLE) $@.tmp rm $@.tmp sed -i "s/{VERSION}/@VERSION@/" $@ dvisvgm.sty: dvisvgm-man.xml $(GEN_DBLATEX_STY) xsltproc -o dvisvgm.sty $(GEN_DBLATEX_STY) $< dvisvgm.txt: dvisvgm.txt.in ../configure.ac sed -e 's/@VERSION[@]/@VERSION@/g' -e 's/@PACKAGE_BUGREPORT[@]/@PACKAGE_BUGREPORT@/g' $< >$@ if [ $< -nt ../configure.ac ]; \ then gtouch -r $< $@; \ else gtouch -r ../configure.ac $@; \ fi
Obviously make
understands that target dvisvgm.1
depends on dvisvgm-man.xml
which depends on dvisvgm.txt
, so it tries to build this target first. It executes the sed
command and then compares the two files to find out which of them is newer (if [ dvisvgm.txt.in -nt ../configure.ac ];
). In the beginning these two files exist:
-rw-r--r-- 1 macports 8409 2021-08-16 12:07:38.000000000 +0200 doc/../configure.ac -rw-r--r-- 1 macports 61026 2021-08-16 12:07:38.000000000 +0200 doc/dvisvgm.txt.in
and have exactly the same date, IMO. Hard to decide which path to go – but this is not the thing that is making /usr/bin/make
fall into the infinite loop. Because both make
and gmake
adjust the sed
produced dvisvgm.txt
file to become the same date as the other two:
-rw-r--r-- 1 macports 8409 2021-08-16 12:07:38.000000000 +0200 doc/../configure.ac -rw-r--r-- 1 macports 61026 2021-08-16 12:07:38.000000000 +0200 doc/dvisvgm.txt.in -rw-r--r-- 1 macports 61025 2021-08-16 12:07:38.000000000 +0200 doc/dvisvgm.txt
gmake
leaves here the doc directory
gmake[2]: Entering directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_dvisvgm/dvisvgm/work/mgieseki-dvisvgm-2e94568/doc' sed -e 's/@VERSION[@]/2.12/g' -e 's/@PACKAGE_BUGREPORT[@]/martin.gieseking@uos.de/g' dvisvgm.txt.in >dvisvgm.txt if [ dvisvgm.txt.in -nt ../configure.ac ]; \ then gtouch -r dvisvgm.txt.in dvisvgm.txt; \ else gtouch -r ../configure.ac dvisvgm.txt; \ fi gmake[2]: Leaving directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_dvisvgm/dvisvgm/work/mgieseki-dvisvgm-2e94568/doc' gmake[2]: Entering directory '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_dvisvgm/dvisvgm/work/mgieseki-dvisvgm-2e94568'
while make
hangs.
In the beginning all files in doc have exactly the same date: And with `gmake` and `make`, when I kill it, quite the same (even Makefile*): -rw-r--r-- 1 macports 21971 2021-12-24 16:15:09.881157339 +0100 Makefile -rw-r--r-- 1 macports 21100 2021-12-24 16:14:55.851002661 +0100 Makefile.in -rw-r--r-- 1 macports 3680 2021-12-24 16:09:26.290946965 +0100 Makefile.am -rw-r--r-- 1 macports 2554 2021-08-16 12:07:38.000000000 +0200 caution.png -rw-r--r-- 1 macports 2554 2021-08-16 12:07:38.000000000 +0200 caution.png -rw-r--r-- 1 macports 718 2021-08-16 12:07:38.000000000 +0200 conf-dblatex-man.xsl -rw-r--r-- 1 macports 718 2021-08-16 12:07:38.000000000 +0200 conf-dblatex-man.xsl -rw-r--r-- 1 macports 2618 2021-08-16 12:07:38.000000000 +0200 conf-dblatex-pdf.xsl -rw-r--r-- 1 macports 2618 2021-08-16 12:07:38.000000000 +0200 conf-dblatex-pdf.xsl -rw-r--r-- 1 macports 5855 2021-08-16 12:07:38.000000000 +0200 db2html.xsl -rw-r--r-- 1 macports 5855 2021-08-16 12:07:38.000000000 +0200 db2html.xsl -rw-r--r-- 1 macports 67474 2021-08-16 12:07:38.000000000 +0200 dvisvgm.1 -rw-r--r-- 1 macports 67474 2021-08-16 12:07:38.000000000 +0200 dvisvgm.1 -rw-r--r-- 1 macports 497 2021-08-16 12:07:38.000000000 +0200 dvisvgm.css -rw-r--r-- 1 macports 497 2021-08-16 12:07:38.000000000 +0200 dvisvgm.css -rw-r--r-- 1 macports 61025 2021-08-16 12:07:38.000000000 +0200 dvisvgm.txt -rw-r--r-- 1 macports 61026 2021-08-16 12:07:38.000000000 +0200 dvisvgm.txt.in -rw-r--r-- 1 macports 61026 2021-08-16 12:07:38.000000000 +0200 dvisvgm.txt.in -rw-r--r-- 1 macports 963 2021-08-16 12:07:38.000000000 +0200 dvisvgm.xpr -rw-r--r-- 1 macports 963 2021-08-16 12:07:38.000000000 +0200 dvisvgm.xpr -rw-r--r-- 1 macports 840 2021-08-16 12:07:38.000000000 +0200 generate-dvisvgm-sty.xsl -rw-r--r-- 1 macports 840 2021-08-16 12:07:38.000000000 +0200 generate-dvisvgm-sty.xsl -rw-r--r-- 1 macports 2730 2021-08-16 12:07:38.000000000 +0200 note.png -rw-r--r-- 1 macports 2730 2021-08-16 12:07:38.000000000 +0200 note.png -rw-r--r-- 1 macports 1013 2021-08-16 12:07:38.000000000 +0200 tweak-db-article.xsl -rw-r--r-- 1 macports 1013 2021-08-16 12:07:38.000000000 +0200 tweak-db-article.xsl -rw-r--r-- 1 macports 1698 2021-08-16 12:07:38.000000000 +0200 tweak-db-refentry.xsl -rw-r--r-- 1 macports 1698 2021-08-16 12:07:38.000000000 +0200 tweak-db-refentry.xsl -rw-r--r-- 1 macports 1041 2021-08-16 12:07:38.000000000 +0200 tweak-dblatex-pdf.xsl -rw-r--r-- 1 macports 1041 2021-08-16 12:07:38.000000000 +0200 tweak-dblatex-pdf.xsl -rw-r--r-- 1 macports 717 2021-08-16 12:07:38.000000000 +0200 tweak-dblatex-tex.py -rw-r--r-- 1 macports 717 2021-08-16 12:07:38.000000000 +0200 tweak-dblatex-tex.py
So all make's make only one file, dvisvgm.txt
(because nothing more that does not exist yet is needed by any make target), and give it the date of dvisvgm.txt.in or ../configure.ac
, which is essentially the same. gmake
now leaves successfuly the doc directory while make
obviously stays there and goes mad therefore.
The problem might not be due to a parallel build…
comment:35 Changed 3 years ago by kencu (Ken)
Replying to kencu:
perhaps adding
.NOTPARALLEL
to the doc Makefile.am just above the touch will do the trick.https://github.com/mgieseki/dvisvgm/blob/master/doc/Makefile.am
https://www.gnu.org/software/make/manual/html_node/Special-Targets.html
Then you can actually PR a fix.
This didn't work? Unfortunate if not; usually does.
comment:36 follow-up: 38 Changed 3 years ago by kencu (Ken)
The .NOTPARALLEL:
workaround works perfectly, with the current make.
$ cat patch-dvisvgm-parallel-make-hang.diff --- doc/Makefile.am.orig 2021-12-24 11:51:24.000000000 -0800 +++ doc/Makefile.am 2021-12-24 11:51:45.000000000 -0800 @@ -73,6 +73,8 @@ dvisvgm.sty: dvisvgm-man.xml $(GEN_DBLATEX_STY) xsltproc -o dvisvgm.sty $(GEN_DBLATEX_STY) $< +.NOTPARALLEL: + dvisvgm.txt: dvisvgm.txt.in ../configure.ac sed -e 's/@VERSION[@]/@VERSION@/g' -e 's/@PACKAGE_BUGREPORT[@]/@PACKAGE_BUGREPORT@/g' $< >$@ if [ $< -nt ../configure.ac ]; \
use this instead of the gtouch patch.
comment:37 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
I tried to install verbose, and here is the output, until it hangs: