FTP hosting source code for wcslib not reachable from NATed VirtualBox machines.

[aszostak-partner-eso-org]

I have run into a problem with the wcslib Portfile. When trying to install the port from within a VirtualBox guest machine the command,

sudo port install wcslib

hands when trying to download the source code, eventually times out and fails. Using the curl command directly I am able to reproduce the connection problem (see listing at the end). This problem only goes away when the "--disable-epsv" options is used with the curl command. I would suggest adding the following configuration option to the wcslib Portfile to solve this issue:

fetch.use_epsv      no

Trace of hanging curl connection:

macosx-10-9:~ vagrant$ curl -v ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib-4.23.tar.bz2 -o /tmp/wcslib-4.23.tar.bz2 
* Adding handle: conn: 0x7fc6e1005e00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fc6e1005e00) send_pipe: 1, recv_pipe: 0
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to ftp.atnf.csiro.au port 21 (#0)
*   Trying 150.229.106.20...
* Connected to ftp.atnf.csiro.au (150.229.106.20) port 21 (#0)
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0< 220-This is the anonymous FTP server operated by
< 220-  CSIRO Astronomy and Space Science and
< 220-  CSIRO ICT Centre.
< 220-
< 220-All transfers from this machine are logged.
< 220 If you don't agree with this policy then please disconnect now.
> USER anonymous
< 230 Anonymous user logged in
> PWD
< 257 "/" is your current location
* Entry path is '/'
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0> CWD pub
< 250 OK. Current directory is /pub
> CWD software
< 250 OK. Current directory is /pub/software
> CWD wcslib
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0< 250 OK. Current directory is /pub/software/wcslib
> EPSV
* Connect data stream passively
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0< 229 Extended Passive mode OK (|||15593|)
*   Trying 150.229.106.20...
* Connecting to 150.229.106.20 (150.229.106.20) port 15593
> TYPE I
  0     0    0     0    0     0      0      0 --:--:--  0:02:04 --:--:--     0^C

[john@…]

Adding fetch.use_epsv no to the Portfile is easy enough, of course, but I worry a little about the long term implications: one day it might cause problems for folks using IPv6. Rather than just blindly putting it in as a quick fix, it might be useful to understand a little more what's actually going wrong here.

Is this problem specific to ftp.atnf.csiro.au, or is there a generic issue with EPSV and VirtualBox NAT? Can you use EPSV with other FTP servers from within your VirtualBox VM?

[aszostak-partner-eso-org]

It does not appear to fail with other FTP servers. I successfully fetched the following and see EPSV commands in all 3 cases: ​ftp://puck.nether.net/pub/janc/cursive.tgz ​ftp://ftp.lip6.fr/pub/gnu/libtool/libtool-2.4.2.tar.gz ​ftp://ftp.perforce.com/jam/jam-2.5.tar

Thus it seems to be a problem with the FTP server itself. I glean that from an exchange I had with some members of the MacPorts mailing list:

From: Jeremy Lavergne,

You’re running into a deadlock with FTP: the machines on both sides of the connection have restrictive firewalls, so additional ports cannot be opened for listening.

From: Dave Horsfall,

But are not FTP servers (not clients) supposed to be active, and outside the firewall (or at least within the DMZ) as a result?

So my proposal is to set "fetch.use_epsv" to "no" for this particular Portfile.

[danielluke (Daniel J. Luke)]

​http://www.ncftp.com/ncftpd/doc/misc/ftp_and_firewalls.html

[aszostak-partner-eso-org]

Was the last comment with the link just for information or was it a suggestion to fix something? I dont control the FTP server and dont think its likely to be resolved any time soon. As for the client, there is no firewall involved only a typical NAT, which works with other FTP servers just fine. Thus, I expect the workaround is the only reasonable alternative for this Portfile.

[danielluke (Daniel J. Luke)]

It's to give you background on what is causing the problem. Your NAT works like a (crappy) firewall in that inbound connections to your NAT'd host get dropped because there is no NAT translation rule for them. Most typical NAT devices have special support for FTP because of this issue.

[john@…]

I'm actually confused as to what the problem is here. I'm no FTP guru, but my understanding is that when the client sends EPSV the server should respond with a port number, and then the client connects back to that port. That seems to be what's happening according to aszostak's log:

> EPSV
* Connect data stream passively
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0< 229 Extended Passive mode OK (|||15593|)
*   Trying 150.229.106.20...
* Connecting to 150.229.106.20 (150.229.106.20) port 15593

But for some reason that connection doesn't seem to be transferring any data.

As far as I can see, this shouldn't be a problem with the NAT/firewall on the client side (since it's initiating the connection, and we already know that it can initiate connections to the server -- it did so to begin the session) or on the server side (it clearly has no problems handling passive connections when VirtualBox isn't in the equation).

Interestingly, when I try this on my own (non-VirtualBox) system, I see:

$ curl -v ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib-4.23.tar.bz2 -o /tmp/wcslib-4.23.tar
[...]
> EPSV
* Connect data stream passively
  0     0    0     0    0     0      0      0 --:--:--  0:00:07 --:--:--     0< 227 Entering Passive Mode (150,229,106,20,237,219)
* Failed EPSV attempt. Disabling EPSV
[...]

I wonder if the VirtualBox NAT is somehow rewriting the server's response so it incorrectly appears that the server supports EPSV when it doesn't. aszostak, could you try running curl outside VirtualBox and capturing a log for comparison?

I continue to be quite reluctant to completely disable EPSV in the Portfile without really understanding what's happening, as it seems like that's just storing up trouble for the future.

[aszostak-partner-eso-org]

I provide below a side by side diff (i.e. using "diff -y -W") of the two curl command traces. The left trace is run on the host ma018258. The right trace is run within the VirtualBox VM guest, itself running on ma018258. They basically the same up to the "TYPE I" command, at which point curl from within the VM just hangs.

ma018258:~ buildacc$ \                                                                | macosx-10-9:~ vagrant$ \
  curl -# -v ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib-4.23.tar.bz2 \            curl -# -v ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib-4.23.tar.bz2 \
  -o /tmp/wcslib-4.23.tar.bz2                                                             -o /tmp/wcslib-4.23.tar.bz2
* Adding handle: conn: 0x7ffc41004e00                                                 | * Adding handle: conn: 0x7ff27a805e00
* Adding handle: send: 0                                                                * Adding handle: send: 0
* Adding handle: recv: 0                                                                * Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1                                                   * Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7ffc41004e00) send_pipe: 1, recv_pipe: 0                                | * - Conn 0 (0x7ff27a805e00) send_pipe: 1, recv_pipe: 0
* About to connect() to ftp.atnf.csiro.au port 21 (#0)                                  * About to connect() to ftp.atnf.csiro.au port 21 (#0)
*   Trying 150.229.106.20...                                                            *   Trying 150.229.106.20...
* Connected to ftp.atnf.csiro.au (150.229.106.20) port 21 (#0)                          * Connected to ftp.atnf.csiro.au (150.229.106.20) port 21 (#0)
< 220-This is the anonymous FTP server operated by                                      < 220-This is the anonymous FTP server operated by
< 220-  CSIRO Astronomy and Space Science and                                           < 220-  CSIRO Astronomy and Space Science and
< 220-  CSIRO ICT Centre.                                                               < 220-  CSIRO ICT Centre.
< 220-                                                                                  < 220-
< 220-All transfers from this machine are logged.                                       < 220-All transfers from this machine are logged.
< 220 If you don't agree with this policy then please disconnect now.                   < 220 If you don't agree with this policy then please disconnect now.
> USER anonymous                                                                        > USER anonymous
< 230 Anonymous user logged in                                                          < 230 Anonymous user logged in
> PWD                                                                                   > PWD
< 257 "/" is your current location                                                      < 257 "/" is your current location
* Entry path is '/'                                                                     * Entry path is '/'
> CWD pub                                                                               > CWD pub
< 250 OK. Current directory is /pub                                                     < 250 OK. Current directory is /pub
> CWD software                                                                          > CWD software
< 250 OK. Current directory is /pub/software                                            < 250 OK. Current directory is /pub/software
> CWD wcslib                                                                            > CWD wcslib
< 250 OK. Current directory is /pub/software/wcslib                                     < 250 OK. Current directory is /pub/software/wcslib
> EPSV                                                                                  > EPSV
* Connect data stream passively                                                         * Connect data stream passively
< 229 Extended Passive mode OK (|||18430|)                                            | < 229 Extended Passive mode OK (|||28823|)
*   Trying 150.229.106.20...                                                            *   Trying 150.229.106.20...
* Connecting to 150.229.106.20 (150.229.106.20) port 18430                            | * Connecting to 150.229.106.20 (150.229.106.20) port 28823
> TYPE I                                                                                > TYPE I
< 200 TYPE is now 8-bit binary                                                        <
> SIZE wcslib-4.23.tar.bz2                                                            <
< 213 1876150                                                                         <
> RETR wcslib-4.23.tar.bz2                                                            <
< 150-Accepted data connection                                                        <
< 150 1832.2 kbytes to download                                                       <
* Maxdownload = -1                                                                    <
* Getting file with size: 1876150                                                     <
{ [data not shown]                                                                    <
######################################################################## 100.0%       <
* Remembering we are in dir "pub/software/wcslib/"                                    <
< 226-File successfully transferred                                                   <
< 226 11.870 seconds (measured here), 154.36 Kbytes per second                        <
* Connection #0 to host ftp.atnf.csiro.au left intact                                 <

[pixilla (Bradley Giesbrecht)]

Changing my VirtualBox vm machines network adapter from "Bridged Adapter" to "NAT" I get the same failure. Changing back to "Bridged Adapter" and curl succeeds. Bridged Adapters also make it possible to ssh to guest vm's from non-host terminals.

[john@…]

It looks as though you can work around this issue by changing the NAT configuration as follows:

$ VBoxManage modifyvm ${your VM name} --nataliasmode1 proxyonly

Given that this seems to be easy enough to fix via VirtualBox configuration, I don't think it warrants a change to the Portfile.

[aszostak-partner-eso-org]

Thanks for finding that workaround. It works on my side also. Out of interest, does anyone know why the default would not be "proxyonly" for VirtualBox? "proxyonly" claims to provide a more transparent behaviour. What disadvantage am I missing?

Do we conclude that the server+firewall for ftp.atnf.csiro.au is OK and that VirtualBox's default NAT configuration is not compatible?

[john@…]

Unfortunately I don't currently have time to spend on tracking down and understanding VirtualBox's NAT behaviour; I'd be similarly curious to know the answer to your question.

Since this issue can be mitigated by tweaking the VirtualBox configuration, I think we conclude that changes to the Portfile which could conceivably store up trouble for the future are not in order. Understanding the details of the interaction of that particular FTP server with your NAT setup is something you could take up with the server admins, should you be so minded: it's not obvious to me that they're entirely in the clear given that you don't see similar problems with other servers.

[aszostak-partner-eso-org]

OK, agreed. I think I have to leave my investigation at that. There is a reasonable solution for changing the VirtualBox NAT configuration which can be used.

I wanted to close the ticket but do not seem to be able to. I guess the assignee will have to do that for me.