My daily routine involves helping customers with programming / scripting issues. Normally support for applications written by anyone outside of our company are not supported by us, however I don’t mind going through people’s code to help out when I get a few spare minutes.

One of our Linux managed hosting customers was creating a PHP script to connect to an FTP server at their office. That should be simple and trivial, right? The important code that they were using looked like:

// create a basic connection
$ftpconn = ftp_connect($ftp_ip, $ftp_port);

// login with username and password
ftp_login($ftpconn, $ftp_user, $ftp_pass);

// Switch into passive mode (required for servers behind firewalls / NAT devices)
ftp_pasv($ftpconn,true);

// Print a directory listing
print_r ftp_nlist($ftpconn, "/");

Unfortunately the code kept timing out at the ftp_nlist() command. After a few minutes the script would timeout and return a warning:

Warning:  ftp_nlist() [function.ftp-nlist]: php_connect_nonb() failed: Operation now in progress (115) in /home/script.php on line 27

My first thought was that there must be a firewall issue with the FTP server that was blocking certain ports. I decided to check if I could connect to the server directly from my local computer using FileZilla. To my surprise, I was able to connect, browse, upload and download without any problems. Now I was intrigued. I decided to see if I could implement the FTP client using PHP’s curl extension – and again I was able to connect, browse, upload and download without any problems. So, given that I was able to connect to the FTP server with FileZilla and curl I knew that the issue was somewhere in PHP’s ftp implementation and not with the FTP server. But I still had no idea what the problem was.

I turned my attention to the FTP logs in FileZilla and noticed:

Command:  PASV
Response: 227 Entering Passive Mode (10,1,2,3,79,212)
Status:   Server sent passive reply with unroutable address. Using server address instead.

The ‘Status’ line clearly hinted to us that FileZilla knew that something was wrong – the FTP server has sent back a reply containing a private RFC1918 IP address (10.X.X.X 192.168.X.X etc) which is not routable over the internet. To understand what is happening here, it is helpful to know about Active FTP vs Passive FTP. The important thing to know is that when using FTP in passive mode, the FTP client sends a ‘PASV’ command to the server. When the server receives a ‘PASV’ command it opens up a random network port for the client to send further data to. The FTP server returns to the client an IP address and port that the client should use for further communications. The problem is, that when connected behind a NAT devices, the FTP server only knows about it’s local RFC1918 IP address (10.X.X.X, 192.168.X.X 172.16.X.X etc) and not it’s public internet IP address – therefore the server responds with the only IP address that it knows about, the RFC1918 IP address. This is problematic because RFC1918 IP addresses are not routable over the internet so any FTP client using the unroutable IP address returned by the FTP server will be unable to send further data packets to the server.

So, I was pretty confident that the problems this customer was seeing were due to PHP attempting to communicate with the FTP server on the private 10.1.2.3 RFC1918 IP address. To test that theory, I fired up tcpdump to take a look at packets going through the network interface and sure enough I found the server attempting twice to send packets to 10.1.2.3:

04:49:42.134381 IP X.X.X.X.36202 > 10.1.2.3.20426: S 1619271447:1619271447(0) win 5840 <mss 1460,sackOK,timestamp 3080816617 0,nop,wscale 7>
04:49:45.133886 IP X.X.X.X.36202 > 10.1.2.3.20426: S 1619271447:1619271447(0) win 5840 <mss 1460,sackOK,timestamp 3080819617 0,nop,wscale 7>

And yes, the time stamp says 4:49AM – I have much more fun debugging than I do sleeping :)

So now we understand why the problem was happening, but how do we solve it? Happily FileZilla told us exactly what it did to solve the problem – ‘Using server address instead’. So FileZilla was basically ignoring the IP address returned by the FTP server in response to the PASV command.

I scoured the PHP ftp extension documentation and source code in a fruitless search for any hints of a setting that would instruction PHP to ignore the IP address returned by the FTP server. Going through the C code that makes up the FTP extension code in PHP, I realized that it would be relatively simple to modify the code and add an option that causes PHP to ignore the unroutable IP addresses returned by an FTP server.

I’ve created a patch for PHP’s FTP extension that adds the USEPASVADDRESS option. When this option is set to TRUE (which is the default setting), PHP will behave as it does now – continuing to use the IP address returned by the FTP server in response to the PASV command.  When this option is set to FALSE, PHP will ignore the IP address returned by the FTP server in response to the PASV command and instead use the IP address that was supplied in the ftp_connect() (or ftp_ssl_connect() call)

This option can be set and retrieved using the ftp_set_option() and ftp_get_options()  functions:

ftp_set_option($ftpconn, USEPASVADDRESS, true);
echo "USEPASVADDRESS Value: " . ftp_get_option($ftpconn, USEPASVADDRESS) ? '1' : '0';
ftp_pasv($ftpconn, true);

The option should be set before calling ftp_pasv($ftpconn, true);

You can download the patch here – it was written for and tested on php 5.3.8 however it also works against PHP 5.2

To apply the patch:

cd /usr/src/php-5.3.8
patch -p0 < ftp_usepasvaddress.patch

Bug #55651 has been submitted to the PHP developers to hopefully include in future releases.

11 Responses to “PHP FTP + Passive FTP Server Behind NAT = nightmare.”

  1. Flyingfenix says:

    That’s exactly what I need to solve exactly the same problem – however, I do not have the slightest idea on how to compile this on Windows…

  2. Flyingfenix says:

    After almost two days trying every thing possible and probably having triggered almost every possible error in MS VC++ compilers, I was able to compile PHP on Windows with the patch applied…

    And it solved completely the problem!

    Thanks again.

    • Avi says:

      I’m glad to hear that you were able to overcome the MS compiler issues and that our patch helped fix your issue :)

      All the best!

  3. Pushpraj Katiyar says:

    i have downloded ftp_usepasvaddress.patch patch file but unable to patch it in existing php(5.3.8).
    can you please tell me how can i patch it??

  4. Jordan says:

    Avi, do you have any idea why I would only have this problem on the *second* pasv ftp connection in my script? Either connection works fine by itself. When I try both — even when closing the ftp connection on the first before trying the second — I get the timeout and error.

  5. aksival says:

    Thanks for the detailed explanation. This helped identify a problem we were scratching our heads on for a couple days. Fortunately, we’re able to simply disable PASV mode for this particular server and were saved from having to patch and recompile PHP. To be honest, I’d rather modify our libraries to use cURL instead of deal with all that.

  6. danel says:

    thanks a lot , problem finally solved.

  7. danel says:

    but how about windows ?
    how can i solve it ?

  8. Fabio says:

    i solved simpy didn’t send the ftp_pasv comand!

    – ftp_connect
    – ftp_login
    – ftp_pasv
    – ftp_put
    DOESNT WORKS.

    – ftp_connect
    – ftp_login
    – ftp_put
    WORKS!

  9. jarred says:

    –ftp-skip-pasv-ip
    (FTP) Tell curl to not use the IP address the server suggests in its response to curlâs PASV command when curl connects the data connection. Instead
    curl will re-use the same IP address it already uses for the control connection. (Added in 7.14.2)

  10. […] guy knows what I'm talking about and actually patched php to fix this […]