<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"><HTML><HEAD><TITLE>rsync firewall tunneling</TITLE></HEAD><!--#include virtual="header.html" --> <H2 align="center">Using rsync through a firewall</H2> <p>If you have a setup where there is no way to directly connect twosystems for an rsync transfer, there are several ways to get a firewallsystem to act as an intermediary in the transfer. <p>This first method should work for any remote-shell (e.g. ssh, rsh, etc).The other methods are all targeted at ssh (which has a lot of flexibilityin making a tunneled connection possible). <h4>Method 1 -- should work with any remote-shell</h4> <p>Use your remote shell (e.g. ssh) to access the middle system and have ituse a remote shell to hop over to the actual target system. <p>To effect this extra hop, you'll need to make sure that the remote-shellconnection from the middle system to the target system does not involve anytty-based user interaction (such as prompting for a password) because thereis no way for the middle system to access the local user's tty. <p>One way that should work for all remote-shell programs is to enable host-basedauthentication, which would allow all connections from the middle system tothe target system to succeed (when the username remains the same).However, this may not be a desirable setup. <p>A better method that works with ssh (and is very safe) is to setup an sshkey (see the ssh-keygen manpage) and ensure that ssh-agent forwarding is turnedon in your ssh client config (e.g. "ForwardAgent&nbsp;yes").  You would putthe public version of your key onto the middle and target systems (in the~/.ssh/authorized_keys file), and the private key on your local system (whichI recommend you encrypt).  With this setup, a series of ssh connections thatstarts from the system where your private key is available will auto-authorize(after a pass-phrase prompt on the first system if your key is encrypted).See also ssh-agent for a way to keep a public key unlocked in memory for anextended time, and the keychain project for a way to manage a system-wide,per-user ssh-agent. <p>To test that you have things setup right, first test a series ofremote-shell connections outside of rsync.  A command such as the followingshould work without issuing multiple prompts (use the appropriate remote-shelland the substitute the real hostnames for "middle" and "target", of course): <blockquote><pre>ssh middle ssh target uptime</pre></blockquote> <p>If you get a password/passphrase prompt to get into the first ("middle")system that's fine, but the extra hop needs to occur without any extra userinteraction. <p>Once that's done, you can do an rsync copy like this: <blockquote><pre>rsync -av -e "ssh middle ssh" target:/src/ /dest/</pre></blockquote> <h4>Method 2 -- requires ssh and nc (netcat)</h4> <p>Assuming you're using ssh as your remote shell, you can configure ssh touse a proxy command to get to the remote host you're interested in reaching.Doing this will allow the multi-hop connection to work with rsync, even ifboth hosts prompt for a password -- this is because both ssh connectionsoriginate from the localhost, and thus both instances of ssh have access tothe local tty to use for an out-of-band password prompt. <p>Here is an example config for your ~/.ssh/config file (substitute "target","target_user", and "middle" as appropriate): <blockquote><pre>Host target  ProxyCommand nohup ssh middle nc -w1 %h %p  User target_user</pre></blockquote> <p>This proxy setup uses "ssh" to login to the firewall system ("middle") anduses "nc" (netcat) to connect to the target host ("target") using the defaultport number.  The use of "nohup" silences a warning at the end of the run, andthe "-w1" option tells nc to shut down when the connection closes. <p>With this done, you can run a normal-looking rsync command to "target" thatwill run the proxy command to get through the firewall system: <blockquote><pre>rsync -av /src/ target:/dest/</pre></blockquote> <h4>Method 3 -- an alternate ssh method for those without nc (netcat)</h4> <p>Assuming you're using ssh as your remote shell, you can configure ssh toforward a local port through your middle system to the ssh port (22) on thetarget system.  This method does not require the use of nc (it uses onlyssh to effect the extra hop), but otherwise it is similar to, but slightlyless convenient than, method 2. <p>The first thing we need is an ssh configuration that will allow us toconnect to the forwarded port as if we were connecting to the targetsystem, and we need ssh to know what we're doing so that it doesn'tcomplain about the host keys being wrong.  We can do this by adding thissection to your ~/.ssh/config file (substitute "target" and "target_user"as appropriate, but leave "localhost" unchanged): <blockquote><pre>Host target  HostName localhost  Port 2222  HostKeyAlias target  User target_user</pre></blockquote> <p>Next, we need to enable the port forwarding: <blockquote><pre>ssh -fN -l middle_user -L 2222:target:22 middle</pre></blockquote> <p>What this does is cause a connection to port 2222 on the local system toget tunneled to the middle system and then turn into a connection to thetarget system's port 22.  The -N option tells ssh not to start a shell onthe remote system, which works with modern ssh versions (you can run asleep command if -N doesn't work).  The -f option tells ssh to put thecommand in the background after any password/passphrase prompts. <p>With this done, you could run a normal-looking rsync command to "target"that would use a connection to port 2222 on localhost automatically: <blockquote><pre>rsync -av target:/src/ /dest/</pre></blockquote> <p><b>Note:</b> starting an ssh tunnel allows anyone on the source systemto connect to the localhost port 2222, not just you, but they'd still needto be able to login to the target system using their own credentials. <h4>Method 4 -- for using rsync in daemon-mode (requires nc)</h4> <p>Install and configure an rsync daemon on the target and use ssh and ncto send the socket data to the remote host. <blockquote><pre>RSYNC_CONNECT_PROG='ssh -l middle_user middle nc %H 873' \    rsync daemonuser@target::module/src/ /dest/</pre></blockquote> <p>(You can also export that variable into your environment if you want toperform a series of daemon rsync commands through the same middle host.) <p>This command takes advantage of the RSYNC_CONNECT_PROG environmentvariable, which tells rsync to pipe its socket data to an external programin place of making a direct socket connection.  The command specifed aboveuses ssh to run the nc (netcat) command on the middle host, which forwardsall socket data to port 873 on the target host (%H).  The "%H" will besubstituted with the target host from the rsync command as long as you'rerunning rsync 3.0.0 or newer (you'll need to replace %H with the actualtarget hostname for earlier rsync versions, which makes the hostnamespecified in the rsync command superfluous). <h4>Method 5 -- for using rsync in daemon-mode (for those without nc)</h4> <p>Install and configure an rsync daemon on the target and use an sshtunnel to reach the rsync sever.  This is similar to method 3, but ittunnels the daemon port for those that prefer to use an rsync daemon. <p>(Note that this method also works to tunnel a socket connectiondirectly to a destination system if you use "localhost" as the targethostname and your destination system's name as the middle hostname.) <p>Installing the rsync daemon is beyond the scope of this document, butsee the rsyncd.conf manpage for more information.  Keep in mind that youdon't need to be root to run an rsync daemon as long as you don't use aprotected port. <p>Once your rsync daemon is up and running, you build an ssh tunnelthrough your middle system like this: <blockquote><pre>ssh -fN -l middle_user -L 8873:target:873 middle</pre></blockquote> <p>What this does is cause a connection to port 8873 on the local system toturn into a connection from the middle system to the target system on port873.  (Port 873 is the normal port for an rsync daemon.) The -N optiontells ssh not to start a shell on the remote system, which works withmodern ssh versions (you can run a sleep command if -N doesn't work).  The-f option tells ssh to put the command in the background after anypassword/passphrase prompts. <p>Now when an rsync command is executed with a daemon-mode command-linesyntax to the local system, the conversation is directed to the targetsystem.  For example: <blockquote><pre>rsync -av --port 8873 localhost::module/src/ dest/rsync -av rsync://localhost:8873/module/src/ dest/</pre></blockquote> <p><b>Note:</b> starting an ssh tunnel allows anyone on the source systemto connect to the localhost port 8873, not just you, so you may want toenable username/password restrictions on your rsync daemon. <!--#include virtual="footer.html" -->