Why can’t I run rsync on my Synology?

I use a Synology NAS at work, and I also happen to have one at home.  They’re a great bit of kit but I ran into a quirk yesterday that I thought I would share.

We run a backup job in Jenkins to copy some files regularly onto the NAS.  It runs a simple command, along the lines of:

ssh login@nas.local rsync remote_files local_folder

Nothing too strange about that, only for some reason it stopped working a few days ago giving an error:

Permission denied, please try again.

Now, that’s not normally a very challenging problem, but it was odd that it was working and then stopped.  It turns out that this part of the mystery was the easy bit.  On the NAS itself I found this:

synology$ cat /var/log/synoupdate.log
2015/07/28 00:15:20 start critical update to buildnumber: 5592 original smallfixnumbre: 0 new_smallfixnumber: 1 build date: 2015/07/09

So the Synology box had done a critical update just before the backups stopped working!  But what was actually going wrong?  After some investigation I found the permission error wasn’t related to the rsync binary, nor to the files I was trying to copy.  In fact I narrowed it down to this rather odd occurrence:

$ ssh login@nas.local "rsync"
Permission denied, please try again.
$ ssh login@nas.local " rsync"
rsync version 3.0.9 protocol version 30
etc.

So any characters in front of the rsync command (in this case a single space) allowed it to work.  W. T. F.   Fortunately at this point I managed to find a post on the Synology forums that revealed everything:  http://forum.synology.com/enu/viewtopic.php?f=160&t=92627

I discovered a new setting in DSM 5.1 that fixes this problem. From Main Menu->Backup & Replication, click on “Backup Services”. There is a setting “Enable network backup services”. Click on this and now rsync works like it always has.

After checking that setting, everything works fine!  So what seems to be happening is that the Synology ssh service has a check in place for the first 5 characters of the command you are running.  If it starts “rsync” and you have not checked the magic checkbox it will not allow the command to run.  It is a pretty lame check, since even a space, or providing the full path or a symlink to another name will defeat it.  When the NAS did its auto update the setting must have become unchecked.  Hopefully this post will save others from the long debugging and Googling session it took me to solve!