Batch Script for Transferring Files from Windows Machines To Linux File Server

Lately I had to come to come up with a way to transfer daily backup files from windows machines to a linux file server automatically. The idea is to keep a backup of the backup files on an external off site backup server. This is pretty much a standard practice, however the Windows machines where the backups came from were of different versions, therefore I had to find a universal (at least for windows) file transfer script. Therefore I settled with Windows batch file and a couple freeware applications.

It took me a bit of research and time to write the batch script, so I thought I’d share this with you all. Especially, some businesses with smaller IT implementation might find this useful.

Basic Concept

To put it simply, the batch script would transfer the nightly files via SCP automatically. The authentication method would be ssh key authentication. Also because the generated backup archives in these Windows machines can be very large, the batch script would also need to compress these files prior to transferring them. This is important in order to speed up the transfer and save cost on bandwidth.

Once the batch script is working properly, the automated task run will be done via windows scheduler.

Since there is no native support for rsa key generation and scp transfer in Windows as far as I know, there are a couple handy applications that needs to be installed. Do leave your comments if you know a good way to generate rsa key and perform scp natively, I normally use Linux therefore not extremely familiar with Windows.

The Applications

As mentioned previously, here are the applications that I needed in order to write the batch script as planned. These applications of course, would need to be able to run from the command line as we are writing a batch script after all.

• Putty
  • This is used to generate the public and private keys on the Windows machine.
• 7zip
  • Used to compress the backup filed.
• WinSCP
  • Used to transfer file using SCP protocol via ssh. I have chosen this method as this is one of those  most secure way for file transfer.

Once all the applications above are installed, make sure that you add their executables into the Windows machine’s environment variables so that it can be run from the command line without providing the full directory paths.

Setup Authentications

EDIT: Here’s a detailed guide on how to setup ssh authentications

I won’t go into too much details as this is pretty much a standard practice to setting up ssh access via ssh keys.

• Generate the public and private key using puttygen. Make sure to save both public key and private key in a directory accessible by the batch script.
• Log into the remote backup server using putty and put the Windows machine’s public key inside this  authorized_keys file in the .ssh directory of the backup server’s account.
• Log out and log back in to the remote server using putty (with private key access setup), at this stage you wouldn’t be asked any password as your private key should be matching the public key in the backup server.

Batch Script Walkthrough

So from here on we start on the batch script codes walkthrough so that it make sense if you decide to use it.

Setup the Variables

In this portion of the code we setup the path to the private key to use, make sure it’s the key that corresponds with the public key that we added earlier to the remote linux server. 

We also setup the backup drive and directory in the Windows machine so we can use it later for the upload. In addition I have also added the config to turn off upload in case we just want to test compression and folder creation.

Move to the Current Backup Directory

We just simply move to the current backup directory so that it’s easier to process the backup files without providing full paths.

Getting Today’s Date and Setting Up Archive Directory

The code snippet above retrieved the date from the server’s time (thanks to this post in stackoverflow) using wmic. We then proceed to create a directory based on today’s date, which will serve as the directory where our archived backups are stored.

Lastly, we loop through all the files in the current backup directories and compress them using 7zip command. Note the %%~nf inside the loop, that is to keep the original filename of the backup file yet excluding the file extension.

Transferring the File to the Backup Server

This is the most important part, is of course transferring the actual files and this can be achieved using WinSCP that we installed previously. The above are the commands for the scp transfer using batch mode, which means the script will not stop for any prompts. Timeout has been set to really high too as the internet connection where I applied these scripts can sometimes stall for a brief moment.

Also note that in this command we also specify the path to the private key to use so that we are able to login to the remote server with no login prompts at all. This is the -privatekey= parameter.

Deleting the Original Files

The codes above will delete the original backup files inside the current directory. If you are not short on space, it’s possible to change this delete command into moving the current backup files into an archived directory.

The Full Codes

So here are the full codes for you to use. I hope that this can be useful to some of you needing a simple way to transfer between windows machine to linux backup server in a more secure manner.

Categorised in: Business Process, Console commands