In this topic I will present
rclone - a simple tool to synchronize/backup files. I will show here a ready-to-use script that allows you to make a copy of selected folders with just one click and the entire copy will work on an incremental basis, so it will be a much better solution than the classic CTRL + C and CTRL + V still used by so many people, also technical ones.
Why is rclone needed at all? Nowadays, electronics are permanently connected to the computer. We store all data digitally - whether there are diagrams, catalog notes, or firmware designs for microcontrollers and documentation. However, all these files are at risk of being lost when least expected. Digital data carriers may fail at any time and then potential data recovery, even if possible, is difficult and problematic. Even if the media does not fail, we are also at risk of ransomware viruses... Of course
rclone this is just one of the options, and I usually recommend using many at the same time (just as you should use many backup copies - the 3-2-1 rule - i.e. "you should have 3 copies, on 2 different media, where 1 of them is on a different location"), but this topic will only talk about rclone.
Where to get rclone? The tool in question is cross-platform and available for download completely free of charge, including: here:
https://rclone.org/downloads/ We simply download and unpack the package for our platform.
This way we get the expected exe:
However, it is not a window program, only a command line tool, so you will need to write a script...
My rclone script The documentation is available on the download page, but for simplicity, I will give you my ready-made script here:
rclone.exe sync W:/ X:/W --progress --transfers=64 --delete-excluded --exclude "Library/**" --exclude "Debug/**" --exclude "Release/**" --exclude "Android/**" --exclude "$RECYCLE.BIN/**" --exclude "Virtual Machines/**" --exclude "GIT/**" --exclude "C-Sky/**" --exclude ".venv" --exclude "NoBackupToDelete/**" --delete-before
pause
Once the above script is run, it will map the contents of the directory
IN:/ In the catalogue
X:/W . After its execution, the target directory will be a 1:1 representation of the source directory, which means that if we delete something from the source directory, it will also disappear in the target directory. The copy progress will be visible in the console pane. The maximum number of file transfers at once here is 64. Additionally, the script excludes several folders from synchronization to which paths are provided - including the trash bin (RECYCLE.BIN). Additionally, it includes an option
--delete-before , which specifies that files deleted on the source are deleted on the copy before new files are transferred to it.
How to use this script? Basically, there are several possibilities:
- you can simply have a second disk in your computer for backups (in laptops you can sometimes replace the CD/DVD drive with a disk compartment), for example HDD, it does not have to be a fast disk
- you can connect an external drive and manually run the script
- you can also map the network drive, then rclone will also work
Script calling can also be automatic or manual. It all depends on what we want to protect ourselves from. Having a copy on a second disk on the computer will not save us, for example, against ransomware...
What are the advantages and disadvantages of this script? Let`s start with the advantages:
+ simplicity, much simpler than copy & paste
+ only what has changed compared to the previous copy is copied
+ deleted files on the source are also deleted on the target
But this solution also has disadvantages that we must be aware of:
- if we edit an important document and overwrite it and make a copy, the previous version will be overwritten (this solution is not SVN/GIT, it does not store the history of changes)
- if we unknowingly delete something and run sync, it will also be deleted from the copy
- if, for example, something encrypts our files, as above, synchronization will spoil the copies
In addition, there is one more problem, but it is not directly related to this script - copying large numbers of small files takes a long time. If we want to keep, for example, a GIT repository with code for a longer period of time, it is worth considering
using a RAR archive with repair data . One large RAR archive will copy much faster than many small files.
Nevertheless, I think the script is useful (or at least it is a step forward compared to the "CTRL + C and CTRL + V" methods) - do you use this type of solutions? Or maybe you can suggest some way to improve this script? I invite you to discuss.
Comments
actually, it t know of) a tool that, in the case of changed files, would allow incremental addition of only changes (more precisely, there are such, but they are rather larger backup systems, not synchronization... [Read more]
You can make an incremental backup, e.g. using Wildebeest tar , encryption GnuPG . tar c -g ~/backup/moje_dane1.snar --totals -X ~/backup/exclude -C ~/moje_dane1 . | pbzip2 | gpg2 --compress-algo=none... [Read more]