Automatic backup - rclone - sample script - how to protect yourself against file loss?

p.kaczmarek2 27 Mar 2024 10:30 2 2724 Cool? (+6)

📢 Listen (AI voice):

TL;DR

rclone command-line sync script creates one-click incremental backups of selected folders instead of manual copy-and-paste.
The script mirrors W:/ to X:/W, uses --delete-before, and excludes folders like Library, Debug, Release, Android, and RECYCLE.BIN.
It allows up to 64 file transfers at once and shows copy progress in the console.
The target becomes a 1:1 representation of the source, so deleted source files also disappear from the backup.
Sync overwrites previous versions, propagates accidental deletions, and can ruin backups after ransomware; many small files also copy slowly.

AI summary based on the discussion. May contain errors.

In this topic I will present rclone - a simple tool to synchronize/backup files. I will show here a ready-to-use script that allows you to make a copy of selected folders with just one click and the entire copy will work on an incremental basis, so it will be a much better solution than the classic CTRL + C and CTRL + V still used by so many people, also technical ones.

Why is rclone needed at all?
Nowadays, electronics are permanently connected to the computer. We store all data digitally - whether there are diagrams, catalog notes, or firmware designs for microcontrollers and documentation. However, all these files are at risk of being lost when least expected. Digital data carriers may fail at any time and then potential data recovery, even if possible, is difficult and problematic. Even if the media does not fail, we are also at risk of ransomware viruses... Of course rclone this is just one of the options, and I usually recommend using many at the same time (just as you should use many backup copies - the 3-2-1 rule - i.e. "you should have 3 copies, on 2 different media, where 1 of them is on a different location"), but this topic will only talk about rclone.

Where to get rclone?
The tool in question is cross-platform and available for download completely free of charge, including: here:
https://rclone.org/downloads/
We simply download and unpack the package for our platform.
This way we get the expected exe:

However, it is not a window program, only a command line tool, so you will need to write a script...

My rclone script
The documentation is available on the download page, but for simplicity, I will give you my ready-made script here:


rclone.exe sync W:/ X:/W --progress --transfers=64 --delete-excluded --exclude "Library/**" --exclude "Debug/**" --exclude "Release/**" --exclude "Android/**" --exclude "$RECYCLE.BIN/**" --exclude "Virtual Machines/**" --exclude "GIT/**" --exclude "C-Sky/**" --exclude ".venv"  --exclude "NoBackupToDelete/**" --delete-before
pause

Once the above script is run, it will map the contents of the directory IN:/ In the catalogue X:/W . After its execution, the target directory will be a 1:1 representation of the source directory, which means that if we delete something from the source directory, it will also disappear in the target directory. The copy progress will be visible in the console pane. The maximum number of file transfers at once here is 64. Additionally, the script excludes several folders from synchronization to which paths are provided - including the trash bin (RECYCLE.BIN). Additionally, it includes an option --delete-before , which specifies that files deleted on the source are deleted on the copy before new files are transferred to it.

How to use this script?
Basically, there are several possibilities:
- you can simply have a second disk in your computer for backups (in laptops you can sometimes replace the CD/DVD drive with a disk compartment), for example HDD, it does not have to be a fast disk
- you can connect an external drive and manually run the script
- you can also map the network drive, then rclone will also work
Script calling can also be automatic or manual. It all depends on what we want to protect ourselves from. Having a copy on a second disk on the computer will not save us, for example, against ransomware...

What are the advantages and disadvantages of this script?
Let`s start with the advantages:
+ simplicity, much simpler than copy & paste
+ only what has changed compared to the previous copy is copied
+ deleted files on the source are also deleted on the target
But this solution also has disadvantages that we must be aware of:
- if we edit an important document and overwrite it and make a copy, the previous version will be overwritten (this solution is not SVN/GIT, it does not store the history of changes)
- if we unknowingly delete something and run sync, it will also be deleted from the copy
- if, for example, something encrypts our files, as above, synchronization will spoil the copies

In addition, there is one more problem, but it is not directly related to this script - copying large numbers of small files takes a long time. If we want to keep, for example, a GIT repository with code for a longer period of time, it is worth considering using a RAR archive with repair data . One large RAR archive will copy much faster than many small files.

Nevertheless, I think the script is useful (or at least it is a step forward compared to the "CTRL + C and CTRL + V" methods) - do you use this type of solutions? Or maybe you can suggest some way to improve this script? I invite you to discuss.

About Author

p.kaczmarek2 wrote 14717 posts with rating 12795 , helped 659 times. Been with us since 2014 year.

Comments

Add a comment

karroryfer 29 Mar 2024 12:15

actually, it t know of) a tool that, in the case of changed files, would allow incremental addition of only changes (more precisely, there are such, but they are rather larger backup systems, not synchronization... [Read more]

Sam Sung 30 Mar 2024 12:49

You can make an incremental backup, e.g. using Wildebeest tar , encryption GnuPG . tar c -g ~/backup/moje_dane1.snar --totals -X ~/backup/exclude -C ~/moje_dane1 . | pbzip2 | gpg2 --compress-algo=none... [Read more]

FAQ

TL;DR: Use rclone sync for a 1:1 backup mirror with 64 simultaneous transfers; as the author puts it, "much better" than manual copy-paste. This FAQ helps Windows users build a one-click backup, understand deletion risk, and choose safer alternatives for ransomware and version-loss scenarios. [#21021543]

Why it matters: A fast mirror backup saves time, but a badly chosen sync method can also replicate accidental deletion, overwrites, or encrypted files.

Method	What it does	Best fit	Main risk
`rclone sync`	Mirrors source to target 1:1	Fast routine backups to disk, USB, or mapped drive	Deletions and overwritten files also propagate
`rsync`	Copies files between disks or LAN systems	Local disk or network copy workflows	Not presented here as the cloud-focused option
`tar` + `pbzip2` + `GnuPG`	Creates encrypted incremental archives	Versioned backup sets and cloud upload	Restore is more manual and step-based

Key insight: rclone sync is useful when you want a simple mirror, not version history. If you need protection from mistakes or ransomware, add offline or encrypted incremental backups instead of relying on one mirrored copy. [#21021543]

Quick Facts

The sample script uses --transfers=64, so up to 64 file transfers can run at once during sync. [#21021543]
The thread recommends the 3-2-1 rule: keep 3 copies, on 2 different media, with 1 copy in another location. [#21021543]
The shown rclone sync job makes the target folder a 1:1 representation of the source, including deletions from the source side. [#21021543]
Suggested exclusions include Library, Debug, Release, Android, $RECYCLE.BIN, Virtual Machines, GIT, C-Sky, .venv, and NoBackupToDelete. [#21021543]
One large RAR archive with recovery data can copy faster than many small files when a project contains huge file counts. [#21021543]

How do I use rclone sync to make a one-click incremental backup of a folder on Windows?

Create a batch file that runs rclone.exe sync source target with your excludes, then launch it manually when needed. The sample uses W:/ as source, X:/W as target, --progress, --transfers=64, and multiple --exclude rules. 1. Download and unpack rclone. 2. Put the command in a .bat file. 3. Run the file to mirror changes in one click. [#21021543]

What does rclone actually do in a backup workflow, and why would I use it instead of plain copy and paste?

rclone sync mirrors a chosen folder so only changed content is copied after the first run. That makes it simpler and faster than repeated manual copy-paste, especially for electronics files such as diagrams, firmware, and documentation. The author explicitly presents it as a step up from CTRL + C and CTRL + V, while still warning that it is only one backup option, not a complete protection strategy. [#21021543]

How does the --delete-before option in rclone affect backup safety when files were removed from the source?

--delete-before removes files from the target before rclone transfers new data. In the sample script, that means anything deleted on the source side also disappears from the backup copy before new files are written. This keeps the target as a strict 1:1 mirror, but it reduces safety if the source deletion was accidental because the backup will reflect the mistake immediately. [#21021543]

What is the 3-2-1 backup rule, and how does it help protect files against disk failure or ransomware?

The 3-2-1 rule means keeping 3 copies, on 2 different media, with 1 copy in another location. The thread recommends it because one local mirror alone does not cover every failure mode. A second disk may help after a disk crash, but it may not save you from ransomware if both copies stay connected to the same computer. [#21021543]

Which folders should I exclude in an rclone backup script, such as $RECYCLE.BIN, Debug, Release, Android, or Virtual Machines?

Exclude folders that are disposable, bulky, or easily rebuilt. The sample excludes Library/**, Debug/**, Release/**, Android/**, $RECYCLE.BIN/**, Virtual Machines/**, GIT/**, C-Sky/**, .venv, and NoBackupToDelete/**. That reduces unnecessary copying and prevents the backup from filling with trash, build outputs, or very large working directories that do not belong in a simple mirror. [#21021543]

Why can a mirrored rclone sync cause file loss after accidental deletion, overwriting, or ransomware encryption?

A mirrored sync can lose data because it reproduces the current source state, including bad changes. If you overwrite an important file and sync, the older version is replaced. If you delete a file by mistake and sync, the target deletion follows. If malware encrypts the source and sync runs, the damaged files can replace the healthy copies too. [#21021543]

What is the difference between rclone sync and rsync for backing up files between local disks, network drives, and the cloud?

In this thread, rclone is framed as the more useful option when cloud upload matters, while rsync is suggested for regular local-disk or local-network copying. One reply argues that disk-to-disk or LAN copies can already be handled with rsync, and that rclone becomes more compelling when you upload backup files to cloud accounts. The practical difference here is workflow focus, not a full feature comparison. [#21025644]

How can I make rclone work with an external drive or a mapped network drive for backups?

Point the script at the drive letter of the destination you want to use. The thread lists three workable targets: a second internal disk, an external drive connected when needed, and a mapped network drive. That means you can keep the same basic rclone sync approach and just change the target path, then run it manually or automatically depending on your risk model. [#21021543]

What is an incremental backup, and how is it different from file synchronization in rclone?

An incremental backup stores new changes as separate backup increments, while rclone sync keeps one current mirror. The thread’s rclone example copies only what changed since the previous run, but it still updates the target into a 1:1 state. That differs from true incremental archives because a mirror does not preserve older file versions after overwrites or deletions. [#21021543]

Are there tools that store only the changes made inside modified files instead of re-copying whole files during backup?

Yes, but the thread says those tools are usually larger backup systems rather than simple synchronization tools. One reply explicitly notes that systems which add only the changed parts of files do exist, but they are not the typical lightweight sync utilities people use for quick folder mirroring. In this discussion, they are treated as a separate class from basic sync workflows. [#21024402]

How do I create encrypted incremental backups with tar, pbzip2, and GnuPG instead of a simple rclone mirror?

Use tar with a snapshot file, compress with pbzip2, then encrypt with gpg2. 1. Run tar c -g ~/backup/moje_dane1.snar ... from the chosen source directory. 2. Pipe the archive through pbzip2. 3. Encrypt the output with gpg2 and save it as an incremental file such as poziom_N.bz2. This creates encrypted incremental backups instead of a live mirror. [#21025644]

What is a tar .snar snapshot file, and how is it used for incremental backups?

".snar is a tar snapshot file that records backup state, enabling later runs to capture only incremental changes instead of rebuilding a full archive each time." In the thread, ~/backup/moje_dane1.snar is passed with tar -g, which is the mechanism used to track successive backup levels like poziom_N. [#21025644]

How should I restore a backup made with tar incremental archives and GnuPG encryption step by step?

Restore it by decrypting, decompressing, and extracting each increment in order into an empty directory. 1. Start with an empty target folder. 2. For each backup file, run gpg2 --decrypt ... | pbzip2 -d | (cd target && tar x -g /dev/null). 3. Apply every increment in sequence, one file after another. The thread states that order matters during restore. [#21025644]

Why does copying many small files take so long, and when is it worth packing a project into a RAR archive with recovery data before backup?

Copying many small files takes longer because the file count itself creates overhead, even when the total size is manageable. The thread gives a practical example: long-term storage of a GIT code repository may benefit from one large RAR archive with recovery data. In that case, a single large archive can copy much faster than thousands of separate small files. [#21021543]

What are some practical ways to improve a simple rclone backup script so it is safer against mistakes and ransomware?

Use more than one backup method and avoid relying on a single always-connected mirror. The thread suggests the 3-2-1 model, warns that a second internal disk does not stop ransomware, and mentions encrypted incremental archives as a safer alternative for versioned protection. A practical improvement is to keep the simple rclone mirror for speed, then add offline or cloud-uploaded encrypted increments for recovery after mistakes. [#21021543]

AI summary based on the discussion. May contain errors.