This would be an interesting video to make if you’re ever inclined, I don’t think I’ve seen this raised by yourself. “When to sync and when to work directly from a server” or “To Sync or not to Sync files”.
I have to admit, this suggestion comes to you after I had an issue with a Workstation that had SyncThing installed. An image from the previous day was restored and because I was worried about file sync issues, I had to be quite careful with the restore procedure.
My server has been super(micro) reliable for many years now, as has SyncThing, so despite the risk of a network or server failure, I am half tempted to edit files directly from the server, instead of from a locally sync’d copy. Alternatively though, I could have a secondary drive on the workstation and let that be for sync’d folders - making restores easier (although they are very infrequent).
Some background if interested
Only TrueNAS is used, Server No. 1 has the ‘hot’ files, these are replicated daily to Server No. 2. (and Backblaze). Weekly Servers No. 3 (cold files) & 4 are booted up to also receive replications of Server No. 1.
I began using SyncThing in 2022, following you bringing it to my attention and it has been brilliant (once I had tested it and familiarised myself). There are occasional moments when it’s misbehaved, but they’re normally easy to resolve - deleted empty folders don’t stay deleted for example.
Not sure how to make a full video on the topic. For files that I always want local, I use Syncthing. For files that I need to share in an office with other via a file share, I don’t use Sycnthing.
Good point of course, this is very much your area. But I had thought it’s an aspect rarely discussed and there may be an audience that are more conflicted than your good self.
For myself, the hot files are ~400GB and it would be helpful if they weren’t included in automated image backups, partly the size but also the aforementioned other issues.
I must admit even after reading your post twice I am still confused a bit, and that’s likely on me. What is your use case?
For config files, HTML pages or what have you on that server that you edit directly, neither editing locally nor Syncthing are the common approaches. One would likely use some sort of git repository as the source of truth, edit from wherever (your local workstation for example), push the changes and then either have a CI/CD auto deploy on the server or manually pull.
Another point Syncthing isn’t a backup solution.
What kind of files are in those 400GB that you edit locally on the server?
Sorry to read of your confusion, the use case is an Architectural practice - I probably should have mentioned that.
Ah, yes there has been a misunderstanding, with my use case the files types are exclusively documents, spreadsheets, pdf’s and drawing files.
The dataset that the SyncThing files resides within, have regular snapshots that are then replicated to 3-4 other servers. In addition every 2 hours during the working day, they are sent to Backblaze. I’m hoping that’s a sufficient backup solution.
Further background
Up until 2019 I solely used cloud, the likes of Google and Dropbox. Then I began exceeding the storage limits of 1TB and juggling files between cloud and local storage became tiresome - I was also finding that cloud didn’t interact well with some of the drawing files that have associated files. Something that also concerned me most was that some of the drawing files are sensitive to file corruption - these are for construction purposes, a partial corruption could alter building dimensions. I heard of ZFS and decided to hop onto that for all my storage. My live files are stored within SyncThing (that behaves very well with the files), while another 600TB are stored on a server and accessed via SMB.
That makes complete sense. Thank you for the background info.
Since all files are in Syncthing, in theory it shouldn’t matter where you do your edits. They will all be kept in sync. Syncthing has versioning you likely know about which would make the eventual conflicts easier to deal with.
Now that I better understood your use case I am with Tom here and would just use TrueNAS shares for the files that need to be shared within your practice between various desktops. And probably Syncthing only for keeping up to date copies of those files you need on your laptop, your tablet, smartphone, etc.
You’re very welcome, but thank you also for following up.
Yes, I’ve been very pleased with SyncThing, though it does take some care to set up and I did test it many times before implementing.
I’ll do that, just put the files on an SMB. I’ll ensure that I set up the correct snapshot routines to allow for that and have them replicate to a secondary machine throughout the day. That way if the main server goes down, I can simply disable read-only and switch to the secondary.
Thanks again for your help and patience Dr Heat
Likely surplus info!
I have two buildings and don’t work in the one with the servers in, I only have 1 CAT cable connecting me, but the future plan is to have 2x 10G fibre and retain at least on conventional CAT cable for redundancy. I would also have a server in the building I work in, perhaps that can be the active server, it can go in a separate room to keep the noise down. Underground conduit is installed already, just need to pull cables. Part of the reason for opting for fibre is to electrically separate the 2 buildings and prevent surges spanning both buildings.
