I have a setup that involves syncing files from my laptop to a server regularly. This has been working pretty well for a long time, apart from the odd verification failures. Both machines are WiFi-connected, so when I attempt to sync from a room far away from the WAP, the failure rates for larger files are higher, I would guess due to packet losses.
Today, I am sitting at the same spot I usually do this successfully with no issues, and I get errors after errors after errors. The odd one will go through after multiple tries, but generally it is just not working properly. I also got a broken pipe today during one attempt. This is not the first time it happens, and I feel crazy for thinking it is correlated to do bad weather, as if that should somehow affect my indoor WiFi quality…
Anyways, I tried to look at the rsync versions on the sender and receiver, and noticed that while both are the same application version number (3.2.7), they operate on different protocol versions (sender: 31, receiver: 32). I found this a bit odd, and I was unable to figure out how I would force my laptop to also use protocol version 32. I know I can pass a --protocol=NUM argument, but that seems to be used to force the sender to use an older version in case the receiver only has an older version, which is the opposite of my current situation.
And what is the likelihood that this is the cause of my woes?
Versions should be fine. Your options matter though, so send the full command you’re using.
Also try this:
- Open one terminal window and run a ping at your remote machine and let it keep running
- Open another term and run your sync. See if ping lags or delay issues start when the rsync issues start in both windows
I use this:
rsync -rah --progress /path/on/source/ user@ip.of.local.server:/path/on/destinationI will try the ping next time I attempt this. There doesn’t seem to be a definite time when the issues start though. It tries to copy the file over, and when it is done it continues to the next. If the first one didn’t succeed, it will retry and if that also fails, it will say “ERROR verification failed - discarding update” (paraphrased) and continue to retry the next file if that also failed.
I do see some fluctuations in the transfer speeds during transfer, which could indicate the times the connection is struggling.
If you’re getting file verification errors, it probably means there are issues with files on one end of the other.
So a few things:
- What’s this remote machine, and have you checked the filesystem recently?
- Have you checked the filesystem on your local machines you’re copying files from?
- Have you tried copying to an empty remote.traget directory and seeing if you still get these errors?
- If 1 & 2 are fine, and then 3 works without problems, make a short list of some of the files that are throwing errors. Do they happen every single time at these same files? What’s unique about these files?
Maybe the one lagging behind had its package updated but still has a running daemon on the old version?
Other people’s wifi can affect yours, and vice versa, if they are occupying the same channel(s).
Most likely something on that channel is spamming multicast. That kills most consumer wifi routers (in default settings). Usually something like a sonos or Google home broadcast group.
Could also be a camera that’s constantly transmitting, also occupying the channel for a long time.
But really, it could be anything.
Use an app like PingTools (Android) that can graph what is on each wifi channel. Check to find the cleanest channels in your area and configure your router to use that channel.
Failing that, if it’s on your network and you don’t know what it is…change your Wi-Fi password to kick everything off, then slowly re-add devices with the new password until you find the culprit.
If you’re curious and technically-minded, I highly recommend this write up: https://www.wiisfi.com/
Most likely something on that channel is spamming multicast. That kills most consumer wifi routers (in default settings). Usually something like a sonos or Google home broadcast group.
I might (i.e. I definitely do) have a non-ideal setup at home that contributes to this, with the router/WPA, a RPi running HA with a Zigbee antenna just next to it, my server in the shelf next over and a Sonos above it. Worst of all, the server is running on WiFi and it is sat in immediate vicinity to my router. Why? Because I could not for the life of me make the ethernet transfer speeds be more than somewhere around 1-5 MiB/s, so I gave up. But considering these issues are so infrequent from this location I assume it is mostly due to outside interference.
And adding to that, …
Use an app like PingTools (Android) that can graph what is on each wifi channel. Check to find the cleanest channels in your area and configure your router to use that channel.
… this scan shows that all discovered networks are occupying the same channels (98-114 it looks like). I have still not figured out what OpenWRT option to go for (the OpenWRT One, which I had originally planned, quickly became very expensive with imports and tolls that I ended up not buying, despite having decided to go for it). So I am using the stock ISP router, which I assume everyone else in my building are as well. And I have been unable to locate an option to change channels in its interface.
If you’re curious and technically-minded, I highly recommend this write up: https://www.wiisfi.com/
That is a resource I didn’t know I needed! Thanks :)
Seems unlikely. It would probably “not work at all” if a protocol mismatch was an issue. I’m willing to bet raync can fall back to older protocols.
Makes sense that it would be able to, especially considering the flag that allows you to force an older version as the sender. Still find it strange that on equal version numbers they default to different protocol numbers. I would have at least thought there was an easy way to tell it which protocol version to use by default, but I have at least not been able to find out how.
I feel crazy for thinking it is correlated to do bad weather, as if that should somehow affect my indoor WiFi quality…
well, if it’s raining more people might stay indoors on their wifi, exacerbating any channel interference problems
Ah, that would make sense. I think my immediate neighbors are home, but about half of my building should be traveling for Easter (judging by the empty parking garage at least).
A bit off topic but i used to had a fiber provider which on rainy days internet would intermittently go down. I think the outside terminal was exposed or something. Also between 2AM and 6AM internet would start dropping packets like crazy.
between 2AM and 6AM internet would start dropping packets like crazy
about 20 years ago I was remotely troubleshooting a microwave connectivity problem that occurred at a clients workplace about 10pm each night. Lasted about an hour. There was no one at work then but data transfers between their server and the mothership would fail.
One night the client went to the site at night to check an alarm and noticed there was a bobtail truck parked next to the building. The aero deflector attachment on its roof blocked line-of-sight with the tower, causing the problem. He asked the driver to nap at some other location in the parking lot and the problem went away.
