[clamav-users] ClamAV mirrors have gotten worse!

Dennis Peterson dennispe at inetnw.com
Tue Nov 27 07:19:52 UTC 2018

I think these reports don't tell you what you think they mean. In fact they're 
pretty much meaningless. The two different servers have different versions of 
the signature. That is perfectly normal - there is simply zero chance and it is 
naive to think they will always be fully synced in the same second of time of 
day. You can infer nothing when this occurs.

In any event these signature serial numbers are associated with the DNS txt 
record. The designed process is entirely serial - freshclam knows your installed 
signature file serial number, it knows the DNS txt record, and it requests 
updates from any of the signature servers if the local version is different from 
the DNS txt record. It will try all the mirrors until success or the list of 
mirrors is exhausted. Other things that mess with the fully synchronized state 
is that DNS caching, TTL, local system clock differences, and policies of 
various name service admins to ignore authoritative TTL suggestions.

The database.clamav.net dns is a round robin of 5 different servers and you 
cannot predict what you will receive. In fact in the best case the list be 
reordered each time you request the A record. And the chances of two different 
clients getting the same A record is very low.

Your own local resolver looks in its own cache to see if it has expired. The TTL 
record for the TXT record is 1800 seconds. If you use the dig command retrieve 
the TXT record you can watch the TTL count down:

     dig  txt current.cvd.clamav.net |grep TXT

To eliminate this as a problem source you can always use host table entries 
rather than dns for your tests. The round robin records ensure reliability for 
the client and crude load balancing for the server farm.

So worst case is the record you see can be 1800 seconds behind an updated TXT 
record. Obviously polling the current.cvd.clamav.net server directly will return 
an uncached record at the expense of recursing queries (use the IP instead of 
the hostname to avoid this).

Because these variables exist, freshclam is somewhat fault tolerant and will 
retry 3 times per mirror (default and is configurable), and if a mirror is in a 
failed state freshclam will map it out of the servers to try next time 
(mirrors.dat). The other variable is some of the sync process is demand-driven. 
In very busy systems (which these are) stale files should not exist very long. 
Your request just might be a trigger to refresh a stale file, and the next 
person to hit that server will retrieve the updated file and your system will 
move to another mirror. This scenario presumes files are pulled to the mirrors, 
not pushed.

I do believe your angst over not having complete system synchronization is 
unwarranted as there are too many uncontrollable variables and it's really not 
critical if the first mirror doesn't respond.

Finally - the current cloudflare process is pretty solid - it is a vast 
improvement over the historical mirror collaboration

On 11/26/18 4:19 PM, Paul Kosinski wrote:
> I believe that the delays we have been observing are due to some
> problem with the Boston Cloudflare servers, or, perhaps, Comcast has a
> "transparent" caching proxy which is causing us trouble.
> I recently installed the same build and configuration of ClamAV 0.100.2
> on our Web server, a virtual machine hosted in NYC. It runs the same
> extra code (curl etc.) to check the cvd version number that we have
> locally. Since Friday, there have been no delays there, although there
> have been several significant delays locally. They check at exactly
> the same time as each other (i.e., via NTP synced cron jobs).
> I also am now running, at each location, simple curls to read the first
> few bytes of the cvd files (to get the version number), *and* to log
> all the headers sent and received. These are also run at exactly the
> same time (as each other) via cron.
> The headers show that our local system uses the 'BOS' Cloudflare server,
> while the remote one uses the 'IAD' server:
>    CF-RAY: 47fd0b7af79dae32-BOS
>    CF-RAY: 47fd0b8064d9c1b8-IAD
> Interestingly, these two cron jobs sometimes show that the BOS server
> is out of date relative to the IAD server. For example, the following
> curls show that one cvd file served by the BOS server is one version
> behind that served by the IAD server at the *same* time. The files'
> "Last-modified" lines are of particular interest. The BOS server says
> the file was last modified on Mon, 26 Nov 2018 at 06:19:22 GMT, while
> the IAD server says the file was last modified on Mon, 26 Nov 2018 at
> 14:15:24 GMT.
> In particular, the BOS "Date:" header says it's already about 14 mins
> *later* than the IAD "Last-modified:" timestamp indicates. In other
> words, the file delivered by the BOS server is, at time of *delivery*,
> already about 14 minutes out of date.

More information about the clamav-users mailing list