[clamav-users] local server takes time to update clamav db

[Paul Kosinski]

"This is one of the IPs which I was expecting to see.  I wouldn't
expect any problems with it, our ClamAV server updated from it at
1818 GMT last night."

Unfortunately, given the way Cloudflare works, the IP address
(e.g., 104.16.218.84) isn't the whole story. A particular Anycast IP
address such as this will route to the "nearest" server for that IP
address, and different servers may behave differently.

The HTTP(S) response header indicates which of the Cloudflare
servers the IP address actually routed to, for example:

  CF-RAY: 433942cde659ae1a-BOS

But I think you have to pretend you are ClamAV, or the server rejects
you, as in:

  User-Agent: ClamAV/0.103.0 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)

(At least this is the way it was in 2018.)

In the summer of 2018 (just after ClamAV started using Cloudflare) we
were having trouble in that our local BOS server was often behind the
latest ClamAV CVD file which was advertised by the DNS TXT record. I
finally gave up trying to have a local mirror for CVD files, and just
changed all our ClamAV machines to use the "scripted update" (CDIFF)
method individually. There are so few machines that it turned out to
*save* bandwidth in practice. 

P.S. There are a lot of emails about this in the ClamAV list for July
2018 et seq with subject lines: "We STILL cannot reliably get virus
updates (since new mirrors)".





On Wed, 9 Dec 2020 11:12:28 +0000 (GMT)
"G.W. Haywood via clamav-users" <clamav-users at lists.clamav.net> wrote:

> Hi there,
> 
> On Wed, 9 Dec 2020, Gal Cohen wrote:
> 
> > 5. here are the full logs of the latest update failure (26011 -> 26012),freshclam run takes 19 sec
> > Tue Dec  8 22:00:02 2020 -> ClamAV update process started at Tue Dec  8 22:00:02 2020
> > ...
> > Tue Dec  8 22:00:02 2020 -> *check_for_new_database_version: Local copy of daily found: daily.cvd.
> > Tue Dec  8 22:00:02 2020 -> *query_remote_database_version: daily.cvd version from DNS: 26012
> > Tue Dec  8 22:00:02 2020 -> daily database available for update (local version: 26011, remote version: 26012)
> > Tue Dec  8 22:00:02 2020 -> *Retrieving https://database.clamav.net/daily.cvd
> > Tue Dec  8 22:00:02 2020 -> *downloadFile: Download source: https://database.clamav.net/daily.cvd
> > Tue Dec  8 22:00:02 2020 -> *downloadFile: Download destination: /data/tmp.7624b/clamav-cde3734f56b3b9351a0261c3b140966f.tmp
> > *   Trying 104.16.218.84:443...  
> 
> This is one of the IPs which I was expecting to see.  I wouldn't expect any
> problems with it, our ClamAV server updated from it at 1818 GMT last night.
> 
> Maybe you have a proxy between you and the Cloudflare servers which is caching
> the data downloads?  Try downloading the 'daily' file with 'wget' from several
> different places and check which versions you receive.
>