Not sure what DNS caching would have to do with this. As I understand
"anycast", it happens at the IP address level. An anycast IP address
gets routed differently depending are where you are -- different
(regional) routers have different "next hops" for the IP address, and
it eventually ends up at a "nearby" server. This is in addition to the
fact that
database.clamav.net resolves to 5 different IP addresses (all
of which are anycast IPs, I would guess).
The Cloudflare servers, although they have the same IP address(es) seem
to identify themselves by means of an HTTP header that they return:
CF-RAY: 4852e02c35ae5a5c-BOS
CF-RAY: 48526af5624356c3-IAD
Finally, AFAIK, I always get the same result for
"dig TXT
current.cvd.clamav.net"
no matter where I 'dig' (or 'host') from.
On Fri, 7 Dec 2018 19:27:20 -0500
Eric Tykwinski <
eric-list@truenet.com> wrote:
This is getting rather technical, and probably some of CloudFlare’s
secret sauce. It sounds like the anycast DNS that cloudflare hosts
isn’t really working, or at least I would assume that they are using
anycast.
So you query current.cvd.clamav.net <http://current.cvd.clamav.net/>
but are getting different results at IAD and BOS. Now next is the
inclusion of Comcast, which may and probably is caching DNS records
beyond normal TTLs which could cause the difference. I personally
always run an Unbound cache server on my mailserver networks to cache
dns for at least an hour for rbls that I’m not rsyncing, but that
could cause an issue with Microsoft’s wonderful 10 second MX
records. So that’s where I’ve run into this issue, but not often
enough since I’m just caching for an hour and probably MS expects it.
So my guess, is probably not anycast, but a caching DNS server that
is still giving older records.
Sincerely,
Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300
On Dec 7, 2018, at 6:20 PM, Paul Kosinski <clamav-users@iment.com>
wrote:
As some of you may be aware, ever since ClamAV began using
Cloudflare, we have seen many occasions when files like daily.cvd
were not available to our LAN until well after the DNS TXT record
implied they should be.
However, we discovered that these same files *are* available to our
Web/email server right away. So what is the difference? The first
difference is that our Web server (a VM) is offsite, and is served
by the "IAD" Cloudflare complex, whereas our local setup is served
by the "BOS" Cloudflare complex.
The second, and likely explanatory difference, is that our local
setup is connected via Comcast (a dynamic IP and all that), while
our Web server (with its static IP etc.) is almost certainly more
directly connected to the Internet as a whole.
The workaround we have adopted is as follows: we installed a
"tinyproxy" server on our offsite VM. To ensure it only proxys for
us, it listens on the encrypted OpenVPN tunnel we already had in
place for FTP uploads etc. Then, instead of directly accessing
database.clamav.net, freshclam uses our remote VM as a proxy,so
that the cvd files are downloaded indirectly from Cloudflare's IAD
server complex (via tinyproxy) rather than directly from
Cloudflare's BOS server complex.
Since switching to this workaround a few days ago, we haven't
observed any delays: the cvd files are available right away when
the DNS TXT query says they should be.
I strongly suspect that Comcast is the culprit in the delays that
had plagued us. This is especially suggested by the fact that
Cloudflare returns a "Cache-Control:" header similar to:
Cache-Control: public, max-age=13672
where the max-age value varies, but is often several hours.
In my opinion, for data like ClamAV virus updates, the
"Cache-Control:" should specify "no-cache". Can Cloudflare do this
for ClamAV?
---------------------------------------------------------------------
Below is a pair of recent (pre-workaround) log excerpts. They show a
delay of over 2.5 hours experienced from BOS (via Comcast) vs no
delay from IAD.
Note that the BOS "Date:" timestamp of 16:49:01 GMT *still* shows
a "Last-Modified:" timestamp of 06:15:18 GMT, while IAD already
shows the up-to-date "Last-Modified:" timestamp of 14:14:30 GMT at
the much earlier "Date:" of 14:29:01 GMT!
IAD
Date: Sun, 02 Dec 2018 14:09:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 14:29:01 GMT
Last-Modified: Sun, 02 Dec 2018 14:14:30 GMT
ClamAV-VDB:02 Dec 2018 09-13
-0500:25173:2167842:63:ba557f61737b9d4b66acc96f7044b524:3nBAOxo97ssSNZb
BOS
Date: Sun, 02 Dec 2018 14:09:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 14:29:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 14:49:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 15:09:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 15:29:02 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 15:49:02 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 16:09:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 16:29:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 16:49:01 GMT
Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT
ClamAV-VDB:02 Dec 2018 01-14
-0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT
Date: Sun, 02 Dec 2018 17:09:01 GMT
Last-Modified: Sun, 02 Dec 2018 14:14:30 GMT
ClamAV-VDB:02 Dec 2018 09-13
-0500:25173:2167842:63:ba557f61737b9d4b66acc96f7044b524:3nBAOxo97ssSNZb
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.nethttp://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml