[clamav-users] We STILL cannot reliably get virus updates (since new mirrors)
clamav-users at iment.com
Sun Jul 1 23:24:45 EDT 2018
The debug flag on the freshclam invocation seems only to report on the
processing that happens *after* the cvd is successfully downloaded.
So... I went to a more basic level and captured the actual network
traffic with pcap and then examined it with wireshark.
I found an update attempt that failed because the mirrors were "not
synchronized" and then went to the capture files to find the DNS that
retrieved the version number, and the HTTP that retrieved the data.
(These are in separate pcap files because our DNS uses our old DSL, but
HTTP uses the faster cable modem.) Anyway, here is a summary of a
typical "not synchronized" failure.
Here is an excerpt from the freshclam log from Sunday 01 July 2018 at
09:36:01 EDT (aka 13:36:01 GMT):
daily.cvd version from DNS: 24713
Ignoring mirror 22.214.171.124 (due to previous errors)
Trying host db.us.clamav.net (126.96.36.199)...
Using ip '10.11.14.160' for fetching.
Trying to download http://db.us.clamav.net/daily.cvd (IP: 188.8.131.52)
Downloading daily.cvd [100%]
WARNING: Mirror 184.108.40.206 is not synchronized.
Indeed, when we look at the captured DNS TXT record and compare it to
the captured downloaded data, we see that it in fact is *not* synchronized!
First, the wireshark decode of the TXT record, showing that the current
version is in fact 24713.
No. Time Source Port Destination Port Length Protocol Info
4723 09:36:02.098521 220.127.116.11 53 10.25.26.60 26793 148 DNS Standard query response 0x5469 TXT
Frame 4723: 148 bytes on wire (1184 bits), 148 bytes captured (1184 bits)
Ethernet II, Src: 6c:72:20:41:d1:32 (6c:72:20:41:d1:32), Dst: fc:aa:14:5a:1b:68 (fc:aa:14:5a:1b:68)
Internet Protocol Version 4, Src: 18.104.22.168 (22.214.171.124), Dst: 10.25.26.60 (10.25.26.60)
User Datagram Protocol, Src Port: 53 (53), Dst Port: 26793 (26793)
Domain Name System (response)
[Request In: 4722]
[Time: 0.044517000 seconds]
Transaction ID: 0x5469
Flags: 0x8400 Standard query response, No error
Answer RRs: 1
Authority RRs: 0
Additional RRs: 1
current.cvd.clamav.net: type TXT, class IN
current.cvd.clamav.net: type TXT, class IN
Type: TXT (Text strings)
Class: IN (0x0001)
Time to live: 30 minutes
Data length: 43
TXT Length: 42
<Root>: type OPT
Type: OPT (EDNS0 option)
UDP payload size: 1680
Higher bits in extended RCODE: 0x0
EDNS0 version: 0
Bit 0 (DO bit): 1 (Accepts DNSSEC security RRs)
Bits 1-15: 0x0 (reserved)
Data length: 0
Next is the HTTP request and (truncated) response from 126.96.36.199,
as provided by wireshark's handy Follow TCP Stream. Note that the
response says that the version is only 24712, not 24713!
GET /daily.cvd HTTP/1.0
User-Agent: ClamAV/0.99.4 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
HTTP/1.1 200 OK
Date: Sun, 01 Jul 2018 13:36:20 GMT
Set-Cookie: __cfduid=d88ed50c4d496c2c73f2f8c6f2579b9bb1530452180; expires=Mon, 01-Jul-19 13:36:20 GMT; path=/; domain=.clamav.net; HttpOnly
Last-Modified: Sun, 01 Jul 2018 04:40:58 GMT
Expires: Sun, 01 Jul 2018 08:40:58 GMT
ClamAV-VDB:01 Jul 2018 00-40 -0400:24712:2000714:63:6b67a9289bf468a7bc9caead0b485bfd:8PaJxsXEfye3s7v74s7ahRQ1IaryM7UxzfE8Cnb8g2l+wLIMC6wCwIvh3wvbLJzDtSMkeerx8s1O1K/TYiCf445F+79Srhgc/Rl1Qokm2IsUNOL4J2VbzRM/Akq0eTQhHk999P7Irz0MyLJWzNeXaKDWN0LmUmcpZrBTjhJYzQg:neo:1530420019
My conclusion is that the cause of this is a typical race condition:
the DNS TXT record is updated before Cloudflare has propagated the new
cvd file to all the mirrors.
I have attached the full freshclam log for this failing update attempt
to this email, The pcap file is far too big for email, but can be
retrieved via the following full URL (directory listing is blocked):
On Sun, 1 Jul 2018 00:37:35 +0000
"Joel Esler (jesler)" <jesler at cisco.com> wrote:
> Ping.clamav.net is an identification lookup. Helps us see what
> versions people are running out there and what version of ClamAV
> people are using. It’s failure shouldn’t stop the update process.
> Please give us a debug.
> Sent from my iPhone
> > On Jun 30, 2018, at 19:28, Paul Kosinski <clamav-users at iment.com>
> > wrote:
> > We are *still* failing to get ClamAV cvd files updates reliably --
> > even after deleting mirrors.dat before each attempt!
> > [... and much, much more ...]
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
More information about the clamav-users