[clamav-users] No good deed goes unpunished, or, why CVD files don't work

J.R. themadbeaker at gmail.com
Sat Dec 15 10:14:22 EST 2018

I seem to recall you said you had comcast, and I'm assuming it is a
business account. Have you tried calling their business support and
talked to someone that is actually local to explain your problem and
see if they possibly have a transparent cache in place and if it would
be possible to exclude you? I also seem to recall you're located in NY
(I could be wrong), but again being in a heavily populated area they
could be doing the caching to try and alleviate an over-saturated
local network. I don't think in reality the BOS cloudflare was always
behind, I think there *has* to be some other caching going on that
simply makes it *look* like it's behind. It also makes sense in that
if others in your area were requesting files from the BOS server, that
would be in the cache... But none would be manually requesting from
another one (IAD) so there wouldn't be an existing cached copy and
thus you get the latest version.

Many years ago the company I worked for used Akami for caching static
content. Their caches were smart and knew when a file was changed
(even if the name was the same), however, web browsers on the other
hand typically had issues caching the older version. Even worse was
when a transparent proxy was somewhere in the mix doing its own
caching and ignoring things like when a file's date changes or
no-cache headers. We found out our company had one in place, and we
had to get our department excluded as it severely interfered with
development work.

I also believe you said that one of the other cloudflare servers had
the correct file when your local one didn't. Did you try changing your
freshclam.conf to point to said other server(s) instead of letting it
geo-locate you to your local cache that has caused you problems?

Third... Have you done a cost-benefit analysis? I know you said you
wanted to help reduce bandwidth, but when you are downloading the
entire daily.cvd file each time there is an update, that's currently a
little over 50MB each update. I downloaded the last 10 cdiff files and
they look to average about 15k... So by that math (I'm still drinking
my coffee this morning, so I could be wildly wrong)... You would need
to have over 3,333 machines to be saving any bandwidth...

Dennis posted what I was thinking about once (but didn't post about
since I've never tried it with clamav). Once you have the data you
need on your local network, you can push it out to clients however you
wish. I was thinking just basic rsync, followed by a notify command
for clamd... Or whatever newer and fancier program you might want to

Lastly, another route would be to setup your own transparent proxy, so
even if X machines were requesting a cdiff, it only gets downloaded
once and your local proxy caches it for all the others... You can do
it even with HTTPS traffic so in theory it should work.

More information about the clamav-users mailing list