[clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

Dennis Peterson dennispe at inetnw.com
Tue Dec 11 17:34:17 EST 2018


You know the daily.cvd file is now larger than the main.cvd file, so you are 
burning up a lot of bandwidth if your world-facing ClamAV mirror is ignoring 
cdiff files. If it is using freshclam then it is using cdiffs and merging them 
as part of the process of mirroring. In that case your clients won't see the 
cdiff files which is perfectly acceptable. I used to use a proxy when many 
systems were co-located and it was very effective and was also being used for 
other purposes. Life is much simpler now that I'm retired.

dp

On 12/11/18 11:45 AM, Paul Kosinski wrote:
> Ever since we set up a local mirror on our LAN, we have not been using
> cdiffs. The reason for this is that I followed the procedure outlined
> on the ClamAV website (about 2/3 down the page) at:
>
>    http://www.clamav.net/documents/clamav-virus-database-faq
>
> where it says:
>
> [Q] I’m running ClamAV on a lot of clients on my local network.  Can I serve the cvd files from a local server
>      so that each client doesn’t have to download them from your servers?
>    
> [A] Sure, you can find more details on our Mirror page.
>    
>     If you want to take advantage of incremental updates, install a proxy server and then
>      configure your freshclam clients to use it (watch for the HTTPProxyServer parameter in man freshclam.conf).
>    
>     The second possible solution is to:
>    
>        Configure a local webserver on one of your machines (say machine1.mylan)
>    
>        Let freshclam download the *.cvd files from http://database.clamav.net to the webserver’s DocumentRoot.
>    
>        Finally, change freshclam.conf on your clients so that it includes:
>    
>        DatabaseMirror machine1.mylan
>    
>        ScriptedUpdates off
>    
>        First the database will be downloaded to the local webserver and then the other clients
>          on the network will update their copy of the database from it.
>    
>        Important: For this to work, you have to add ScriptedUpdates off on all of your machines!
>
> Since I didn't want to set up a proxy server for this purpose, I used
> the 2nd solution (and a very trivial web server). Thus, cvd files only.
>
> P.S. I am now thinking about trying the BOS vs IAD test for cdiff
> files. But, even if cdiff files always work without any delays, doesn't
> "scripted update" on occasion have to back off to downloading full cvds?
>
> P.P.S. Thanks for the curl help!
>
>
>
> On Mon, 10 Dec 2018 20:34:45 -0800
> Dennis Peterson <dennispe at inetnw.com> wrote:
>
>> You were using curl (I did remember that after I posted as I'd helped
>> you sort out curl options to do what you wanted) to explore what was
>> available on the servers compared to what was on the DNS TXT record,
>> and that was outside process. It also ignored cdiff files that may
>> have been available in a version that matched the TXT record. The
>> purpose of the cdiff files is to cut down on bandwidth.
>>
>> dp
>>
>> On 12/10/18 6:34 PM, Paul Kosinski wrote:
>>> We ARE using freshclam to perform the actual update. And always have
>>> been!
>>>
>>> We've only been using curl (not wget, if that matters) to pull the
>>> first few bytes of the cvd to see if its version number matches
>>> what the DNS TXT query said.
>>>
>>> We do this because, after the conversion to Cloudflare, we were
>>> getting lots of FAILURES where *freshclam* said things were out of
>>> sync (and eventually disabled all the mirrors).
>>>
>>> And we have recently seen that our Web server sometimes can get the
>>> new updates (from IAD) *hours* before our main LAN does (from BOS).
>>>
>>> P.S. It's been quite frustrating getting some replies seemingly
>>> based on assumptions that we are doing things we shouldn't, when we
>>> aren't in fact doing those things. (Like not using freshclam.)
>>>
>>>
>>>
>>> On Mon, 10 Dec 2018 16:46:42 -0800
>>> Dennis Peterson <dennispe at inetnw.com> wrote:
>>>
>>>> Exactly right. We can't be blaming the ClamAV process when we don't
>>>> use the ClamAV process. People that don't use freshclam should have
>>>> no expectation of high reliability. In fact any expectations are
>>>> baseless when the wrong tools are employed.
>>>>
>>>> dp
>>>>
>>>> On 12/9/18 5:44 AM, Joel Esler (jesler) wrote:
>>>>> As it should be.  No one should be downloading the daily and main,
>>>>> (although thousands are), cdiffs were created for a reason.
>>>>>
>>>>> Sent from my  iPhone
>>>>>
>>>>>> On Dec 9, 2018, at 06:58, Eric Tykwinski <eric-list at truenet.com>
>>>>>> wrote:
>>>>>>
>>>>>>    From back in archives, I think he’s using wget to just pull the
>>>>>> files, but freshclam would just pull the cdiffs and keep you up
>>>>>> to date on the next check.
> _______________________________________________
> clamav-users mailing list
> clamav-users at lists.clamav.net
> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml





More information about the clamav-users mailing list