[clamav-users] Detected duplicate databases

Al Varnell alvarnell at mac.com
Sun Jan 14 09:36:29 UTC 2018


On Sat, Jan 13, 2018 at 04:12 AM, Hugo Deprez wrote:
> 
> hello,
> 
> sorry I wasn't clear :
> 
> - I used solution number 3 or more than one year, but freshclam on clients
> was randomly reporting error on downloading cvd files. I never found the
> solution :
> 
> *Tue Dec  1 17:54:34 2015 -> WARNING: Can't download daily.cvd from *
> 
> *mirror*
> 
> - So now I switch to solution number 2 from
> https://www.clamav.net/documents/private-local-mirrors
> 
> My "/etc/clamav/freshclam.conf" is :
> 
> *AllowSupplementaryGroups falseBytecode trueChecks
> 24CompressLocalDatabase noConnectTimeout 30DNSDatabaseInfo
> current.cvd.clamav.net
> <http://current.cvd.clamav.net>DatabaseDirectory
> /var/lib/clamavDatabaseMirror mirror.recette.localDatabaseOwner
> clamavDebug falseForeground falseLogFacility LOG_LOCAL6LogFileMaxSize
> 0LogRotate trueLogSyslog trueLogTime yesLogVerbose falseMaxAttempts
> 5PidFile /var/run/clamav/freshclam.pidPrivateMirror
> mirror.recette.localReceiveTimeout 30ScriptedUpdates noTestDatabases
> yesUpdateLogFile /var/log/clamav/freshclam.log*
> 
> But when I run freshclam, it still downloading the cld files :

Because you changed PrivateMirror to mirror.recette.local. See freshclam.conf:

# This option allows you to easily point freshclam to private mirrors.
# If PrivateMirror is set, freshclam does not attempt to use DNS
# to determine whether its databases are out-of-date, instead it will
# use the If-Modified-Since request or directly check the headers of the
# remote database files. For each database, freshclam first attempts
# to download the CLD file. If that fails, it tries to download the
# CVD file. This option overrides DatabaseMirror, DNSDatabaseInfo
# and ScriptedUpdates. It can be used multiple times to provide
# fall-back mirrors.
# Default: disabled
#PrivateMirror mirror1.mynetwork.com

Try commenting that line out.

> *# freshclam*
> 
> *ClamAV update process started at Sat Jan 13 13:07:47 2018Reading CVD
> header (main.cld): OK (IMS)Reading CVD header (main.cvd): OK
> (IMS)main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60,
> builder: sigmgr)Reading CVD header (daily.cld): OKDownloading
> daily.cld [100%]daily.cld updated (version: 24217, sigs: 1823727,
> f-level: 63, builder: neo)Reading CVD header (bytecode.cld):
> OKbytecode.cvd is up to date (version: 319, sigs: 75, f-level: 63,
> builder: neo)[LibClamAV] Detected duplicate databases
> /var/lib/clamav/daily.cvd and /var/lib/clamav/daily.cld. The
> /var/lib/clamav/daily.cvd database is older and will not be loaded,
> you should manually remove it from the database directory.Database
> updated (6390051 signatures) from mirror.recette.local (IP:
> 192.168.1.15)*If I remove cld files from my private mirror :
> 
> *# freshclamClamAV update process started at Sat Jan 13 13:10:18
> 2018Downloading main.cld [100%]main.cld updated (version: 58, sigs:
> 4566249, f-level: 60, builder: sigmgr)Downloading daily.cld
> [100%]daily.cld updated (version: 24217, sigs: 1823727, f-level: 63,
> builder: neo)WARNING: getfile: bytecode.cld not found on
> mirror.recette.local (IP: 192.168.1.15)WARNING: Can't download
> bytecode.cld from mirror.recette.localDownloading bytecode.cvd
> [100%]bytecode.cvd updated (version: 319, sigs: 75, f-level: 63,
> builder: neo)Database updated (6390051 signatures) from
> mirror.recette.local (IP: 192.168.1.15)*
> 
> I just want to run freshclam without any error.
> 
> * Any idea ? *
> 
> On 11 January 2018 at 10:37, Al Varnell <alvarnell at mac.com> wrote:
> 
>> On Thu, Jan 11, 2018 at 01:15 AM, Hugo Deprez wrote:
>>> Hello,
>>> 
>>> thank you for the answer.
>>> I don't think my freshclam is trying to download the cdiff file because I
>>> use : ScriptedUpdates no
>>> 
>>> Is this parameter you are talking about incremental updates ?
>> 
>> Yes, that's what I was referring to.
>> 
>>> I know this is not bandwith efficient, but I had to many issues with
>>> freshclam and my local repository made by clamdownloader.pl
>> 
>> So you are using an option 3 private local mirror as described in <
>> https://www.clamav.net/documents/private-local-mirrors>. Make sure you
>> also change freshclam.conf "DatabaseMirror machine1.mylan" where
>> machine1.mylan is the name of your mirror server.
>> 
>> Also make sure you are using the latest version of the clamavdownloader.pl
>> <https://github.com/akissa/clamav-faq/blob/master/
>> mirrors/clamdownloader.pl>
>> 
>> The existing clamdownloader.pl script does not have any error correction
>> it simply bails out if a downloaded file is not valid and is unable to
>> retry different mirrors if one fails. That is the most likely reason for
>> those 404 errors.
>> 
>> I have no experience with private local mirrors, but you might have better
>> luck with the clamavmirror script <https://pypi.python.org/pypi/
>> clamavmirror/0.0.3>.
>> 
>> -Al-
>> 
>>> Best regards,
>>> 
>>> On 10 January 2018 at 10:20, Al Varnell <alvarnell at mac.com <mailto:
>> alvarnell at mac.com>> wrote:
>>> 
>>>> The first time freshclam pulls down a daily.cdiff file, your daily.cvd
>>>> file will be decompressed to daily.cld and the .cdiff file added to it.
>>>> From that point on you should only have the daily.cld file. One
>> exception
>>>> is that if for some reason freshclam is unable to find needed .cdiff
>> files
>>>> on a mirror, it will download a new .cvd file which will start the
>> process
>>>> again.
>>>> 
>>>> It is possible to reconfigure freshclam to download .cvd files by
>>>> disabling incremental updates, but that would be very inefficient use of
>>>> bandwidth and mirror server time, so unless you have a overriding need
>> for
>>>> that, I don't recommend it.
>>>> 
>>>> -Al-
>>>> 
>>>> On Wed, Jan 10, 2018 at 01:12 AM, Hugo Deprez wrote:
>>>>> Hello,
>>>>> 
>>>>> I have a question about daily.cvd and daily.cld files. If I understood
>>>>> correctly, those two files are almost the same : one is compressed, the
>>>>> other is not.
>>>>> Still I have an issue in my setup :
>>>>> 
>>>>> If I put both filtes in apache2 server (which act as PrivateMirror) :
>>>>> 
>>>>> -rw-r--r-- 1 www-data www-data 117892267 07.06.2017 23:38 main.cvd
>>>>> -rw-r--r-- 1 www-data www-data    153228 07.12.2017 03:17 bytecode.cvd
>>>>> lrwxrwxrwx 1 root     root             8 05.01.2018 10:14 main.cld ->
>>>>> main.cvd
>>>>> lrwxrwxrwx 1 root     root             9 05.01.2018 10:14 daily.cld ->
>>>>> daily.cvd
>>>>> lrwxrwxrwx 1 root     root            12 05.01.2018 10:14 bytecode.cld
>> ->
>>>>> bytecode.cvd
>>>>> -rw-r--r-- 1 www-data www-data  43804052 10.01.2018 06:17 daily.cvd
>>>>> 
>>>>> On the client side I have this Warning :
>>>>> 
>>>>> LibClamAV Warning: Detected duplicate databases
>> /var/lib/clamav/daily.cvd
>>>>> and /var/lib/clamav/daily.cld, please manually remove one of them
>>>>> 
>>>>> 
>>>>> But If I remove the *.cld files on my PrivateMirror I got multiples 404
>>>>> errors from the Freshclam clients.
>>>>> 
>>>>> Is there a way to configure Freshclam in order to grabe only the cvd
>>>> files
>>>>> ?
>>>>> 
>>>>> Best regards,
>>>> 
>>>> _______________________________________________
>>>> clamav-users mailing list
>>>> clamav-users at lists.clamav.net <mailto:clamav-users at lists.clamav.net>
>>>> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>>>> 
>>>> 
>>>> Help us build a comprehensive ClamAV guide:
>>>> https://github.com/vrtadmin/clamav-faq
>>>> 
>>>> http://www.clamav.net/contact.html#ml
>>>> 
>>> _______________________________________________
>>> clamav-users mailing list
>>> clamav-users at lists.clamav.net <mailto:clamav-users at lists.clamav.net>
>>> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>>> 
>>> 
>>> Help us build a comprehensive ClamAV guide:
>>> https://github.com/vrtadmin/clamav-faq
>>> 
>>> http://www.clamav.net/contact.html#ml
>> 
>> -Al-



More information about the clamav-users mailing list