[clamav-users] Detected duplicate databases

Hugo Deprez hugo.deprez at gmail.com
Thu Jan 18 18:07:07 UTC 2018


Hello,

I don't think I'm the only one having this issue. It seems that I used the
official configuration from http://www.clamav.net/doc/mirrors-faq.html
Should I report the issue somewhere ?

The only way I see to avoid this issue is to create a shell script and to
push the updates my self on all servers, but this is not the proper way to
do this...

Hugo

On 15 January 2018 at 11:44, Hugo Deprez <hugo.deprez at gmail.com> wrote:

> Hello,
>
> by using this freshclam.conf  :
>
> AllowSupplementaryGroups false
> Bytecode true
> Checks 24
> CompressLocalDatabase no
> ConnectTimeout 30
> DNSDatabaseInfo current.cvd.clamav.net
> DatabaseDirectory /var/lib/clamav
> DatabaseMirror mirror.recette.local
> DatabaseOwner clamav
> Debug false
> Foreground false
> LogFacility LOG_LOCAL6
> LogFileMaxSize 0
> LogRotate true
> LogSyslog true
> LogTime yes
> LogVerbose false
> MaxAttempts 5
> PidFile /var/run/clamav/freshclam.pid
> #PrivateMirror mirror.recette.local
> ReceiveTimeout 30
> ScriptedUpdates no
> TestDatabases yes
> UpdateLogFile /var/log/clamav/freshclam.log
>
> I got the old issue :
>
> root at server:/var/lib/clamav# rm -f *
> root at server:/var/lib/clamav# freshclam
> ClamAV update process started at Mon Jan 15 11:38:48 2018
> Downloading main.cvd [100%]
> main.cvd updated (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr)
> Downloading daily.cvd [100%]
> daily.cvd updated (version: 24223, sigs: 1825336, f-level: 63, builder:
> neo)
> Downloading bytecode.cvd [100%]
> bytecode.cvd updated (version: 315, sigs: 75, f-level: 63, builder:
> raynman)
> Database updated (6391660 signatures) from mirror.recette.local (IP:
> 192.168.1.15)
> root at raadmin:/var/lib/clamav# freshclam
> ClamAV update process started at Mon Jan 15 11:40:00 2018
> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder:
> sigmgr)
> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63,
> builder: neo)
> Downloading bytecode.cvd [100%]
> WARNING: Mirror 192.168.8.15 is not synchronized.
> Trying again in 5 secs...
> ClamAV update process started at Mon Jan 15 11:40:05 2018
> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder:
> sigmgr)
> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63,
> builder: neo)
> WARNING: Can't download bytecode.cvd from mirror.recette.local
> Trying again in 5 secs...
> ClamAV update process started at Mon Jan 15 11:40:10 2018
> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder:
> sigmgr)
> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63,
> builder: neo)
> WARNING: Can't download bytecode.cvd from mirror.recette.local
> Trying again in 5 secs...
> ClamAV update process started at Mon Jan 15 11:40:15 2018
> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder:
> sigmgr)
> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63,
> builder: neo)
> WARNING: Can't download bytecode.cvd from mirror.recette.local
> Trying again in 5 secs...
> ClamAV update process started at Mon Jan 15 11:40:20 2018
> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder:
> sigmgr)
> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63,
> builder: neo)
> ERROR: Can't download bytecode.cvd from mirror.recette.local
> Giving up on mirror.recette.local ...
> Update failed. Your network may be down or none of the mirrors listed in
> /etc/clamav/freshclam.conf is working. Check http://www.clamav.net/doc/
> mirrors-faq.html for possible reasons.
>
>
>
> On 14 January 2018 at 10:36, Al Varnell <alvarnell at mac.com> wrote:
>
>>
>> On Sat, Jan 13, 2018 at 04:12 AM, Hugo Deprez wrote:
>> >
>> > hello,
>> >
>> > sorry I wasn't clear :
>> >
>> > - I used solution number 3 or more than one year, but freshclam on
>> clients
>> > was randomly reporting error on downloading cvd files. I never found the
>> > solution :
>> >
>> > *Tue Dec  1 17:54:34 2015 -> WARNING: Can't download daily.cvd from *
>> >
>> > *mirror*
>> >
>> > - So now I switch to solution number 2 from
>> > https://www.clamav.net/documents/private-local-mirrors
>> >
>> > My "/etc/clamav/freshclam.conf" is :
>> >
>> > *AllowSupplementaryGroups falseBytecode trueChecks
>> > 24CompressLocalDatabase noConnectTimeout 30DNSDatabaseInfo
>> > current.cvd.clamav.net
>> > <http://current.cvd.clamav.net>DatabaseDirectory
>> > /var/lib/clamavDatabaseMirror mirror.recette.localDatabaseOwner
>> > clamavDebug falseForeground falseLogFacility LOG_LOCAL6LogFileMaxSize
>> > 0LogRotate trueLogSyslog trueLogTime yesLogVerbose falseMaxAttempts
>> > 5PidFile /var/run/clamav/freshclam.pidPrivateMirror
>> > mirror.recette.localReceiveTimeout 30ScriptedUpdates noTestDatabases
>> > yesUpdateLogFile /var/log/clamav/freshclam.log*
>> >
>> > But when I run freshclam, it still downloading the cld files :
>>
>> Because you changed PrivateMirror to mirror.recette.local. See
>> freshclam.conf:
>>
>> # This option allows you to easily point freshclam to private mirrors.
>> # If PrivateMirror is set, freshclam does not attempt to use DNS
>> # to determine whether its databases are out-of-date, instead it will
>> # use the If-Modified-Since request or directly check the headers of the
>> # remote database files. For each database, freshclam first attempts
>> # to download the CLD file. If that fails, it tries to download the
>> # CVD file. This option overrides DatabaseMirror, DNSDatabaseInfo
>> # and ScriptedUpdates. It can be used multiple times to provide
>> # fall-back mirrors.
>> # Default: disabled
>> #PrivateMirror mirror1.mynetwork.com
>>
>> Try commenting that line out.
>>
>> > *# freshclam*
>> >
>> > *ClamAV update process started at Sat Jan 13 13:07:47 2018Reading CVD
>> > header (main.cld): OK (IMS)Reading CVD header (main.cvd): OK
>> > (IMS)main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60,
>> > builder: sigmgr)Reading CVD header (daily.cld): OKDownloading
>> > daily.cld [100%]daily.cld updated (version: 24217, sigs: 1823727,
>> > f-level: 63, builder: neo)Reading CVD header (bytecode.cld):
>> > OKbytecode.cvd is up to date (version: 319, sigs: 75, f-level: 63,
>> > builder: neo)[LibClamAV] Detected duplicate databases
>> > /var/lib/clamav/daily.cvd and /var/lib/clamav/daily.cld. The
>> > /var/lib/clamav/daily.cvd database is older and will not be loaded,
>> > you should manually remove it from the database directory.Database
>> > updated (6390051 signatures) from mirror.recette.local (IP:
>> > 192.168.1.15)*If I remove cld files from my private mirror :
>> >
>> > *# freshclamClamAV update process started at Sat Jan 13 13:10:18
>> > 2018Downloading main.cld [100%]main.cld updated (version: 58, sigs:
>> > 4566249, f-level: 60, builder: sigmgr)Downloading daily.cld
>> > [100%]daily.cld updated (version: 24217, sigs: 1823727, f-level: 63,
>> > builder: neo)WARNING: getfile: bytecode.cld not found on
>> > mirror.recette.local (IP: 192.168.1.15)WARNING: Can't download
>> > bytecode.cld from mirror.recette.localDownloading bytecode.cvd
>> > [100%]bytecode.cvd updated (version: 319, sigs: 75, f-level: 63,
>> > builder: neo)Database updated (6390051 signatures) from
>> > mirror.recette.local (IP: 192.168.1.15)*
>> >
>> > I just want to run freshclam without any error.
>> >
>> > * Any idea ? *
>> >
>> > On 11 January 2018 at 10:37, Al Varnell <alvarnell at mac.com> wrote:
>> >
>> >> On Thu, Jan 11, 2018 at 01:15 AM, Hugo Deprez wrote:
>> >>> Hello,
>> >>>
>> >>> thank you for the answer.
>> >>> I don't think my freshclam is trying to download the cdiff file
>> because I
>> >>> use : ScriptedUpdates no
>> >>>
>> >>> Is this parameter you are talking about incremental updates ?
>> >>
>> >> Yes, that's what I was referring to.
>> >>
>> >>> I know this is not bandwith efficient, but I had to many issues with
>> >>> freshclam and my local repository made by clamdownloader.pl
>> >>
>> >> So you are using an option 3 private local mirror as described in <
>> >> https://www.clamav.net/documents/private-local-mirrors>. Make sure you
>> >> also change freshclam.conf "DatabaseMirror machine1.mylan" where
>> >> machine1.mylan is the name of your mirror server.
>> >>
>> >> Also make sure you are using the latest version of the
>> clamavdownloader.pl
>> >> <https://github.com/akissa/clamav-faq/blob/master/
>> >> mirrors/clamdownloader.pl>
>> >>
>> >> The existing clamdownloader.pl script does not have any error
>> correction
>> >> it simply bails out if a downloaded file is not valid and is unable to
>> >> retry different mirrors if one fails. That is the most likely reason
>> for
>> >> those 404 errors.
>> >>
>> >> I have no experience with private local mirrors, but you might have
>> better
>> >> luck with the clamavmirror script <https://pypi.python.org/pypi/
>> >> clamavmirror/0.0.3>.
>> >>
>> >> -Al-
>> >>
>> >>> Best regards,
>> >>>
>> >>> On 10 January 2018 at 10:20, Al Varnell <alvarnell at mac.com <mailto:
>> >> alvarnell at mac.com>> wrote:
>> >>>
>> >>>> The first time freshclam pulls down a daily.cdiff file, your
>> daily.cvd
>> >>>> file will be decompressed to daily.cld and the .cdiff file added to
>> it.
>> >>>> From that point on you should only have the daily.cld file. One
>> >> exception
>> >>>> is that if for some reason freshclam is unable to find needed .cdiff
>> >> files
>> >>>> on a mirror, it will download a new .cvd file which will start the
>> >> process
>> >>>> again.
>> >>>>
>> >>>> It is possible to reconfigure freshclam to download .cvd files by
>> >>>> disabling incremental updates, but that would be very inefficient
>> use of
>> >>>> bandwidth and mirror server time, so unless you have a overriding
>> need
>> >> for
>> >>>> that, I don't recommend it.
>> >>>>
>> >>>> -Al-
>> >>>>
>> >>>> On Wed, Jan 10, 2018 at 01:12 AM, Hugo Deprez wrote:
>> >>>>> Hello,
>> >>>>>
>> >>>>> I have a question about daily.cvd and daily.cld files. If I
>> understood
>> >>>>> correctly, those two files are almost the same : one is compressed,
>> the
>> >>>>> other is not.
>> >>>>> Still I have an issue in my setup :
>> >>>>>
>> >>>>> If I put both filtes in apache2 server (which act as PrivateMirror)
>> :
>> >>>>>
>> >>>>> -rw-r--r-- 1 www-data www-data 117892267 07.06.2017 23:38 main.cvd
>> >>>>> -rw-r--r-- 1 www-data www-data    153228 07.12.2017 03:17
>> bytecode.cvd
>> >>>>> lrwxrwxrwx 1 root     root             8 05.01.2018 10:14 main.cld
>> ->
>> >>>>> main.cvd
>> >>>>> lrwxrwxrwx 1 root     root             9 05.01.2018 10:14 daily.cld
>> ->
>> >>>>> daily.cvd
>> >>>>> lrwxrwxrwx 1 root     root            12 05.01.2018 10:14
>> bytecode.cld
>> >> ->
>> >>>>> bytecode.cvd
>> >>>>> -rw-r--r-- 1 www-data www-data  43804052 10.01.2018 06:17 daily.cvd
>> >>>>>
>> >>>>> On the client side I have this Warning :
>> >>>>>
>> >>>>> LibClamAV Warning: Detected duplicate databases
>> >> /var/lib/clamav/daily.cvd
>> >>>>> and /var/lib/clamav/daily.cld, please manually remove one of them
>> >>>>>
>> >>>>>
>> >>>>> But If I remove the *.cld files on my PrivateMirror I got multiples
>> 404
>> >>>>> errors from the Freshclam clients.
>> >>>>>
>> >>>>> Is there a way to configure Freshclam in order to grabe only the cvd
>> >>>> files
>> >>>>> ?
>> >>>>>
>> >>>>> Best regards,
>> >>>>
>> >>>> _______________________________________________
>> >>>> clamav-users mailing list
>> >>>> clamav-users at lists.clamav.net <mailto:clamav-users at lists.clamav.net>
>> >>>> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>> >>>>
>> >>>>
>> >>>> Help us build a comprehensive ClamAV guide:
>> >>>> https://github.com/vrtadmin/clamav-faq
>> >>>>
>> >>>> http://www.clamav.net/contact.html#ml
>> >>>>
>> >>> _______________________________________________
>> >>> clamav-users mailing list
>> >>> clamav-users at lists.clamav.net <mailto:clamav-users at lists.clamav.net>
>> >>> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>> >>>
>> >>>
>> >>> Help us build a comprehensive ClamAV guide:
>> >>> https://github.com/vrtadmin/clamav-faq
>> >>>
>> >>> http://www.clamav.net/contact.html#ml
>> >>
>> >> -Al-
>> _______________________________________________
>> clamav-users mailing list
>> clamav-users at lists.clamav.net
>> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>>
>> http://www.clamav.net/contact.html#ml
>>
>
>



More information about the clamav-users mailing list