[clamav-users] Problem with freshclam updating daily-25380.cdiff
Maarten Broekman
maarten.broekman at gmail.com
Wed Mar 6 15:41:29 UTC 2019
I'm not sure if the safebrowsing.cld is included in the daily cdiff, but
the current safebrowsing.cld takes between 50 and 70 seconds to *load* into
clamscan, where a copy from February loads in <5 seconds.
safebrowsing data:
Old (fast): ClamAV-VDB:13 Feb 2019 13-16
-0500:48472:3041760:63:X:X:google:1550081775
New (slow): ClamAV-VDB:05 Mar 2019 19-20
-0500:48473:3229612:63:X:X:google:1551831615
Anyone know what might have changed in there to so drastically increased
the load time?
This happened after freshclam ran last night.
# /opt/clamav/clamav/bin/clamscan -d ~/safebrowsing.cld
samples/clam_test.html
samples/clam_test.html: OK
----------- SCAN SUMMARY -----------
Known viruses: 3041760
Engine version: 0.100.2
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 2.423 sec (0 m 2 s)
# /opt/clamav/clamav/bin/clamscan -d
/opt/clamav/var/lib/clamav/safebrowsing.cld samples/clam_test.html
samples/clam_test.html: OK
----------- SCAN SUMMARY -----------
Known viruses: 3229612
Engine version: 0.100.2
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 64.429 sec (1 m 4 s)
On Wed, Mar 6, 2019 at 10:17 AM Micah Snyder (micasnyd) via clamav-users <
clamav-users at lists.clamav.net> wrote:
> I confirmed with our signature management team that the extended time
> processing daily-25380 is because this change is significantly larger than
> a standard update.
> This update drops 768053 hash-based signatures for malware that is
> detected by other more efficient logical signatures. The net result will
> be a leaner database that should load a little faster and take up less
> memory.
>
> The validation stage when creating the daily had estimated less than 26
> minutes for the cdiff to apply. You may be correct that it's much faster
> on x86 than on Sparc. 3h15m is definitely worse than expected, and I
> apologize for the inconvenience.
>
> Regards,
> Micah
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>
>
> On 3/6/19, 9:31 AM, "Pierre Dehaen" <dehaenp at drever.be> wrote:
>
> Yes Micah, it finished while I was checking the computer because of
> the messages received
> on the mailing list.
>
> $ tail -50 /var/log/freshclam.log
> ...
> --------------------------------------
> ClamAV update process started at Wed Mar 6 11:37:46 2019
> WARNING: Your ClamAV installation is OUTDATED!
> WARNING: Local version: 0.100.0 Recommended version: 0.101.1
> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav
> securiteinfo.hdb is up to date (version: custom database)
> securiteinfo.ign2 is up to date (version: custom database)
> Downloading javascript.ndb [*]
> javascript.ndb updated (version: custom database, sigs: 45008)
> securiteinfohtml.hdb is up to date (version: custom database)
> securiteinfoascii.hdb is up to date (version: custom database)
> securiteinfopdf.hdb is up to date (version: custom database)
> Downloading spam_marketing.ndb [*]
> spam_marketing.ndb updated (version: custom database, sigs: 24199)
> main.cld is up to date (version: 58, sigs: 4566249, f-level: 60,
> builder: sigmgr)
> Downloading daily-25380.cdiff [100%]
> daily.cld updated (version: 25380, sigs: 1503528, f-level: 63,
> builder: raynman)
> bytecode.cld is up to date (version: 328, sigs: 94, f-level: 63,
> builder: neo)
> Database updated (6139078 signatures) from db.be.clamav.net (IP:
> 104.16.219.84)
> Clamd successfully notified about the update.
>
> $ ls -l /var/log/freshclam.log
> -rw-r--r-- 1 clamav clamav 701634 Mar 6 14:51
> /var/log/freshclam.log
>
> It ran from 11:37 to 14:51. It might run faster on x86 computers
> though.
>
> Pierre
>
> On 6 Mar 2019 at 14:20, Micah Snyder (micasnyd) via clamav-users wrote:
>
> Pierre,
>
> So you're saying it actually did finish after 3 hours, 15 minutes on
> its own? That is good news
> for all of the automated systems, even if this is a potentially
> terrible bug.
>
> I'm still investigating the cause, and asking our signature management
> team if they have any
> additional details.
>
> Micah
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>
>
>
> On 3/6/19, 9:06 AM, "clamav-users on behalf of Pierre Dehaen"
> <clamav-users-
> bounces at lists.clamav.net on behalf of dehaenp at drever.be> wrote:
>
> Here too: it took about 3 hours and 15 minutes to calm down
> (SPARC, Solaris 11,
> v0.100.0)... without noticiable error in freshclam.log.
>
> On 6 Mar 2019 at 6:27, J.R. via clamav-users wrote:
>
> > When crontab execs freshclam
> > CPU server goes to 100%
> > Hanged finishing Downloading daily-25380.cdiff [100%]
>
> Just checked my server and it happened to me too! A little after
> 5am
> central time. :(
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20190306/d414e0aa/attachment.htm>
More information about the clamav-users
mailing list