[clamav-users] Scan very slow

Mark Allan markjallan at gmail.com
Mon Mar 25 06:52:46 EDT 2019


Hi all,

We've been experiencing this slowdown too.  We run every DB update through
an extra FP test against a number of recent Mac OS installs (OS X 10.6 -
10.14, as well as some well-known 3rd party apps) just to weed out any
potentially overzealous signatures. On the new Mac Minis this FP test used
to take around 45 minutes to complete, it now takes almost 3 hours (176
minutes).

>From our logs, looking only at the smallest disk we check (Mac OS X 10.6.8)
I've managed to compile the following list of dates, scan times and ClamAV
DB version numbers. For months, the 10.6 disk used to take around 3m 20s to
scan. It always jumped up and down a bit, but really hasn't been right
since around the middle of February.

The list is best viewed using a mono-spaced font. I've marked (with 3
asterisks) scans where the time seems to indicate an issue with the DB
update.

Hopefully this helps someone to narrow things down a bit.
Mark

dd/mm/yy duration DNS Txt
5/2/19 3m 14s TXT from DNS: 0.101.1:58:25351:1549376940:1:63:48440:328
6/2/19 3m 20s TXT from DNS: 0.101.1:58:25352:1549466941:1:63:48444:328
11/2/19 3m 20s TXT from DNS: 0.101.1:58:25356:1549837740:1:63:48460:328
11/2/19 3m 25s TXT from DNS: 0.101.1:58:25356:1549877342:1:63:48462:328
11/2/19 3m 19s TXT from DNS: 0.101.1:58:25357:1549881900:1:63:48462:328
12/2/19 3m 22s TXT from DNS: 0.101.1:58:25357:1549963741:1:63:48466:328
13/2/19 3m 22s TXT from DNS: 0.101.1:58:25358:1550050141:1:63:48470:328
14/2/19 3m 22s TXT from DNS: 0.101.1:58:25359:1550140140:1:63:48472:328
16/2/19 6m 38s TXT from DNS: 0.101.1:58:25361:1550269740:1:63:48472:328 ***
17/2/19 7m 35s TXT from DNS: 0.101.1:58:25362:1550348940:1:63:48472:328
18/2/19 7m 41s TXT from DNS: 0.101.1:58:25363:1550442540:1:63:48472:328
18/2/19 4m 22s TXT from DNS: 0.101.1:58:25364:1550492940:1:63:48472:328
19/2/19 4m 28s TXT from DNS: 0.101.1:58:25365:1550579340:1:63:48472:328
20/2/19 4m 30s TXT from DNS: 0.101.1:58:25365:1550658540:1:63:48472:328
21/2/19 4m 28s TXT from DNS: 0.101.1:58:25366:1550744940:1:63:48472:328
22/2/19 4m 36s TXT from DNS: 0.101.1:58:25368:1550842141:1:63:48472:328
24/2/19 7m 51s TXT from DNS: 0.101.1:58:25370:1551040140:1:63:48472:328 ***
25/2/19 4m 31s TXT from DNS: 0.101.1:58:25371:1551092103:1:63:48472:328
26/2/19 4m 41s TXT from DNS: 0.101.1:58:25372:1551177619:1:63:48472:328
27/2/19 4m 29s TXT from DNS: 0.101.1:58:25373:1551277740:1:63:48472:328
28/2/19 4m 28s TXT from DNS: 0.101.1:58:25373:1551349740:1:63:48472:328
1/3/19 4m 39s TXT from DNS: 0.101.1:58:25374:1551443340:1:63:48472:328
3/3/19 8m 14s TXT from DNS: 0.101.1:58:25376:1551558540:1:63:48472:328 ***
3/3/19 8m 45s TXT from DNS: 0.101.1:58:25377:1551644940:1:63:48472:328
4/3/19 4m 51s TXT from DNS: 0.101.1:58:25377:1551691742:1:63:48472:328
4/3/19 4m 52s TXT from DNS: 0.101.1:58:25378:1551709740:1:63:48472:328
5/3/19 5m 6s TXT from DNS: 0.101.1:58:25379:1551796140:1:63:48472:328
6/3/19 5m 7s TXT from DNS: 0.101.1:58:25380:1551868140:1:63:48473:328
7/3/19 5m 15s TXT from DNS: 0.101.1:58:25381:1551953509:1:63:48474:328
8/3/19 5m 14s TXT from DNS: 0.101.1:58:25382:1552048140:1:63:48478:328
9/3/19 5m 7s TXT from DNS: 0.101.1:58:25383:1552163340:1:63:48482:328
11/3/19 5m 14s TXT from DNS: 0.101.1:58:25384:1552253340:1:63:48485:328
11/3/19 5m 24s TXT from DNS: 0.101.1:58:25385:1552302125:1:63:48487:328
12/3/19 5m 42s TXT from DNS: 0.101.1:58:25386:1552388890:1:63:48490:328
13/3/19 5m 44s TXT from DNS: 0.101.1:58:25386:1552465741:1:63:48492:328
14/3/19 7m 24s TXT from DNS: 0.101.1:58:25388:1552559341:1:63:48495:328 ***
15/3/19 8m 56s TXT from DNS: 0.101.1:58:25389:1552645741:1:63:48498:328 ***
18/3/19 10m 49s TXT from DNS: 0.101.1:58:25392:1552904941:1:63:48507:328 ***
19/3/19 10m 19s TXT from DNS: 0.101.1:58:25393:1552991341:1:63:48510:328
20/3/19 10m 43s TXT from DNS: 0.101.1:58:25394:1553074140:1:63:48513:328
22/3/19 10m 58s TXT from DNS: 0.101.1:58:25395:1553180408:1:63:48517:328
22/3/19 10m 58s TXT from DNS: 0.101.1:58:25396:1553246940:1:63:48519:328




On Sat, 23 Mar 2019 at 23:26, Al Varnell via clamav-users <
clamav-users at lists.clamav.net> wrote:

> Sorry, I misinterpreted the meaning of "crawled" thinking it referred to
> some sort of compromise of the data.
>
> -Al-
>
> On Mar 23, 2019, at 09:42, Jean-Michel via clamav-users <
> clamav-users at lists.clamav.net> wrote:
>
> See Maarten Broekman tests above
> https://lists.clamav.net/pipermail/clamav-users/2019-March/007737.html
>
> *De :* Al Varnell <alvarnell at mac.com>
> *Envoyé :* samedi 23 mars 2019 10:55
> *À :* ClamAV users ML <clamav-users at lists.clamav.net>
> *Objet :* Re: [clamav-users] Scan very slow
>
> Reference? First I'm hearing of any such thing.
>
> -Al-
>
>
> On Mar 23, 2019, at 02:26, Jean-Michel via clamav-users <
> clamav-users at lists.clamav.net> wrote:
>
> Hi,
> Micah Snyder, Do you know if Clamav was able to trace the orgine of
> getting crawled in the database "daily.cld" and was able to fix the problem?
> Regards
>
> *De :* Micah Snyder (micasnyd) <micasnyd at cisco.com>
> *Envoyé :* lundi 18 mars 2019 18:09
> *À :* ClamAV users ML <clamav-users at lists.clamav.net>
> *Objet :* Re: [clamav-users] Scan very slow
>
> Maarten,
>
> This is very concerning, and the details you’ve provide are quite
> helpful.  Thank you for investigating.
> Hopefully we can figure out why the newer daily.cld/cvd is scanning
> significantly slower than before. Any other details you can provide would
> probably be helpful.  If you’re aware if any specific file types are
> causing the issue, or if all files appear to scanning slower that will also
> help.
>
> -Micah
>
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>
>
>
> *From: *clamav-users <clamav-users-bounces at lists.clamav.net> on behalf of
> Maarten Broekman via clamav-users <clamav-users at lists.clamav.net>
> *Reply-To: *ClamAV users ML <clamav-users at lists.clamav.net>
> *Date: *Monday, March 18, 2019 at 10:37 AM
> *To: *ClamAV users ML <clamav-users at lists.clamav.net>
> *Cc: *Maarten Broekman <maarten.broekman at gmail.com>
> *Subject: *Re: [clamav-users] Scan very slow
>
> We've noticed a marked increase in scan times over the last couple of
> weeks as well. From the look of it, there's something in the daily file
> that's causing it. Whether this is similar to the safebrowsing issue (where
> the ordering of entries in the file caused a 3000% increase in time) is
> unclear.
>
> --Maarten Broekman
>
> Full scans without the daily cvd/cld: Scan time ~60seconds
> Full scans with the daily from March 11th: Scan time: 84seconds
> Full scans with the daily from March 17th: Scan time: 109seconds
>
> ~/clamav# ls -larth  /tmp/clamdtest*/daily.cld
> -rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld
> -rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld
>
> ~/clamav# wc /tmp/clamdtest*/daily.cld
>   1514589   1517471 115031552 /tmp/clamdtest2/daily.cld
>   1524782   1527664 118202368 /tmp/clamdtest/daily.cld
>
> Single file scans with JUST the daily.cld:
> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d
> /tmp/clamdtest2/daily.cld test42.js
> test42.js: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 1504423
> Engine version: 0.100.2
> Scanned directories: 1
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.00 MB
> Data read: 0.00 MB (ratio 0.00:1)
> Time: 5.255 sec (0 m 5 s)
>
> real    0m5.260s
> user    0m5.044s
> sys    0m0.192s
> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld
> test42.js
> test42.js: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 1514543
> Engine version: 0.100.2
> Scanned directories: 1
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.00 MB
> Data read: 0.00 MB (ratio 0.00:1)
> Time: 9.300 sec (0 m 9 s)
>
> real    0m9.329s
> user    0m9.100s
> sys    0m0.204s
>
>
>
>
>
>
> On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu at utahime.org> wrote:
>
> From: Jean-Michel via clamav-users <clamav-users at lists.clamav.net>
> Subject: Re: [clamav-users] Scan very slow
> Date: Mon, 18 Mar 2019 12:30:49 +0100
>
> > Isn't it the second scan result ? The second analyse on same file is
> faster.
> > Could tou try to restart clamav-daemon and re-do the analyse with
> clamdscan.
> > I've tried it on 3 computers, all are above 40seconds
>
> It was first trial. But after restarting clamav-daemon result changed
> as following.
>
> yasu at kusanagi[1716]% clamdscan esploso_A3TH.pdf
> /home/yasu/tmp/esploso_A3TH.pdf: OK
>
> ----------- SCAN SUMMARY -----------
> Infected files: 0
> Time: 60.551 sec (1 m 0 s)
> yasu at kusanagi[1717]%
>
> ---
> Yasuhiro KIMURA
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20190325/5e354f8c/attachment.html>


More information about the clamav-users mailing list