Hi all,

We've been experiencing this slowdown too.  We run every DB update through an extra FP test against a number of recent Mac OS installs (OS X 10.6 - 10.14, as well as some well-known 3rd party apps) just to weed out any potentially overzealous signatures. On the new Mac Minis this FP test used to take around 45 minutes to complete, it now takes almost 3 hours (176 minutes).

From our logs, looking only at the smallest disk we check (Mac OS X 10.6.8) I've managed to compile the following list of dates, scan times and ClamAV DB version numbers. For months, the 10.6 disk used to take around 3m 20s to scan. It always jumped up and down a bit, but really hasn't been right since around the middle of February.

The list is best viewed using a mono-spaced font. I've marked (with 3 asterisks) scans where the time seems to indicate an issue with the DB update.

Hopefully this helps someone to narrow things down a bit.
Mark

dd/mm/yy duration DNS Txt
5/2/19 3m 14s TXT from DNS: 0.101.1:58:25351:1549376940:1:63:48440:328
6/2/19 3m 20s TXT from DNS: 0.101.1:58:25352:1549466941:1:63:48444:328
11/2/19 3m 20s TXT from DNS: 0.101.1:58:25356:1549837740:1:63:48460:328
11/2/19 3m 25s TXT from DNS: 0.101.1:58:25356:1549877342:1:63:48462:328
11/2/19 3m 19s TXT from DNS: 0.101.1:58:25357:1549881900:1:63:48462:328
12/2/19 3m 22s TXT from DNS: 0.101.1:58:25357:1549963741:1:63:48466:328
13/2/19 3m 22s TXT from DNS: 0.101.1:58:25358:1550050141:1:63:48470:328
14/2/19 3m 22s TXT from DNS: 0.101.1:58:25359:1550140140:1:63:48472:328
16/2/19 6m 38s TXT from DNS: 0.101.1:58:25361:1550269740:1:63:48472:328 ***
17/2/19 7m 35s TXT from DNS: 0.101.1:58:25362:1550348940:1:63:48472:328
18/2/19 7m 41s TXT from DNS: 0.101.1:58:25363:1550442540:1:63:48472:328
18/2/19 4m 22s TXT from DNS: 0.101.1:58:25364:1550492940:1:63:48472:328
19/2/19 4m 28s TXT from DNS: 0.101.1:58:25365:1550579340:1:63:48472:328
20/2/19 4m 30s TXT from DNS: 0.101.1:58:25365:1550658540:1:63:48472:328
21/2/19 4m 28s TXT from DNS: 0.101.1:58:25366:1550744940:1:63:48472:328
22/2/19 4m 36s TXT from DNS: 0.101.1:58:25368:1550842141:1:63:48472:328
24/2/19 7m 51s TXT from DNS: 0.101.1:58:25370:1551040140:1:63:48472:328 ***
25/2/19 4m 31s TXT from DNS: 0.101.1:58:25371:1551092103:1:63:48472:328
26/2/19 4m 41s TXT from DNS: 0.101.1:58:25372:1551177619:1:63:48472:328
27/2/19 4m 29s TXT from DNS: 0.101.1:58:25373:1551277740:1:63:48472:328
28/2/19 4m 28s TXT from DNS: 0.101.1:58:25373:1551349740:1:63:48472:328
1/3/19 4m 39s TXT from DNS: 0.101.1:58:25374:1551443340:1:63:48472:328
3/3/19 8m 14s TXT from DNS: 0.101.1:58:25376:1551558540:1:63:48472:328 ***
3/3/19 8m 45s TXT from DNS: 0.101.1:58:25377:1551644940:1:63:48472:328
4/3/19 4m 51s TXT from DNS: 0.101.1:58:25377:1551691742:1:63:48472:328
4/3/19 4m 52s TXT from DNS: 0.101.1:58:25378:1551709740:1:63:48472:328
5/3/19 5m 6s TXT from DNS: 0.101.1:58:25379:1551796140:1:63:48472:328
6/3/19 5m 7s TXT from DNS: 0.101.1:58:25380:1551868140:1:63:48473:328
7/3/19 5m 15s TXT from DNS: 0.101.1:58:25381:1551953509:1:63:48474:328
8/3/19 5m 14s TXT from DNS: 0.101.1:58:25382:1552048140:1:63:48478:328
9/3/19 5m 7s TXT from DNS: 0.101.1:58:25383:1552163340:1:63:48482:328
11/3/19 5m 14s TXT from DNS: 0.101.1:58:25384:1552253340:1:63:48485:328
11/3/19 5m 24s TXT from DNS: 0.101.1:58:25385:1552302125:1:63:48487:328
12/3/19 5m 42s TXT from DNS: 0.101.1:58:25386:1552388890:1:63:48490:328
13/3/19 5m 44s TXT from DNS: 0.101.1:58:25386:1552465741:1:63:48492:328
14/3/19 7m 24s TXT from DNS: 0.101.1:58:25388:1552559341:1:63:48495:328 ***
15/3/19 8m 56s TXT from DNS: 0.101.1:58:25389:1552645741:1:63:48498:328 ***
18/3/19 10m 49s TXT from DNS: 0.101.1:58:25392:1552904941:1:63:48507:328 ***
19/3/19 10m 19s TXT from DNS: 0.101.1:58:25393:1552991341:1:63:48510:328
20/3/19 10m 43s TXT from DNS: 0.101.1:58:25394:1553074140:1:63:48513:328
22/3/19 10m 58s TXT from DNS: 0.101.1:58:25395:1553180408:1:63:48517:328
22/3/19 10m 58s TXT from DNS: 0.101.1:58:25396:1553246940:1:63:48519:328




On Sat, 23 Mar 2019 at 23:26, Al Varnell via clamav-users <clamav-users@lists.clamav.net> wrote:
Sorry, I misinterpreted the meaning of "crawled" thinking it referred to some sort of compromise of the data.

-Al-

On Mar 23, 2019, at 09:42, Jean-Michel via clamav-users <clamav-users@lists.clamav.net> wrote:

See Maarten Broekman tests above 
 
De : Al Varnell <alvarnell@mac.com> 
Envoyé : samedi 23 mars 2019 10:55
À : ClamAV users ML <clamav-users@lists.clamav.net>
Objet : Re: [clamav-users] Scan very slow
 
Reference? First I'm hearing of any such thing.
 
-Al-


On Mar 23, 2019, at 02:26, Jean-Michel via clamav-users <clamav-users@lists.clamav.net> wrote:
 
Hi,
Micah Snyder, Do you know if Clamav was able to trace the orgine of getting crawled in the database "daily.cld" and was able to fix the problem?
Regards
 
De : Micah Snyder (micasnyd) <micasnyd@cisco.com> 
Envoyé : lundi 18 mars 2019 18:09
À : ClamAV users ML <clamav-users@lists.clamav.net>
Objet : Re: [clamav-users] Scan very slow
 
Maarten,
 
This is very concerning, and the details you’ve provide are quite helpful.  Thank you for investigating.
Hopefully we can figure out why the newer daily.cld/cvd is scanning significantly slower than before. Any other details you can provide would probably be helpful.  If you’re aware if any specific file types are causing the issue, or if all files appear to scanning slower that will also help.
 
-Micah
 

Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.
 
 
 
From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Maarten Broekman via clamav-users <clamav-users@lists.clamav.net>
Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
Date: Monday, March 18, 2019 at 10:37 AM
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: Maarten Broekman <maarten.broekman@gmail.com>
Subject: Re: [clamav-users] Scan very slow
 
We've noticed a marked increase in scan times over the last couple of weeks as well. From the look of it, there's something in the daily file that's causing it. Whether this is similar to the safebrowsing issue (where the ordering of entries in the file caused a 3000% increase in time) is unclear.
 
--Maarten Broekman
 
Full scans without the daily cvd/cld: Scan time ~60seconds
Full scans with the daily from March 11th: Scan time: 84seconds
Full scans with the daily from March 17th: Scan time: 109seconds
 
~/clamav# ls -larth  /tmp/clamdtest*/daily.cld
-rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld
-rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld
 
~/clamav# wc /tmp/clamdtest*/daily.cld
  1514589   1517471 115031552 /tmp/clamdtest2/daily.cld
  1524782   1527664 118202368 /tmp/clamdtest/daily.cld
 
Single file scans with JUST the daily.cld:
~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld test42.js
test42.js: OK
 
----------- SCAN SUMMARY -----------
Known viruses: 1504423
Engine version: 0.100.2
Scanned directories: 1
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 5.255 sec (0 m 5 s)
 
real    0m5.260s
user    0m5.044s
sys    0m0.192s
~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld test42.js
test42.js: OK
 
----------- SCAN SUMMARY -----------
Known viruses: 1514543
Engine version: 0.100.2
Scanned directories: 1
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 9.300 sec (0 m 9 s)
 
real    0m9.329s
user    0m9.100s
sys    0m0.204s
 
 
 
 
 
 
On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org> wrote:
From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net>
Subject: Re: [clamav-users] Scan very slow
Date: Mon, 18 Mar 2019 12:30:49 +0100

> Isn't it the second scan result ? The second analyse on same file is faster.
> Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
> I've tried it on 3 computers, all are above 40seconds

It was first trial. But after restarting clamav-daemon result changed
as following.

yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
/home/yasu/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 60.551 sec (1 m 0 s)
yasu@kusanagi[1717]%

---
Yasuhiro KIMURA

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
 

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml