[clamav-users] Slow database loading

[Ferdinand Goldmann]

Hello Arnaud!

Thank you for your input!

On Tue, 22 Nov 2016, Arnaud Jacques / SecuriteInfo.com wrote:

>> I made some adjustments like giving the clamd process higher priority
>> and providing my VMs with more CPU cores.
>
> Could you please provide more RAM to your VMs ?

I don't think that's the problem. The VMs all have 16GB RAM, of which
plenty is available:

Mem: 1808M Active, 436M Inact, 2532M Wired, 1860K Cache, 1600M Buf, 11G Free

The clamd process uses roughly 1GB of memory:

PID USERNAME      THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
782 spamd           3 -52   r0   947M   852M uwait   3  14:44 0.00% clamd

I really don't think memory is the issue here. Also there are no memory
limitations imposed on the spamd user:

# limits -m -U spamd
Resource limits for class default:
   memoryuse            infinity kB

>> Does anyone else experience such high db reload times with when using the
>> unofficial signatures?
>
> Yes, maybe javascript.ndb too big for your VM. Can you remove it and retry
> please ?

I just tried that, and the difference it made is tremendous! With
javascript.ndb removed, reload time is down to under 20 seconds:

Nov 23 08:26:53 mail2 clamd[782]: Reading databases from /var/db/clamav
Nov 23 08:27:09 mail2 clamd[782]: Database correctly reloaded (7942253 signatures)

After I put it back, reloading took over one minute again:

Nov 23 08:28:41 mail2 clamd[782]: Reading databases from /var/db/clamav
Nov 23 08:29:49 mail2 clamd[782]: Database correctly reloaded (8008706 signatures)

While reloading with the javascript.ndb in place the CPU usage of the clamd process really goes up:

PID USERNAME      THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
782 spamd          13 -52   r0   968M   858M uwait   0  15:47 1099.71% clamd

If you insist :) I can try more RAM for the VM, but as I said, I don't
think that too little RAM is the problem here. Rather the way clamd
handles signatures internally.

Kind regards,
Ferdinand
-- 
Ferdinand Goldmann
System Administrator
Information Management

JOHANNES KEPLER
UNIVERSITY LINZ
Altenberger Straße 69
Hochschulfond Building, HF9902
4040 Linz, Austria
P +43 732 2468 3925
ferdinand.goldmann at jku.at
www.jku.at/im
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3900 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20161123/4cf99fe1/attachment.bin>