[clamav-users] High CPU load during startup/reload of sigs for a long time.

[Ralf Hildebrandt]

> 1. Does clamd scan memory during startup and/or restart?[1] The
>    problem seems to occur less with less committed memory in the VM.

I'm not authoritative on this, but I doubt it.
 
> 3. Does ClamAV use more than one CPU core during startup/reload?

Just tried that, I don't see more then 100%, so it's merely using one
core.

>    Because if my problem occurs, htop shows a load of more than 100%
>    for the ClamAV process, sometimes up to 500.

Odd.

Dec 28 08:06:12 proxy-cbf-2 clamd[56735]: SelfCheck: Database modification detected. Forcing reload.
Dec 28 08:06:12 proxy-cbf-2 clamd[56735]: Reading databases from /var/lib/clamav
...
Dec 28 08:06:24 proxy-cbf-2 clamd[56735]: Database correctly reloaded (6534998 signatures)

and:

Dec 28 14:07:12 proxy-cbf-2 clamd[56735]: SelfCheck: Database modification detected. Forcing reload.
Dec 28 14:07:12 proxy-cbf-2 clamd[56735]: Reading databases from /var/lib/clamav
Dec 28 14:07:24 proxy-cbf-2 clamd[56735]: Database correctly reloaded (6535004 signatures)

so it takes about 12s on a Intel(R) Xeon(R) CPU E5-2609 v2 @ 2.50GHz
on a busy proxy (physical hardware).

> 5. What should be most likely the bottleneck during startup/reload,
>    available time on one CPU core or I/O to read sigs? I don't seem to
>    have any reasonable I/O when the high CPU load occurs.

Maybe it's a memory issue? I've had some machines with low memeory
which took a long time to reload sigs.

I used "strace -c -p 2906" and issued a "kill -SIGUSR2 2906" in
another window and got these stats for the reload of the signatures:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 99.67    0.102712         194       529         1 poll
  0.24    0.000248           0      2096           munmap
  0.08    0.000080           0     32141           read
  0.01    0.000010           0      2094           mmap
  0.00    0.000000           0         7           write
  0.00    0.000000           0        37           open
  0.00    0.000000           0        43           close
  0.00    0.000000           0        32           stat
  0.00    0.000000           0        43           fstat
  0.00    0.000000           0       143           lseek
  0.00    0.000000           0         3           mprotect
  0.00    0.000000           0         6           brk
  0.00    0.000000           0         1         1 rt_sigreturn
  0.00    0.000000           0         4         4 ioctl
  0.00    0.000000           0         8         6 access
  0.00    0.000000           0         6           dup
  0.00    0.000000           0       341           recvmsg
  0.00    0.000000           0         1           uname
  0.00    0.000000           0         6           fcntl
  0.00    0.000000           0         6           getdents
  0.00    0.000000           0         2           getcwd
  0.00    0.000000           0       480           futex
  0.00    0.000000           0         1           restart_syscall
------ ----------- ----------- --------- --------- ----------------
100.00    0.103050                 38030        12 total


-- 
Ralf Hildebrandt                   Charite Universitätsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
https://www.charite.de             Hindenburgdamm 30, 12203 Berlin
Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155