[clamav-users] How to boost clamav? Reloading database results in a talking timeout?

[Micah Snyder (micasnyd)]

The database load process reads signatures and uses the data to populate a couple of pseudo-tries (https://en.wikipedia.org/wiki/Trie).  The tries themselves could only be modified by a single thread at a time, with a mutex around each trie.  There might be some performance to be gained by using multiple threads.  I'm not certain.  Definitely a bunch of thread safety code would need to be written. 

-Micah

On 9/4/19, 1:40 PM, "clamav-users on behalf of Thomas Barth via clamav-users" <clamav-users-bounces at lists.clamav.net on behalf of clamav-users at lists.clamav.net> wrote:

    Am 2019-09-01 19:30, schrieb Joel Esler (jesler) via clamav-users:
    > Alright. I think we’ve beat the proverbial dead horse here.  The devs
    > know this is a request and they will get it into their dev queue for
    > examination.
    
    I saw that clamd use just one core at a time to load the databases.
    
    top - 16:09:43 up 23:33,  2 users,  load average: 0.47, 0.13, 0.04
    Tasks: 176 total,   2 running, 174 sleeping,   0 stopped,   0 zombie
    %Cpu0  :  0.0 us,  0.0 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.0 hi,  0.0 si,  
    1.0 st
    %Cpu1  :  1.0 us,  1.0 sy,  0.0 ni, 98.0 id,  0.0 wa,  0.0 hi,  0.0 si,  
    0.0 st
    %Cpu2  : 85.4 us,  6.8 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  
    7.8 st
    %Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  
    0.0 st
    %Cpu4  :  0.0 us,  1.0 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.0 hi,  0.0 si,  
    0.0 st
    %Cpu5  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  
    0.0 st
    MiB Mem :  16042.2 total,  14207.5 free,    813.4 used,   1021.3 
    buff/cache
    MiB Swap:      0.0 total,      0.0 free,      0.0 used.  14921.4 avail 
    Mem
    
       PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ 
    COMMAND
       455 clamav    20   0  469352 207432  10840 R 100.0   1.3  30:33.51 
    /usr/sbin/clamd --foreground=true
    22861 root      20   0   11316   3648   3108 R   2.0   0.0   0:24.91 top
    
    Always 2 minute loading time
    Wed Sep  4 16:09:17 2019 -> Reading databases from /var/lib/clamav
    Wed Sep  4 16:11:24 2019 -> Database correctly reloaded (10966440 
    signatures)
    
    
    Why not using half of the cores to also reduce the loading time? Many 
    years ago when I used eMule for downloading big files, I was so 
    fascinated by the download mechanism: one big file, many download 
    sources to get the file together piece by piece. And it didn't have to 
    follow any order. That would be fun to programm for loading the 
    databases, am I right? :-)
    
    
    
    _______________________________________________
    
    clamav-users mailing list
    clamav-users at lists.clamav.net
    https://lists.clamav.net/mailman/listinfo/clamav-users
    
    
    Help us build a comprehensive ClamAV guide:
    https://github.com/vrtadmin/clamav-faq
    
    http://www.clamav.net/contact.html#ml