[clamav-users] Spam 3.041 clamd using 100% CPU in Fedora 30 with sendmail & clamav-milter, : Probe for slot 1 returned: failed

Wed Jul 31 14:28:46 UTC 2019

Sorry forgot to include the hive in my responses. So increasing the timeout value to 900 did work. I didn’t time it but it definitely seems like 4-5 minutes to finally start. We rebooted and it started fine.

Should a big report be created? Would this be in Fedora’s Bugzilla, or Clamav’s bug tracker? Are there any other optimization settings?

> On Jul 31, 2019, at 2:47 AM, Reio Remma <reio at mrstuudio.ee> wrote:
> 
> Just curious, did you note how long it actually took to fully load clamd afterwards?
> 
> It might be worth taking this to CentOS devs, because the signatures database keeps growing and clamd loading time with it.
> 
> But it's really an issue with older machines like the one I have here. :D
> 
> Good luck!
> Reio
> 
> 
> On 30/07/2019 23:30, Robert Kudyba wrote:
>> I did but then I also increased from 600 to 900 and that started the daemon. Any idea why this wouldn't be considered a bug?
>> 
>> Thanks for the response.
>> 
>> On Tue, Jul 30, 2019 at 3:48 PM Reio Remma <reio at mrstuudio.ee <mailto:reio at mrstuudio.ee>> wrote:
>> Did you do "systemctl daemon-reload" before restarting the service again?
>> 
>> On 30.07.2019 22:23, Robert Kudyba wrote:
>>> No luck:
>>> 
>>>  systemd[1]: Starting Generic clamav scanner daemon...
>>>  journalctl -xe
>>> -- Defined-By: systemd
>>> -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_systemd-2Ddevel&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=HeyWNpOta-zU4iUgsT6qIc2Inh2JoVpyP8g7tjAJKuc&e=>
>>> --
>>> -- An ExecStart= process belonging to unit clamd at scan.service <mailto:clamd at scan.service> has exited.
>>> --
>>> -- The process' exit code is 'killed' and its exit status is 15.
>>> Jul 30 15:20:21 storm.cis.fordham.edu <http://storm.cis.fordham.edu/> systemd[1]: clamd at scan.service <mailto:clamd at scan.service>: Failed with result 'timeout'.
>>> -- Subject: Unit failed
>>> -- Defined-By: systemd
>>> -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_systemd-2Ddevel&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=HeyWNpOta-zU4iUgsT6qIc2Inh2JoVpyP8g7tjAJKuc&e=>
>>> --
>>> -- The unit clamd at scan.service <mailto:clamd at scan.service> has entered the 'failed' state with result 'timeout'.
>>> Jul 30 15:20:21 storm.cis.fordham.edu <http://storm.cis.fordham.edu/> systemd[1]: Failed to start Generic clamav scanner daemon.
>>> -- Subject: A start job for unit clamd at scan.service <mailto:clamd at scan.service> has failed
>>> -- Defined-By: systemd
>>> -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_systemd-2Ddevel&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=HeyWNpOta-zU4iUgsT6qIc2Inh2JoVpyP8g7tjAJKuc&e=>
>>> --
>>> -- A start job for unit clamd at scan.service <mailto:clamd at scan.service> has finished with a failure.
>>> --
>>> -- The job identifier is 331899 and the job result is failed.
>>> 
>>> It's as if clamd continues to try to start as running 'top' shows 100% CPU:
>>>   PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
>>>  4949 root      20   0  774044 727648   7736 R  93.8   1.5   1:16.88 clamd
>>> 
>>> status shows it's still trying to start:
>>>  systemctl status clamd at scan.service <mailto:clamd at scan.service>
>>> * clamd at scan.service <mailto:clamd at scan.service> - Generic clamav scanner daemon
>>>    Loaded: loaded (/usr/lib/systemd/system/clamd at scan.service; enabled; vendor preset: disabled)
>>>    Active: activating (start) since Tue 2019-07-30 15:21:52 EDT; 26s ago
>>>      Docs: man:clamd(8)
>>>            man:clamd.conf(5)
>>>            https://www.clamav.net/documents/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.clamav.net_documents_&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=2whNina19JXLnZkRg0ik9mRMN2az0Rq8MSra-kr4rGo&e=>
>>> Cntrl PID: 5175 (clamd)
>>>     Tasks: 1 (limit: 4915)
>>>    Memory: 244.0M
>>>    CGroup: /system.slice/system-clamd.slice/clamd at scan.service
>>>            `-5175 /usr/sbin/clamd -c /etc/clamd.d/scan.conf
>>> 
>>> Jul 30 15:21:52 ourdomain systemd[1]: Starting Generic clamav scanner daemon...
>>> 
>>> And just to be sure:
>>> cat  /lib/systemd/system/clamd at .service
>>> [Unit]
>>> Description = clamd scanner (%i) daemon
>>> Documentation=man:clamd(8) man:clamd.conf(5) https://www.clamav.net/documents/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.clamav.net_documents_&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=2whNina19JXLnZkRg0ik9mRMN2az0Rq8MSra-kr4rGo&e=>
>>> # Check for database existence
>>> # ConditionPathExistsGlob=@DBDIR@/main.{c[vl]d,inc}
>>> # ConditionPathExistsGlob=@DBDIR@/daily.{c[vl]d,inc}
>>> After = syslog.target nss-lookup.target network.target
>>> 
>>> [Service]
>>> Type = forking
>>> ExecStart = /usr/sbin/clamd -c /etc/clamd.d/%i.conf
>>> Restart = on-failure
>>> TimeoutSec=600
>>> 
>>> On Tue, Jul 30, 2019 at 3:12 PM Reio Remma via clamav-users <clamav-users at lists.clamav.net <mailto:clamav-users at lists.clamav.net>> wrote:
>>> I suspect it's might be the same issue I had a few days back.
>>> 
>>> Check out the thread "Clamd fails to start with daily.cvd".
>>> 
>>> As suggested by user Axb: 
>>> 
>>> in file clamd.service 
>>> to section: 
>>> [Service] 
>>> add 
>>> TimeoutSec=900 
>>> 
>>> restart clamd service 
>>> 
>>> I personally increased the limit to 300 seconds. :)
>>> 
>>> I suspect systemd is killing the process because it goes over the timeout threshold when loading the signatures.
>>> 
>>> Good luck!
>>> Reio
>>> 
>>> 
>>> On 30.07.2019 21:58, Robert Kudyba wrote:
>>>> rpm -qa clamav-milter
>>>> clamav-milter-0.101.2-2.fc30.x86_64
>>>> rpm -qa clamd
>>>> clamd-0.101.2-2.fc30.x86_64
>>>> 
>>>> See some logs and statuses below. clamd takes up all of the CPU. clamd does appear to start based on the ps command but you can see the status shows no running;
>>>> 
>>>>   PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
>>>> 26618 root      20   0  214188 207576   7996 R  99.0   0.4   0:10.76 clamd
>>>> 
>>>> Tue Jul 30 14:30:17 2019 -> WARNING: No clamd server appears to be available
>>>> Tue Jul 30 14:31:16 2019 -> Failed to establish a connection to clamd
>>>> Tue Jul 30 14:31:16 2019 -> Probe for slot 1 returned: failed
>>>> Tue Jul 30 14:31:16 2019 -> WARNING: No clamd server appears to be available
>>>> Tue Jul 30 14:32:15 2019 -> Failed to establish a connection to clamd
>>>> Tue Jul 30 14:32:15 2019 -> Probe for slot 1 returned: failed
>>>> Tue Jul 30 14:32:15 2019 -> WARNING: No clamd server appears to be available
>>>> 
>>>>  ps -auwx|grep clam
>>>> clamav    2538  0.0  0.0  18348  3156 ?        Ss   Jul29   0:00 /usr/bin/freshclam -d -c 4
>>>> clamav   24692  0.0  0.0  19852 10044 ?        Ss   14:10   0:00 /usr/lib/systemd/systemd --user
>>>> clamav   24697  0.0  0.0 181296  5200 ?        S    14:10   0:00 (sd-pam)
>>>> clamav   24717  0.0  0.0 113064  3312 ?        Ss   14:10   0:00 /bin/sh -c [ -x /usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash /usr/local/sbin/clamav-unofficial-sigs.sh > /dev/null
>>>> clamav   24718  0.0  0.0 113848  3908 ?        S    14:10   0:00 /usr/bin/bash /usr/local/sbin/clamav-unofficial-sigs.sh
>>>> clamilt  26222  0.0  0.0  88488   588 ?        Ssl  14:18   0:00 /usr/sbin/clamav-milter -c /etc/mail/clamav-milter.conf
>>>> root     26227 99.6  0.5 263348 251924 ?       Rs   14:18   0:20 /usr/sbin/clamd -c /etc/clamd.d/scan.conf
>>>> clamav   26360  1.8  0.0 126316 12992 ?        S    14:18   0:00 /usr/bin/wget --no-check-certificate --quiet --connect-timeout=60 --random-wait --tries=3 --timeout=180 --output-document=/var/lib/clamav-unofficial-sigs/dbs-si/securiteinfo.hdbhttps://www.securiteinfo.com/get/signatures/6651194e2baf9979742029c715d7dd90c94e25355ca57fdf22c81828f6fe7a3fc01bfbee6c9a20efa17559c52a04cc4aab1cbe6810596bb16afae8518a9400d1/securiteinfo.hdb\ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.securiteinfo.com_get_signatures_6651194e2baf9979742029c715d7dd90c94e25355ca57fdf22c81828f6fe7a3fc01bfbee6c9a20efa17559c52a04cc4aab1cbe6810596bb16afae8518a9400d1_securiteinfo.hdb-255C&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=1r9FHRsyaDucqFLyd9_yj6gSAcLUUdcJpWaCvwUs0Lk&e=>
>>>> 
>>>> systemctl  status clamd at scan.service <mailto:clamd at scan.service>
>>>> * clamd at scan.service <mailto:clamd at scan.service> - Generic clamav scanner daemon
>>>>    Loaded: loaded (/usr/lib/systemd/system/clamd at scan.service; enabled; vendor preset: disabled)
>>>>    Active: inactive (dead) since Mon 2019-07-29 13:24:11 EDT; 24h ago
>>>>      Docs: man:clamd(8)
>>>>            man:clamd.conf(5)
>>>>            https://www.clamav.net/documents/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.clamav.net_documents_&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=hP9lw6TqqvMhxIcjF0PBc9GJj8nl0wNqMlhW1kf4Y7c&e=>
>>>> 
>>>> Jul 29 13:24:09 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd at scan.service:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead.
>>>> Jul 29 13:24:11 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: clamd at scan.service <mailto:clamd at scan.service>: Control process exited, code=killed, status=15/TERM
>>>> Jul 29 13:24:11 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: clamd at scan.service <mailto:clamd at scan.service>: Succeeded.
>>>> Jul 29 13:24:11 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: Stopped Generic clamav scanner daemon.
>>>> Jul 30 04:53:06 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd at scan.service:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead.
>>>> Jul 30 11:13:50 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd at scan.service:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead.
>>>> Jul 30 11:19:10 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd at scan.service:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead.
>>>> Jul 30 14:05:05 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd at scan.service:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead.
>>>> Jul 30 14:05:07 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd at scan.service:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead.
>>>> Jul 30 14:05:08 ourdomain.edu <https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd at scan.service:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead.
>>>> 
>>>> systemctl status clamav-milter
>>>> * clamav-milter.service - Milter module for the Clam Antivirus scanner
>>>>    Loaded: loaded (/usr/lib/systemd/system/clamav-milter.service; enabled; vendor preset: disabled)
>>>>    Active: active (running) since Mon 2019-07-29 13:23:46 EDT; 24h ago
>>>>  Main PID: 4350 (clamav-milter)
>>>>     Tasks: 3 (limit: 4915)
>>>>    Memory: 2.6M
>>>>    CGroup: /system.slice/clamav-milter.service
>>>>            `-4350 /usr/sbin/clamav-milter -c /etc/mail/clamav-milter.conf
>>>> 
>>>> Jul 29 13:23:45 ourserver systemd[1]: Starting Milter module for the Clam Antivirus scanner...
>>>> Jul 29 13:23:46  ourserver  systemd[1]: Started Milter module for the Clam Antivirus scanner.
>>>> 
>>>> Tue Jul 30 14:20:11 2019 -> +++ Started at Tue Jul 30 14:20:11 2019
>>>> Tue Jul 30 14:20:11 2019 -> Received 0 file descriptor(s) from systemd.
>>>> Tue Jul 30 14:20:11 2019 -> clamd daemon 0.101.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)
>>>> Tue Jul 30 14:20:11 2019 -> Log file size limited to 1048576 bytes.
>>>> Tue Jul 30 14:20:11 2019 -> Reading databases from /var/lib/clamav
>>>> Tue Jul 30 14:20:11 2019 -> Not loading PUA signatures.
>>>> Tue Jul 30 14:20:11 2019 -> Bytecode: Security mode set to "TrustSigned".
>>>> 
>>>> The uncommented directives in /etc/clamd.d/scan.conf are:
>>>> LogFile /var/log/clamd.scan
>>>> LogTime yes
>>>> LogSyslog yes
>>>> DatabaseDirectory /var/lib/clamav
>>>> TCPSocket 3310
>>>> TCPAddr 127.0.0.1
>>>> 
>>>> I had to disable it in sendmail where I had this in sendmail.mc <https://urldefense.proofpoint.com/v2/url?u=http-3A__sendmail.mc&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=Nm05g7xhCGkbvqrPNmiCxoaZIINskP8j9ub976PYskg&e=>:
>>>> INPUT_MAIL_FILTER(`clamav-milter', `S=inet:6666 at 127.0.0.1 <mailto:inet%3A6666 at 127.0.0.1>, F=, T=S:4m;R:4m')dnl
>>>> 
>>>> This all starting happening after a reboot. Any ideas what may be wrong?
>>>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20190731/4358f6ed/attachment.htm>

[clamav-users] ***Spam 3.041*** clamd using 100% CPU in Fedora 30 with sendmail & clamav-milter, : Probe for slot 1 returned: failed

[clamav-users] Spam 3.041 clamd using 100% CPU in Fedora 30 with sendmail & clamav-milter, : Probe for slot 1 returned: failed