[clamav-users] Occasional sendmail queue delay when using clamav-milter

Aaron Paetznick aaronp at critd.com
Tue May 1 15:41:03 EDT 2018


This and Ted's comment about falling back to a permissive OnFail were 
both very helpful, thanks! I'm sorry for the late reply, but problems 
like this require small course corrections and long periods of observation.

Here are my log levels:

define(`confMILTER_LOG_LEVEL', `8')
define(`confLOG_LEVEL', `10')


For the record, I'm running a distinct copy of clamd using a local unix 
domain socket on each server. I'd be happy to consider switching to TCP, 
but because I'm running a distinct clamd per server I think it's 
unlikely the overhead would be worth it. I could be wrong though.

The VM I'm using for testing has 12GB of RAM allocated right now, and 
averages about 70% memory utilization and under 1% CPU utilization. The 
pool is setup in a DNS round-robin, so the rest should be similar.

Here's some threaded top output for my clamd process:

[root at smtp ~]# top -b -n 1 -H -p 1324
top - 13:50:11 up 6 days,  1:25,  1 user,  load average: 0.14, 0.15, 0.14
Threads:   4 total,   0 running,   4 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  
0.0 st
KiB Mem : 12121752 total,  3800616 free,   966264 used, 7354872 buff/cache
KiB Swap:  1679356 total,  1649968 free,    29388 used. 10741208 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
   1324 clamav    20   0  975436 709588   3264 S  0.0  5.9 3:17.37 clamd
   1678 clamav    20   0  975436 709588   3264 S  0.0  5.9 0:08.38 clamd
  57290 clamav    20   0  975436 709588   3264 S  0.0  5.9 1:17.77 clamd
  57438 clamav    20   0  975436 709588   3264 S  0.0  5.9 0:56.31 clamd
[root at smtp ~]#


Reloading the clamd databases typically takes under 10 seconds.

One more thing. I am running two total milters, the other being 
opendkim. Your comment got me thinking that what I'm seeing may not be 
clamd itself. Maybe clamd is aggravating a problem with opendkim, or 
maybe something is going wrong with the interaction between the two? 
Here's how I'm running them side-by-side:

INPUT_MAIL_FILTER(`clamav', 
`S=local:/var/run/clamav-milter/clamav-milter.sock, F=T, T=S:4m;R:4m')
INPUT_MAIL_FILTER(`opendkim', `S=local:/var/run/opendkim/opendkim.sock')


All things considered, I may just decide to temporarily set the OnFail 
option to Accept in my clamav-milter.conf file. It's certainly not an 
ideal solution though.


On 5/1/2018 11:34 AM, G.W. Haywood wrote:
> Hi there,
>
> On Tue, 1 May 2018, Aaron Paetznick wrote:
>
>> Occasionally a small percentage of email will seemingly unnecessarily
>> get held in the queue when using clamav-milter, although it will get
>> delivered successfully on the first attempt with the next queue run. The
>> size, time, sender, and recipient all seem to be irrelevant. Our
>> work-around is to simply process the queue every 5 minutes, but this is
>> not sustainable. We've conclusively narrowed it down to ClamAV, as the
>> problem vanishes when we comment out the INPUT_MAIL_FILTER line in our
>> sendmail.cf file. Here's that milter line:
>>
>> INPUT_MAIL_FILTER(`clamav',
>> `S=local:/var/run/clamav-milter/clamav-milter.sock, F=T, T=S:4m;R:4m')
>
> The intermittent ones are the trickiest. :(
>
> I haven't seen anything like this issue, although our queues won't be
> nearly as busy as yours.
>
> Typically database reloads here (modest hardware) take a couple of
> minutes, and your T=S and T=R timeouts for clamav-milter are, like
> ours, much longer than that.  But if you have other milters for which
> the timeouts are not so long, perhaps it could be an issue.  Even so,
> one might then expect that you'd have noticed a correlation with the
> reloads, and apparently you haven't.  A puzzle. :)
>
> What log levels are you using for Sendmail and the milters?
>
> Are you using a single clamd instance for all the servers, or one per
> server, or ...?
>
> Do you know how long your database reloads take?  Are they reliably
> taking that long or do they sometimes stall?
>
> What connections type(s) are you using for clamd?
>
> Have you checked that clamd always responds quickly/reliably to PINGs
> on the socket?
>
> Can we take it that you do have enough memory?
>
> -- 
>
> 73,
> Ged.
> _______________________________________________
> clamav-users mailing list
> clamav-users at lists.clamav.net
> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>




More information about the clamav-users mailing list