[clamav-users] problem reading socket while updating database

[Bowie Bailey]

On 7/7/2015 4:31 PM, Kris Deugau wrote:
> Jingo Administrator wrote:
>> Already more than a week ago I posted my first question to the list. I
>> must admit I'm a bit disappointed that nobody responds. Is it that I
>> asked a silly question? Or is the issue just to hard to solve and just
>> nobody wants to burn his fingers on it?
>>
>>
>> On 06/30/2015 08:38 AM, Jingo Administrator wrote:
>>> Dear clamav-users,
>>>
>>> I am struggling with this problem now for quite some time and can't get
>>> a solution. Reason for asking the user list is that I couldn't get a
>>> clue solving the issue even after thorough searching on the internet and
>>> the clamav-users lists archive.
>>> The situation is as follows. I am running a mail server (exim4) on a
>>> Debian Wheezy 32-bit Linux machine. The freshclam daemon is running the
>>> update every hour. What I have noticed is that clamav is taking quite
>>> some time to actually update the database. Currently this is about 5
>>> minutes. Also the cpu is occupied by the clamd daemon during the update
>>> to almost 100%. I can reduce this to every percentage I want by for
>>> example utilizing cpulimit, as a side effect of this the update would
>>> only take longer.
>>> The problem I have with this is that, when during the update exim4 sends a
>>> message to the daemon to be checked by clamav, I get an error message in
>>> /var/log/exim4/paniclog stating :
>>> 2015-06-19 13:51:10 [21601] 1Z5unF-0005cP-Lg malware acl condition:
>>> clamd: unable to read from socket (Connection timed out)
>>> At first I thought the cause of the problem was in some misconfiguration
>>> of exim4, but then I noticed messages during the same time in the
>>> clamav.log :
>>> Fri Jun 19 13:51:24 2015 -> Client disconnected (FD 12)
>>> This behavior and synchronicity is reproduced. I am running this server
>>> for quite a while now, the reason I only lately noticed this problem is
>>> that the size of the database has grown, due to including some 3rd party
>>> descriptions, in this case securiteinfo. In ram (resident memory) it now
>>> takes about 0.5 Gb, total memory is 2 Gb. I recently added 1 Gb of ram but
>>> that doesn't make any difference. In the past only now and then
>>> I got the same error message in the paniclog of exim4, but I did not pay
>>> much attention. Now that's occurring more frequently I do. Maybe there
>>> are ways to reduce the time it takes for clamav to update, but this
>>> nevertheless does not take away the fact that during the clamav update
>>> the socket isn't accessible by exim. And that's the whole point.
>>> No matter how short this time is, the problem is still there.
>>> As I use this mail server for my own use only, it's not very busy in terms
>>> of handling a lot of e-mails. If it were then the problem would have been
>>> much bigger I guess.
>>> When trying to solve the issue I more than quadruple checked all the
>>> relevant options in clamav.conf, like setting AllowSupplementaryGroups
>>> to yes, checking the socket path, permissions, ownership etc. I am out
>>> of options.
>>> So if someone has a clue I would be more than happy.
>>> Thanks in advance,
>>> Wouter Berkepeis
> It's like more a case of "this is nothing new";  clamd has IIRC always
> gone nonresponsive when reloading the signatures, and adding gobs of
> third-party sigs doesn't help speed things up.
>
> There was a suggestion at one point for clamd to do the reload in
> parallel with continuing to process with the previous loaded set of
> signatures, but that would require at least double the RAM usage
> periodically, and there have also been ongoing grumbles about clam's
> memory usage - any change that would cause clamd to periodically double
> its RAM usage would seriously upset some people who run it on low-memory
> systems.
>
> About the best you can do is to make sure whatever talks to clamd
> tempfails sanely when it can't connect or communicate, and that it will
> retry later.  MTAs using the milter interface to mediate between mail
> server and clamd can be set up to do this at the milter layer and some
> can retry the actual call to clamd as well IIRC.  I can't say about exim
> because the first thing I do on Debian installs where I care even the
> slightest bit about mail traffic is to drop Exim in favour of Postfix or
> sendmail.

It all depends on your server.  I have two servers that run Clam. The 
first, an old PII-400 that's no longer in use, takes 2 minutes to reload 
the base signatures and with only 512M RAM, went into swap when I tried 
to load all the 3rd party sigs.  The second server is a much newer 
dual-core Pentium G620 running at 2.5GHz with 4G RAM.  It loads the main 
sigs as well as a ton of 3rd party sigs in about 30 seconds.

You said you have 2G RAM.  What CPU are you using?

-- 
Bowie