[clamav-users] clamav-milter: Failed to create temporary file

[J. David Rye]

On Thu, 2014-08-21 at 19:22 -0400, Steven Morgan wrote:
> Hi Urban,
> 
> I took a look at this code. The real problem is the inability to
> create a
> temporary file. The second message just results from the return code
> of the
> function that attempts to create the temp file. We need to find out
> why the
> temp file creation fails. There should also be a clamav error message
> written from:  cli_errmsg("cli_gentempfd: Can't create temporary file
> %s:
> %s\n", *name, strerror(errno)); Can you find this message?
> 
> Otherwise, it is a memory allocation failure for space for the temp
> file
> name, which seems unlikely.
> 
> Steve

I am also seeing this issue. Mostly intermitant but see further down.

cli_errmsg wont work if clamav-milter has daemonezed.
it only writes to STDERR and the function daemonize closes standard
error even if you recompile with CL_DEBUG set. 

Only way to get is to get the error messages from  cli_gentempfd seams
to be to uncomment the line

"#Foreground yes"

In clamav-milter.conf, then run in foreground from command line.

As an aside I wonder why cli_gentempfd does not use the function logg()
and output to file or syslog depending on configuration file.

I am running clamav-milter on a VM. OS is CentoOS 6.5
VM has 4 vcpu, and 2GB RAM
clamav-milter is version 0.98.4-1.el6.rf installed from rpmforge
repository.

Looking at he logs if time stamps in syslog for calls to clamav-milter
are two seconds or more apart the problem never shows.

However if 4 or more messages arrive in two seconds problem always shows
up, the failure to create temp file is usually time stamped 2 seconds
after the first message in the burst that triggered it.

On a sustained burst of traffic pretty much all the messages trip the
issue.
In a 1 hour period last week when I had a lot of messages due to a
different issue. I had 20,000 temp file failures, and 23 messages
delivered.

[root at mailhost-c6 etc]# clamav-milter
--config-file=/etc/clamav-milter.conf.foreground
Local socket unix:/var/run/clamav/clamd.sock added to the pool (slot 1)
Probe for slot 1 returned: success
LibClamAV Error: cli_gentempfd: Can't create temporary
file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
ERROR: Failed to create temporary file
ERROR: Failed to initiate streaming/fdpassing
LibClamAV Error: cli_gentempfd: Can't create temporary
file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
ERROR: Failed to create temporary file
ERROR: Failed to initiate streaming/fdpassing
LibClamAV Error: cli_gentempfd: Can't create temporary
file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
ERROR: Failed to create temporary file
ERROR: Failed to initiate streaming/fdpassing
LibClamAV Error: cli_gentempfd: Can't create temporary
file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
ERROR: Failed to create temporary file
ERROR: Failed to initiate streaming/fdpassing
LibClamAV Error: cli_gentempfd: Can't create temporary
file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
ERROR: Failed to create temporary file
ERROR: Failed to initiate streaming/fdpassing
Message from <n0r3ply812361 at scotland117.wanadoo.co.uk> to
<stevensonbros> infected by Heuristics.Phishing.Email.SpoofedDomain
Message from <n0r3ply620601 at aughamullan.dungannon.ni.sch.uk> to
<brett01> infected by Heuristics.Phishing.Email.SpoofedDomain
Probe for slot 1 returned: success


I think

clamav-milter does a lot of initialization, including setting up a
structure with a list of function entry points it then calls smfi_main.

smfi_main in turn forks one thread for each message, and calls the entry
points in the context of the thread.

call back to function clamfi_header
that calls sendchunk
which calls nc_connect_rand
which calls cli_gentempfd which prints the EEXISTS errors to stderr
shown above.

File name looks like it is supposed to be based on a 16 byte MD5 digest
printed in hex.

cli_gentempfd builds the name by calling cli_gentemp
	which adds 32 bytes of random data from cli_rndnum to a 16 byte seed
	passes a pointer the the 48 buffer to cli_md5buff
	which in turn calls cl_hash_data
	digest returned is used to create the file name, 
	and update the seed for the next name.


Note cli_gentemp is common to all programs in the clam set.

Note cl_hash_data is calling openssl library functions to calculate the
digest.

Note cli_md5buff does not check for an error in cl_hash_data