[clamav-users] Upgrade to 0.100.0 disables CL_TYPE_ZIP regex signatures for Office files

[David Shrimpton]

Upgrade of clamav to 0.100.0 disables Container CL_TYPE_ZIP regex signatures
for Office 2007+ files.  Eg signatures attempting to match a contained file
of an Office zip.

Prior to 0.100.0 the Container for Office files was classified only as CL_TYPE_ZIP.
With 0.100.0 the  Container is classified as both
(CL_TYPE_OOXML_WORD or CL_TYPE_OOXML_XL or CL_TYPE_OOXML_PPT) and CL_TYPE_ZIP.
Existing signatures pre 0.100.0 would all be CL_TYPE_ZIP.

With 0.100.0 it appears regexes are pooled as one regardless of Container type 
and files are first scanned with Container set to one of the CL_TYPE_OOXML_* then same pool
of regexes is run with Container set to CL_TYPE_ZIP.

But with no hits on a file during the  CL_TYPE_OOXML_*  run the file  md5 is cached as clean 
so that file is  not re-scanned with container set to CL_TYPE_ZIP .
Thus the  CL_TYPE_OOXML_* run disables  the CL_TYPE_ZIP run.

The only time a CL_TYPE_ZIP signature may work is with -z and if an CL_TYPE_OOXML_* sig is hit 
as caching is turned off for the rest of the files during the  CL_TYPE_OOXML_* Container run
when there is a sig hit.   This would also need the file that triggers the CL_TYPE_ZIP to be
the same file as that  triggering the CL_TYPE_OOXML_* sig or to be a file scanned after
that file (so the file  is not in the clean cache). 

--disable-cache for clamscan or 'DisableCache yes' in clamd.conf fixes the problem.

The fix would be to not cache files as clean until all Container types are tested.

Same problem I expect would apply in other multiple Container situations.

The problem might impact a large pool of existing signatures as well as new ones using
CL_TYPE_ZIP and not one of the CL_TYPE_OOXML_*.

Another unrelated problem is that Flash used to be container  CL_TYPE_ZIP but are now CL_TYPE_SWF so
some sigs for flash using CL_TYPE_ZIP may no longer work.

David Shrimpton