[clamav-users] Scanning a zip file fails, extract it, scan with the same options and it passes
Mark Fortescue
mark.lists at thurning-instruments.co.uk
Mon Oct 4 10:21:09 UTC 2021
Hi Max,
Please be aware that at present, ClamAV can't scan files above 2G
because of internal variable limitations.
If your ZIP files are around 2G then this may be the limit you are
encountering.
Regards
Mark.
On 04/10/2021 09:37, Max Allan via clamav-users wrote:
> Thanks Laurent,
>
> The results say : "Infected files: 1". Therefore our automated systems
> cannot differentiate between this file being infected and not. And if
> the AV scanner has stopped scanning before everything was scanned, it
> MAY be infected and I cannot allow the file in.
>
> If it went above a limit, which limit do I need to increase to make it
> scan. There is only one extra level of zip nesting and none of the
> files are large. So I can presumably increase one of those limits by
> "1" to count for the extra level of nesting perhaps. But I increased
> the limits related to recursion massively and it still fails. All of
> those limits are far, far bigger than the content (mostly 2GB and
> hundreds of thousands of files). And content that it CAN scan when
> extracted, so clearly there is nothing there that is beyond a limit.
>
> I do not want files beyond a limit to remain unscanned, I want them
> all to be scanned. And I can achieve that manually by extracting the
> .zip. But it fails to scan when the zip still compressed. It is not
> feasible to have an automated AV system uncompress every zip file it
> finds outside of the scan solution and feed in all found zip files as
> exclusions from the scan.
>
> So I ask again, why does it hit a limit when in a .zip file but not
> when the zip is expanded, when all the limits are clearly much higher
> than anything it will encounter?
>
> Max
>
> On Fri, 1 Oct 2021 at 18:06, Laurent S. via clamav-users
> <clamav-users at lists.clamav.net> wrote:
>>
>> Dear Max Allan,
>>
>> Heuristics.Limits.Exceeded doesn't mean the file is infected, but it's
>> only a warning telling you that something went above the limits you set.
>> It give the warning this way because of --alert-exceeds-max=yes
>>
>> ClamAV managed to go search into those files in each case as you can see
>> from the scan summaries. It will count the zip as a single file.
>>
>> I would recommend against copy-pasting all those parameters without
>> having given proper thought into what you are doing.
>>
>> Best regards,
>> Laurent
>>
>> On 01.10.21 18:09, Max Allan via clamav-users wrote:
>>> Hi,
>>> I have a requirement (from the business) to AV scan all docker
>>> containers we create.
>>> I started experimenting with tomcat:latest, which is handy because you
>>> can follow along at home easily!
>>> Someone else has already recommended a scan command :
>>>
>>> clamscan <file> \
>>> --infected \
>>> --recursive=yes \
>>> --alert-exceeds-max=yes \
>>> --max-recursion=2000000 \
>>> --max-dir-recur
>> sion=2000000 \
>>> --max-files=2000000 \
>>> --max-filesize=2000M \
>>> --max-scansize=2000M \
>>> --max-embeddedpe=2000M \
>>> --max-htmlnormalize=2000M \
>>> --max-htmlnotags=2000M \
>>> --max-scriptnormalize=2000M \
>>> --max-ziptypercg=2000M \
>>> --max-partitions=2000000 \
>>> --max-iconspe=2000000 \
>>> --max-rechwp3=2000000 \
>>> --pcre-match-limit=2000000 \
>>> --pcre-recmatch-limit=2000000 \
>>> --pcre-max-filesize=2000M -a
>>>
>>> So, if you run the tomcat:latest container, apt update, apt install
>>> clamav, freshclam and run that scan command against
>>> /usr/local/openjdk-11/lib/src.zip you will probably get a failure :
>>>
>>> /usr/local/openjdk-11/lib/src.zip: Heuristics.Limits.Exceeded FOUND
>>> /usr/local/openjdk-11/lib/src.zip!(0)ZIP:jdk.zipfs/jdk/nio/zipfs/ZipInfo.java:
>>> Heuristics.Limits.Exceeded FOUND
>>> ---------- SCAN SUMMARY -----------
>>> Known viruses: 8570214
>>> Engine version: 0.103.3
>>> Scanned directories: 0
>>> Scanned files: 1
>>>
>> Infected files: 1
>>> Data scanned: 290.07 MB
>>> Data read: 55.52 MB (ratio 5.22:1)
>>> Time: 260.438 sec (4 m 20 s)
>>> Start Date: 2021:10:01 13:39:47
>>> End Date: 2021:10:01 13:44:07
>>>
>>>
>>> However, if I extract that zip file to /src and then run clamscan on
>>> /src then it passes without a problem :
>>>
>>> ----------- SCAN SUMMARY -----------
>>> Known viruses: 8570214
>>> Engine version: 0.103.3
>>> Scanned directories: 2076
>>> Scanned files: 18415
>>> Infected files: 0
>>> Data scanned: 333.04 MB
>>> Data read: 170.92 MB (ratio 1.95:1)
>>> Time: 320.573 sec (5 m 20 s)
>>> Start Date: 2021:10:01 13:23:39
>>> End Date: 2021:10:01 13:29:00
>>>
>>> (There are indeed 18415 files in that .zip according to unzip -l)
>>>
>>> Or even scan the single file :
>>>
>>> clamscan ZipInfo.java --infected --recursive=yes
>>> --alert-exceeds-max=yes --max-recursion=2000000
>>> --max-dir-recursion=2000000 --max-files=2000000
>>> --max-filesize=2000M --max-scansize=2000M --max-embeddedpe=2000M
>>> -
>> -max-htmlnormalize=2000M --max-htmlnotags=2000M
>>> --max-scriptnormalize=2000M --max-ziptypercg=2000M
>>> --max-partitions=2000000 --max-iconspe=2000000
>>> --max-rechwp3=2000000 --pcre-match-limit=2000000
>>> --pcre-recmatch-limit=2000000 --pcre-max-filesize=2000M -a
>>>
>>> ----------- SCAN SUMMARY -----------
>>> Known viruses: 8570214
>>> Engine version: 0.103.3
>>> Scanned directories: 0
>>> Scanned files: 1
>>> Infected files: 0
>>> Data scanned: 0.01 MB
>>> Data read: 0.01 MB (ratio 1.50:1)
>>> Time: 68.326 sec (1 m 8 s)
>>> Start Date: 2021:10:01 16:03:14
>>> End Date: 2021:10:01 16:04:22
>>>
>>>
>>>
>>> Clearly the content of src.zip (ZipInfo.java) IS scannable, when
>>> extracted, but for some reason not scannable when it is in a zip
>>> file... Is this a bug? Or am I specifying some options that are
>>> causing it??
>>>
>>> (clamscan -V
>>> ClamAV 0.103.3/26309/Fri Oct 1 09:03:53 2021 )
>>>
>>> _______________________________________________
>>>
>>> clamav-users mailing list
>>> clam
>> av-users at lists.clamav.net
>>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>>
>>>
>>> Help us build a comprehensive ClamAV guide:
>>> https://github.com/vrtadmin/clamav-faq
>>>
>>> http://www.clamav.net/contact.html#ml
>>>
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users at lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>>
>> http://www.clamav.net/contact.html#ml
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
More information about the clamav-users
mailing list