[clamav-users] Input Stream Scanning for very large files

Micah Snyder (micasnyd) micasnyd at cisco.com
Fri Jan 25 15:48:22 EST 2019


By no means am I happy with the 2-4GB file size limit, but it will take a fair amount of work to properly change all of the integer types throughout the codebase to handle larger file offsets.  The specific task of raising the limit for 64bit machines isn't on our roadmap, but is definitely something I want to do.

-Micah


On Jan 25, 2019, at 1:12 PM, Paul Kosinski <clamav-users at iment.com<mailto:clamav-users at iment.com>> wrote:

I understand that it's impractical for ClamAV to scan exceedingly large
files, as it could fill up RAM and/or page forever. But the current 4GB
hard limit is overly restrictive, especially since 32-bit addresses and
numbers are ancient history in current OSes.

In particular, scanning big archives immediately after downloading is
desirable, since there can be malware that attacks the de-compressing or
de-archiving mechanism itself. Thus simply scanning the individual
contained files isn't completely adequate.

Is there any plan to allow scanning bigger files? There still are,
after all, size limits specifiable in the config file(s), and warnings
about the consequences of setting limits too big can be documented.


On Fri, 25 Jan 2019 14:32:55 +0000
"Micah Snyder (micasnyd)" <micasnyd at cisco.com<mailto:micasnyd at cisco.com>> wrote:

Regarding specific limits, I'm sorry to say that ClamAV is presently
limited to max file size of 4GB on most systems (and, I think
unintentionally, 2GB on some systems).

-Micah


On Jan 24, 2019, at 4:23 PM, J.R.
<themadbeaker at gmail.com<mailto:themadbeaker at gmail.com><mailto:themadbeaker at gmail.com>> wrote:

I think I framed my problem statement differently.
So, our requirement is similar the one asked by John in the
below link. I do not know if the solution proposed is a correct one..
Also, how do you propose I should scan an archive of 100GB ( let's
say) size. Does clamav have any limitations on scanning a single file
of such huge size ??

Without knowing more about this "archive" it's hard to say if ClamAV
will even pick up anything, due to the reason Micah gave in his reply.
But another issue is if this is just one humongous file you are trying
to shove through and say it *does* trigger some virus... How are you
going to know what / where the virus is? All you know is its somewhere
in your massive archive file...

You would be much better off scanning the individual files as you
assemble said archive, and obviously only need to scan files where an
infection would make sense (i.e. a text file isn't going to contain a
virus)...

There are stream settings in the clamd.conf, but I don't know what the
hard upper-limits are.

In cases like this, it's probably best to assemble you own sample
archives, one clean & one infected, and run through your proposed
process. If it works as intended, then create a few more samples and
re-test... If it doesn't work as intended then you'll need to re-think
your process...


_______________________________________________
clamav-users mailing list
clamav-users at lists.clamav.net<mailto:clamav-users at lists.clamav.net>
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20190125/4b5d55f0/attachment.html>


More information about the clamav-users mailing list