[clamav-users] Scanning very large files in chunks
alvarnell at mac.com
Thu Aug 4 22:14:05 EDT 2016
Does anybody have any evidence of malware that exceeds 4GB? Although I can certainly see the utility of the proposed capability as a hedge for the future, it would seem to be a waste of time and compute power to scan such large files today.
With the ever increasing malware issues we face today, it’s important to consider this:
Risk = threat x vulnerability x consequence
We all need to focus on fixing the high risk items first.
Sent from Janet's iPad
On Aug 4, 2016, at 4:40 PM, sapientdust+clamav at gmail.com wrote:
> I've recently run into the issue of clamd not being able to scan files
> that are larger than a small number of GB, and I have seen the
> warnings in `man clamd.conf` that say specified limits above 4GB are
> Could developers or other folks familiar with the clamd codebase
> comment on the feasibility of scanning large files in multiple pieces
> as a way of handling larger files?
> For example, given a file that is 6GB, does using multiple INSTREAM
> calls (that's how I'm interacting wth clamd currently) to check the
> full 6GB seem like it should work reliably?
> INSTREAM: bytes 0-1000MB
> INSTREAM: bytes 900MB-1.9GB
> INSTREAM: bytes 1.8GB-2.8GB
> INSTREAM: bytes 2.7GB-3.7GB
> INSTREAM: bytes 3.6GB-4.6GB
> INSTREAM: bytes 4.5GB-5.5GB
> INSTREAM: bytes 5.4GB-6.0GB
> There is overlap above, wherein the 100MB of data that starts at the
> 900MB position is scanned twice, once in the first call (as the last
> 100MB of that stream) and once in the second call (as the first 100MB
> of that stream), to reduce the possibility of a virus being split into
> two pieces and therefore not recognized.
> If Clamav needs the first bytes in order to know
> what kind of file it is scanning and trigger filetype-specific heuristics, then
> something like the above could be adapted so that the first N bytes of the
> first chunk are prepended to each subsequent chunk that is checked for
> that file.
> Thanks for any guidance or feedback you can provide.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 2370 bytes
Desc: not available
More information about the clamav-users