[clamav-users] Scanning very large files in chunks

G.W. Haywood clamav at jubileegroup.co.uk
Tue Aug 9 12:40:11 EDT 2016

Hi there,

On Tue, 9 Aug 2016, sapientdust+clamav at gmail.com wrote:
On Thu, Aug 4, 2016 at 7:14 PM, Al Varnell <alvarnell at mac.com> wrote:

>> ... Risk = threat x vulnerability x consequence
> I agree. In my case, the consequence factor is very large ...

Perhaps you can elucidate the consequences.  If the consequence factor
is as you say very large, then you have a problem to solve.

> I have to scan even the large files somehow.

This will not solve the problem.  It can never and will never solve it.
You need to find another way of going about things.

> Skipping large files would just provide an easy attack vector ...

Then you have to fix the system so that it wouldn't be easy.

> Does anybody have any feedback on the proposed solution to scanning
> large files in chunks?

Stop worrying about it, it's a waste of time and effort.  The probability
that you will actually find what you're looking for is very small.

> ... are there any reasons that the method wouldn't work for all file
> types, assuming that the initial bytes of the file are prepended to
> each chunk so that ClamAV knows what type of file it is?

Yes.  Because of what I wrote above.  Forget prepended bytes and fancy
ways of doing things that won't solve the problem.  Look at the problem
in a different way.  I'm sure this isn't what you want to hear, but it's
the way things are.

I don't worry about viruses.  The reason for that is that I don't use
Windows boxes.  The main reason I use ClamAV is to stop spam and similar
junk which third-party databases do pretty well.  Scanning for viruses
and similar is just a bonus as far as I'm concerned, it means that if
something is found then we might be able to alert somebody to a problem
that they might have, or we might be able to avoid passing something
on from one correspondent to another through our mail.

But of course we might not find it.

Like all virus scanners, ClamAV performance is not 100% and it never
will be.  I suspect it's nearer 30% of the viruses that my servers
see, but that's just my personal experience in what are probably very
atypical systems -- for a start, 25% of the internet address space is
firewalled and if a packet gets past the firewalls it gets harder from
there; spammers and purveyors of malware get firewalled for a single
offence, permanently, and their entire network gets firewalled, not
just the one IP that tried it on.  Very atypical.  But the point is
that I still see *new* threats which will not usually be found by any
scanner and if the system is vulnerable it will succumb.

If you want to test ClamAV performance, set up a mail server and grab
all the cr at p it sees for a few months.  Run all that past a couple of
dozen virus scanners such as you can find on jotti.org and then come
back and tell us what you've found.



More information about the clamav-users mailing list