[clamav-users] ClamAV® blog: ClamAV, CVDs, CDIFFs and the magic behind the curtain

[Andrew C Aitchison]

On Fri, 19 Mar 2021, Joel Esler (jesler) via clamav-users wrote:

> https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html
> 
> ClamAV, CVDs, CDIFFs and the magic behind the curtain

3. ... This is an expensive operation in terms of bandwidth
    because daily.cvd and main.cvd are, currently, 105 MB and 117 MB,
    respectively.
    ... For example, for an update where 10,000 signatures were removed
    from daily, the corresponding CDIFF was only around 60 KB in size.
    ...
    To update via CDIFF, FreshClam determines the version of the database
    on disk and requests every CDIFF between that version and the latest.
    Assuming each of those CDIFFs exists on the server (only the last
    90 days worth are currently kept) ...

    60KB * 90 ~= 5MB << 100MB.

    A zero-byte CDIFF indicates that FreshClam should download the CVD
    instead. This is sometimes preferred to patching when a significant
    portion of the CVD changes, like when a large portion of daily is
    migrated to main in a single update.

So a machine which is 100 updates behind will download 100+MB of .cvd
instead of <10MB of .cdiff files :-(

I think I may have read that the 90 CDIFF files was being reviewed
which sounds like a good idea
(except of course when there has been a large daily -> main migration).

Is it possible to configure freshclam to keep the (verified) cdiffs if the
update fails, so that they don't have to be downloaded on the next update
attempt ?

Thanks,

-- 
Andrew C. Aitchison                                     Kendal, UK
                         andrew at aitchison.me.uk