[Clamav-devel] Building ClamAV 0.99 with PCRE support

Kevin Lin klin at sourcefire.com
Wed Dec 9 11:42:39 EST 2015


In a short explanation, there's a pcre compiler flag we use to suppress UTF
regex patterns which is only available in certain versions of lpcre. The
transfer of binaries across different machines causes this flag to be
dropped by the lpcre binary though I'm not certain why even the direct
importing of the lpcre binary would result in the flag being dropped.

In the future, it might be best to remove the enforcement of the UTF
suppression and leave the issue up to the signature writer.

-Kevin

On Wed, Dec 9, 2015 at 6:19 AM, Mark Allan <markjallan at gmail.com> wrote:

> Hi Kevin,
>
> Yes, that's fixed it. Thanks so much.
>
> I still can't fathom why it would work differently on different versions
> of OS X, but it looks like you've solved the problem I'm seeing for now.
>
> Thanks again
> Mark
>
> > On 8 Dec 2015, at 8:21 pm, Kevin Lin <klin at sourcefire.com> wrote:
> >
> > Can I ask you to try this patch and tell me if it fixes the issue? If the
> > issue persists, please submit the debug log. Thanks.
> >
> > -Kevin
> >
> > On Tue, Dec 8, 2015 at 2:00 PM, Mark Allan <markjallan at gmail.com> wrote:
> >
> >> Hi Kevin,
> >>
> >> Thanks.
> >>
> >> Yes, the configure options are definitely the same. In fact PCRE isn't
> >> installed on OS X by default, so I'm compiling it as well and copying
> all
> >> the binaries & libraries etc over to the destination machines at the
> same
> >> time as copying the ClamAV binaries.
> >>
> >> Mark
> >>
> >>> On 8 Dec 2015, at 6:36 pm, Kevin Lin <klin at sourcefire.com> wrote:
> >>>
> >>> It appears that the PCRE library is correctly linking in and ClamAV is
> >>> making calls to it. The error message:
> >>>
> >>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >>> unknown option bit(s) set
> >>>
> >>> results directly from a failed compilation of PCRE regex which in this
> >> case
> >>> is due to an unknown option bit being set.
> >>>
> >>> Looking into it, the options that can be passed to pcre_compile are
> >> fairly
> >>> common ones; the only real exception is PCRE_NEVER_UTF which was added
> in
> >>> 8.33. It's possible that the flag existed on the source machine but not
> >> the
> >>> destination.Are the PCRE configure options consistent across the source
> >> and
> >>> all the destination machines?
> >>>
> >>> -Kevin
> >>>
> >>>
> >>>
> >>> On Tue, Dec 8, 2015 at 12:15 PM, Mark Allan <markjallan at gmail.com>
> >> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> Are there any recommendations for compilation options on/with pcre?
> >>>>
> >>>> I've tried several things but can't seem to get the build to work on
> >>>> anything other than OS X 10.11.  I never have a problem moving my
> ClamAV
> >>>> builds between machines, but something's going wrong with PCRE
> support -
> >>>> even when I build PCRE on the destination machine, I always end up
> with
> >> the
> >>>> following error from clamscan:
> >>>>
> >>>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >>>> unknown option bit(s) set
> >>>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> >>>> ERROR: Database initialization error: Malformed database
> >>>>
> >>>>
> >>>> Here's a section of the output with --debug on:
> >>>>
> >>>> LibClamAV debug: Ignoring signature Email.Trojan-417
> >>>> LibClamAV debug: main.ndb loaded
> >>>> LibClamAV debug: main.zmd loaded
> >>>> LibClamAV debug: main.fp loaded
> >>>> LibClamAV debug: in cli_tgzload_cleanup()
> >>>> LibClamAV debug: /usr/local/share/clamav/main.cvd loaded
> >>>> LibClamAV debug: Using filter for trie 0
> >>>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >>>> unknown option bit(s) set
> >>>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> >>>> ERROR: Database initialization error: Malformed database
> >>>> LibClamAV debug: Cleaning up phishcheck
> >>>> LibClamAV debug: Freeing phishcheck struct
> >>>> LibClamAV debug: Phishcheck cleaned up
> >>>>
> >>>> These are the options I'm passing to pcre's configure phase:
> >>>> ./configure --prefix=/usr/local --enable-newline-is-any --enable-utf
> >>>> --enable-unicode-properties --enable-rebuild-chartables
> --enable-pcre16
> >>>> --enable-pcre32 --enable-jit
> >>>>
> >>>> This is what I'm passing to ClamAV's configure phase:
> >>>> ./configure --disable-dependency-tracking  --enable-llvm=no
> >>>> --enable-clamdtop --with-user=_clamav --with-group=_clamav
> >>>> --enable-all-jit-targets --with-pcre=/usr/local --prefix=/usr/local
> >>>>
> >>>> I get the same results regardless of what options I pass to PCRE's
> >>>> configure script. I've also tried pcre-8.37 and pcre-8.38.
> >>>>
> >>>> Can anyone suggest anything?
> >>>>
> >>>> Many thanks
> >>>> Mark
> >>>>
> >>>>> On 20 Nov 2015, at 6:01 pm, Mickey Sola <msola at sourcefire.com>
> wrote:
> >>>>>
> >>>>> Hi Mark,
> >>>>>
> >>>>> Unfortunately, as of right now the only way to get pcre 8.38 is via
> >> their
> >>>>> rc1 candidate (check the pcre-dev mailing list for a tarball).
> >>>>>
> >>>>> In practice, the pcre exploit ClamAV warns about (
> >>>>> http://www.securitytracker.com/id/1032453) relies upon an explicitly
> >>>>> malicious regex, so you don't have to worry too much unless you're
> >> using
> >>>>> untrusted sigs. Everything should still compile and run just fine,
> even
> >>>>> with 8.37.
> >>>>>
> >>>>> - Mickey
> >>>>>
> >>>>> On Fri, Nov 20, 2015 at 8:08 AM, Mark Allan <markjallan at gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I saw the blog post about v0.99 rc 2 and have downloaded it for
> >> testing.
> >>>>>>
> >>>>>> It looks like bug 11411 [
> >>>>>> https://bugzilla.clamav.net/show_bug.cgi?id=11411 ] is still open,
> >> so I
> >>>>>> decided to download and build PCRE as well.
> >>>>>>
> >>>>>> I initially tried the PCRE2 branch but it wasn't recognised by
> >> ClamAV's
> >>>>>> configure script, so I went with the most up-to-date version of PCRE
> >>>> (which
> >>>>>> is currently 8.37) but now configure outputs the following:
> >>>>>>
> >>>>>> configure: WARNING: The installed pcre version may contain a
> security
> >>>> bug.
> >>>>>> Please upgrade to 8.38 or later: http://www.pcre.org
> >>>>>>
> >>>>>> There is no 8.38 that I can see:
> >>>>>>      https://sourceforge.net/projects/pcre/files/pcre/
> >>>>>>
> >>>>>> Are you just assuming that 8.38 will be coming soon to fix the bug,
> or
> >>>> is
> >>>>>> there a download somewhere that I'm not seeing?
> >>>>>>
> >>>>>> Thanks
> >>>>>> Mark
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> http://lurker.clamav.net/list/clamav-devel.html
> >>>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>>>>
> >>>>>> http://www.clamav.net/contact.html#ml
> >>>>>>
> >>>>> _______________________________________________
> >>>>> http://lurker.clamav.net/list/clamav-devel.html
> >>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>>>
> >>>>> http://www.clamav.net/contact.html#ml
> >>>>
> >>>> _______________________________________________
> >>>> http://lurker.clamav.net/list/clamav-devel.html
> >>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>>
> >>>> http://www.clamav.net/contact.html#ml
> >>>>
> >>> _______________________________________________
> >>> http://lurker.clamav.net/list/clamav-devel.html
> >>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>
> >>> http://www.clamav.net/contact.html#ml
> >>
> >> _______________________________________________
> >> http://lurker.clamav.net/list/clamav-devel.html
> >> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>
> >> http://www.clamav.net/contact.html#ml
> >>
> > <pcre_flags_try.patch>_______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>


More information about the clamav-devel mailing list